SQL — a Common Language for the Whole Data Team

Today, I wanted to share how we’ve empowered colleagues outside the data engineering team to write their own data transformations by overcoming coding language barriers. Within our Data team, there are several specializations: Data Analysts create dashboards and analyze data for business leads and product teams across the company.Data Scientists apply machine learning technology at … Continue reading SQL — a Common Language for the Whole Data Team

Bulk Log Analytics With Hive

Leveraging the distributed powers of MapReduce to perform custom log analysis or some one-time queries on the raw data is fast and easy and you don't even have to build a complicated ETL process to do it. The data engineering team at WordPress.com recently used this approch to query tens of billions of log lines with just a couple minutes of work.

Continue reading