Anomaly detection and time series forecasting are valuable in monitoring the financial and technical health of an organization. Proper modeling of time series requires accounting for periodic fluctuation; malicious users; data irregularity, saturation or scarcity; sudden peaks and drops. To account for these parameters, the modeler needs to select the proper model family, optimize the model parameters, validate the assumptions, and refine the process as needed. The task is even more complicated when one needs to build a self-service application that supports "slicing and dicing" any metric to its underlying components. In such a case, where the number of possible models can be counted by thousands, manual tuning is impossible. In this lecture, I show how a series of assumptions and simplifications allowed completing the modeling task in one week, using open source Python packages. I will review all the assumptions, their implications, and limitations. I will also show which modeling approach worked, and which didn't work in case of Automattic, the company behind WordPress.com, Jetpack and other projects, that serves more than 180,000,000 unique visitors a month in the US alone. I hope that this information will be useful in many data-driven organizations.
This week, Charles, Xaio, Chris, and Boris share pieces on machine learning, MySQL, pop lyrics, and career advice.
Charles Earl reports on Elfbot, a machine learning project geared to helping Happiness Engineers provide fast, efficient, wonderfully human support to WordPress.com users.
Two great reads and a YouTube channel await you in this installment of "This Week in Data Reading."
This week, Boris Gorelik shares his thoughts on a NYU study of "The Persuasive Power of Data Visualization."
Boris Gorelik shows how we use data at Automattic to visualize social connections between Automatticians.
In this week in data reading, Demet offers up some digital anthropology over at FiveThirtyEight while Charles shares two papers he enjoyed in the field of natural language processing, and Carly offers a piece on back propagation.