Anomaly detection and time series forecasting are valuable in monitoring the financial and technical health of an organization. Proper modeling of time series requires accounting for periodic fluctuation; malicious users; data irregularity, saturation or scarcity; sudden peaks and drops. To account for these parameters, the modeler needs to select the proper model family, optimize the model parameters, validate the assumptions, and refine the process as needed. The task is even more complicated when one needs to build a self-service application that supports "slicing and dicing" any metric to its underlying components. In such a case, where the number of possible models can be counted by thousands, manual tuning is impossible. In this lecture, I show how a series of assumptions and simplifications allowed completing the modeling task in one week, using open source Python packages. I will review all the assumptions, their implications, and limitations. I will also show which modeling approach worked, and which didn't work in case of Automattic, the company behind WordPress.com, Jetpack and other projects, that serves more than 180,000,000 unique visitors a month in the US alone. I hope that this information will be useful in many data-driven organizations.

… Continue reading