Experimentation


  • Challenge Hiring Assumptions with Data

    We use experiments to model reality (sometimes to create alternative realities as in A/B experiments), to understand reality, and ultimately, to make decisions moving us ever closer to our goals. Improving iteratively, we learn not only from successful experiments but also from failed attempts. Experiments are important because they provide us with measurements. And measurements…

  • Architecting ExPlat: Automattic’s New Experimentation Platform

    Last month, my colleague Aaron Yan published a broad overview of Automattic’s new Experimentation Platform (ExPlat). This month, we’ll dive deeper into ExPlat’s architecture and design, explain how the landscape at Automattic informed our architectural decisions, and describe the platform’s main components. Future posts will share details on each component and other aspects of experimentation…

  • ExPlat: Automattic’s Experimentation Platform

    Over the past 18 months, the Decision Science team has been building the Experimentation Platform (ExPlat): a tool to help our colleagues run experiments to improve customer experiences, inform product decisions, and quantify the impact of newly released features on many of Automattic’s products such as WordPress.com, Jetpack, Akismet, and more. In this multi‑part series, we…

  • How to Avoid Statistical Bootstrapping Pitfalls

    Yanir Seroussi shares some insight into the common pitfalls of statistical bootstrapping and how to avoid them.

  • Christo Wilson Discusses the Ethics of Online Behavioral Experiments

    Charles Earl shares the insights he gleaned after hearing a talk from Dr. Christo Wilson about ethics in online behavioral experiements.