Experimentation and A/B Testing


  • Challenge Hiring Assumptions with Data

    We use experiments to model reality (sometimes to create alternative realities as in A/B experiments), to understand reality, and ultimately, to make decisions moving us ever closer to our goals. Improving iteratively, we learn not only from successful experiments but also from failed attempts. Experiments are important because they provide us with measurements. And measurements…

  • ExPlat’s Development Principles and Practices

    When we started working on Automattic’s new experimentation platform (ExPlat), it became clear that we needed to agree on the development practices for the project. Achieving consensus was critical, given Automattic’s fractal nature and its strong emphasis on autonomy. In practice, these attributes lead to a high variability of project development cultures across the company, depending…

  • Architecting ExPlat: Automattic’s New Experimentation Platform

    Last month, my colleague Aaron Yan published a broad overview of Automattic’s new Experimentation Platform (ExPlat). This month, we’ll dive deeper into ExPlat’s architecture and design, explain how the landscape at Automattic informed our architectural decisions, and describe the platform’s main components. Future posts will share details on each component and other aspects of experimentation…

  • ExPlat: Automattic’s Experimentation Platform

    Over the past 18 months, the Decision Science team has been building the Experimentation Platform (ExPlat): a tool to help our colleagues run experiments to improve customer experiences, inform product decisions, and quantify the impact of newly released features on many of Automattic’s products such as WordPress.com, Jetpack, Akismet, and more. In this multi‑part series, we…

  • Christo Wilson Discusses the Ethics of Online Behavioral Experiments

    Charles Earl shares the insights he gleaned after hearing a talk from Dr. Christo Wilson about ethics in online behavioral experiements.