Using a Toggle Router for Multivariate Testing and A/B Experiments

A toggle is a switch that has two positions, either on or off. It’s commonly used in hardware and software to enable or disable system functionality such as the Caps Lock and Num Lock keys on a keyboard. Toggles are also often used to configure settings or preferences (i.e. states of system functionality). They are the preferred control over radio buttons as they take less screen estate and provide clearer context for users as to the current state.

Toggles are also well-suited for multivariate testing or A/B experiments where we need to measure the impact of a change on a specific subset of our audience. This is done by configuring a Toggle Router which consistently sends a user down one or the other codepath depending upon which cohort they belong to. It’s common to use the same approach for things like the purchase flow of an ecommerce site or Call To Action wording on a button.

When using a Toggle Router it’s important to ensure that the configuration can be modified at runtime, especially for more dynamic toggles such as those involved in a Champagne Brunch or Permissioning Toggle. This reduces the need to re-deploy artifacts and re-test. Many savvy teams have taken this a step further and created some type of admin UI that allows team members to modify toggle configuration from source.

When implementing a Toggle Router it’s recommended to test the toggle configuration that is expected to go live in production, as well as a fallback configuration with all toggles flipped Off. Some teams also implement some form of time bombs which will fail a release if the feature flag has not been manually re-flipped after an expiration date.