Dynamic allocation is an option on the Traffic allocation step of all A/B Tests.
Dynamic traffic allocation is based on a statistical research topic called “multi-armed bandit”. The goal of this algorithm is to limit loss due to the lowest performing variations.
However, when the campaign has just been launched, the performance of the variations is not known yet. When data collection begins, the performance of variations is known progressively, but these measures are not fully significant.
At this point, it would be an error to send all traffic to the apparent winner. However, some formulas can help compute the right amount of traffic to send to each variation with the following goal: sending more traffic to the variation that could win, but not too much, otherwise there wouldn’t be enough traffic to ensure the measurement of the supposed losers.
The solution is to periodically change the allocation regarding previous data, following a statistical formula that ensures the goal mentioned above.
Before switching to dynamic allocation, you need to define at least a primary goal in the Goal step of the test creation flow. The Dynamic allocation algorithm is exclusively based on this primary goal. This means that it detects the best performing variation based on the increase in conversions of the primary goal. For more information, refer to Configuring goals.
The primary goal can be any type of goal except Browsing metrics (Revisit rate, Pages per session and Bounce rate).
To activate dynamic allocation, click the Change/switch to Dynamic allocation button at the top of the page.
A confirmation message is displayed. By confirming, the traffic allocated to each variation will be determined based on the predictive algorithm and on the primary goal you configured for your test. At this point, you won’t be able to manually adjust the percentage of traffic allocated to each variation. However, you are still able to define the percentage of traffic you want to allocate to your test and the percentage of traffic you want to leave untracked.
🚩 Heads up
If you didn’t select a primary goal, you won’t be able to switch/change to dynamic allocation. In this case, you will be asked to define at least a primary goal and will be redirected to the goal step.
Once the test has been launched, you can access the reporting anytime, as you would for any campaign with static allocation.
From the test dashboard, tests using dynamic allocation are easily identifiable thanks to the icon.
Once your test has been launched with dynamic allocation, you won't be able to perform the following actions:
- Change the primary goal
- Add, remove or duplicate a variation within the test
- Switch back to static allocation
If you want to perform any of these actions, you need to pause the test and duplicate it.
💡 Use case
Dynamic Allocation can be very useful in the following cases:
- When you want to optimize micro conversions that are expected to occur within a short period of time after the user has been exposed to a variation. For instance, e-commerce, prefer the “add to cart” goal rather than the transaction, as the primary goal.
- When you have very little time to run a test (ex: testing the headline of a media article, testing the message of a countdown banner).
- When you have really low traffic on the page you want to test, but still want to do some optimizations. Dynamic allocation carries out optimization even before reaching enough data to be fully significant. This would not be very interesting because the traffic is low.
But it may become interesting if carried out on a high number of pages. For instance, if you have low traffic on each individual product page, you can’t carry out a classic A/B test. But it may be interesting to implement multiple dynamic allocation tests, one for each product page. The gain will be low for each individual page, but it can be more significant when multiplied by the number of pages.
- When you have a lot of variations to test (more than 6), dynamic allocation enables you to quickly identify the lowest performing variations in order to run the test on the most relevant ones.
However, we recommend not using dynamic allocation in the following case:
- To optimize a meal ordering website. Since allocation changes over time, the primary goal should be stable during the experiment.
Need additional information?
Submit your request at firstname.lastname@example.org
Always happy to help!