A Multivariate test enables you to test combinations of changes simultaneously on your website. You can change several elements on a page simultaneously (e.g., the color and wording of a CTA) and identify which of the possible combinations performs best according to the goal you wanted to reach (e.g., increasing the number of clicks, the number of pages viewed, decreasing the bounce rate). You can then apply these changes directly to your website.
Unlike an A/B test, which involves testing each hypothesis in a different test, a multivariate test allows you to run hypotheses at the same time and to find the best combination of all.
The purpose of multivariate tests is to measure the interactive effects between several supposedly independent elements (e.g. page title and visual illustration).
A subtest contains variations that will be tested independently across multiple combinations. For example, if your goal is to test a simple CTA, you might want to create three subtests: Color, Shape, Wording, with as many variations of colors, shapes and wordings as necessary. AB Tasty will create all the possible combinations and display them randomly to your traffic.
⭐ Good to know
The number of combinations created is calculated as follows: [Number of variations of the 1st subtest]*[Number of subtests].
The more subtests with variations you configure, the more randomly your traffic is allocated between the existing combinations.
In this case, the readiness of your campaign and data reliability takes more time to be reached than a classic A/B test.
Here are the 9 tips for a successful set-up:
- MAIN INFORMATION: We recommend establishing a hypothesis of your test according to the following model: If I apply [these changes on my webpage] to [this audience], then [it will impact] and enables to enhance [this goal]. And to add it in the description field.
By default, only one subtest is created, but you can add as many subtests as necessary. For each subtest, enter the URL you want to load in the editor. In most cases, you must enter the same URL for all subtests.
- EDITOR: For each subtest, you can add as many variations as you want. For example, if your subtest one is related to the color of a button. In a variation 1, you can test the blue color and in a variation 2, a red color.
- GOALS: You can’t select any goals for an MVT, only the transaction goal is available and preselected.
- TARGETING: Be careful when entering the targeted pages (in the Where section) of your test to ensure that it will display correctly on the pages you want to target (e.g. Product pages).
To trigger the test only for visitors using a specific device, use the Device criterion in the triggers section. When the Who, How and Where sections have the same configuration for each sub-test, you can use the Replicate targeting option.
- TRAFFIC ALLOCATION: All users must be tracked. Each percentage of the targeted traffic is assigned to a subtest.
- QA: QA is a fundamental step to make sure that the modifications appear correctly on all targeted devices and that your action trackings have been configured correctly. For more information, refer to Using the QA assistant.
- Before launching your test into production (that is to say, making it visible to your audience), make sure the QA mode is disabled and the targeting of your test has been saved.
- We recommend that you let your MVT run for at least 15 days before analyzing it and wait until at least 5,000 unique visitors are tested per variation and per combination (with around 500 conversions) in order to guarantee reliable results. For more information, refer to How long does a test take?
- REPORT: It displays a breakdown of the weight of each variation within the combination. The weight of a variation is the impact a particular variation has on the improvement identified in this combination.
💡 Use cases
Multivariate tests can be used in the following cases:
🖊️ Action / modification
Testing the wording and color of a CTA
Increasing the conversions on a CTA
Testing the layout of several elements on a page (coupon code field, CTA, etc.)
Optimizing the shopping cart
Need additional information?
Submit your request at firstname.lastname@example.org
Always happy to help!