Multivariate tests enable you to test combinations of changes simultaneously. You can change several elements on a page simultaneously and identify which of the possible combinations performs best.
Unlike an A/B test, which involves testing each hypothesis in a different test, a multivariate test allows you to run hypotheses at the same time and to find the best combination of all.
The purpose of multivariate tests is to measure the interactive effects between several supposedly independent elements (e.g. page title and visual illustration).
Creating a multivariate test involves configuring 5 steps (including 4 mandatory steps).
🚩 Heads up
You can’t select any goals for an MVT, only the transaction goal is available and preselected. Also, you can’t select which transaction tag is to be considered. This means that if you have several transaction tags implemented on your account, all transaction data will be mixed up.
In the main information step, you need to enter a name, an optional description and as many subtests as you need in your MVT.
A subtest contains variations that will be independently tested across multiple combinations. For example, if your goal is to test a simple CTA, you might want to create three subtests: Color, Shape, Wording, with as many variations of colors, shapes and wordings as necessary.
For each subtest, you need to enter the URL and a name. A multivariate test generally concerns one page only, however, it can be applied to multiple pages.
By default, only one subtest is created but you can add as many subtests as necessary.
An MVT remains active for longer than a standard A/B test because a certain amount of traffic is required on the tested pages in order to guarantee reliable results.
In a multivariate test, all users are tracked. Each percentage of the targeted traffic is assigned to a subtest and tracked in the report. Some subtests may include 3 variations while others only include 2.
The report of an MVT includes a card at the top of the page that indicates the total number of combinations
As well as a tab displaying the results.
There is a row for each generated combination with the corresponding number of conversions and total visitors count along with the conversion rate.
The tab also displays a breakdown of the weight of each variation within the combination. The weight of a variation is the impact a particular variation has on the improvement identified in this combination. The weight is measured on a scale from 1 to 5.
💡 Use case
For example, you want to increase the number of conversions on the Order button of your product pages. To do so, you need to test several combinations of changes on this unique element. For example, you can test the label and the color of the button.
In the Main information step, enter the name of your test and create the following subtests:
- Subtest 1: button label test
- Subtest 2: button color test
Enter the same URL for both subtests.
In the Variation editor, select the button label subtest from the drop-down list. In variation 1, change “Order” to “Buy”. In variation 2, replace the text with “Purchase”.
Select the button color subtest from the drop-down list. In variation 1, change the button color from green to blue. In variation 2, change the button color from green to red.
Here, two variables have been changed (the label and the color) and each of these variables features three versions (the original and the two variations).
If you change 2 variables and each of these features three versions, there will be 9 combinations in the final ranking (number of variations of the first variable multiplied by the number of variations of the second variable).
In the Targeting step, include the URL you entered in the first step for subtests 1 and 2.
In the Traffic allocation step, leave the default traffic allocation as it is.
Need additional information?
Submit your request at firstname.lastname@example.org
Always happy to help!