Here’s what you need to know about setting up your first experiment
Before you get started with creating and running experiments, please ensure that you have integrated our latest SDKs with your games and have the remote config calls setup correctly.
You can find out exactly how-to here
Setup and Segmentation
When you create an A/B test, you will first need to define an audience group that you would like to run the experiment on by selecting a combination of:
- Target Criteria:
- Target Users Percentage
- Filter: Country and OS
- Exclude offline users
Your experiment will only be targeted at New Users – existing players will not be included in any A/B tests. Once a new player has entered an A/B test, they will remain exclusively part of that experiment until you stop the experiment.
You can optionally include or exclude certain Countries and OS Versions in your experiment.
If you’d like the experiment to exclude users who were offline in their first session, select ‘exclude offline users’. You can read about how offline users are treated in our FAQs.
Create Config and Variants
Now you can define the remote config and variant groups:
- Config Key:
- Create the key that you want to perform the A/B test on. This key will need to be supported in all the build previously selected.
- Each different config value to be tested.
- You can create up to 3 variants, allowing for 4 test groups in total, including the control group.
- Players included in the control group, will not receive any value for the A/B test config key.
- Players will be randomly allocated to the control group or a variant.
Decide if you want to start the test right away or at a future date/time. You will also have the ability to select the goal metric now to determine the experiment winner. Metrics available now are Conversion, Retention, Playtime per user, Playtime per session and Session count. We are working on adding more goal metrics to this list.
Here you can:
- See an overview of the A/B test specification
- Give your test a title
- Start the test
After a test has been stopped, you can chose to delete or copy the test.
Copying a test will open the creation flow, with pre-populated selections based off the original test.