Setup | A/B Testing
Before you get started with creating and running experiments, please ensure that you have integrated our latest SDKs with your games and have the Remote Config call setup correctly. To access A/B Testing, select it from the feature menu. Once in the A/B Testing menu, create a test by selecting the Create button.
Setup and segmentation
When an A/B test is created the first thing you will need to:
- Give it a name
- Select what build does the test target
- Choose whether to include untracked versions automatically in the test (builds that have not been tracked in the past 12 months)
- Choose enrollment rate (e.g: 10 meaning 10% chance a user will be added to the experiment)
- Include or exclude users based on filters.
Only new users will be added to the experiment.
Create configs and variables
The next step is to set up your Remote Configs and variables.
The Config Key needs a name and values. In addition it’s also possible to add a description, but this is optional.
The maximum value length of the variable is 256 characters. The maximum description length is 2000 characters, and the maximum length for the name is 16 characters.
Maximum number of variants that can be added is 10, and you can also change the percentage of users that will be added in each variant by selecting Change size.
Changing sample size distribution can be useful when a group that performs well is identified in one test. For example, a group was identified that is performing very well in terms of revenue and in app purchases. When running a new test, a larger number of users could be set in this group to maximize the gains from these particular users.
Scheduling
These options allow you to configure when the A/B test will start, when it will end, and what metric is considered the Goal metric for the experiment. In addition the End date can also be used to stop the test after a predetermined time.
It’s possible to have custom start and end dates, and the completion criteria can be expanded to show the list of Goal metrics. The list of Goal metrics is as follows:
| Area | Goal metrics |
|---|---|
| Engagement |
|
| Ad activity |
|
| Revenue |
|