Skip to main content

Setup

Here’s what you need to know about setting up your first experiment

Technical Integration​

Before you get started with creating and running experiments, please ensure that you have integrated our latest SDKs with your games and have the remote config calls setup correctly.

Setup and Segmentation​

When you create an A/B test, you will first need to define an audience group that you would like to run the experiment on by selecting a combination of:

  • Build
  • Target Criteria:
    • Target Users Percentage
    • Filter: Country and OS
    • Exclude offline users

Setup & Segmentation

Your experiment will only be targeted at New Users – existing players will not be included in any A/B tests. Once a new player has entered an A/B test, they will remain exclusively part of that experiment until you stop the experiment.

You can optionally include or exclude certain Countries and OS Versions in your experiment.

Create Config and Variants​

Now you can define the remote config and variant groups:

  • Config Key:
    • Create the key that you want to perform the A/B test on. This key will need to be supported in all the build previously selected.
  • Variants:
    • Each different config value to be tested.
    • You can create up to 3 variants, allowing for 4 test groups in total, including the control group.
    • Players included in the control group, will not receive any value for the A/B test config key.
    • Players will be randomly allocated to the control group or a variant.

Configs & Variants

Scheduling​

Decide if you want to start the test right away or at a future date/time. You will also have the ability to select the goal metric now to determine the experiment winner. Metrics available now are:

  • Ad clicks
  • Ad impressions
  • Ad revenue per user
  • Ad revenue per impression
  • Conversion
  • Playtime per session
  • Playtimer per user
  • Retention D1, D3, D5, D7, D14, D28
  • Session count

We are always working on increasing the number of metrics available for testing. If you feel you are missing any metric, let us know here.

Scheduling

Summary​

Here you can:

  • See an overview of the A/B test specification
  • Give your test a title
  • Start the test

Summary

Delete/Copy Test​

After a test has been stopped, you can chose to delete or copy the test.

Copying a test will open the creation flow, with pre-populated selections based off the original test.