Product page optimization metrics

Product page

A page on the App Store that informs people about your app and lets them download it onto their device.

Test

An experiment in which different treatments of your product page are shown to a percentage of users so their performance can be compared to the performance of your original product page during the same timeframe.

Variant

An instance of your product page (either the original or a treatment).

Treatment

A variant of your product page that’s compared to your original product page.

Original

Your default product page, which serves as the control for your test.

Referral traffic

Visits to any variant of your product page from a direct URL.

Apply treatment

The action of replacing the metadata on your original product page with the metadata from one of your treatments.

Confidence level

The probability that the data gathered in a test suggests that two variants are performing differently.

Baseline

The variant that the other pages are being compared against.

Collecting data

Data is still being collected to determine whether this variant is performing better than the baseline.

Performing better

This variant is performing better than the baseline with at least 90% confidence.

Performing worse

This variant is performing worse than the baseline with at least 90% confidence.

Likely to be inconclusive

Based on the current results, there likely won’t be enough data after 90 days to determine how this variant is performing compared to the baseline.

Estimated conversion rate

The estimated percentage of people that downloaded or pre-ordered your app from a certain product page variant. This estimate incorporates the data observed during the test, as well as existing data.

Estimated relative lift

The estimated relative increase in conversion rate for a variant as compared to the selected baseline. It requires fewer weeks of data to determine significance on higher improvement values (e.g. 30%) as opposed to lower improvement values (e.g. 5%). Your actual test duration will depend on the level of improvement detected during your specific test.

Bayesian reasoning

A statistical method where probability expresses a degree of confidence in an event; used to justify confidence in your conversion rate lift.

Credible interval

The probable range of your lift or conversion rate. This represents a 90% interval, meaning there’s a 90% probability that the conversion rate or lift falls within this range.