ASO Tool Free: Run an App Store Screenshot A/B Test

An aso tool free workflow can produce a useful screenshot A/B decision when you test one first-frame change, measure store-page conversion rate, and wait for a pre-declared sample size. In App Store Connect, use Product Page Optimization to compare the control with up to three treatments. On Android, use Google Play Store Listing Experiments. Keep every later screenshot, locale, price, and release condition fixed; do not select a winner because it collected more impressions or because a dashboard looks favourable early.

Key takeaways

• •Apple Product Page Optimization tests a control against up to 3 treatments.
• •Google Play Store Listing Experiments is the Android equivalent of Apple’s screenshot-testing feature.
• •Change one variable per test; the first screenshot frame is usually the highest-value variable.
• •Pre-declare the visitor or conversion sample required before starting the test.
• •Use conversion rate, not impressions, as the decision metric.

This guide is for mobile product designers, growth founders, and engineers who can prepare store assets and access App Store Connect or Google Play Console.

Time: 45 minutes to configure, then the pre-declared test duration · You'll need: App Store Connect access with an App Manager, Admin, Marketing, or Sales role, Google Play Console access with Store presence permissions, A spreadsheet for the hypothesis, sample target, and decision rule, A source image editor such as Figma, Sketch, or Adobe Photoshop, Device screenshots for the exact iOS and Android builds being marketed

What's on this page

1. •Write one app store optimization test hypothesis
2. •Set the sample size before the screenshot A/B test
3. •Build matched screenshots manually
4. •Configure Apple Product Page Optimization
5. •Run Google Play Store Listing Experiments
6. •Apply the decision rule and archive the result
7. •Reference table

• •Common mistakes
• •FAQs

Run an ASO tool free screenshot A/B test

1. Write one app store optimization test hypothesis

Write a one-line hypothesis that names one screenshot variable, the audience, and the expected conversion outcome. Use the first screenshot frame unless research identifies a clearer bottleneck. For example: “For US English visitors, a first frame showing the receipt-scan result will increase product-page conversion rate by at least 1 percentage point versus the control frame showing the home screen.”

Freeze the rest of the listing before launch: app name, subtitle or short description, icon, video, price, offer state, and screenshots 2 through 10. A first-frame test may change its device image, headline, background colour, or benefit claim, but not all four independently. If you change several elements, a winning treatment cannot tell the team which element caused the lift.

Record the control asset filename and each treatment filename in a sheet. Also record the locale, countries, start date, baseline conversion rate, minimum meaningful lift, required sample, and winner rule. This document prevents a favourable-looking result from becoming a post-hoc decision.

Tip: Do not test a dark-mode visual treatment against a light-mode control if the app’s actual first-run experience has not changed; the store promise must match the shipped app.

2. Set the sample size before the screenshot A/B test

Set a stopping target from the control conversion rate and the smallest lift worth shipping. Use a two-sided comparison of conversion proportions with 80% power and a 5% significance level as a practical starting rule. For a rough equal-size two-arm estimate, use n ≈ 2 × p × (1−p) × (1.96+0.84)² ÷ d², where p is baseline conversion rate and d is the absolute lift you need to detect.

For example, at a 10% baseline conversion rate and a 1 percentage-point lift (d = 0.01), the estimate is about 14,112 store-page visitors per arm. Treat this as a planning figure, then use your team’s approved calculator if it supports unequal traffic or multiple variants. If Apple has three treatments, pre-declare how you will control false positives across the comparisons; a Bonferroni rule uses 0.05 ÷ 3 = 0.0167 per treatment-versus-control comparison.

Do not stop when a treatment reaches a nice round number, gets a temporary lead, or produces a compelling chart. Stop when each relevant arm reaches the declared sample or when the platform’s permitted test window ends.

Tip: Choose a minimum meaningful lift that can repay the engineering and creative work; a 0.1-point gain may be statistically detectable but commercially irrelevant.

3. Build matched screenshots manually

Export the control and treatment screenshots from the same source file. Keep the screenshot count identical and preserve the same order after frame 1. If the control has 10 screenshots, each treatment should also have 10; only the first frame changes in this test. Use identical copy casing, device mockup scale, margins, localisation, and image compression for every unchanged frame.

For the manual route, make one artboard per required store-device size, place the approved screenshot image inside each artboard, and export PNG files with a naming pattern such as en-US_control_01.png and en-US_treatment-A_01.png. Upload only sizes accepted by the relevant App Store Connect or Play Console asset slot. Preview every asset at small phone scale before upload: the first frame must communicate its benefit without relying on unreadable body copy.

Keep the editable master at a single aspect ratio only if your composition survives the required crops. Otherwise create size-specific compositions and document the changed crop. The test variable must remain recognisable across every uploaded size.

Tip: Use a 2×2 review sheet showing control and treatment at actual phone scale; side-by-side review catches accidental copy, crop, and order changes before submission.

4. Configure Apple Product Page Optimization

In App Store Connect, create a Product Page Optimization test for the selected app and locale. The existing product page is the control. Create up to 3 treatments and upload each treatment’s screenshot set, keeping every non-tested element equal to the control. Apple Product Page Optimization can run for up to 90 days, so check that the expected store-page traffic can reach the declared sample within that window.

Name treatments by the actual variable, not by an optimistic label. Use names such as First frame—scan result and First frame—budget total; avoid Winner candidate. Select the same localisation and markets represented in the hypothesis. Do not combine a screenshot test with an icon, app preview, or promotional-text change in the same treatment if the goal is to learn about the first frame.

After the test is approved and live, log its launch date and treatment definitions in the experiment sheet. Monitor implementation problems such as a rejected asset or an unintended locale, but do not make an outcome call before the pre-declared sample threshold.

Tip: If the first frame needs a different headline length in German or Japanese, run that locale as a separate hypothesis rather than treating translated layouts as identical assets.

5. Run Google Play Store Listing Experiments

In Google Play Console, create a Store Listing Experiment for the matching Android listing. Google Play Store Listing Experiments is the Android equivalent of Apple Product Page Optimization. Use the current store listing as the control and create no more than 3 variants for a direct comparison with the iOS plan.

Upload the control-equivalent screenshot sequence and variants that differ only in frame 1. Keep the Play listing’s short description, full description, icon, feature graphic, and app version stable while the screenshot experiment runs. If your app has country-specific or language-specific listings, test within the exact listing that supplies the traffic; do not pool assets from different localisations and assume their conversion behaviour is interchangeable.

Use conversion rate as the evaluation metric: compare the share of store-listing visitors who become installers for each variant against the control. Impressions describe exposure and can change with traffic mix, campaign spend, or store placement. They do not answer whether a visitor who reached the listing was persuaded by the screenshots. Export or record the final result with the visitor counts and conversion rates for every arm.

Tip: Avoid launching a paid acquisition campaign targeted at one creative message during the test unless the campaign traffic is distributed evenly across control and variants.

6. Apply the decision rule and archive the result

At the declared sample threshold, apply the decision rule written in step 2. Ship a treatment only when it exceeds the control on conversion rate by at least the minimum meaningful lift and meets the pre-declared confidence threshold. Keep the control when no treatment clears both thresholds. “No decision” is a valid result when the available traffic cannot distinguish the planned lift before Apple’s 90-day Product Page Optimization limit or the business context changes.

Record four values for each arm: store-page visitors, conversions or installers, conversion rate, and absolute lift versus control. For example, a control at 10.0% and a treatment at 11.2% has a +1.2 percentage-point lift, not a 1.2% lift. Save screenshots of the final platform report, the uploaded asset files, and the exact decision rule.

Turn the next test into a new single-variable question. If a receipt-scan result frame wins, test its benefit headline against a second headline; do not reopen the old experiment by adding a new colour, device, and copy combination.

Tip: Archive losing assets too; a treatment that loses in the United States can still provide a precise hypothesis for a different locale or audience segment.

Reference table

Store screenshot experiment limits

Do it with the free App Store Screenshot Generator

Caption and frame up to ten screenshots, then export them at every App Store and Play Store size at once.

Open the App Store Screenshot Generator → — free, no sign-up.

Common mistakes

Changing the first frame, icon, and short description in one treatment.

Change the first frame only. A combined treatment can improve conversion, but it cannot identify which asset should become the new control.

Calling the treatment with the most impressions the winner.

Compare conversion rate among store-page visitors. Impression volume is affected by acquisition source and store exposure, not only screenshot persuasion.

Stopping when a variant reaches 1,000 visitors because the chart is positive.

Use the visitor target calculated from baseline conversion and minimum meaningful lift. Early leads reverse often when more conversions arrive.

Uploading different screenshot counts or reordered frames across variants.

Match the sequence exactly after the changed frame. A different second frame turns a first-frame test into a multi-variable experiment.

Frequently asked questions

How many variants can I test in Apple Product Page Optimization?

Apple Product Page Optimization lets you test up to three treatments against the existing product page control. Use those treatments for distinct alternatives to one hypothesis, such as three first-frame benefit messages. If traffic is limited, testing one treatment against the control reaches a useful sample sooner than splitting traffic among three treatments.

What is the Android equivalent of an App Store screenshot A/B test?

Google Play Store Listing Experiments is the Android equivalent of an App Store screenshot A/B test. It compares variants of your Google Play store listing with the current listing used as the control. Keep the Android test’s changed variable and decision rule aligned with the iOS test, but treat results as platform-specific rather than assuming they will match.

Should I use impressions or conversion rate for product page optimization?

Use conversion rate, not impressions, to decide a product page optimization test. Conversion rate measures the share of store-listing visitors who install or download, while impressions only measure how often the store showed the listing. Record both for diagnosis, but require the conversion-rate threshold before replacing the control.

How long should a screenshot A/B test run?

A screenshot A/B test should run until each arm reaches its pre-declared sample target, subject to the platform’s limits. Apple Product Page Optimization tests can run for up to 90 days. Set the target from baseline conversion rate and minimum meaningful lift before launch, rather than choosing a duration because it sounds reasonable.

Where this leaves you

A screenshot test becomes a decision tool when its scope is narrow enough to explain the result and its stopping rule exists before traffic arrives. Start with the first frame, because it is the asset most likely to establish the app’s value before a visitor scrolls. Use Apple Product Page Optimization for iOS and Google Play Store Listing Experiments for Android, then judge every arm on conversion rate at the planned sample. Your next action is to create the control and one first-frame treatment. You can prepare up to 10 captioned and framed screenshots, then export App Store and Play Store sizes from the free App Store Screenshot Generator at /free-tools/app-store-screenshot-generator, or follow the manual export method in this guide.

---

Design the screens first

Describe the screen you need in plain English and floow.design generates production-ready iOS and Android layouts you can iterate on by chat, then export to Figma or code. Design your mobile app screens in floow.design first, then build.

Start designing free →

---

Related

• •WebP Converter — Batch-convert PNG, JPEG, WebP and AVIF in your browser, with a quality slider, optional resize and a live size comparison.
• •SVG to PNG Converter — Rasterise any SVG at 1x to 4x, or export every iOS scale and Android density as one archive.
• •App Store Screenshot Sizes for iOS and Google Play — App store screenshot sizes for iPhone and Google Play: export 6.9-inch assets, meet image
• •App Store Screenshot Generator: 5 Frames That Convert — Use an app store screenshot generator workflow to build five benefit-led frames that sell
• •Gemini 3.1 for Mobile App UI Design: 4 Real Prompts Tested Hands-On (2026) — I tested Gemini 3.1 Pro with 4 real mobile app UI prompts — fitness tracker, food delivery

Sources

• •Apple Developer — About Product Page Optimization
• •Apple Developer — Create a Product Page Optimization test
• •Google Play Console Help — Run store listing experiments