How to Run a Valid A/B Test for Size Adviser

A comprehensive guide to measuring the true impact of Size Adviser on sales and return rates while eliminating self-selection bias.

Understanding Why an A/B Test Is Necessary

Directly comparing shoppers who used Size Adviser against those who didn’t—or running a simple “before vs. after” comparison—produces misleading results due to self-selection bias:

  • Shoppers who use Size Adviser often lack confidence in their fit, which makes them an inherently higher return risk from the start.
  • Shoppers who skip Size Adviser are frequently repeat customers who already know their size and naturally return fewer items.

Comparing these two cohorts directly conflates Size Adviser’s actual performance with pre-existing shopper behavior. A properly randomized A/B test isolates the algorithm’s true effect on conversion rates and return reduction.

Test Setup & Recommended Methodology

To ensure accurate, statistically valid results, adhere strictly to the following testing protocol:

  • Traffic Split: Assign 50% of product detail page (PDP) visitors to the Control Group and 50% to the Test Group.
  • Strict Experience Isolation: Ensure users consistently see only their assigned variant throughout their entire journey and across sessions:
    • Recommended Test: Test Group (Size Adviser) vs. Control Group (Standard Static Size Guide).
    • Alternative Test: Test Group (Size Adviser) vs. Control Group (No Sizing Tool), if a static guide variant is not technically feasible.
  • Full-Funnel Tracking: Monitor both cohorts across every stage of the purchasing journey: Page Visit → Interaction/Click → Recommendation Received → Conversion → Return.
  • Order & Return Integration: Fully integrate order and return data with uSizy before launching the test. This allows you to split and analyze returns into size/fit error vs. other reasons (e.g., color, preference, late delivery).

Common Implementation Pitfalls

Many self-managed A/B tests produce invalid conclusions due to technical or methodological setup errors:

  • Flicker and Delayed Script Loading: In improper setups, Control elements briefly appear to Test group visitors (or vice versa) before the script loads and hides them. This brief delay leaves users with no sizing solution for seconds and corrupts user behavior.
  • Session Leakage: Visitors switching devices or clearing cookies might see different variants across sessions, contaminating cohort isolation.
  • Comparing Clicks vs. Non-Clicks: Comparing “users who clicked Size Adviser” against “users who didn’t” within the same group fails to account for self-selection. Always compare the total Control Group (Intent to Treat) against the total Test Group.
  • Leverage Native Methodology: Use uSizy’s built-in internal control group features, which are engineered specifically to account for self-selection bias and handle DOM script timing cleanly.
  • Use Dedicated Reporting: Evaluate test results using the dedicated A/B testing section within the uSizy dashboard rather than relying on ad-hoc analytics queries.
  • Pre-Launch Technical Review: If you decide to run the test using a third-party platform (e.g., VWO, AB Tasty, Optimizely), confirm that your setup satisfies all isolation requirements and review your configuration with the uSizy technical team prior to launch.