All problems

Planning an A/B Test's Sample Size

mediumPythonA/B Testing

June's checkout test at Marlow & Finch ended without an answer, and the post-mortem found the reason: it was too small to have worked. The team now sizes every test before a single visitor arrives, and the same calculation has to serve whatever page is being tested next.

Two numbers define what a test is trying to catch: the rate the current page converts at, and the improved rate that is the smallest rise worth detecting - anything smaller would not pay for the redesign, so the test is not required to find it.

Two more numbers define how sure the team wants to be. The significance level supplies one z value; a 5% two-sided test gives 1.96. The power supplies the other: power is the chance the test actually catches the improvement given that the improvement is really there, and at the usual 80% - four reruns in five find it, one misses it - the z value is 0.84. Raising the ambition to 90% power changes it to 1.28.

The conventions:

  • Both arms get the same number of visitors, and the answer is the figure per arm, not the total traffic the test needs.
  • Use the ordinary planning approximation, which treats both arms as sharing a single conversion rate - the midpoint of the two supplied rates - when it sizes how much the results will wobble.
  • Visitors come in whole numbers and being slightly short defeats the purpose, so round up to the next whole visitor.

Task: Print the required number of visitors per arm, as a whole number rounded up.

Input

Four lines: the current conversion rate, the improved rate worth detecting, the z value for the significance level, and the z value for the power. The two rates are decimals between 0 and 1 with the second larger than the first; the two z values are positive decimals.

Output

One line holding the visitors needed per arm, as a plain integer - 3777, never 3777.0.

Example:

Input:
0.05
0.065
1.96
0.84

Output:
3777

Sign in to solve this problem

Reading problems is free for everyone — solving them (Run, Submit, and tracking what you've solved) needs an account.

Sign in

Discussion

Sign in to join the discussion — reading is open to everyone.

Loading comments…