A/B Tests: How To Verify Difference In Average Order Value?
-
Hi there! When the data from an A/B test shows a difference in AOV between the variants, how do you determine if this difference is statistically probable or not? Best Regards, Martin
-
Thank you, David.
-
Perhaps this will get your on the right track, from http://flintanalytics.com/how-to-determine-if-you-should-advertise-higher-priced-goods-to-mac-users/
Average Order Value Statistical Significance
To do this you need to download the transaction data for both variants, because you need to be able to determine the variance in order value. You can download this data by going to conversions > ecommerce > transactions in Google Analytics. Once there, Google only lets you export 500 rows of transactions at one time, so to keep it simple I ensured my date ranges were under 500 transactions.
Once you get the order values for each set of customers in Excel you need to perform a t-test so that you can compare the means of the two groups. This t-test will give you the p-value that you will use to determine if the difference in average order values between the groups is significant. Yout formula in excel should look like the following:
=t.test(array of group 1, array of group 2,two-tailed distribution, two-sample equal variance homoscedastic)
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
When migrating to https should you unlink the verified http version from my GA account?
Hi We had a new site design at the end of April and at the same time changed to https. Over the last 6 months, according to our analytics account the traffic has dramatically dropped especially from mobile. We were at 3607 new visitors from mobile in April and now 32! in September. Technically (other than page speed and some meta issues that I'm aware of) there doesn't initially appear to be anything wrong. I have just been into search console and notice that the old http site still has the GA code on it, could it be that this is why the data is showing such a massive drop? Should we have taken that off when we migrated the site? The same analytics account is also on the new site. Thanks in advance Janine
Reporting & Analytics | | Janine10 -
Backlinks Tracking Websites/Tools/Software
I have multiple websites that I need to keep track of their backlinks. How do you guys keep track of your backlinks? What are some cool tools that you use ?
Reporting & Analytics | | AngelosS0 -
Google Analytics Goal/Event/SOMETHING to show only Wordpress "Posts", not pages, etc
Hi all, Our site is build on Wordpress and formerly the post URL's had the typical date format at the beginning. This made it easy for me to look at, for example, all search traffic to the blog. I would just view URL's containing /2014/ and /2015/ and boom. We have since removed the dates from the URL's with proper redirects etc, which is great, but now I can't figure out a way to look at ONLY the blog in GA. I like to track a KPI of 'search visits to blog posts' and I can't figure out how to now. Can I set up a GA event that only fires when the post type template for blog posts loads? Some other solution? I'm lost here, and there's gotta be a good way to do it...
Reporting & Analytics | | 3DR0 -
Index.php and /
Hello, We have a php system and in the MOZ error report our index.php shows up as a duplicate for / (home page). I instituted a rel canonical on the index.php because the / gets better rank than the other. This said, the error report through MOZ still shows them as duplicates. Should I be using a 301 instead? Please help! Also, I would love a good technical SEO book (for bridging the gap between SEO and programmer) if someone can recommend one? Thanks in advance!
Reporting & Analytics | | lfrazer0 -
Why am I showing a difference in Goal completion for Adwords and Analytics?
I have conversion tracking setup for Adwords and Goal tracking setup in Analytics. There is a large discrepancy between the two. Anyone have any ideas?
Reporting & Analytics | | EcommerceSite0 -
Which Algorithm Change Hurt the Site? A causation/correlation issue
The attached graph is from google analytics, a correlation of about 14 months of Organic Google visits with algo changes, data from moz naturally 🙂 Is there any way to tell from this which will have affected the site? for example #1 or #2 seems to be responsible for the first dip, but #4 seems to fix it and it broke around 6, or is the rise between 4 and 7 an anomaly and actually 1 or 2 caused a slip from when it was released all the way to when 7 was released. Sorry if the graph is a little cloak and dagger, that is partly because we don't have permissions to reveal much about the identity, and partly because we were trying to do a kind of double blind, separating the data from our biases 🙂 We can say though the different between the level at the start and end of the graph is at least 10,000 visits per day JarMzoK.png
Reporting & Analytics | | Fammy0 -
Core audience, daily / weekly uniques
In Google analytics or site catalyst, is there anyway to target IP addresses to see how many of my daily uniques are daily, weekly, monthly and what % are just 'tourists'/'one-offs'? My site is a local news site... Or is there another way to find this info. I am trying to work out what my core audience is (that keeps coming back every day or every week). Kind regards
Reporting & Analytics | | MirandaP0 -
Is the link data from Open Site Explorer in real time or an average?
I just started using Open Site Explorer to track internal and external link data. Is this information given in real time or is it an average over a specified period of time?
Reporting & Analytics | | mequoda0