We often make choices based on our feelings, past experiences, or habits. However, when vital decisions rise and uncertainty exists or there are multiple stakeholders to please, most decision-makers see value in a science-based approach to decision making. eCommerce Optimization helps you leverage the scientific method for making decisions about changes to your web site. Make better informed decisions, choose the best course of action, and avoid unintended consequences.
Why are orders up, but revenue is down? Our customer saw a higher volume of orders, but a decreased AOV, and thus overall less total revenue.
After some careful analysis, we discovered one small but potentially significant user page flow change that had been made. The new site had added the capability for the site to have a "floating cart". This feature is a shopping cart that is persistent on all site pages and accessed via hover state in the header. Here is an example of it in action:
Note the "Checkout" button at the bottom of the floating cart. This button allowed a user to skip the cart page entirely and proceed directly into the checkout process. On the old site, there was no floating cart and the user was forced through the cart page when making a purchase. The cart page was merchandised with targeted cross-sells and sale items. This divergence in page flow experience formulated the hypothesis for this experiment: "Shoppers are skipping directly to the checkout page via the floating cart. They are not exposed to the Special Offers and Cross-Sells that are on the cart page, causing missed opportunities to add additional products to the cart."
A solution for this seemed rather apparent: force everyone through the cart again. However, we still wanted to test and be absolutely sure. The predicted result of the experiment was: "Forcing all shoppers through the cart page during the checkout process will increase revenue by increasing items-per-cart."
The experiment itself was rather straightforward, a A/B split test.
Variation A was the baseline: 50% of traffic experiences current behavior.
Version B was the comparison: the other 50% of traffic is forced through the cart page by removing the "Checkout" button.
The experiment ran for a little over 2 months during an average (typical) traffic period for the site. Based on information collected from Optimizely, Google Analytics and internal platform data, it was determined that version B resulted in a higher AOV. However, revenue for Version B was down 1.2% versus baseline. How are we seeing a higher AOV but lower revenue? We took a look at page view conversion rates for the conversion funnel pages being tested: cart, checkout, and summary.
Version A - 7.05% conversion rate
Version B - 9.66% conversion rate (+36.9% improvement)
Version A - 8.38% conversion rate
Version B - 7.36% conversion rate (-12.2% improvement)
Version A - 5.54% conversion rate
Version B - 5.32% conversion rate (-4.0% improvement)
For Version B, higher conversion was expected on the cart page since the variation forced users through that page. However, lower checkout and summary page conversions was unexpected! Version B was causing higher cart abandonment between the cart and checkout + summary pages.
These results allowed us to infer that the hypothesis was indeed correct; users are missing out on the offers and cross-sells presented on the cart page, as demonstrated by the higher AOV on Version B. However, the predicted solution was not the correct one; users preferred the new page flow and abandoned more when forced through the cart, as demonstrated by the cart and checkout page views.
If our customer had proceeded with implementing their proposed solution without testing, they would have ended up causing even more revenue loss. Instead, further experiments eventually allowed us to find a solution that both increased order size while maintaining order conversion rates, leading to an increase in revenue and solving their AOV migration issue.
Protect yourself from unintended consequences. Learn about Kalio eCommerce Optimization Service.