Portada AB

THE CHALLENGE

Create an A/B testing experiment to decide whether or not a chain restaurant should launch a new menu.

MY ROLE

Sole analist, part of a Udacity Nanodegree.

TOOLS

Alteryx, Tableau and Excel.

A coffee chain restaurant wants to launch a new menu to improve their sales, to let everyone know of the new product offerings the restaurant will expand their marketing budget significantly. In order to minimize risk, management wants to realize an A/B test of their new menu before rolling it out to every store.  The new menu will be tested on 10 stores in key locations for 12 weeks.


I was asked to analyze the results of the experiment to determine whether the menu changes should be applied to all stores. The predicted impact to profitability should be enough to justify the increased marketing budget: at least 18% increase in gross margin.

A coffee chain restaurant wants to launch a new menu to improve their sales, to let everyone know of the new product offerings the restaurant will expand their marketing budget significantly. In order to minimize risk, management wants to realize an A/B test of their new menu before rolling it out to every store.  The new menu will be tested on 10 stores in key locations for 12 weeks.


I was asked to analyze the results of the experiment to determine whether the menu changes should be applied to all stores. The predicted impact to profitability should be enough to justify the increased marketing budget: at least 18% increase in gross margin.

WIREFRAMESMesa de trabajo 1 copia@2x

The variables we will use to match the treatment stores  selected for the A/B test  and the control stores are as similar as possible are: Average Monthly Sales and Region, two variables with a high correlation with Gross Margin, our predictor variable. Using these two variables, I matched the treatment stores with two control stores each.

TABLE TREATMENT-CONTROL-01

After analyzing the results obtained in treatment and control stores I would recommend to rollout the new menu to the remaining stores, it had a great reception in the treatment stores and could greatly improve profit. It manages to cover the cost of the marketing campaign needed to inform the customers about the new items, it has an incremental lift of 39%, way more than the 18% needed to cover the costs of the marketing campaign. Also, the significance value is as high as it can be which means this improvement is caused due to the introduction of the new menu.

TR_CONMesa de trabajo 2 copia 2
TR_CONMesa de trabajo 2 copia 3
AB-01
AB-02
AB-03

Above we see that the treatment and control stores behaved very similarly during the comparison period and  the contrast with the test period in which the treatment stores gained a great performance boost is significant. Overall, as a group, the treatment stores had more profits than the control stores during the test period.

After the analysis of the results, we can give our recommendation to rollout to all stores the new menu and marketing strategy.

*Both the expected lift and significance level are unusually high, this is due to the fact this was an academic exercise, not a real life situation.

Did you enjoy looking at this project? Here are some other projects you might enjoy as well.

Predicting Store Sales

Udacity Bussiness Analyst's Capstone Project

Peipeile! - Web Design

Online sales platform for Peipeile dough outside of China.

Meta UX - Hotel User Research

Redesign of the login and room reservation interaction at a hotel website.