Industry
CPG
- Improved resource utilization by automating repetitive tasks
- Achieved 50% reduction in time-to-design market testing experiments
- Enhanced experiment scheduling by allowing multiple tests to run without overlaps
New opportunities unlocked with Math + Engineering + Design
- Eliminated process complexity by standardizing procedures for all subsequent testing experiments
- Automated scheduling for multiple tests, simultaneously computing individual results
- Leveraged pre-built ML algorithms to further save data migration time
- Facilitated intelligent sampling suggestions to optimize A/B testing efforts
In recent years, the brewing industry has witnessed significant developments. With the craft beer market's niche supersizing, global breweries have had to rethink product development and test marketing strategies to better gauge consumer behaviors and cost implications.
The client, one of the world’s largest brewing companies with several hundred popular brands in its portfolio, wanted to optimize and streamline its test marketing efforts to make informed decisions on product marketing and new product launches.
Propelling this need for an upgrade was the fact that several experiment designing functions, such as sample identification, test scheduling, and split testing, were still being carried out manually by the product management team. Further, the business lacked capabilities such as running multiple tests, smart stat monitoring, and result visualization, increasing overall time-to-design and, subsequently, time-to-market.
Our partners from the global brewery, the Global Lead for Business Intelligence and the Head of Analytics, were looking for an agile, automated, and highly scalable solution that would help optimize their market testing initiatives.
To secure their brand's chances of successfully transitioning into niche markets, they wanted a custom tool that could streamline experiment design and inform marketing decisions and campaigns for localized beverage flavors in select geographical segments.
For the solution to achieve this, it had to meet 4 key expectations:
1. Process Standardization: A homogenized testing process to be leveraged and applied to all future test marketing experiments.
2. Test Automation: An auto-enabled platform to schedule and launch multiple tests without the need for manual intervention.
3. Stat Analyzation: An intelligent framework to collate, compute, and compare results from multiple tests, enabling better decisions from experiment performances.
4. Design Optimization: An agile engine to execute end-to-end design operations ranging from sample size identification and split creation to overlap monitoring and cost estimation.
The client's global footprint demanded that the tool offer optimal scalability for all test and learn initiatives across the globe. MathCo.’s team regularly interfaced with the client’s analytics team to gather data on whom-to-, how-to-, and why-to-market.
To meet the client’s scalability requirements, the team developed a data-light solution that could be used to design the experiment with different scopes such as region, price, and POC, while also being highly modifiable to cater to the business’s diverse cross-functional needs.
Considering the client’s primary objective of pinpointing and remodeling their test marketing strategy, TheMathCompany adopted an iterative approach to develop a highly scalable tool that could read the true impact of any potential marketing initiative, while offering actionable decision-making support to meet shifting needs.
The Test & Learn Tool, a cloud-based experimentation platform, was developed for the client using a set of pragmatic ML algorithms that ingest training data, such as expected lift and tuning parameters, to enable multi-purpose A/B and Difference in Difference testing.
In a real-world scenario, the tool’s Power Analysis capability can
- Suggest statistically split test and control groups;
- Compare and monitor the performance of different beverage flavors;
- Evaluate packaging and aisle placement; and even
- Predict lift rates and break-even values unique to each permutation and combination
to markedly reduce time and costs involved in a typical experiment.
Thanks to the tool’s filter and audience list generation features, the client could custom-target potential consumers. This could be further optimized by harnessing the tool’s intelligent cost and ROI estimations, helping make precise pricing decisions.
Enabling teams with the capabilities to evaluate various data points, the tool also provided early in-market product performance data to define product distribution strategy in stores and predict potential sales velocities.
Another key functionality specifically enabled to enhance the client’s in-prospect testing requirements was the automated scheduling mechanism. This mechanism helped factor in technical, event-, or experiment-related pre-requisites, avoiding overlaps among tests during the implementation phase.
To enable consumption for the client teams, MathCo. developed a business-view React-powered dashboard, allowing for a highly personalized display. This was accompanied by flexible filters in report view to generate actionable insights about potential sales and target samples.
An end-to-end solution, the tool was also purpose-built to provide the client a seamless pacing view of ongoing experiments, automated summaries, stats, and results.
