Rapid Product Experimenting at Scale: The Tech Underneath Snapdeal

With a new business model, you need to win customers & that’s why you experiment. A scalable experimentation framework becomes the need of the hour. Experimentation becomes unavoidable after a certain stage & it is necessary to optimize every last thing you can. In the case of rolling out new features, experimenting requires a critical mass. It’s best for organizations to be focused on the outcomes of new features.

“I know something, but I’ll have to experiment for the best results” is the best approach to experimentation. There’s no definitive “this will work”. Experimentation and validation is the way to go. What’s innovative and exciting from your perspective may perhaps not be what customers are excited about.

The ideal product feature cycle is: controlled rollout → measure → optimize → repeat

You’ll need the right technology for:

• Rollout
• Measurement
• Changes — rapid experimentation
• Conclude — keep the tenure of experiments in check. You need to have checks in place for what qualifies as a success, and what qualifies as a failure.

Technology around experimentation should be:

1. Easy to set up
2. Easy to measure
3. Controlled
4. Configurable
5. Avoid latency to end-user
6. Scalable

Snapdeal has, for instance, about 300 microservices, over three billion messages every day, low latency, and a single-responsibility design. Every feature is built on solid thinking and planning.

At Snapdeal, we’ve created our own data platform to process all the data we need. All user events and transactional events are handled here. This could track user history, creating opportunities for fruitful experiments regarding user behavior. We split experiments internally into ‘inbound’ and ‘outbound’ experiments.

Inbound experiments concern those features that are shown to users who visit our site or use the app. We choose what to show to users. Things such as entire pages like the home page and the listings page are fully configurable & are driven by experiments.

Outbound experiments include emails, messages, and other communication that we send to people. These include notifications and corresponding content. We also experiment with target demographics and the best time for communication.

Most of the experiments at Snapdeal focus on improving experience, increase order value, revenue for sellers, shippability, or serviceability. Ultimately, the results benefit the consumers and the sellers.

Originally published on the orangescape