Recently, Josh Elman, Product Guy Extraordinaire and Principal at Greylock Partners wrote this fantastic article about metrics. If you get a chance to bend Josh’s ear about product or scaling a product at any time please take it. It could change your perspective on things forever. Anyways, the article itself is a must-read.

We had been thinking about metrics for some time at Band of the Day HQ and we made some changes internally about 2-3 months ago to try and focus on a few that mattered to us while we embarked on a pretty ambitious redesign of the product.

We would be evaluating our execution based on some results that we could clearly measure after the new version was out. Product changes focused on results sounds pretty simple and obvious, but, in reality the entire process of executing towards it while maintaining a product with millions of users is absolutely non-trivial. Why change something that clearly is working already? What are the risks involved in drastic product “enhancements”? How would existing users respond and react?

Ask digg. (they were at a different scale, but, I do empathize with what they may have gone through). Massive product changes are scary and nerve racking to execute.

Before I discuss our learnings (in another post): here’s a brief glimpse at our results. I’ve hidden out the specific metric/s since that’s not the interesting bit of learning here…

[to give you some reference, the numbers were in the high 10s of thousands before and now are in the higher 100s of thousands a day. We are outperforming our most optimistic of estimates by an order of magnitude and we appear to be gaining or staying in sync at these new levels]