I have written a couple of times about the consequences of incentives. Metrics work the same way. As some people were quick to (correctly) point out, not all incentives are monetary and business metrics that are monitored, even without a bonus attached, become incentives in and of themselves. When I was a consultant a lifetime ago, we had a saying that “What gets measured, gets done”. We are working our way through one of those moments right now with my interactive team. The good news is that although we had a bit “metric myopia”, there were no bad consequences to speak of…
For our online businesses, our primary focus is being efficient lead generators for our advertisers. Since we can easily track email leads in Omniture (our web analytics tool), that metric had become the primary focus point for all of our dashboards (Email Leads, Traffic and Conversion). It has worked extrodinarily well for us – email leads in all of our brands are up. Personally, I check on our email lead production 5 or 6 times a day. In Apartment Finder, where we have made the most significant changes, email leads are up more than 300% since Septembrer. So, what is wrong with that?
The measure is incomplete. Take the example of Apartment Finder. Our advertisers ascribe significant value to 3 consumer actions; phone leads, email leads and “click thru’s” from our site to theirs. When we did our A/B testing on the new site design we monitored email leads and click-thru’s – but it was too complex to try to measure phone leads. We watched phone lead counts in aggregate, and they were not negatively impacted by any of the changes. I checked again tonight and we have shown lead lift in all 3 categories, which, in the case of phone leads is somewhat lucky in that we thought about it during design, but did not specifically test the results.
While it will be harder to do, it is imperative that we bring all of the valued consumer actions to the “dashboard” – because I would suspect that while we have done nothing to hurt phone leads, if it had been on the dashboard every day (or 6x a day for someone compulsive), we would have found many more ways to drive that result.
Metric Myopia. I know better, but I defined our metric too narrowly. A holistic view of leads (or better yet, holistic advertiser cost per lead), would have better perspective.
Here is a great example from the long ago past. At one of my consulting clients in 1996 I interviewed the credit department at a sales organization to understand the complete process flow of the sales division. I remember the head of the credit department saying, “we have not had to write off one penny of revenue in the past 12 months”. I had also just interviewed a regional sales manager who was angry because it took so long to get credit approval and for most new customers the terms came back “cash-in-advance”. Somehow, the sales conversation had not made it to the credit office. The credit department was not bonused based on bad debt at $0.00, it was just their driving metric. A risk averse company, they were clearly walking away from revenue and profit.
