Average Performance can be Decieving

I still remember first semester at business school when my professor, Paul Glasserman, handed back the first stats test of the year and dryly said, “Remember, not everyone can be above average.” This is certainly true for Apartment Communities and lead generation – not all properties perform above average…

Averages can be deceiving and using averages to make business decisions can leave a ton of value on the table. Similarly, understanding the factors that drive the outliers in performance can create significant value. This is why there can be INCREASING return in analyzing marginally more granular data.

Before I go to the apartment data (which is very cool, by the way), here are a couple of examples from my career that reinforce the point.

When I was at Cendant and was responsible for our Membership Travel business, our cost center cost drivers were sales-per-call and “minutes per completed call” – those metrics described most of our variable cost (agent time, telephone time, etc.). When I began running the business, every decision was made on “average call time”. I don’t remember the real number, but let’s call it 12 minutes for an average sale. So, cool, if we can get some tools to make it 11 minutes or 10 minutes, that will drive profit per call. We ran with monitoring the average for a while, and then I asked to see the raw telephone switch data. One thing that leapt out at me was that there were some CRAZY long calls. No one had ever mentioned it to me, but when we looked back at reports we found that a portion of our calls, call it 10%, were what we called “Long Calls” – calls lasting 25+ minutes. Most of those calls were significantly unprofitable for us. By zeroing in on the cause of the long calls (mostly picking hotels in big cities like NYC, LA or Chicago) we could significantly reduce our cost. While the average was important, the outliers were the key lever for improved performance.

Now for the cool Apartment stuff. Here is what we know:

• Like Paul Glasserman said, not all properties can perform above average.
• At NCI we have TONS of data about how different properties perform in terms of lead generation.
• We can hire a really smart stats PhD to CRUNCH this data. (So we did)
• We can ask the question, “Why do some properties perform better than average and others worse?” Where performance is measured in leads generated.

First some descriptive data:

Our data represented more than 10,000 properties with phone and email leads generated over an 11 month period. That is tens of millions of magazine copies distributed, tens of millions of unique visitors, millions of leads.

On average (oooh, I hate that word), the properties advertised on Apartment Finder:

• Ask $844 per month in rent
• Have 964 square feet
• Have 1.8 bedrooms
• Have 1.4 bathrooms

When taking a look at the key amenities offered, here is the % frequency of each facet.

It seems that offering a pool is “table stakes” for a community.

Our analysis focused on “observable characteristics” – like asking price, the amenities offered, square footage, bedrooms, leads, etc. We know that there are “unobservable characteristics” that change these results. For example, ugly pictures hurt lead production. Being in a “bad part of town” hurts lead production.

First we asked the question – what amenities drive leads? In our first assessment we took price out of the regression. So there are two ways to read the chart. One way is to say that by offering a garage, you should expect 11% fewer leads than average and by offering a Washer Dryer Hookup, you should expect roughly 20% more leads than average. However, another way to look at the data is the signaling of price. Higher priced apartment complexes generate fewer leads than average and tend to have Garages, Fitness Centers, Washer/Dryers, etc. Most interesting is that allowing pets provides a significant lift in leads without being associated with price.

Next, we asked how does price fit into the equation. For the following chart, you can read it as “if you add a given amenity to an average property, how much can you change monthly asking rent and receive average leads ?”

In this chart, Fitness Center, Gated Community, Garage all indicate a more expensive property, where you can ask for more rent and still receive average leads. With Washer Dryer Hook-Up, you are signaling a less expensive property and need to reduce asking rent by a significant amount (more than the cost of a washer dryer) in order to maintain average leads.

We are getting close to having a model that we can run against the “observable characteristics” of an apartment community advertiser to predict how many leads that property should generate. We can then help our advertisers to understand if they are an “outlier”. If they are getting more leads than they should expect per the model, should they test raising the asking rent? If they are getting fewer leads than the model predicts, do they need to rewrite their property description or take new photos?

By helping the properties who perform below predicted lead volumes improve their “unobservable characteristics” we can help them drive their business. We believe that generating more, high quality leads for all of our advertisers will help drive our business.

Tagged as: Average, Forecast Model, Lead generation, marketing, Mulit Family, Outliers, Statistics