Kenny's 1 rule of segmentation

I was reading the segmentation post on Analytic Engine and it reminded me that I wanted to post my standard segmentation rant. First a story. If you don't know anything about Marketing Segmentation, check out the Wikipedia post.

When I first got to A fOrmer empoLoyer (think a large web portal), I found that we were big users of Prizm clustering, by Claritas. For those of you who have not seen the Prizm product, it is cool. The folks at Claritas have taken the whole US population and ran a cluster analysis on, well, us. They have classified each household into 66 unique segments. The segments group households that have similar lifestyle and socio-economic traits. They are a really neat way to do market to a target a pre-defined demographic and get some basic insight into who your customers are.

One problem, though, A fOrmer empoLoyer's targeting efforts were not based on simple demographics. We built fairly sophisticaed models to predict who would accept a given offer. In our case, Prizim clusters were rarely predictive, over an above the other variables we had available. People at the company had tried to use Prizm (and some of the other Claritas products) for all kinds of marketing-y things and they just did not provide value.

So, some bright person at A fOrmer empoLoyer said "we need to develop our own segmentation scheme. We'll provide a Universal Segmentation" (that is really what it was called) that can be used for any marketing or targeting activity." Strike two. The Universal Segmentation was a worse solution than Prizm and never made it out of the lab. Actually, A fOrmer empoLoyer took several runs at building a custom segmentation scheme using cluster analysis. None of them were found to be useful. As an aside, I mentioned the Universal Segmentation project to an expert in segmentation and he laughed and laughed. It was a bonding moment.

So why did A fOrmer empoLoyer have some much difficulty in using a tool that is in use by marketers everywhere? In both cases, the segments were not built with A fOrmer empoLoyer's needs in mind. They tried to do too much.

To finish the story. A fOrmer empoLoyer had been sending out a mass email to the whole customer base as a way of stimulating engagement and generating page views. The program was, uh, less than effective. I convinced the Customer Engagement folks to let us build a custom segmentation scheme for their email newsletter program that categorized each customer on the basis of the content they visited. In the spirit of transparency, Omniture did most of the work, they had the data. The thought was that the content someone viewed was a good proxy for their interests.

From the segmentation, we found that A fOrmer empoLoyer had less than 20 unique customer segments, but that very few segments described most of our customers. The Member Engagement staff started to create newsletters for each of the large segments. We had just started to use the segmentation scheme before A fOrmer empoLoyer stopped marketing, but the first campaign had much higher open rates than the mass mailing approach.

The lesson here is that when you initiate a segmentation project, you need to be really thoughtful about what you are trying to accomplish. Don't use every variable that you have available and just start cranking the k-means. Your segments will not be interpretable, and thus (you never see thus any more) won't be actionable. You need focus.

Instead, think about what you are trying to accomplish (e.g., be able to classify your customers into demographic groups) and build datasets that only include variables that are actionable or would give you insight about your customers. Build a segmentation scheme for one purpose. And when the guys at Claritas say "we have been doing this for 20 years. We have this nut cracked. Use our segmentation scheme", ask yourself, why do they have several segmentation products?

I could say some other stuff about creating segments, but if you focus on what you are trying to accomplish and build your dataset accordingly, you should have good results.

Break Down The Wall!

The other day, someone asked me what I do to break down silos between functional teams. First, let me say that I don't think breaking down silos is an academic exercise; you almost always generate value by talking to folks outside your group, department, business unit, whatever. I can think of at least 10 examples of really impactful projects that have come out of "Silo Busting." Second, creating an environment for silo busting is not hard. It just requires some time. However, the way you attack your silos varies depending on the construction material.

Just keep in mind, some silos are harder to bust than others. If structural conflicts (e.g., working together means that a rational person would do worse off by cooperating) or long standing animosity between silos exist, the problem becomes a lot tougher.

Set expectations.
When I first take responsibility for a team, I set the expectation that we are going to make things better. Every day. And that means innovating. And that in my experience, innovation often comes when two people who are very different talk to each other about their areas of expertise and their problems. So, there is just an incentive to getting to know your colleagues and what they do; Opportunity for improvement often results.

As an aside, I actually have an expectations document that I share with all of my staff; innovation in the service of continuous improvement, etc is one the core values listed in the document. Also, I set an example. I am often going out to lunch to meet with folks whom I have no obvious business connection and I encourage my staff to do the same.

Information sharing
In most cases, people are working in silos not becuase they enjoy it but becuase they are too busy to share information. There are some easy tactics that you can employ to facilitate information sharing. A couple of things I do is have whole team staff meetings where and monthly round tables. During these meetings, we discuss the projects we are working on if there are ways that the group can help. Barriers fall very naturally. Also, as I mentioned above, I try to go out to lunch with people outside my business unit and encourage my staff to take their clients (and anyone else they think they should get to know better) out to lunch. To provide an incentive, I offer to pay for it.

Personality or Competence
Other times, it is a personality or competence issue. Here, I run at the problem head on and really set the expectation that the staff has to work together effectively. I then put the staff members in situations where they have to work together to be successful (e.g., some special business initiative.) The expectation of being an effective collaborator also becomes part of their development plans and makes sure the problem gets fixed. If we get to the development plan stage, then I am going to be paying close attention and actively trying to help the staff member be an effective collaborator. Typically, by working together, people reach some accommodation, build some level of relationship, and break down the silos.

If it turns out that the problem is due to competence, well, no amount of relationship building is going to matter. If the staff member on my team is causing the problem, then a performance plan is going to be put in place. If it is a staff member outside my influence, I will likely have a conversation with their boss about their performance. Good luck.

The Analytic Value Chain - Understand the relationships in your data

One the dataset is together and the data is QA'ed, New analysts want to dive right in and start mining the data or building statistical models . More experienced hands just putter. The run descriptives, they check to see relationships that they thought they would see actually exist (age and income are positively related, for example), creating some histograms to look at the frequency distribution of the variables.

Why all the putterage? An effective analyst needs to develop an intuition about the data. Without that intuition, they won't have the common sense required to make some of the judgment calls they are going to need to make when they get down to modeling. Data analysis requires judgement (which is why you hire experts to do it) and without a good intuitive understanding of how the data behaves, the judgement calls to come are going to be on an unstable foundation. And you get to understand the business dynamics (by understanding what variables drive outcomes) in a way that the people who are running the business can not.

Another point, you are going to find relationships that were unexpected. These unexpected relationships are things you should take note of, you are going to want to follow-up on them and understand if they are real. These unexpected relationships are the beginning of finding what I call "Game Changing Analyses", the home run of data analytics. What do I mean by game changing? Analyses that lead your business partners into changing their business strategy, that indicate that activities that are fundamental to the business need to be reassessed. A good analyst should be able to have this kind of impact multiple times a year.

For those data analysts who read the blog, developing the intuition is critical for your ability to have impact and your reputation. Imagine that you are presenting some work to a senior executive and they ask a question that you can't answer based on your analysis. If you have a good understanding of the data you can say "I can't answer your question definitively, but given what I know of the data, I believe this to be the answer to your question. Of course, I will check the answer as soon as I get back to my desk." To the extent that you consistently get these kinds of questions right, you be come a trusted resource for the executive. I know more than one analyst whose job was saved because they proved over and over again that they were an expert in the dynamics of a business unit.

Upshot, spend the time getting to know your data . If you are a manager, build time to explore data into your project plans. Understanding the relationships in the data is a critical part of the data analysts and managers job.