Every year, marketing campaigns get bigger and better. However, measuring the real impact of marketing campaigns has not gotten easier. That’s because most tools used to measure the success of marketing efforts still rely on traditional methods of survey research. Like some of the other brand tracking tools on the market.
Traditional methods of survey research usually involve collecting a sample of respondents. They then make and conclusions about the opinions of specific groups within the sample (i.e. millennials) based on the answers given by those specific respondents in the sample.
For example, to find out how much millennials love a brand, traditional brand tracking methods first collect a large sample and then zoom into the millennials in the sample to find out what they think.
These methods struggle to overcome a major tradeoff between detail and quality. This leads to the following problem: the more you zoom in on the opinions of specific audience groups, the more quality you lose in the results. It becomes difficult to track the opinions of the people that matter — the target audience. Knowing the opinion of the target audience is key to discovering whether or not a marketing campaign is actually working.
This article will explain how Latana uses a cutting-edge method in data science — MRP — to overcome this limitation. It will shed light on how we are using this technology to power brand tracking, the next-generation brand tracking tool that paints a fine-grained picture of how specific types of customers perceive brands.
Under the hood of MRP: What is it?
Let’s kick off this article by discussing what MRP is. MRP, also known as Mr.P, stands for Multilevel Regression Poststratification. It has been mostly used in election forecasts and had its publicized breakthrough with this paper, where the authors predicted the 2012 Elections election for all 50 US States without actually having representative samples from each state. Latana is now applying MRP to brand tracking.
At its core, MRP is a model-based method used to generate estimates for responses in a survey. First, it measures the relationship between the respondent’s characteristics in a dataset (i.e. demographics) and their responses to a survey question. From this, it builds a model that makes predictions for responses, given a set of respondent characteristics. To put it simply, if you give the model a set of respondent characteristics (i.e. young, highly educated, mid-income) it will produce an estimate for how that type of respondent would answer a survey question. That’s the “regression” part.
The “multilevel” part is used to organize the respondent characteristics in the model into groups so that their relationships can better capture how the variables interact in real life. (i.e. a common “level” for grouping is “country”, meaning that the model will measure if respondent characteristics have different relationships to the variable of interest depending on the country).
Lastly is the “post-stratification” part. Given that the model can predict the response for any type of respondent (conditional on the respondent’s’ characteristics), the last step required to generate representative results is to take weighted averages of all the predictions. The weighted average is the postratification part, and it is used to make sure that the overall predictions for the responses of a group of people take into account the right proportions of respondent characteristics (i.e. young, highly educated, mid-income) that belong in that group in order to better represent this population in real life.
How Traditional Brand Tracking Measures Marketing Campaigns
Now, that we know a bit more about MRP, let’s look at how traditional brand tracking works.
Imagine a company preparing to launch a massive marketing campaign to increase awareness of its brand and product. After a lot of user-testing and message-testing, the marketing team has a good sense of what customer audiences are potentially most interested in the product.
For the purpose of this article, imagine the company is a craft beer company and their target customers are tech-savvy, soccer-playing, millennials.
This craft beer company launches a marketing campaign to appeal specifically to these tech-savvy, soccer-playing millennials in order to maximize the impact of the marketing investment.
The company needs a tool that measures whether or not this campaign actually reaches this niche audience and makes them more aware of the brand and product. This is generally what we call brand tracking.
The Problem with Traditional Brand Tracking
To track brand awareness over time, the company uses the following method: they conduct a nationally representative survey once a month, each time with a sample size of 1000 respondents. From this large monthly sample, it’s easy to get an overall picture of brand awareness for the general population. However, the marketing campaign is designed specifically for tech-savvy, soccer-playing millennials. Our craft beer company wants to zoom in and get results for this audience specifically, not a more general audience.
To track the opinions of the target audience, the company starts zooming into the sample. It finds 300 tech-savvy respondents, 400 millennials, and 150 soccer players. In the end, it identifies the 20 respondents in the sample who are part of the tech-savvy, soccer-playing millennial target audience.
Of course, with a group of respondents this small, the tradeoff between detail and quality becomes very apparent: even though we see that 35% of these respondents are aware of the brand in a given month, the small number of respondents makes the results very unreliable: brand awareness can be anywhere within the large confidence bounds, ranging from 15% and 55%, making it virtually impossible to track progress over time.
MRP overcomes this tradeoff between detail and quality by taking a different approach: instead of relying on the 20 respondents in the sample to draw an unreliable estimate for the target audience’s brand awareness, MRP uses information from the entire sample to make a prediction.
This is how it works:
Starting with the same sample of 1000 respondents, this method focuses on respondent characteristics instead of the individual respondents themselves. It recognizes that the tech-savvy, soccer-playing millennial target audience can be separated into three characteristics: tech-savvy people, soccer players and millennials.
The method then identifies how each of these individual characteristics is related to brand awareness. For example, to find out if there’s a relationship between playing soccer and being aware of the brand, the model looks at all the soccer players in the sample (n=150) and compares their brand awareness with the 850 non-soccer players. Whatever difference it finds becomes the “effect” of being a soccer player on brand awareness.
Next, the model does the same for millennials: it looks at all the 400 millennials in the sample and compares their brand awareness to the 600 non-millennials. This difference helps to define the “effect” of being a millennial. Lastly, the model does the same for tech-savviness.
Once the model is built based on these characteristics, it can now predict the brand awareness level for a respondent with any combination of these characteristics — aka, our target audience: the tech-savvy, soccer-playing millennial.
Notice that the MRP model uses information from the entire sample when it makes predictions. For example, finding the “effect” of soccer-playing characteristic relied on comparing the 150 soccer players to the 850 soccer players, therefore using information from all 1000 respondents in the sample. That means the predicted results from MRP will borrow strength from the entire sample, and produce much more reliable results.
In this case, we see that the predicted brand awareness for the target audience is 35% — but now the confidence interval is much smaller, ranging from 34% to 36%. With reliable results like this it’s much easier to track changes over time, and therefore measure if the marketing campaign was actually successful for this target audience.
To Recap: Why We Use MRP for Brand Tracking
When traditional brand trackers try to measure opinion in small target audiences, they face problems because they slice and dice the sample and narrow down on specific respondents within the target audience. They only use this small amount of information to make an estimate for brand awareness for the target audience, which leads to very unreliable results with a large margin of error.
We solve this problem by using MRP: this method does not restrict itself to the small number of respondents in the target audience. Instead, it builds a model from information in the entire sample and uses this model to predict brand awareness based on a respondent’s characteristics. In other words, to estimate the brand awareness of someone in the target audience — tech-savvy, soccer-playing millennials — , MRP looks at all soccer players, all tech-savvy people and all millennials in the sample. It then combines the effects of these characteristics to produce an estimate for respondents who have all three characteristics — our target audience. Because it uses the entire sample to build the model, the estimate for the target audience is actually very precise, even though there are only technically 20 respondents of the target audience in the sample.
It’s much easier to draw reliable insights from the data when you have higher precision with a smaller margin of error.
Next time a company launches a marketing campaign and wants to increase brand awareness for a specific type of target audience, they should look for Latana.
MRP-based brand tracking is the best way to get reliable insights into the opinions of the people that matter — the target audience.
If you have any specific questions about MRP or brand tracking, please contact us here.
MRP FAQ
Why did the company conduct a nationally representative survey, to begin with, if their intention was to look at the opinion of the target audience (tech-savvy, soccer-playing millennials)?
Finding a large sample of very specific people is quite difficult in survey research — and quite expensive. Furthermore, in reality, companies often have several target audiences that they are interested in. If they want to track the opinion of each of these groups with traditional methods, they would need a large sample of each of these audiences — an even more difficult challenge. This is why MRP is so powerful for predicting the responses for small target groups (also referred to as “small area estimations” in the academic community).
What if the company doesn’t know exactly what type of audience it wants to target to begin with?
With MRP it’s possible to generate an estimate for each of these specific groups, all from the same general sample. The main condition is that the general sample has enough respondents from each of the main characteristics used in the model (i.e. enough tech-savvy respondents, enough soccer players, enough millennials), which is much easier to satisfy than finding respondents with specific combinations of characteristics (i.e. tech-savvy, soccer-playing millennials).
One of the additional strengths of MRP is that it is very flexible. The model looks at all the available characteristics and uses whichever characteristics are most important. Latana’s brand tracking software includes a list of variables in the survey, ranging from socioeconomic variables to behavioural variables, in order to cover the widest range possible. In the end, it can make predictions for a respondent with any combination of these characteristics. So if a company doesn’t know what target audience they are most interested in, the MRP model can provide the predicted brand awareness for all the possible combinations of characteristics. This helps the company identify which types of respondents actually reacted most positively during the campaign — thus helping the company identify its target audience.
Originally published at https://www.latana.com.
