Every year, marketing campaigns get bigger and better. However, measuring the real impact of marketing campaigns has not gotten easier.
That’s because most tools used to measure the success of marketing efforts still rely on traditional methods of survey research — like some of the other brand tracking tools on the market.
Latana, however, uses a completely different method: MRP. This article will take a look at why traditional methods of survey research are lacking, what MPR is, and how it creates more valuable brand tracking data.
It will also shed light on how Latana is using MRP technology to paint a more nuanced picture of how specific types of customers perceive brands. Let’s dive in!
Why Are Traditional Methods of Survey Research Problematic?
Traditional methods of survey research usually involve collecting data from a sample of respondents and, then, using the opinions of specific groups (i.e. Millennials) within the sample to draw conclusions.
For example, to find out how aware Millennials are of a certain brand, traditional brand tracking methods would first collect a large sample and, then, zoom in on the Millennials in the sample to find out what they think.
Traditional methods of survey research first collect a large sample size and then zoom into only the Millennials within the sample.
So, what’s wrong with this traditional method? Well, it struggles to overcome a major tradeoff between detail and quality. This leads to the following issue: as you zoom in closer on specific audience groups, the quality of your results decreases.
Therefore, it becomes more difficult to track the opinions of the people that matter — aka your target audience. Knowing the opinion of your target audience is key to discovering whether or not a marketing campaign is actually working.
Seems pretty important, right?
What is MRP?
Let’s kick off this article by discussing what MRP is. MRP, or Multilevel Regression Poststratification, is a form of advanced statistical modeling that has been previously used as a means of forecasting election results.
MRP’s most publicized breakthrough came via Microsoft Research’s 2014 paper, “Forecasting elections with non-representative polls”. In this paper, the authors predicted the 2012 U.S. election results for all 50 states — without having representative samples from each state.
Latana is now using MRP for brand tracking purposes.
At its core, MRP is a model-based method used to generate estimates for responses in a survey. First, it measures the relationship between the respondent’s characteristics in a dataset (i.e. demographics) and their responses to a survey question.
Given a set of respondent characteristics, it then builds a model that makes predictions for responses. To put it simply, if you give the model a set of respondent characteristics (i.e. young, urban, mid-income) it will produce an estimate for how that respondent would answer a survey question. That’s the “regression” part.
The “multilevel” part is used to organize the respondent characteristics in the model into groups so that their relationships can better capture how the variables interact in real life.
For example, a common “level” for grouping is “country” — meaning that the model will measure if respondent characteristics have different relationships to the variable of interest depending on the country.
Lastly, we have the “poststratification” part. Given that the model can predict the response for any type of respondent -conditional on the respondent’s characteristics — the last step required to generate representative results is to take weighted averages of all the predictions.
The weighted average is the “poststratification” part, and it’s used to make sure that the overall predictions for the responses of a group of people take into account the right proportions of respondent characteristics (i.e. young, urban, mid-income) that belong in that group in order to better represent this population in real life.
How Traditional Brand Tracking Measures Marketing Campaigns
Now, that we know a bit more about MRP, let’s take a deeper look into how traditional brand tracking works.
Imagine a brand is preparing to launch a massive marketing campaign with the goal of increasing brand awareness. After a lot of user and message-testing, the Marketing team has a good sense of what target audiences are potentially most interested in the product.
For the purpose of this article, imagine the company is a craft beer company, called “Maltoast”, and their target customers are urban Millennial men.
Maltoast launches a marketing campaign to appeal specifically to these urban Millennial men, in order to maximize the impact of the marketing investment.
But the brand needs a tool that measures whether or not this campaign actually reaches this niche audience and makes them more aware of the brand and product. This is generally what we call brand tracking.
The Problem with Traditional Brand Tracking
To measure brand awareness over time, Maltoast uses the following method: they conduct a nationally representative survey once a month, each time with a sample size of 1000 respondents. From this sample, it’s easy to get an overall picture of brand awareness for the general population.
However, the marketing campaign in question was designed specifically for urban Millennial men. Therefore, Maltoast wants to zoom in and get results for this audience specifically — not just the general population.
Here’s an illustrative example of how traditional brand tracking would work:
To track the opinions of the chosen target audience, Maltoast starts zooming in on the sample and finds 300 male respondents, 500 Millennials, and 200 that live in an urban area. In the end, it identifies 20 respondents within the sample who have all three characteristics and, thus, can be included in the target audience of urban Millennial men.
Of course, with a group of respondents this small, the tradeoff between detail and quality becomes very apparent. Even though we see that 35% of these respondents are aware of the brand in a given month — the small number of respondents makes the results very unreliable.
Brand awareness can be anywhere within the large confidence bounds seen in the graphic above — ranging from 15% and 55% and making it virtually impossible to reliably track progress over time.
The MRP Solution
MRP overcomes this tradeoff between detail and quality by taking a different approach.
Instead of relying solely on the 20 respondents in the sample who meet all three demographic requirements and drawing an unreliable estimate for the target audience’s brand awareness — MRP uses information from the entire sample to make a prediction.
How does it work? Well, starting with the same sample of 1000 respondents, MRP focuses on respondent characteristics instead of the individual respondents themselves. It recognizes that a target audience of urban Millennial men can be separated into three characteristics: urban, men, and Millennials — as seen in the graphic below.
MRP then identifies how each of these individual characteristics is related to brand awareness.
For example, to find out if there’s a relationship between male respondents and brand awareness levels, the model looks at all the male respondents in the sample (n=300) and compares their brand awareness with the 700 non-male respondents.
Whatever difference it finds then becomes the “effect” (i.e. the strength of the relationship) of being male on brand awareness.
Next, the model does the same for Millennials. It looks at the 500 Millennials in the sample and compares their brand awareness to the 500 non-millennials. This difference helps to define the “effect” of being a Millennial. Lastly, MRP does the same for the characteristic of living in an urban area.
Once the MRP model is built based on these characteristics, it can now predict the brand awareness level for a respondent with any combination of these characteristics — aka, our target audience: urban Millennial men.
Notice that the MRP model uses information from the entire sample when it makes predictions. For example, finding the “effect” of education characteristics relied on comparing the 200 urban respondents to the 800 non-urban respondents — therefore using information from all 1000 respondents in the sample.
That means the predicted results from MRP will borrow strength from the entire sample and produce much more reliable results.
In this case, we see that the predicted brand awareness for the target audience is 35% — but now the confidence interval is much smaller, ranging from 34% to 36%. With reliable results like this, it’s much easier to track changes over time, and, therefore, measure if marketing campaigns are actually successful with specific target audiences.
Why We Use MRP for Brand Tracking
When traditional brand trackers try to measure opinion in small target audiences, they face problems because they slice and dice the sample and narrow it down to specific respondents within the target audience.
Then, they use a very small amount of information to make an estimate for brand awareness for a target audience — which leads to unreliable results with large margins of error.
We solve this problem by using MRP, a method that does not restrict itself to the small number of respondents in the target audience. Instead, it builds a model from information in the entire sample and uses this model to predict brand awareness based on a respondent’s characteristics.
Ultimately, it’s much easier to draw reliable insights from the data when you have higher precision and smaller margins of error.
Final Thoughts
Next time a company launches a marketing campaign and wants to increase brand awareness for a specific type of target audience, we highly recommend choosing a brand tracking tool that uses MRP — like Latana.
MRP-based brand tracking is the best way to get reliable insights into the opinions of the people that matter — your target audience.
If you have any specific questions about MRP or brand tracking, check out our recently updated article “ What is MRP? And More FAQs” or contact us here.
Updated by: Cory Schröder on 18.03.22.
Originally published at https://latana.com.
