Polling Vs. Research: What’s the Difference?
Polling vs. Research: What’s the Difference?
By Jeff Johnson and Geri True
Regardless of whether you lean to the left or lean to the right, there is one thing about the 2016 presidential election which is tough to deny: the pollsters got it wrong. Very wrong. According to Kenneth P. Vogel and Alex Isenstadt (reporting for Politico), “By the time polls had closed nationwide on Tuesday night, those projections had been left in shambles — just like the ones a year ago that all-but ruled out the possibility of Donald Trump winning the Republican Party’s presidential nomination.”¹
They go on to quote veteran GOP operative Ned Ryun, who said, “The very premise of polling is based on the idea that voters will be completely honest with total strangers.”² I would take this notion a step further, and suggest that (in this case), many poll participants said one thing in public, and voted another way in private. Imagine how much more skewed the results would have/could have been if those polled declared their preference directly to one of the candidates? If complete honesty with total strangers is difficult, how likely is it to occur when there is no anonymity at all?
If we compare the worlds of politics and franchising, it’s not difficult to see some similarities. In both cases, a leader (of a political party or a franchise company) sets forth a defined agenda for those in their jurisdiction/franchise. They lead (with varying levels of experience, skill, diplomacy, etc.) those under them according to this agenda. The citizens/franchisees have (by virtue of citizenship or contractual obligation) agreed to follow the “rules” set forth in this agenda (i.e., laws, operating procedures….you get the picture), or deal with the consequences.
This may seem to paint a rather dreary picture, but let me remind you that citizenship in the United States comes with certain obligations (taxes), but also benefits (public schools, roads, law enforcement, and so on). Likewise, franchise fees and royalties are the tradeoff for having (ideally) a time-tested product/service, and a road map for delivering said product/service to consumers, brand recognition, national advertising, and more. In other words, franchises provide the opportunity to be in business “for yourself, but not by yourself.”
So what does this have to do with political polls? Before you invest your savings, time, energy, credit, and reputation in a franchise opportunity, you would be wise to take a serious look “behind the curtain” of any and all franchises you are considering.
It is simply not enough to only consider the equivalent of a “popularity poll” (information without sound research methodology, or a high enough response rate to back up the results). As shared by Joseph P. Williams in USNews.com, “Three different public-opinion polls, three important elections, three decisively erroneous results: President Barack Obama blindsided Romney in the 2012 presidential elections, winning a second term by five points; McConnell crushed Alison Lundergan Grimes en route to becoming Senate majority leader in 2014; and Scots last year overwhelmingly chose to keep ties with the United Kingdom, an outcome that stunned the polling establishment.” Williams goes on to quote Michael Traugott, (a University of Michigan political science professor who specializes in polling and opinion surveys), ’Polls “give the public an independent voice that’s not generally present” otherwise in politics and political news coverage,’ Traugott said. But he says the recent errors, and a steep decline in the number of people responding to opinion surveys, is “a worrisome trend because one of the main claims of polling is that it represents the people’s views.”’³
So….we know some critical problems with regard to polls/surveys (lack of honesty, low response rates). How do you (as a franchisee candidate) use franchisee satisfaction feedback effectively in deciding whether or not to invest in an opportunity?
It’s logical to draw the conclusion that the best source of information on what it’s like to be a franchisee with a specific brand comes from (drum roll)….the franchisees with a specific brand! They are truly the only ones looking at their investment through “franchisee goggles,” a perspective which is ideally aligned with their franchisor, but can often be at odds with the brand’s leadership. Before making this giant leap, you should absolutely follow the due diligence process. Of course, this includes a thorough legal review of the FDD, Item 19, and contracts. It also includes finding out about the real-life franchisee experience.
The most common way this information-gathering process is approached is to contact selected franchisees as suggested by the franchisor. Although this may seem like a fine approach, keep in mind these potential issues:
- Franchisees can be difficult to reach (particularly if business is booming, and/or if they are working with an owner/operator business model).
- If they are hand-picked, it is likely that they are among the most established and successful ‘zees in the system. They may have been early adopters of the brand, which may or may not have hit market saturation in more recent years.
- You may be speaking with someone from an entirely different type of territory than where you would be located. Urban vs. rural locations, geographic and socioeconomic variables, local/regional demographics and customs; any or all of these can play a significant role in the success (or lack thereof) of a given brand and its franchisees. Feedback from a broad cross-section of all franchisees is critical to getting “the big picture”. This also may include single-unit and multi-unit licensees.
- There may indeed be a reward offered to franchisees (tangible or intangible) for providing a positive review of their experience, or there may be fear of retribution for a negative one. If you decide against the opportunity based upon a negative franchisee, it is not difficult to determine the source.
Given these hurdles, what other options do you have for obtaining the intelligence you need about franchisee satisfaction? It lies in research. In this case (as has been shown with polling), research means something far beyond a smattering of “surveys.” To become a wise consumer of research, consider the following:
Published Methodology: Simply put, if methodology is a mystery, how can the results be trusted? All too often, “proprietary” methodologies or arbitrary point scales or indexes would not be acceptable to credible experts in the field.
Sample: Is the sample size a defined population? Ideally, this should include all franchisees in a given franchise system, including new ‘zees and those who fall in the category of “sold, but not open”. It is important to include both of these groups, as they provide valuable insight on how a franchisor treats those who have yet to find locations or open their businesses. Too often, these upstarts “die on the vine” for lack of sufficient capital or an appropriate site.
Response Rate: In order to reduce the margin of error to +/- 4%, there must be a response rate of 70%. With a large population, a random sample may be used (in accordance with published, science-based methodology). A low response rate begs the question of whether the respondents are remotely representative of the whole group; are the respondents hand-picked, particularly happy, especially angry….? There is no way to know.
Anonymity: As cited earlier, there appeared to be a significant issue with honest responses in the 2016 political polls. How then, would the issues of fear/favor impact answers to a survey within a franchise system, unless the responses were guaranteed to be confidential?
Reporting: Research/survey results must be reported in a manner which is meaningful and easily understood. If that isn’t the case, what good can it provide? You should be able to tell from the results:
- The specific question (worded as it was for the respondent)
- How many responded to the question
- How many responses fell into each response category (i.e. “Excellent,” “Very Good,” “Good,” “Acceptable,” and “Poor”)
- The overall positive and negative scores (as a percentage)
- The total respondents for the survey, total number of franchisees in the system, and error range
Sounds like a lot of detail to manage? Perhaps, but these details matter. According to a study conducted by FranchiseGrade (which compiled survey results from 158 franchise experts who were asked to rate the importance of certain factors in determining the value of a franchise opportunity,) the #1 factor was independent franchisee satisfaction data. It ranked above the Financial Disclosure Document and the Item 19 Financial Performance Representation.⁴
Before you cast your vote to “invest” or “not invest” in a franchise opportunity, be sure your decision is based not on a sales pitch, but on actual “behind the scenes” insight. This may be your only chance to vote; make it count!
¹Vogel, Kenneth P. and Isenstadt, Alex, Politico, 11/09/2016, http://www.politico.com/story/2016/11/how-did-everyone-get-2016-wrong-presidential-election-231036
²Vogel, Kenneth P. and Isenstadt, Alex, Politico, 11/09/2016, http://www.politico.com/story/2016/11/how-did-everyone-get-2016-wrong-presidential-election-231036
³Williams, Joseph P. US News, 9/28/2015, http://www.usnews.com/news/the-report/articles/2015/09/28/why-public-opinion-polls-are-increasingly-inaccurate
⁴2015 Franchise Grade® Franchise Expert Survey, available at firstname.lastname@example.org