Out of focus: the pitfalls of political market research

Jan 31, 2022
Support among females is the lowest in the Liberal Party’s history. (Image: AAP/Mick Tsikas)

Political parties spend big on focus group research with the approach of an election – but changes in methodology raise questions over its worth.

During the next few months millions of dollars will be spent on political market research – much of it on focus groups which take the form of group discussions.

Allegedly it allows political parties to identify what people are thinking and how to respond to that. It is not a consultative project but rather one often devoted to testing ideas which form the basis of slogans. It also won’t be much about policy unless it is to elicit ideas for attacks on opponents’ promises.

The focus group is a product of the distinctions between qualitative and quantitative research which date back to social science developments in the US and Australia.

In Australia, researchers such as University of Melbourne professor A.F. “Foo” Davies and pioneering political psychologist Graham Little, who was also at UoM, interviewed people at depth for the Images of Australia project. Social researcher Hugh Mackay was also an Australian pioneer of such interviews, which he used to develop quite detailed pictures of Australian society.

But over decades focus group research has evolved from that approach and Judith Brett’s new book, Doing Politics, has an important chapter (“Writing Ordinary People’s Politics”) that looks at the weaknesses of much political quantitative and qualitative research and outlines alternative approaches.

Graham Chant, a former principal at ChantLink consultants and retired academic psychologist, published a paper (Qualitative Research: Issues & Techniques) in 2012 which also described methodological problems in qualitative research and recommended measures to address them.

Commenting recently on the paper, he said: “While I have not been directly involved in market research now for at least three years, my observations prior to retiring is that usage/application of group discussions has progressively got worse. And this applies to Australia but even more so to the USA.

“The direction of group discussions in Australia has been negatively impacted by group recruiting agencies having large databases of people who are willing to attend groups, perhaps because they get paid or because they like the idea of participating. But this means that these people represent a biased set of respondents and do not necessarily reflect the broad Australian society. For instance, you could argue that wealthy people are less likely to participate as would be the case of those who hold more senior roles in business or the public service.

“The environment in which groups are conducted is now very much focused on specialist group rooms that contain one-way screens, cameras and recording devices. When respondents arrive they sit outside in waiting rooms prior to being ushered into the group room. The environment is certainly not conducive to open discussions and I was certainly aware in the past that many groups had at least eight and often more participants.

“The original approach to groups was to conduct them in people’s homes or offices, environments which reflected something people were used to and encourage ‘real’ interactions. This is certainly no longer the case – it’s probably due to pressure from clients wanting to see the ‘real thing’ and not necessarily trusting reporting from the market researcher.” 

Chant also raises questions about the now generally standard group of eight when in normal society “real groups rarely consist of more than four to six people”. 

He also warned that even if groups are conducted according to the practices his paper recommends, the researcher still needs to beware of accepting the outputs as representative of a large population. This is particularly a problem with commercial research, which generally has results based on a limited number of groups although political attitudinal research usually draws on many more groups over longer periods of time.

But both commercial and political attitudinal research have a specific purpose. It is not consultation about what should be done but rather what can be done.

There is no doubt that political attitudinal research can be both excellent and valuable but, given the secrecy surrounding it, there is much about it that is worrying.

However, close observers can sometimes discern what the political parties are seeing in their attitudinal research by reverse engineering what they are saying by asking the perennial Lasswellian questions: who says what, which channel, to whom, with what effects, for what purpose and under what circumstances?

The questions are also a useful way to deconstruct articles by political reporters quoting sources – particularly those sources who insist on being quoted without attribution. In Australia the main source of this sort of stuff is the prime minister or the prime minister’s office. Journalists continue to co-operate with these surreptitious efforts to distort reality because they don’t want to get taken off the drip.

The reporting of quantitative research is also problematic and it is not unusual to see a political journalist pontificating on a 0.5 per cent change in a survey with a statistical error range of 2 to 3 per cent. Because media outlets preference their own polling and tend to mention other polls so rarely, you do not see the detailed comparisons, aggregation and probability estimates such as those produced in the US by FiveThirtyEight.

Retrospective analysis – if based on actual research and not opinion – is often more robust and some of that has implications for our views of the 2019 “‘miracle” election result and possible 2022 outcome.

For instance, if you tried to reach a consensus about the 2019 “miracle” from media reporting, you would conclude any or all of the following: the polling was wrong; voters didn’t trust Bill Shorten; Scott Morrison was a miracle worker; tradies swung from Labor to Liberals; coal miners hated Bob Brown; Labor’s policy platform was too big and threatening; and most Australians (whether or not they owned investment properties or shares) were terrified of losing their negative gearing rorts and their franking credits.

Yet some post-election Liberal Party analysis, discussed by Judith Brett in her essay on Morrison’s “quiet Australians”, suggested that the decisive factor was the vote swings of suburban working mothers in the 35 to 54 age group. This doesn’t seem to have been discussed much anywhere – but if it’s right then Morrison could be in big trouble this year as all the indications are that support among females is the lowest in the current Liberal Party’s history.

Indeed, election night might see Grace Tame, Brittany Higgins and a few former female Liberal MPs dancing on Morrison’s political grave along with many others.

Share and Enjoy !

Subscribe to John Menadue's Newsletter
Subscribe to John Menadue's Newsletter

 

Thank you for subscribing!