Who hasn’t been asked to share their opinion in a survey or poll? Such tools are among the most common ways of finding out what different social groups think about specific issues. But is simply asking, “Do you think that…?” enough to truly capture the moods and attitudes that shape society?
Research by Prof. Karolina Hansen and Dr. Aleksandra Świderska from the Faculty of Psychology at the University of Warsaw shows that the way questions are asked can strongly influence respondents’ answers. Replace a simple yes/no question with an open-ended one, and the picture of public attitudes suddenly becomes far more varied and nuanced.
What do Poles think about refugees?
To find out how the form of a question shapes the answers people give, Prof. Karolina Hansen and Dr. Aleksandra Świderska from the Faculty of Psychology at the University of Warsaw designed two surveys focusing on a highly contentious issue: refugee policy. Each survey involved more than 250 participants.
The respondents were selected to reflect Poland’s demographic structure in terms of gender, age, and place of residence. The surveys were conducted a year apart to capture possible shifts in opinion over time: the first in 2016 and the second in 2017.
The surveys began with an open-ended question:
“What strategy, in your opinion, should Poland adopt concerning refugees who want to come to Poland?”
Respondents were then asked a series of closed-ended questions, among them:
- Do you think that the arrival of refugees to Poland would make Poland a better or worse country to live in?
- Do you think that Poland should accept some of the refugees coming to Europe?
The scale ranged from strongly negative to strongly positive responses.
The open-ended responses were analyzed both manually and with automated tools, which helped to identify recurring themes and determine the tone of each statement.
Let them in, but…
The results differed depending on how the question was asked. In the first survey, when respondents were asked the closed-ended question “Do you think Poland should accept some of the refugees coming to Europe?”, 28% answered yes, 49% were opposed, and 23% were undecided. In the second survey, 26% responded positively, 56% negatively, and 17% remained undecided.
The open-ended question about Poland’s refugee strategy revealed a somewhat different picture. In the first survey, as many as 54% of respondents expressed support for accepting refugees. In the second survey, support stood at 45%.
Importantly, this support often came with reservations about the conditions refugees should meet in order to live in Poland. In the first survey, respondents frequently mentioned the need to assimilate, find employment, or accept low social benefits.
A year later, these reservations took a slightly different form. For the first time, respondents spoke about accepting only selected individuals or groups – for example, women with children.
How can we accurately capture public opinion?
The research shows that both open-ended and closed-ended questions have their limitations.
Closed-ended questions with predefined answer options can limit respondents’ ability to express the full complexity of their views. As a result, they may oversimplify – or even distort – people’s actual attitudes toward a given issue.
Open-ended questions, by contrast, make it easier to uncover motivations and reveal subtle differences in opinion. Yet surveys do not always provide the time or practical opportunity to include them.
According to Karolina Hansen and Aleksandra Świderska from the University of Warsaw, the most effective way to understand social attitudes is to combine both types of questions. This approach allows researchers to capture a far more nuanced picture of public sentiment – one that may prove especially valuable when designing social policies that respond to society’s real concerns and challenges.
As Prof. Hansen explains:
“Such research helps us better understand what is happening in society. Although we observed a strong influence of narratives present in the media and politics on our respondents’ statements, the open nature of the question still allowed for a freedom of expression that is not possible with a closed question. A closed yes/no question does not allow answers such as ‘but…’ or ‘OK, but on the condition that…’ – and these are extremely important considerations. Proposed policies, in turn, should recognize and understand society as fully as possible.”
The text was originally published in Polish on the Serwis Naukowy UW website on February 26, 2025.
