Context effect in the study on the perceptions of violence against LGBT people

On 27th December 2019 by Mohamad Al-Hakim in Attitudes, Call it Hate, Context Effect, discrimination, hate crime, Hate Crime Policy, homophobia, LGBT hate crime, Research, Uncategorized, Victims, Violence and tagged Attitudes, Context effect, Hate, Hate Crimes, LGBT, Research, Social Attitudes, Statistics

By: Jacek Mazurczak and Piotr Godzisz

Jacek Mazurczak is the chief statistician in the Call It Hate research. He is currently an expert at the Institute of Social Security.

Piotr Godzisz, PhD, is the principal investigator in the Call It Hate research. He sits on the management board of Lambda Warsaw and the advisory board of the INHS. He has recently started lecturing at the Birmingham City University.

_________________________________

While often regarded as a bias or error impacting the quality of the data, the context effect in attitude surveys may also serve as a useful tool. In the design of the recently published Call It Hate study, this effect was predicted and used to analyse the gender dimension of respondents’ attitudes towards the victims of anti-LGBT violence.

Understanding attitudes toward LGBT people as victims of crimes – see the key findings of the Call It Hate project!

A bit about the context effect

The context effect in attitude surveys is generally considered as a methodological issue. As this explainer video and this encyclopedia entry argue, earlier questions may influence (prime) how people interpret later questions, reducing the accuracy of responses.

One of the best-known examples of how this works is a study conducted in 1948 in the USA, reported by Schuman and Presser. Respondents were provided with a set of two questions: (A) “Do you think the United States should let Communist newspaper reporters from other countries come in here and send back to their papers the news as they see it” and (B) “Do you think a Communist country like Russia should let American newspaper reporters come in and send back to America the news as they see it”. Of those who were asked about the presence of Communist journalists in the USA first (AB), 36.5% agreed. In the reverse order (BA), the percentage of admitting Communist journalists in the USA increased to 73%.

To improve the quality of the data, the context effect should normally be countered, e.g. by rotating the order of the questions in different versions of the survey. If used consciously, however, the context effect may serve as a tool to interpret social facts. Tourangeau and Rasinski argue that “[c]ontext can, for example, suggest a standard to which subsequent items are then compared”. As such, the first question, and the response to it, may provide a framework of reference for subsequent questions.

Context effect in the Call It Hate study

The priming effect was factored in the construction of the Call It Hate survey, conducted among 10,612 respondents in 10 EU member states, and looking at attitudes towards LGBT people as victims of crimes.

Respondents were asked how much empathy they feel for the victim(s) of crime in specific scenarios, which probed for differing emotional reactions to crimes according to the sexual orientation or gender identity of the victim(s), according to the victims’ behaviour at the time of the incident, and the characteristics of the perpetrator(s). Respondents answered the questions under one of three routes to which they were randomly assigned. The routes referred to either gay men, lesbians or transgender persons. In the UK and Ireland, a fourth route, about bisexual people, was provided.

In order to provide a framework of reference, respondents on the gay and lesbian routes were first asked about their empathy for a heterosexual couple assaulted on the street. This statement was followed by one about a similar situation but involving either a gay or lesbian couple. This allowed the respondents to calibrate their responses. Together, both statements served as a reference for the remainder of the statements in the question, whose order, in turn, was rotated. The responses for the lesbian and the gay routes are presented in table 1 below.

	Gay route	Lesbian route
A heterosexual couple who is physically assaulted on the street when holding hands REFERENCE CASE	8.61	8.77
A [gay man / lesbian] who is physically assaulted by a group of people who are members of a far-right extremist organisation	8.05	8.45
A [gay / lesbian] couple who is physically assaulted on the street when holding hands REFERENCE CASE	7.64	8.24
A [gay man / lesbian] participating in [national name of Pride event] who is physically assaulted by counterdemonstrators	7.32	7.74
	n = 3,221	n = 3,115

Table 1. People may feel more or less empathy for victims of crime depending on the context in which the crime occurs. To what degree do you feel empathy for people who experience crime in each of the following situations? Please answer using a scale, where 0 means that you feel no empathy at all for the person against whom the crime is committed, and 10 means that you feel complete empathy for that person. Selected results. Source: Call It Hate.

Table 1 shows that, in both routes, the average empathy for the straight couple assaulted on the street when holding hands was significantly higher than for a gay or lesbian couple assaulted in a comparable situation. In fact, the empathy for gay and lesbian victims was lower across all scenarios – even in the case of an attack by members of a far-right group. A pattern is also visible in which lesbian victims received more empathy than gay male victims, regardless of the context of the crime.

This shows that, apart for sexual orientation, the gender of the victim(s) is also an important factor influencing the level of empathy they elicit.

Empirically, the above findings document one more context in which anti-LGBT prejudice is manifested, adding to body of research on discrimination. For example, similarly to the Call It Hate study, the recent Eurobarometer, which probed for comfort with couples expressing affection in public, revealed both the negative attitude towards same-sex couples and a gender bias.

Importantly, however, the averages for the straight couple also significantly differed between routes. Respondents assigned to the gay route declared having less empathy also for the heterosexual couple (M = 8.61), compared to people who drew the lesbian route (M = 8.77). This shows that the gender bias extended beyond the gay / lesbian division and affected declared empathy for straight couples.

The difference turned out to be statistically significant at the level of p = 0.05.

We interpret this unexpected finding as an effect of reverse priming, in which the response to the prime is corrected for bias. This may happen as respondents have a tendency to provide responses deemed to be more socially acceptable than the “true” answers (social desirability bias). In line with this explanation, respondents in the gay route corrected the levels of empathy for the straight couple in order not to be perceived as homophobic. This was possible as all statements were visible for the respondents at the same time.

We found a similar regularity in the question about witnessing a crime. There, the respondents on the lesbian route were more likely to declare readiness to intervene on behalf of the victim than the respondents on the gay or transgender routes. The gendered difference extended to all items in the question (i.e. all categories of victims), including the reference category “someone”, which was placed first in the question. Interestingly, however, unlike in the gay and transgender route, there was no significant difference between the level of support for a lesbian victim vs the reference category. This suggests that, unlike empathy, where both the sexual orientation and the gender of the victim play a role, the readiness to react is influenced primarily by the victim’s gender, with (presumably cisgender) women being more likely to be helped by witnesses than men or transgender people.

Conclusion

Instead of counteracting the context effect, the Call It Hate survey embraced it, making it part of the methodology. By using reference cases to calibrate the answers to questions about the attitudes towards LGBT victims, we expected to find different reactions to anti-LGBT violence vs other kinds of violence. We also expected to find more positive attitudes towards lesbian victims (as women) compared to gay male victims. We managed, however, to find more than that: The different reactions to anti-gay and anti-lesbian violence extended to reference cases, e.g. violence against straight victims or an undescribed “someone”, which shows that respondents corrected their responses to hide bias. This unexpected finding would not be discovered if we did not consciously employ the context effect as a tool in the design of the research.

***

The whole report Awareness of Anti-LGBT Hate Crime in the European Union, edited by Godzisz and Viggiani, is available on the Call It Hate project website.

The project is co-funded by the European Commission under the Rights, Equality and Citizenship programme 2014-2020.

***