Bias in Research
Bias can unintentionally shape study outcomes and interpretations, making it important to recognize and address. Understanding bias helps researchers create fair, accurate studies and allows readers to critically evaluate research.
If you like a refresher on bias, please refer to Unit 1 Lesson 3.1 and Unit 3 Lesson 1.5!
1. Common Types of Bias in Research Design
Subconscious bias
Subconscious bias is a bias that researchers have without realizing it, usually influenced by their background or experiences. It can affect decisions like choosing research topics or selecting participants.
Framing bias
Framing bias occurs in a study when the way questions are asked can influence how participants respond (e.g. leading questions). This can unintentionally lead to answers that match the researcher’s expectations.
Confirmation bias
Confirmation bias happens when researchers focus on results that support what they already believe, which may lead to them ignoring evidence that challenges their views.
Identify the type of bias in the following scenarios:
1. A survey asks, "Don’t you agree that organic food is healthier?" Framing Bias
2. A researcher unknowingly selects participants from a similar cultural background. Subconscious Bias
2. Funding and Grant Bias in Research Design
Funding sources, especially when linked to industries, governments, or private organizations, can heavily influence the direction and focus of research. After all, research of high quality usually requires a lot of funding support to complete. Therefore, where research money comes from can impact how studies are designed and reported.
Hover over the boxes to see an example for each scenario!
Selective Research Focus
Funding organizations may support topics that benefit them while ignoring less profitable subjects.
E.g. A pharmaceutical company funds research on a new painkiller but chooses not to fund studies on alternative, non-drug treatments like physical therapy or acupuncture, which might compete with their products.
Publication Pressure
Some researchers may feel pressure to produce findings that please their funders.
E.g. A food industry sponsor funds a study on the health effects of artificial sweeteners. The researchers may feel pressure to design the study in a way that is more likely to show the sweetener as safe, such as by selecting comparison groups that minimize potential negative effects.
Skewed Data Reporting
Studies may only highlight positive results, leaving out negative or neutral findings.
E.g. A biotech company funds a clinical trial for a new supplement. The study finds some benefits but also uncovers potential side effects. The company publishes only the positive results, leaving out data on the side effects to maintain public confidence in their product.
3. Design Bias and Failure to Control Design
Biases and flaws in a study’s setup can lead to certain outcomes being favoured, sometimes unintentionally. These flaws or inconsistencies in the structure of the study can be caused by:
Sampling Bias
-
If the sample population is not representative of the larger population, the results may not be generalizable.
-
Example: If a new medication is only tested on young people, it may not work the same for older adults.
4. Errors in Recruitment Design
Bias can be controlled during the recruitment of participants or selection of samples for a particular study.
Design of the study needs to have a well defined population of interest, with well defined inclusion/exclusion criteria.
-
Bias occurs if the study population does not closely represent the target population and if subjects are selected in a non-random way
-
Bias can occur in patient recruitment if you are recruiting in a limited way
-
E.g. Recruitment through an online survey posted only on Facebook - this is only reaching those with a Facebook account and Internet access
-
Some participants are more or less likely to enter a study than others as they may have a personal connection or gain, and due to this, it may not be representative of the entire population. Their personal motivation could also influence their responses and decisions.
It can be quite challenging to carry out the recruitment of participants given layers of stringent criteria that are imposed to help minimize bias in the process.
1
For instance, an overly strict recruitment criteria may lead to a lack of eligible participants.
2
In addition, if a research study is well planned but is hard for participants and/or researchers to execute, it can also lead to disinterest in participation of the study or participant withdrawal from the study.
3
Sometimes, even funding limitations (discussed above) and time limits can further impede the recruitment of participants.
4
Further down the line, all of this will result in insufficient sample size when performing data analysis.
What impact does insufficient sample size have on your research study?
Not meeting sample size increases the possibility of having type 2 errors (i.e. false negative - concluding there is no significant difference between treatment groups when there might be, see Unit 1 Lesson 1.3 for further discussion on this topic!). In the case of clinical trials, the lack of a sufficient sample size leads to a level of uncertainty about treatment efficacy, which can lead to delays in future trials or next stages, limiting the effectiveness of the research being conducted.
5. How Researchers’ Backgrounds Affect Studies
The final concept we will introduce in this section is researcher positionality and how it may impact their research.
Positionality refers to a researcher’s position or stance in relation to their research and participants. It is essentially their worldview - how they view reality and knowledge. This is impacted by factors such as social identity, professional background, and research context.
Researcher positionality affects all stages of research study, from the choice of research topic to selection of methodology and methods, data collection approach, and even analysis and interpretation of results.
For example, consider the following hypothetical study:
​
A researcher studying school discipline is a former teacher who believes strict policies improve student behaviour. Their views might shape:
Research topic: They focus on benefits of strict policies rather than potential harms.
Selection of methods: They believe that conducting interviews and recording qualitative data is more effective than collecting quantitative measurements.
Data Collection: Students may hesitate to share negative experiences due to perceived power differences between students and teachers.
Interpretation of results: The researcher may emphasize teacher viewpoints while overlooking student concerns, diminishing the validity and applicability of the study.
Researcher positionality also exists relative to the participants of a study.
Insiders: closer to the community they are studying
-
Easily build rapport between participants, better understanding of cultural nuances, able to ask more insightful questions due to background knowledge
-
Difficulty maintaining objectivity, role conflicts, thereby creating bias or assumptions
Outsiders: non-member of studied community
-
Bring a different perspective, can ask “dumb” questions, perhaps able to be more objective
-
May not easily build rapport among participants, misunderstandings/miscommunications, no insider knowledge (goes back to data collection approach)
-
Typically, an insider’s strength is an outsider’s weakness and vice versa
Nowadays, we have less of a binary view of the insider-outsider positionality and rather a continuum - researchers may shift from one view to another throughout the duration of the study.
It is important that researchers are explicit with their positionality in their work to enhance transparency and allow readers to better understand potential biases or influences on the research.
Reflexivity is the process of reflecting and acknowledging one's own positions, beliefs, judgements, and practices throughout the research process and how these will ultimately affect the results and design of their study.​
The practice of reflexivity requires that the researcher is sensitive to their own cultural, political, and social context. This is often the toughest part as most early researchers are not exposed to this type of requirement. True objectivity cannot be achieved as there will always be a degree of subjectivity person to person, however, by practicing positionality statements and reflexivity researchers will become better at recognizing their biases and take it into account. It does not guarantee higher quality research, but is meant to help them become a better researcher.
So, how exactly can you find your positionality if you choose to conduct a research study?
-
First, you will need to acknowledge your personal positions that may influence the study. In other words, take note of how your worldview can impact different aspects of your study.
-
Then, try to locate yourself in relation to the participants. Consider how you would view yourself and how the participants view you.
-
Finally, acknowledge that the research will ultimately be influenced by these factors even if utmost care is taken, however, the process of identifying positionality can help reduce bias in your study.
This is an ongoing process and takes time! Researchers are urged to reflect deeply and not rush this process. Positionality will also change and evolve as researchers move through their career.
As a final take-home message, researchers are encouraged to clearly state the limitations of their study, while readers need to cautiously evaluate the conclusions a study proposes to avoid overinterpretation of results.
About Us
Science for Everyone is a Canadian Nonprofit Organization that provides educational resources to help raise the level of scientific literacy in the general population.