Selective Observation
Let's begin this lesson with an example. Suppose you want to conduct a survey on the general public's music preference and then write a summary report. The survey simply asks: list your top 5 music genres. You walked around your high school for a few hours and in the end collect 100 responses from only students.
You will use your survey results to conclude a trend for the general population. Do you think you obtained a good representation of the population group?
​
Hover over to reveal the answer.
No - you only ended up asking 100 high school students. This introduced sampling bias to your study - where you are selectively choosing the things you want to measure or observe, in this case, participants (1).
You went back home and analyzed the responses. You are an avid enjoyer of jazz music, so you expected many people to have put down “jazz.” But it turned out that only 15 people out of 100 listed jazz in one of their top 5 music genres. In your report, you then claim that “about a quarter of people who responded believe that jazz is among their top 5 favourite music genres.”
Now this sounds like you may have been influenced by a trend that you expected to see. This is called observer bias, because the conclusions you made here significantly rounded up the number of people who thought jazz music was among their top 5 genres. Being able to objectively state what you observed will prevent your survey report from including bias (2). In this case, the correct claim in your statement would be “15% of the people who responded believe that jazz is among their top 5 favourite music genres.”
At the end of your report, you wanted to briefly highlight other less frequently mentioned genres recorded in the responses. It turns out that you also dislike rap music. Despite 7 people answering “rap,” and only 3 people answering “lofi study music,” you decide to only write about lofi study music in your report.
Wait... I think you are introducing confirmation bias into your report!
Confirmation bias occurs when the observations which agree with preexisting beliefs are chosen, and ones that disagree are ignored. Just because you do not like rap music, and only a small minority of people made that response, you introduced confirmation bias by not including it, even though you decided to list out all the other less frequent responses (3).
Other types of bias exist because there are many ways a person can deviate, distort, or incorrectly process information they are trying to understand. Towards the end of this learning module, you will be introduced to various ways that can help alleviate bias.
About Us
Science for Everyone is a Canadian Nonprofit Organization that provides educational resources to help raise the level of scientific literacy in the general population.