Overcoming Errors in Reasoning
In this final lesson, we will explore four main ways that are implemented in the modern day scientific method which help overcome the errors in reasoning introduced in the previous lessons.
1. Randomization
First, let’s discuss how randomization in the scientific method can reduce bias in science, which involves the concept of blinding (16).
Imagine a group of patients being given omega-3 supplements to treat bipolar disorder. What the researchers want to investigate is whether there is a correlation between receiving the supplement and an improvement in bipolar disorder symptoms. The patients will be given either supplement pills or placebo pills (which have no effect), and their symptoms will be re-evaluated in 30 days. Blinding, or minimizing the observer and/or the participant's pre-existing knowledge about what is being observed, is a useful tool to help eliminate bias in the study (17). Click on the buttons to the right to examine this study and see the differences between the absence and presence of blinding.
Supplement
Placebo
Blinding is one common way of introducing randomization and other methods do exist to also help alleviate bias.
2. Deductive reasoning
Have you ever played the board game CLUE or watched Scooby-Doo? Imagine you are a detective trying to solve a mystery. Detectives must start with having a large amount of clues. In order to solve the mystery, a detective must narrow down the possible evidence through the given clues until a confident conclusion is reached and the mystery can be solved! This is deductive reasoning. Deductive reasoning is a logical approach where general ideas progress to specific conclusions (top-down approach) (18). It begins with a premise, and adds other premises to form a conclusion (18). The inferences and conclusions are reliant on the truth of the premise (10).
​
​
​
​
​
​
​
​
Deductive reasoning is commonly used in scientific research through the testing of hypotheses to see if predictions are supported by experimental data (20). Deductive research starts with a premise or research problem (20). A hypothesis is created based on this premise or theories of the research problem (20). As data is collected and analyzed, more premises are added to the research problem (20). Eventually, a conclusion is made or a hypothesis is accepted or rejected (20).
For example, all dogs are mammals, all mammals have hearts, therefore all dogs have hearts (19). This overcomes various illogical reasoning errors (18).
Can you use deductive reasoning to pick the right coin?
Guess the coin!
Four coins are laid out in front of you - a penny, a nickel, a dime and a quarter.
​
​
​
Clue #1: The correct coin is silver in colour. Which one can be eliminated?
3. Repetition
Understanding how scientists conduct research helps us understand how errors in scientific reasoning, like overgeneralization, can occur.
There are many avenues in which scientific research can be conducted. For example, in pre-clinical trials, research can be conducted in cells growing in a flask or in a dish or it can be conducted in animals like mice, rats, and fish. Once there is enough research done in this stage, clinical trials in humans can be conducted.
Once a scientific discovery is made in a certain animal model or cell-line, that phenomenon cannot be generalized to all cells or all animal models.
For example, if a drug worked at killing cancer cells in a breast cancer cell line, the statement that “this drug kills all cancer cells” cannot be made. This would be an overgeneralized conclusion. Instead, we can say that this drug kills breast cancer cells. The same experiment must be repeated in other cancer cell types like brain cancer cells. Once this is done, then we can make the statement that “this drug kills breast and brain cancer cells.”
By repeating the same observation in multiple cell lines or animal models, we can ensure that the conclusions made are not overgeneralized beyond the parameters of the study.
For example, if we look back to the example in Lesson 2, the same experiment must be repeated in multiple different cancers before making the statement that this study provides a universal gene marker for all cancers.
In a study in 2005, scientists discovered a set of genes that links to the breast cancer’s spread to the lungs. This finding, according to the researchers, is still in its early stages (Brechman, 2009). The media, however, displayed the same study to the public in a completely different light. The press declared the finding to be a landmark because it proved the existence of genetic signature for “each type of cancer and the organ it spreads to.” This single sentence represents media overgeneralization on multiple levels.
Hover over the box to be reminded of the example
Similarly, when looking at research in human participants, it is important to ensure that the study sample (i.e., the group of individuals enrolled in a study is representative of the population being studied) (5).
Before moving on, hover over the box for important definitions: population versus sample
Population → The population of a study is the whole group that the study is investigating and trying to make an observation or conclusion about. The population is usually too big to include every single member (21).
Sample → The sample of a study is a small subset of the population that is enrolled in a study to make these conclusions. The sample is always smaller than the population but should always be representative of it (21).
It is also important for researchers to disclose the characteristics of the sample that is enrolled in the study.
Details of the study sample must be disclosed when stating what discovery was made. For example, the statement that all adults have higher risk of developing X-disease cannot be made if the study enrolled only male participants. This would be considered an overgeneralization as the study results are being generalized on female adults when they are not represented in the sample. The same study should be repeated in female participants.
For example, between 1996-2005, 80% of research articles published in the scientific journal Pain were experiments done on males ONLY. A study also showed that 80% of pain drugs recalled by the FDA in 2005 were due to adverse effects in females (22).
So this shows the danger of overgeneralizing results done in a sample that is not representative of the population e.g., a study in only males (sample) for drugs that are intended to be used by both sexes (population).
Therefore...
Repetition of a study across different contexts and different methods to get a better idea of the generalizability of a finding helps to prevent overgeneralization (5).
4. Communication
Lastly, communication plays a huge role in mitigating errors in reasoning, especially with resistance to change. The communication of findings is important to both the scientific community and the general public as it allows for opportunities for feedback and repetitions of a study (23). Lacking communication can affect the implementation and quality of findings and create an atmosphere of uncertainty leading to negative feelings around change and even leading to resistance. Examples of good communication include a healthcare professional explaining the importance of vaccination to the public using minimal jargon, a science teacher teaching photosynthesis using lots of analogies, and a nonprofit organization creating science literacy learning modules by citing many sources.
​
Here are some tips to keep in mind for good communication (24):
1. Avoiding jargon, euphemisms, clichés, wordplays, and puns;
2. Using analogies and examples;
3. Only including important details;
4. Spending a lot of time revising and rewriting by seeking feedback;
5. And citing your sources.
These crucial components of the scientific method, randomization, deductive reasoning, repetition, and communication, are excellent ways to overcome errors in reasoning.
About Us
Science for Everyone is a Canadian Nonprofit Organization that provides educational resources to help raise the level of scientific literacy in the general population.