Data Collection: Introduction and Sample Generation Methods

Imagine you wanted to survey the happiness level of every single person in the country you live in. Collecting data from the entire population would be unreasonable and take too much time and too many resources. This is why in science, it is very common that sampling is used. Sampling is identifying a subgroup of the population to have data collected from based on the strategy of sampling selected.

Before even starting the data collection process, you must consider the sample size you want to survey, i.e., how many participants you need to recruit for your sample in order to be able to make accurate conclusions about the population. It is important to perform sample size calculations to get an estimate of how many individuals are needed in your study.

The sample size must not be too big or too small. It needs to be large enough to capture all characteristics of interest. However, an overly large sample is inconvenient to collect, while too small of a sample would not have enough statistical power to answer the research question [8].

The sample size will dictate the generalizability of your results and affect the interpretation of statistical analyses [9]. More on that later…

E.g., If one of your friends told you that the restaurant across from where they live is good, do you think all of your friends will like it? If you thought that was not enough to tell, then your sample size is not large enough to make your results generalizable.

Some researchers account for expected dropout, incomplete records, or other study related problems in their sample size calculation [9]. The more considerations you make to find a suitable sample size, the more likely it is to produce meaningful data.

Let’s go through various sample generation methods that you can apply (as you find suitable) to a wide range of studies. The methods are divided into two subgroups: probability sampling and non-probability sampling. Probability sampling is when every individual in a population has an equal chance of being selected for the sample (random selection). Non-probability sampling involves non-random selection. The sample is recruited in a non-systematic process and not every individual in the population has an equal chance of being selected [10,11,12].

As this will be a dense section of the learning module, feel free to take breaks!

Probability Sampling

Non-Probability Sampling

Where to apply this method?

Usually used in quantitative research.

Usually used for exploratory or qualitative research or in underrepresented or difficult to recruit populations.

Can be used in quantitative research as well.

Advantages

You can make stronger statistical conclusions about the whole population when sampling this way and it is more representative of the target population you are hoping to research.

Allows for easier collecting of data and cheaper to conduct.

Disadvantages

May be more challenging to do.

Not as representative of the target population, has a higher risk of sampling bias.

Types

Simple Random Sampling
Systematic (Interval) Sampling
Stratified Sampling
Cluster (Multistage) Sampling

Convenience Sampling
Voluntary Response (Self-Selection) Sampling
Purposive/Judgemental Sampling
Snowball Sampling
Quota Sampling

Simple Random Sampling

Used when the whole population is accessible (e.g., members of a specific company, or school).
- Researcher should have a list of all subjects in the population (called the sampling frame).
Every member of the population has an equal chance of being selected.
Draw random samples through tools like a random number generator, lottery list, computer generated random list.
Example: You want to conduct a study on all the undergraduate students at a university. Using a list of all enrolled students at that school in all undergraduate programs, you can assign each student a number and then select a sample using a random number generator.

Systematic (Interval) Sampling

Similar to simple random sampling, but easier to conduct.
Used when the whole population is accessible, and researcher should have a list of all subjects in the population (not always needed, see below).
Instead of randomly generating numbers, individuals are chosen at fixed intervals (e.g., every 5th, 10th, 20th, etc., person).
- When using this method, ensure that there is no bias in the way names are listed (e.g., If there is an order to the list other than just alphabetical, like by program or year, then certain groups may be skipped over with this method).
- Sometimes a sampling frame is not needed (e.g., in the case of a hospital, trying to sample all patients visiting the ER on that day, can just include every 5th person that arrives).

Stratified Sampling

Modified version of simple random sampling. Involves dividing the population into subpopulations that may differ in important ways (e.g., gender identity, age, socioeconomic status, education, etc.).
Can make more precise conclusions by making sure that every subgroup is represented in the sample.
Also requires a sampling frame (list of the total population being used).
Divide the population into subgroups (strata) based on the relevant trait.
Based on the proportions of these characteristics in the population, you can calculate how many people should be included using random or systematic sampling to select from each group.
Advantages: can obtain enough people from each strata separately, and better represents minority or under-represented populations in the research.
Disadvantages: may not represent all key subgroups and can be based on the perspective of the researcher.
Example: The population you are looking to research has 70 women and 30 men, you can get this represented in your sample by recruiting 7 women and 3 men.

Cluster (Multistage) Sampling

Used when you cannot obtain a sampling frame due to the large size of a population.
Population is divided by geographical location into clusters. A list of all clusters is made and the researcher selects a random number of clusters to be included.
List of individuals in these clusters is obtained and then use another method of random sampling to get the final sample.
Advantage: can sample large and dispersed populations.
Disadvantage: more chance for error if there is a chance that the clusters differ in an important way.
Example: you want to sample all elementary school students in Ontario, but there are many schools. It would be extremely difficult to obtain a list of students from all schools, so you select a few schools (clusters) across the province, and conduct simple random sampling at a few schools that were also randomly selected.

Convenience Sampling

Most applicable and widely used method in clinical research.
Researchers include individuals based on their availability and accessibility (e.g., proximity to research).
Advantages: quick, inexpensive, and convenient.
Disadvantages: may not be representative of the whole population, so not generalizable. At risk for sampling and selection bias.

Voluntary Response (Self-Selection) Sampling

Similar to a convenience sample, as you obtain your sample from ease of access. People volunteer themselves for the research by responding to posters or surveys being sent out.
Advantage: time saving and convenient. Participants are more likely to be committed to the research and provide more truthful responses.
Disadvantage: samples may be biased as some people will be more likely to volunteer than others (self-selection bias). Can lead to exaggerated or misleading findings.
Example: you want to research students at a university. You put up a poster in one of the science buildings with a link to the survey. With this method, you are only sampling individuals who may have a class in this building and who want to complete the survey. This may be geared towards students in science programs, and may bias those who have strong opinions on the topic and is not representative of all students at this university.

Purposive/Judgemental Sampling

Researchers use their expertise on the research question of interest and select a sample based on the characteristics they think will be most useful.
Used when a researcher wants to get detailed knowledge on a specific phenomenon or when the population is very small and specific.
Important to have specific and clear criteria and rationale for why you include certain people.
Advantages: not many advantages to this method, but may be necessary to use with certain populations.
Disadvantages: prone to researcher bias.
Example: you want to conduct a study on the leadership styles used by principals in schools with high levels of student achievement. You would want to select a sample of only principals who work at schools that indeed have high levels of student achievement.

Snowball Sampling

Used when the population may be difficult to access
Sample gets recruited through other participants (the sample “snowballs” by adding more people as time passes with more participants)
Advantages: can conduct research on difficult to reach populations
Disadvantages: cannot be certain how representative your sample is of the general population, leading to sampling bias
Example: you want to conduct a study on individuals who are homeless. It would be difficult to obtain this sample, and thus by asking one individual in this population to share the survey, you may be able to gain access to others in the population.

Quota Sampling

Similar to stratified sampling, except that random sampling is not used
Used in order to try and control who makes up your sample
First you identify strata with characteristics of the population and then select the sample with these characteristics in mind until you reach your quota
Advantages: quicker and easier than stratified sampling. Can look at differences in subgroups
Disadvantages: sampling error can occur because random sampling is not used

Pause and Check

1. Which type of sampling method provides you with most control over the composition or diversity of your sample population? Stratified sampling. Recall that this sampling method allows you to recruit participants from different subpopulations (e.g., age, height, and gender identity). It is particularly useful when you need more representation from under-represented populations in your study, or when you require more input from certain populations.

2. You work at a local café where they just released a new type of drink. You hope to get any passerby to try out a free sample and give you some quick feedback. Which type of sampling method best describes this? Convenience sampling. Since you are trying to get whoever walks past the café to try your new drink, your sample population will consist of people who are most available to you at the time.

Previous Lesson

Differences

Next Lesson

Data Collection Bias

About

About Us

Science for Everyone is a Canadian Nonprofit Organization that provides educational resources to help raise the level of scientific literacy in the general population.