A friend told me he wanted to find out if people liked the food at KFC. He created a survey, went to KFC and distributed it to the people who had gone there to buy food. After a few days, he came to me and told me he found that majority of people liked the food there. I wasn't surprised at his conclusion because I knew it was going to be so. He distributed his survey only in KFC. I mean, why will people be going to KFC if they don't like the food? To get a more accurate result, he would have given his survey to people in different locations.
What has happened here is called survey bias because the survey was only distributed to a particular set of people. In this article, you will learn what survey bias is, the types of survey bias and how it can be avoided.
Definition of a Biased Survey
Let's start by remembering the definition of a survey.
A survey is a method of data collection where relevant questions are used to gain knowledge about a specific topic or subject.
A survey is one of the data collection methods. To find out more about surveys and how to collect data using them, see Conducting a Study and Survey Sampling Methods.
During surveys, questions are asked to groups of people to gain an understanding of the research subject. The questions may be asked face to face, a form containing the questions can be distributed to the people to fill or it can be done online. Surveys give information about behavior and preferences and this can be very essential in product design. Even though surveys are useful and easy to analyze, there is still room for bias. As you read on, you will see that sometimes, survey bias is a result of the way sampling is being done.
So what is survey bias?
Survey bias means that the results of a survey are inaccurate because of things done to consciously or unconsciously influence the answers given.
Some of the things that can affect the result of the survey are the structure of the questions, the method of sampling and the way the data is being analyzed.
So then a biased survey is one in which the questions or results of a survey are biased in some way.
Let's learn about the types of survey bias.
Types of Survey Bias
The following are some of the types of survey bias:
Voluntary Response Bias.
Non-response Bias.
Response Bias.
Under-coverage Bias.
Question Wording Bias.
Non-random Bias.
Voluntary Response Bias
Let's start with the meaning of voluntary bias.
Voluntary response bias occurs when the participants of a survey consist only of those who are willing to or choose to participate in the survey.
Below is an example of voluntary response bias.
If you put up a poll on your Instagram story for people to answer, the only people that are going to respond are people who want to. You are most likely to get answers from people close to you, some of your loyal followers or people who have strong opinions about the topic.
The bias here is that only a particular group of people will respond just because they want to or they have an opinion. The result from this poll will be inaccurate because it doesn't represent the whole population. This is voluntary response bias.
Voluntary response bias is a result of voluntary sampling. Voluntary sampling is a form of convenience sampling. the samples are not random and you do not have a good representation of the entire population or target population. You only have volunteers and you are making do with what is available. So, there will be a lot of bias.
Convenience sampling is a sampling method where the samples collected are based on their availability and ease to get. Avoid convenience sampling whenever possible since it almost always introduces bias!
Non-response Bias
Bias can occur when there is not enough response to a survey.
Non-response bias is a result of a major difference between those who respond to a survey and those who do not.
There are various reasons why people will not respond to a survey. The survey may be poorly designed, the survey may be about or contain sensitive questions that people may not want to answer, the survey may be inaccessible to the target population, or people cannot relate to the subject or the survey is sent by mail and ends up thrown away.
An example of non-response bias is a survey carried out to find out about the popular nightclubs in a city. If this survey is distributed to people under 10, you may not get any responses because they cannot relate to the subject.
Response Bias
Having lots of people respond to your survey can be good and encouraging but you have to watch out for response bias. This bias occurs when the answers given in your survey are not truthful. Participants may give wrong or untruthful answers consciously or unconsciously and this will affect the accuracy of your result.
For example if you distributed the survey about nightclubs to 10 year old people, they may find it funny to fill in random responses and send it back rather than just throwing it away.
Under-coverage bias
Under-coverage bias is another type of survey bias.
Under-coverage bias is a type of sampling bias that happens when important information and sampling population are poorly represented or not represented at all.
Let's look at an example.
If you are conducting a survey for a product evaluation, the sample population should include past and current users of the product. If either of these samples is poorly represented or not represented at all, then under-coverage bias will be present.
The following are ways in which under-coverage bias occurs.
When the researcher only gathers samples that are easily accessible, then bias occurs. This is called convenience sampling. Avoid convenience sampling whenever possible!
The bias can occur when there is little or no knowledge about what is to be sampled and how it should be sampled. The persons involved in gathering the data may not even understand the topic enough to gather adequate samples.
Poor survey design can contribute to under-coverage bias. The way the survey is structured and distributed can affect the response and stop people from participating.
Time can also be a factor. The time available may not be enough for the researcher to reach as many people as desired to take the survey.
Lack of resources can cause under-coverage bias. The way the sample data is to be collected may be too expensive and the researcher may not be able to afford it or get funding. The samples that will be collected here, will be inadequate.
Question Wording Bias
As the name applies, the question wording bias has to do with the questions that are in a survey.
Question wording bias is the bias that occurs when the wordings of the questions in a survey is structured in a way that systematically influences the answer of the participants.
If the answers to a survey are influenced, then the results obtained will be inaccurate. This survey bias can be introduced intentionally or unintentionally.
The wording of the questions can be formulated in a way that leads the participant to a particular answer. The questions may contain too much information that is likely to make the participant choose a particular answer and in other cases, the words are structured confusingly and cannot be understood.
Let's use one question to show question-wording bias.
QUESTION - Should a shopping center that will cause traffic in your area be built? YES/NO
For this question, the participant is likely to choose NO because they wouldn't want traffic in their area. The question contains too much information and it shows a negative effect of what you are proposing which will influence the answer given.
QUESTION - Should a shopping center that will create job opportunities be built in your area? YES/NO
For this question, you are most likely to get YES as the answer because the question contains information on the positive outcome of the shopping center. This should not be so because you are influencing the answer of the participant.
A better way to frame the question is below.
QUESTION - Should a shopping center be built in your area? YES/NO
Any participant answering this question may choose YES or NO for reasons best known to them. You can only know this reason if there is a follow-up question asking for the reason for their answer. The reason why this question structure is better than the two above is that there is no excess information and the wording is not in any way influencing the answer.
Non-random Bias
Non-random bias is as a result of non-random sampling.
Non-random sampling is a sampling method where samples are taken based on availability, convenience, time or resources.
The samples here are not taken at random and when samples are taken this way, there's a lot of room for bias. Significant categories of people will be excluded because factors are limiting how far and how well the samples are taken. When the samples are not taken at random, there can be misrepresentation, over-representation or under-representation which is a bias and will negatively affect the results of the survey.
Non-random bias can also be a result of convenience sampling. This is yet another reason to avoid convenience sampling.
These various types of bias can affect the accuracy and validity of your research.
Survey Bias Examples
Let's take a look at some examples.
A popular company whose product is widely used wanted to know how well people are interacting with their product. So, they sent out a survey to people on their mailing list.
Which of the following is the most concerning type of bias here?
- Non-response bias.
- Non-random sampling.
- Question-wording bias.
Solution:
The correct answer is option B.
Option B is non-random sampling, One of the reasons why non-random sampling occurs is because the samples are taken based on convenience. The company is a popular one and its products are widely used. Some people who are not on the mailing list use the product. If the surveys are sent to only people on their mailing list, it means the samples are taken out of convenience and the conclusion from the survey will be inaccurate. There are so many people who are not on that list with helpful opinions and those people have been excluded from the survey because they were not selected among the samples.
Non-random sampling can be limited by making sure your samples are well represented and all significant categories are included and by finding out the best way to reach out to your target population to avoid getting the wrong response or no response at all.
Option A is non-response bias. It is possible for some people on the mailing list to not respond and it is likely that the number of non-response will not be too concerning because everyone that has subscribed to that list probably loves the product. The non-response here can be a little source of bias but it is not the most concerning source of bias.
Option C is question-wording bias. We can't tell if this bias is present because we have no information on the question given in the survey.
Let's take a look at another example.
During a research, a survey was distributed to the target population through various means. The amount of non-response they got was insignificant.
What kind of bias is possible here?
Answer:
The question says that the amount of non-response was insignificant which means that almost everyone responded to the survey. Response bias can occur here. While it can be encouraging to get a lot of responses, it is also possible for some of the responses to be false. Participants may give false answers to the questions for various reasons and this will affect the outcome of your research.
Be sure to watch out for this type of bias when preparing, distributing and analyzing a survey.
Let's see a look at another example.
The authorities in an area decide to call its citizens over the phone to gather information on an issue.
Which of the following best describes an under-coverage bias?
- People may not pick up the phone
- They may only contact people whose phone numbers they already have.
- People may not give accurate answers to the questions asked.
Solution:
The correct option is B. They may only contact people whose phone numbers they already have. If this happens, it means a lot of people will be left out. The authorities may not get some important information and a significant part of the population will be excluded from the survey. This is under-coverage bias.
To limit or avoid this type of bias, make sure your sample is well represented and all significant categories are included and find out the best way to reach out to your target population to avoid getting no response.
For option A, people not picking up the phone is not an under-coverage bias but a non-response bias.
For option C, people not giving accurate answers will affect the accuracy of the survey results but this is not under-coverage bias.
Let's see one more example.
Why is the question below an example of question wording bias?
QUESTION - What problems did you have with the service you received?
Solution:
The question is an example of question wording bias because it suggests that there was a problem in the first place. So, the participant is made to look for problems even when they don't feel like there are any. This will influence the answer given and lead to inaccurate results.
To limit or avoid this type of bias, make sure that the questions are properly structured and neutral. The questions should not contain words that may influence the answers of the participant.
What about bias in polling?
The government announced that it is no longer mandatory to wear face masks in public because of the pandemic. An online poll was created and people were asked if they agree with the government or not. The poll showed that 80% disagreed with the government.
Which of the following best describes the bias in this poll?
- People who disagree are more likely to respond than people who agree. So, 80% is an underestimation of what it is.
- People who disagree are more likely to respond than people who agree. So, 80% is an overestimation.
Solution:
The correct option is B. People who disagree are more likely to respond than people who agree because they are angry. The people who agree may not respond because the government has done what they like. So, 80% will be an overestimation of the people who disagree.
There is voluntary sampling bias and non-response bias here. To avoid this, an appropriate sampling method should be used to ensure that every category of people required is represented.
Avoiding Bias in Survey Questions
Let's take a look at some ways we can avoid bias in surveys.
Survey questions should be kept short and simple.
The wording of survey questions should be structured properly.
Survey questions should not contain too much information that may influence the answers given.
The wording of the survey questions should be kept neutral.
Keeping the survey anonymous may encourage the participant to answer truthfully.
Avoid asking leading questions. For example, in a survey for a product evaluation, you should not ask if they are satisfied with the product instead ask about the quality of the product.
The question and the answer options should not be confusing. Simple and easy to understand language should be used.
The survey should not contain assumptive or loaded questions. These are questions that will force the participant to choose an answer, especially a defensive one so they don't look bad. It is like tricking or forcing the participants to give an answer that you desire.
Limiting Survey Bias
Survey bias cannot be eliminated but there are things you can do to limit it. The following are ways in which survey bias can be limited:
Make sure your sample is well represented and all significant categories are included.
Find out the best way to reach out to your target population to avoid getting the wrong response or no response at all.
Avoid including personal opinions when structuring a survey.
Try not to frame survey questions in a way that will influence a response.
Be vigilant! Watch out for any type of bias when preparing, distributing and analyzing a survey.
Allow your survey to be reviewed by someone else. This will help spot any bias you may have missed.
Ensure that sample data is not misinterpreted to avoid wrong conclusions.
Sources of Bias in Surveys - Key takeaways
- Survey bias means that the results of a survey are inaccurate because of things done to consciously or unconsciously influence the answers given.
- The following are some of the types of survey bias we will be looking at.
- Voluntary Bias.
- Non-response Bias.
- Response Bias.
- Under-coverage Bias.
- Question-wording Bias.
- Non-random Bias.
- The following are some ways we can avoid bias in surveys.
- Survey questions should be kept short and simple.
- The wording of survey questions should be structured properly.
- Survey questions should not contain too much information that may influence the answers given.
- The wording of the survey questions should be kept neutral.
- Keeping the survey anonymous may encourage the participant to answer truthfully.
Explanations 
Exams 
Magazine 
