Questionnaire Construction

Questionnaire Construction: Psychology

Researchers often use questionnaires because they are relatively inexpensive, and it is easy to recruit and analyse many responses and their data.

Most often, researchers use established questionnaires in research. But what if no questionnaire effectively measures what the researcher intends to measure? Well, this is where questionnaire construction comes in.

Psychology research aims to be scientific, so researchers can't randomly select questions and form questionnaires. Instead, researchers have to follow a standardised process, which we'll get into later.

Construction of Questionnaire in Research Methodology

Researchers use questionnaires when there are no established questionnaires that can be used to measure the variables of interest. Thus, the researchers must develop their own reliable and valid questionnaire.

Questionnaire construction is likely needed when studying novel ideas or specific or investigating unique cases.

Questionnaire Construction: Process

When it comes to the questionnaire construction process, it's not as simple as just putting a bunch of questions together. There are more sophisticated and standardised procedures involved in the questionnaire construction process.

Let's explore some things researchers should remember during the questionnaire construction process.

The first thing a researcher needs to determine is the variable(s) that will be measured using the questionnaire.

Next, the researcher should determine whether they want to collect qualitative (non-numerical) or quantitative (numerical) data.

Open-ended questions in questionnaires collect qualitative data, and close-ended questions collect quantitative data.

When the researcher wants to understand the respondents' thoughts or behaviours, they use open-ended questions. In contrast, a close-ended question is used when a simple, fixed response is sufficient, e.g. yes or no.

Fixed-response questions should have no overlaps.

Figure 1: Some questionnaires contain a mixture of both open- and close-ended types of questions.

Questionnaire Construction: Comparison of Open/Close-Ended Questions

Let's compare the strengths and weaknesses of open- and close-ended questions in questionnaires.

Although the table suggests that close-ended questions have the most advantages, that is not necessarily true. If researchers are trying to get in detail knowledge of participants' thoughts, feelings, behaviours, and motives, wouldn't it be more appropriate to use open-ended questions?

Principles of Questionnaire Construction

As we've learned earlier, there are standardised procedures to ensure that questionnaires are constructed to a high scientific standard.

Before a questionnaire is used in an investigation, a pilot study must be conducted.

In terms of reliability, researchers want to ensure that questionnaires have high internal reliability.

There are several types of validity, one of which is constructed validity. The questionnaire measures the variables stated in the hypothesis and if the scores measured reflect the hypothesis. Researchers must also ensure that the questions cover all the variables under study. If this is the case, the experiment has high validity.

Principles of Questionnaire Construction: Testing Reliability and Validity

One way to measure a questionnaire's reliability is the test-retest method. The process involves the same participants completing the same questionnaire but on different days. If there is a high correlation between test scores on each data, then it has high internal reliability.

A common way to measure the validity of a questionnaire is to distribute a questionnaire to two groups:

1. One group consists of experts.
2. The other has limited knowledge about the phenomenon under investigation.

If the two groups have similar results, the questionnaire is easy to understand, and the questions are not open to interpretation. Therefore, the questionnaire has high validity.

Another way to measure the validity of a questionnaire is to check whether it has concurrent validity. It is verified by inspecting whether the new questionnaire's results match those of a previously created questionnaire that measures a similar concept.

Questionnaire Construction: Strengths and Issues

Overall, questionnaires are an inexpensive and quick way to collect data for analysis, and as they are usually, anonymous respondents are more likely to be truthful. Additionally, a higher response rate is likely in comparison to interviews.

It becomes problematic when questions are not carefully constructed, as it can lead to biased, invalid and unreliable results.

Ultimately, questionnaires should avoid influencing or suggesting the correct answers (to avoid participants making the socially desirable choice).

Questionnaire Construction: Example

Let's look at an example of what type of questions may be asked when researchers are trying to measure how depressed respondents' are.

In the example, only close-ended questions are used, i.e. they only rely on fixed responses. Thus, the question collects quantitative data.

So what's good and bad about this questionnaire construction example?

The good things are that:

• It quantifies how far back their responses should think. Imagine if you don't ask this, someone may reflect on how they've been feeling in the past two weeks, and someone else may think back to the past two years; this isn't consistent. Thus, it causes reliability issues.
• The questions are not leading and not open to interpretation.
• The questions are written coherently, logically, and easy to follow.

However, the issue is that the questionnaire does not measure each symptom of depression; therefore, it lacks validity.