Probability and Significance

Imagine a dice on five sides has the number five and one side has a six. Can we say with 100% confidence that it will land on the number five when the dice is rolled? No, right, because there is still a one in six chance, it will land on the number six. Researchers have the same restrictions; they can never say with 100% confidence that their results are due to manipulation of the independent variables. To understand this further, we need to understand the concepts of probability and significance in psychology.

• We will start by exploring probability and significance: psychology and how probability and significance can be explained in terms of psychological research.
• Then we will take a look at the types of significance tests.
• Finally, we will take a look at some levels of significance examples.

Probability and Significance Psychology

Psychologists aim to establish if their findings support or negate their hypothesis. Assuming that the hypothesis should be accepted if a pattern/ trend is found would be too simple.

Significance tests give researchers a statistical value to measure how likely/ the probability the research results are due to chance. Another word for this is the alpha level. In psychology, this is written as$\alpha$.

Significance and Probability Explained

The main research method used in psychology involves manipulating the independent variable to observe its effect on the dependent variable. However, researchers can't control everything, even when the experiment is conducted in a lab setting. Researchers cannot determine 100% that their results are due to the variables they are studying because extraneous and confounding variables are likely to influence observed effects.

Researchers have a consensus about what alpha level is acceptable, which is 0.05. If a significance level above this is found, the hypothesis that states an observed difference (alternative hypothesis) should be rejected.

But what does a significant level of 0.05 exactly mean, and how can it be interpreted? A significance value of 0.05 indicates a 95% probability that the results are not due to chance.

Level of Significance Psychology

The significance level in psychology ranges from 0 to 1 and is expressed as a p-value. The closer the value is to 0, the more likely the results are not due to chance. The smaller the number, the less likely the researcher will reject the alternative hypothesis and be more likely to reject the null hypothesis.

In psychology, the accepted level of significance is .05.

Types of Significance Tests

The significance value is a quantitative value that tells the researcher and readers how likely the results are due to chance.

The significance values commonly used in psychological research are:

• 0.05 - there is a 5% chance results are due to chance.
• 0.01 - there is a 1% chance: results are due to chance.
• 0.001 - there is a 0.01% chance results are due to chance.

Anything above the 5% threshold means that the hypothesis should be rejected. It is because there is a too high probability that the results are due to external factors rather than the independent variable.

Two-Tailed Probability and Significance

The hypothesis proposed at the start of the study also influences the probability and significance level.

There are many types of hypotheses in psychology research:

• The null hypothesis proposes the IV (independent variable) will not affect the DV (dependent variable).
• A non-directional - the researcher does not state how the IV will change.
• A directional - the researcher proposes the IV will affect the DV; this can be further subdivided into one-tailed to two-tailed.

As we learned earlier, the type of hypothesis accepted depends on the level of significance found.

Acceptance of:

• The alternative hypothesis is accepted if a value is found at or below the significance level.
• The null hypothesis is accepted if a value above the significance level is found.

Rejection of:

• The alternative hypothesis is rejected if a value above the significance level is found.
• The null hypothesis is rejected if a value at or below the significance level is found.

Significance Tests and Methodology

Occasionally, non-significant results are due to problems with the methodology used. Once these have been corrected and the experiment retested, significant results can be found.

These errors can lead to type 1 and type 2 errors.

A Type 1 error is when we incorrectly reject the null hypothesis when it is, in fact, true. If the significance level is at 0.05, there is a 5% chance of a Type 1 error.

A Type 2 error is when we incorrectly reject the alternative hypothesis when it is true.

Level of Significance Example

Let's quickly take a look at the symbols you may come across during your studies:

• p = significance value.

• < = the significance value is less than the number reported (e.g. < 0.05 = significance is less than 0.05).

• > = the significance value is greater than the number reported (e.g. > 0.05 = significance is greater than 0.05).

Now let's apply significance to psychology and some examples.

If a significant value is found, the inferential statistic can be used to make inferences about the target population supported by evidence. However, if this is not the case, the study should not make inferences about the target population.

If a non-significant value is not found, it does not mean there is no relationship/interaction between the independent and dependent variables.

Instead, it means that other variables are also influencing the dependent variable. Therefore the study cannot provide an accurate (reliable or valid) measure of the interaction between the independent and dependent variables.

An example of a significance level for a correlation analysis that would be accepted is:

This example shows:

• A correlational analysis was performed (r).

• A positive correlation was found (.63).

• There were 56 participants (56).

• The significance value was accepted because the significance value is below the accepted alpha value of 0.05.

• The alternative hypothesis should be accepted.

On the other hand, a correlational analysis that suggests the researcher should reject the alternative hypothesis and accept the null hypothesis is...

This example shows:

• The significance value is non-significant as it is above .05.

• The null hypothesis should be accepted.