Chi Square Test for Independence

Say your city is trying to encourage its residents to recycle their household trash, so they come up with two methods for asking them to do so:

1. mailing an educational pamphlet; and

2. calling each resident.

Then, the city randomly selects \(200\) households and randomly assigns them to one of three categories:

1. receiving the pamphlet;

2. receiving a phone call;

3. the control group (no form of intervention).

Finally, the city will use the results of this test to decide what is the best way to ask their residents to recycle more.

Can you guess which hypothesis test they will use to make this decision? A Chi-square test for independence!

Chi-Square Test of Independence Definition

Occasionally, you want to know if there is a relationship between two categorical variables.

Think of it this way:

If you know something about one variable, can you use that information to learn about the other variable?

You can use a Chi-square test of independence to do just that.

If there is a relationship between the two categorical variables, then knowing the value of one variable tells you something about the value of the other variable.

If there is no relationship between the two categorical variables, then they are independent.

Assumptions for a Chi-Square Test of Independence

All the Pearson Chi-square tests, for independence, homogeneity, and goodness of fit, share the same basic assumptions. The main difference is how the assumptions apply in practice. To be able to use this test, the assumptions for a Chi-square test of independence are:

• The two variables must be categorical.

  • This Chi-square test uses cross-tabulation, counting observations that fall in each category.

• Groups must be mutually exclusive; i.e., the sample is randomly selected.

  • Continuing from the introductory example, three months after the city's intervention methods are tested, they look at the outcome and put the data into a contingency table. The groups that must be mutually exclusive are the subgroups: (Recycles-Pamphlet), (Does Not Recycle-Control), etc.

Table 1. Contingency table, Chi-square test for independence.

• Expected counts must be at least \(5\).

  • This means the sample size must be large enough, but how large is difficult to determine beforehand. In general, making sure there are more than \(5\) in each category should be fine.

• Observations must be independent.

  • This is about how the data is collected. In the city recycling example, the researcher should not sample houses that are near each other. That is, it is more likely that a street of household recycle than that households chosen from different neighborhoods recycle.

Null Hypothesis and Alternative Hypothesis for a Chi-Square Test of Independence

When it comes to independence of variables, you almost always assume that two variables are independent, then try to prove that they aren’t.

• The null hypothesis is that the two categorical variables are independent, i.e., there is no association between them, they are not related.\[ H_{0}: \text{“Variable A” and “Variable B” are not related.} \]

• The alternative hypothesis is that the two categorical variables are not independent, i.e., there is an association between them, they are related.\[ H_{a}: \text{“Variable A” and “Variable B” are related.} \]

Replacing “Variable A” and “Variable B” with the variables in the city recycling example, you get:

Expected Frequencies of a Chi-Square Test of Independence

As with other Chi-square tests, a Chi-square test of independence works by comparing your observed and expected frequencies. You calculate expected frequencies using the contingency table. So, the expected frequency for row \(r\) and column \(c\) is given by the formula:

\[ E_{r,c} = \frac{n_{r} \cdot n_{c}}{n} \]

where,

• \(E_{r,c}\) is the expected frequency for population (or, row) \(r\) at level (or, column) \(c\) of the categorical variable,

• \(r\) is the number of populations, which is also the number of rows in a contingency table,

• \(c\) is the number of levels of the categorical variable, which is also the number of columns in a contingency table,

• \(n_{r}\) is the number of observations from population (or, row) \(r\),

• \(n_{c}\) is the number of observations from level (or, column) \(c\) of the categorical variable, and

• \(n\) is the total sample size.

Continuing with the city recycling example:

Degrees of Freedom for a Chi-Square Test of Independence

Like in the Chi-square test for homogeneity, you are comparing two variables and need the contingency table to add up in both dimensions.

The formula for the degrees of freedom is the same in both the homogeneity and independence tests:

\[ k = (r - 1) (c - 1) \]

where,

• \(k\) is the degrees of freedom,

• \(r\) is the number of populations, which is also the number of rows in a contingency table, and

• \(c\) is the number of levels of the categorical variable, which is also the number of columns in a contingency table.

Formula for a Chi-Square Test of Independence

The formula (also called a test statistic) for a Chi-square test of independence is:

\[ \chi^{2} = \sum \frac{(O_{r,c} - E_{r,c})^{2}}{E_{r,c}} \]

where,

• \(O_{r,c}\) is the observed frequency for population \(r\) at level \(c\), and

• \(E_{r,c}\) is the expected frequency for population \(r\) at level \(c\).

The Chi-square test statistic measures how much your observed frequencies differ from your expected frequencies if the two variables are unrelated.

Steps to Calculate the Test Statistic for a Chi-Square Test of Independence

Step \(1\): Create a Table

Using your contingency table, create a table that separates your observed and expected values into two columns.

Table 3. Table of observed frequencies and expected frequencies, Chi-square test for independence.

Step \(2\): Subtract Expected Frequencies from Observed Frequencies

Add a new column to your table called “O – E”. In this column, put the result of subtracting the expected frequency from the observed frequency.

Table 4. Table of observed frequencies and expected frequencies, Chi-square test for independence.

Step \(3\): Square the Results from Step \(2\)

Add a new column to your table called “(O – E)²”. In this column, put the result of squaring the results from the previous column.

Table 5. Table of observed frequencies and expected frequencies, Chi-square test for independence.

Step \(4\): Divide the Results from Step \(3\) by the Expected Frequencies

Add a new column to your table called “(O – E)²”/E. In this column, put the result of dividing the results from the previous column by their expected frequencies.

Table 6. Table of observed frequencies and expected frequencies, Chi-square test for independence.

Step \(5\): Add the Results from Step \(4\) to get the Chi-Square Test Statistic

Finally, add up all the values in the last column of your table to calculate your Chi-square test statistic:

\[ \begin{align}\chi^{2} &= \sum \frac{(O_{r,c} - E_{r,c})^{2}}{E_{r,c}} \\&= 0.9793 + 0.1907 + 1.0761 + 0.4966 + 0.04299 + 0.1269 \\&= 2.91259\end{align} \]

The Chi-square test statistic for the Chi-square test of independence in the city recycling example is:

\[ \chi^{2} = 2.91259 \]

Steps to Perform a Chi-Square Test of Independence

If your calculated test statistic is large enough, then you can draw the conclusion that the observed frequencies are not what you would expect if the variables are indeed unrelated. But what is considered “large enough”?

To determine whether the test statistic is large enough to reject the null hypothesis, you compare the test statistic to a critical value from a Chi-square distribution table. This act of comparison is the heart of the Chi-square test of independence.

Follow the \(6\) steps below to perform a Chi-square test of independence.

Step \(1\): State the Hypotheses

• The null hypothesis is that the two categorical variables are independent, i.e., there is no association between them, they are not related.\[ H_{0}: \text{“Variable A” and “Variable B” are not related.} \]

• The alternative hypothesis is that the two categorical variables are not independent, i.e., there is an association between them, they are related.\[ H_{a}: \text{“Variable A” and “Variable B” are related.} \]

Step \(2\): Calculate the Expected Frequencies

Use your contingency table to calculate the expected frequencies using the formula:

\[ E_{r,c} = \frac{n_{r} \cdot n_{c}}{n} \]

Step \(3\): Calculate the Chi-Square Test Statistic

Use the formula for a Chi-square test of independence to calculate the Chi-square test statistic:

\[ \chi^{2} = \sum \frac{(O_{r,c} - E_{r,c})^{2}}{E_{r,c}} \]

You have two options for finding the critical value:

1. use a Chi-square distribution table, or

2. use a critical value calculator.

Either way, there are two pieces of information you need to know to find the critical value:

1. the degrees of freedom, \(k\), given by the formula:

   \[ k = (r - 1) (c - 1) \]

2. and the significance level, \( \alpha \), which is usually \( 0.05 \).

Referring back to the city recycling example, find the critical value.

Step \(5\): Compare the Chi-Square Test Statistic to the Critical Chi-Square Value

Now for the moment of truth! Is your test statistic large enough to reject the null hypothesis? Compare it to the critical value you just found to find out.

Again, continuing with the city recycling example, compare the test statistic to the critical value.

Step \(6\): Decide Whether to Reject the Null Hypothesis

Finally, decide whether to reject the null hypothesis.

• If the Chi-square value is greater than the critical value, then the difference between the observed and expected frequencies is significant; \( (p < \alpha) \)

  • This means you reject the null hypothesis that the variables are unrelated, and you have support that the alternative hypothesis is true.

• If the Chi-square value is less than the critical value, then the difference between the observed and expected frequencies is not significant; \( (p > \alpha) \)

  • This means you do not reject the null hypothesis, but you do not have support that the alternative hypothesis is true.

Decide whether to reject the null hypothesis for the city recycling example.

Using Critical Value VS Using P-Value

In the steps to perform a Chi-square test of independence, you calculated and used the critical value to decide whether to reject the null hypothesis.

It is important to know, however, that there is another option you can use: the \(p\)-value.

The image below sums up the critical value approach vs. the \(p\)-value approach.

Figure 1. A diagram showing how you can use either a \(p\)-value or a critical value to determine whether to reject the null hypothesis.

Chi-Square Test for Independence – Example

Many jobseekers are applying via online job boards these days. Sites like Indeed, ZipRecruiter, and CareerBuilder have thousands of enticing posts inviting people to apply. It’s never been easier for fraudulent recruiters to lure in unsuspecting and vulnerable people.