A store that sells lucky charms claims that the more charms you buy in their store, the better you do in school exams. As proof of this, they conducted a survey of \(5\) customers, and thanks to statistics, they concluded that there was a positive linear relationship between the number of lucky charms and the grades obtained in school.
Does this sound convincing to you? They are probably just saying that so they can sell more. The good thing is that, in situations like the one above, you can use a hypothesis test for the slope of a regression model to test how useful a regression line is for modeling the behavior between two sets of data.
Meaning of the Hypothesis Test for Regression Slope
Suppose that to find the relationship between two variables, you have used linear regression to obtain an equation \[\hat{y}=\alpha+\beta x.\]
In theory, this equation should allow you to predict values of \(y\), by evaluating at \(x\), that is, \(y\approx\hat{y}(x)\).
But how can you be confident that the linear regression equation obtained is good at predicting \(y\) values? As mentioned at the beginning, a hypothesis test can help you.
Hypothesis testing is based on calculating how likely it is to obtain a sample like yours, if certain conditions are assumed, in this case, assuming the regression slope obtained, what is the probability of obtaining the sample in question.
Recall that the slope \(\beta\) represents the average change of the variable \(y\) with respect to the change per unit of the variable \(x\).
Importance of Hypothesis Test for Regression Slope
Whenever you use linear regression to model the behavior of two datasets that are related, the regression slope that you get is an estimate of how one data changes regarding the other.
Normally, this linear regression equation changes each time you take a different sample, so it makes sense to ask yourself if the actual slope value of the population is similar to the one you get from the sample using linear regression.
The following images show the scatter plots of \(2\) sets of data with their respective regression line.
A good regression line should allow you to predict \(y-\)values knowing the \(x-\)values quite accurately. Looking at the first image, you can notice that since the points are close to the line, then the regression line is good.
Fig. 1 - A scatter plot with a good regression line
On the other hand, in the second image, several values are far from the values predicted by the regression line. For this reason, you can say that the regression line is not so good.
Fig. 2 - A scatter plot with a bad regression line
In situations like the graph above, it makes sense to doubt how good the obtained regression line is.
Hypothesis Test for Regression Coefficients
There are many hypothesis tests that can be performed on the slope of the regression line. These consist of having a null hypothesis, which it can be
\[H_0:\; \beta=\beta_0,\]
that is, that the regression slope is equal to a certain value.
While the alternative hypothesis will be some form of negation of the null hypothesis, such as
\( H_a:\;\beta>\beta_0 \);
\(H_a:\; \beta<\beta_0 \); or
\( H_a:\; \beta\neq\beta_0 \).
Although the slope of a regression line can have many values, hypothesis testing generally only focuses on answering: Is the slope different from zero? If it is different from zero, then you will be able to use it to make predictions. Therefore, this article will only focus on making this type of hypothesis.
Why can't you use a regression line with zero slope to make predictions? A regression line with zero slope means that the data for \(y\) does not depend on \(x\), in other words, knowing the value of \(x\) does not allow you to predict the value of \(y\) using the regression line. This means the regression line is not useful.
Conditions for Hypothesis Test for Regression Slope
To be able to make inference about the coefficients of the regression line, you must make sure that your data meets the following conditions:
Linearity: The scatter plot of the data looks straight.
Independence: The residuals must be independent (see the article Residuals for more information about this).
Equal variance: The standard deviation of the \(y\)-values should be nearly equal for all values of \(x\).
Normal population: The \(y\)-values are distributed normally for any value of \(x\).
Methods of Hypothesis Test for Regression Slope
Recall that in this article you will only learn how to perform the hypothesis test to prove that the slope of the regression line is non-zero. So, the procedure is as follows:
Step 1. State the hypotheses.
The null hypothesis and the alternative hypothesis are given by
\[\begin{align} &H_0\; :\beta=0 \\ &H_a:\;\beta\neq 0. \end{align}\]
The null hypothesis states that the slope is zero, which is equivalent to saying that there is no useful linear relationship between \(x\) and \(y\) while the alternative hypothesis states that there is a useful linear relationship.
Step 2. Determine a significance level to use.
Normally, the significance level \(\alpha\) is taken as \(0.05\), but you can also consider \(0.01\), or \(0.1\).
Step 3. Find the test statistic and the corresponding \(p-\)value.
For this step, you need the standard error of the slope, the slope of the linear regression, the degrees of freedom (for samples having \(n\) pairs of data, the degrees of freedom are \(n-2\)) and the \(p-\)value associated to the test statistic.
The test statistic is given by
\[t=\frac{b}{s_b},\]
where \(b\) is the slope of the sample regression line, and the standard error \(s_b\) is given by
\[s_b=\frac{s_e}{\sqrt{\sum\limits_{i=1}^n(x_i-\mu_x)^2}}\]
where
\[s_e=\sqrt{\frac{\sum\limits_{i=1}^n(y_i-\hat{y})^2}{n-2}}.\]
Remember that for a small sample size, or when you don't know the population variance, you use the \(t\)-distribution rather than a normal distribution.
You will also need the degrees of freedom for the \(t\)-distribution. Since it is paired data (the value of \(x\) is paired with a value of \(y\)), there are \(n-2\) degrees of freedom.
Step 4. Interpret results.
If the result obtained in the sample is unusual, given the null hypothesis, then the null hypothesis is rejected.
This step involves comparing the \(p\)-value obtained with the significance level, and the null hypothesis is rejected if the \(p\)-value is less than the significance level. Otherwise you will be unable to reject the null hypothesis.
See the article Hypothesis Testing for an explanation of why you don't say things like "the null hypothesis is true".
Example of Hypothesis Test for Regression Slope
Ana wants to know if there is a useful linear relationship between hand size and foot size. So, she decided to collect data from her family. Below is the table with the hand and foot sizes in centimeters of different members of her family.
| Hand size | 15 | 17 | 18 | 19 | 21 |
| Foot size | 17 | 24 | 26 | 25 | 28.5 |
Is there a significant linear relationship between hand and foot size? Use a significance level of \(\alpha=0.05\).
Solution:
The very first thing to do is check the conditions for making a hypothesis test. By making a quick graph of the data you can see that it will satisify the conditions of linearity, independence, equal variance and normal population
Step 1. Since you want to know if there is a significant linear relationship between the two data, the null hypothesis is
\[H_0:\;\beta=0,\]
which says that there is no useful linear relationship. The alternative hypothesis is
\[H_a:\;\beta\neq0 ,\]
which says that there is a useful linear relationship.
Step 2. In this case, the significance level is \(\alpha=0.05\).
Step 3. Using a statistical calculator you can obtain that the regression line for the above data.
If you would like to calculate the regression line by hand, see the article Least-Squares Regression for information on how to do so along with an example.
The regression given by
\[\hat{y}=1.775x-7.85,\]
and the standard error is
\[s_b=0.43.\]
Next, you calculate the test statistic using the formula:
\[\begin{align} t&=\frac{b}{s_b}\\ &=\frac{1.775}{0.43}\\ &=4.128.\end{align}\]
Since you have \(5\) pairs of data, your test statistic follows a \(t\)-distribution with \(5-2=3\) degrees of freedom.
Step 4. If you use a \(t\)-table, you can see that the \(p\)-value associated with \(4.128\), with \(3\) degrees of freedom, is between \(0.01\) and \(0.025\). Since the \(p\)-value is less than the significance level \((0.05)\), the null hypothesis is rejected.
For more information on how to use the \(t\)-table, see our article \(t\)-Distribution.
Therefore, there is evidence that there is a useful linear relationship between hand size and foot size.
Hypothesis Test for Regression Slope - Key takeaways
- The hypothesis test for the regression slope consists of testing whether there is a useful linear relationship between the data.
- The null hypothesis used when doing a hypothesis test for the slope of a regression line is \(H_0:\; \beta=0\), and the alternative hypothesis is \(H_a:\; \beta\neq 0\), where \(\beta\) is the slope of the regression line.
- To perform the hypothesis test for the slope of a regression line, the conditions of linearity, independence, equal variance and normal population must be verified.
Explanations 
Exams 
Magazine 
