In everyday life there are many quantities that are related to each other, for example, the price of fruits is related to their weight, someone's weight may be related to their height, or the temperature in degrees Fahrenheit has its equivalence in degrees Celsius.
Explore our app and discover over 50 million learning materials for free.
Lerne mit deinen Freunden und bleibe auf dem richtigen Kurs mit deinen persönlichen Lernstatistiken
Jetzt kostenlos anmeldenNie wieder prokastinieren mit unseren Lernerinnerungen.
Jetzt kostenlos anmeldenIn everyday life there are many quantities that are related to each other, for example, the price of fruits is related to their weight, someone's weight may be related to their height, or the temperature in degrees Fahrenheit has its equivalence in degrees Celsius.
Now, if you have two sets of numbers that might be related, how can you find the equation that relates them? If you suspect that the relationship is linear, you can use linear regression.
In this article, you will understand what linear regression is, what the model for linear regression is, what the equation for linear regression is, and what assumptions need to be considered.
Recall that the equation of a straight line is given by \[y=a+bx,\] where \(b\) is called the slope of the line and \(a\) is called the \(y\)-intercept (the value of \(y\) where the line crosses the \(y\)-axis).
As mentioned above, some quantities are related to others in a linear way. For example, the price of mangos. In the United States, the price per kilogram of mango is approximately \(\$1.80\), so the price for \(2\) kilograms would be \(\$3.60\). Thus, the relationship between the price and the weight of the mango is given by the equation \[y=1.80x,\]
where \(x\) is the number of kilograms (the independent variable) and \(y\) is the price (the dependent variable).
Now, suppose you are interested in people's addiction to cell phones. So in your afternoon arts class, you asked \(5\) people how many text messages they sent over the duration of the class, and you got the following information.
Age | Number of text messages sent |
\(17\) | \(35\) |
\(18\) | \(27\) |
\(20\) | \(29\) |
\(22\) | \(23\) |
\(27\) | \(18\) |
Table 1. Ages and number of text messages data.
Is there a relationship between a person's age and the number of text messages they send? Is the relationship linear? How can I find the equation that relates them?
Before defining what linear regression is, let's look at the following scatter plot showing the distribution of the data obtained in the text message example.
At a glance, you can see that you can draw several lines that can approximate the behavior of the points.
Linear regression is a statistical technique that consists of finding the best straight line that describes the relationship between a dependent variable and one or more independent variables.
The most commonly used model is the so-called least squares regression line.
With linear regression, you can make a prediction of the data you do not know, from the behavior of the data you obtained in the sampling.
One way to know if two sets of data are linearly related is to look at the scatter plot. The other is by calculating the Pearson correlation coefficient or correlation coefficient.
Let \(x\) and \(y\) be the independent and dependent variable, respectively. If \(\mu_{x}\) and \(s_x\) are the mean and the standard deviation of the sample of \(x\) and \(\mu_{y}\) and \(s_y\) are the mean and the standard deviation of the sample of \(y\).
Then, if the sample has size \(n\), the correlation coefficient is calculated by:
\[r=\frac{\sum z_xz_y}{n-1
where
\[z_x=\frac{x-\mu_{x}}{s_x}\,\text{ and }\, z_y=\frac{y-\mu_{y}}{s_y}\]
To calculate the correlation coefficient for the text messages example, let \(x\) be the variable denoting the age (the independent variable), and \(y\) the number of text messages (the dependent variable).
Then \(\mu_{x}=20.8\) and \(s_x=3.96\) are the sample mean and standard deviation for the age, and \(\mu_{y}=26.4\) and \(s_y=6.39\) are the sample mean and standard deviation for the number of text messages.
Thus, the correlation coefficient is:
\[\begin{align} r&=\frac{(-0.96)(1.35)+(-0.7)(0.09)+(-0.20)(0.41)+(0.3)(-0.53)+(1.56)(-1.31)}{4} \\&=-0.91\end{align}\]
Review our articles on sample mean, standard deviation and \(z\)-score to remember these topics!
1. The correlation coefficient takes values from \(-1\) to \(1\).
2. If \(|r|=1\) then the relationship between the variable \(x\) and \(y\) is completely linear.
3. If \(r=0\), then there is no linear relationship between \(x\) and \(y\).
4. If \(r>0\), then when \(x\) increases, \(y\) tends to increase and when \(x\) decreases, \(y\) tends to decrease (also called positive correlation).
5. If \(r<0\), then when \(x\) increases, \(y\) tends to decrease and when \(x\) decreases, \(y\) tends to increase (also called negative correlation).
❗❗ Correlation does not imply causation.
To apply linear regression, you first have to check the following conditions:
1. Quantitative variable condition: Correlation only applies if both variables are quantitative.
2. Straight enough condition: Look at the scatter plot and make sure your data has an approximately linear relationship. Correlation only measures the strength of a linear association.
3. Outlier condition: Outliers can ruin the correlation. When outliers are present, it is best to calculate one correlation including the outliers and another excluding the outliers.
The regression line is not perfect. It does not pass through all the points, some points will be above and some will be below, but it is the best in the sense that the sum of squares of the residuals (see the Residuals article for more information) is the smallest possible.
The calculations for finding the regression line are often tedious and time-consuming, that's why there are statistical software and graphing calculators that you can use to help you do the calculations.
The line of best fit is called the least square regression line.
The equation of the least squares regression line is given by:
\[\hat{y}=a+bx,\] where \[b=\frac{\sum(x-\mu_{x})(y-\mu_{y})}{\sum(x-\mu_{x})^2}\,\text{ and }\,a=\mu{y}-b\mu_{x}\]
In the previous line equation, the value \(\hat{y}\) is the predicted value of \(y\) that results from substituting a particular value \(x\) into the equation. Since \(\hat{y}\) is only a prediction, the difference between the value \(\hat{y}\) and the actual value of \(y\) is called residual and is given by:
\[\varepsilon=y-\hat{y}\]
For the text messages example, the equation for the least square regression line is given by:
\[\hat{y}=-1.47x+57.1\]
Using this equation, you can predict how many text messages a \(25\) years old would send. Then:
\[\hat{y}=-1.47(25)+57.1=20.35\]
that is, a \(25\) years old would send \(20.35\) text messages.
The linear regression line should only be used to predict values that are within the domain of the \(x\) values in the sample. Otherwise, in the example of text messages, you could erroneously conclude that a \(1\)-year-old sends \(55\) text messages!
Let's look at an example where outliers can change the regression line.
The following scatter plots show the grades obtained by \(20\) students in a calculus exam and the hours of study dedicated.
In the first image, the linear regression was done with all the data, while in the second image the outliers were omitted, that is, the student who studied \(15\) minutes and scored \(95\) and the student who studied \(6\) hours and scored \(60\) were omitted.
Which line best fits the data?
Solution:
Note that in the first image because of the outliers, many data were far away from the regression line. While in the second image, by not considering the outliers, the data are closer to the regression line.
Therefore, the second line is a better fit to the data.
Multiple linear regression is used to estimate the relationship between one dependent variable and two or more independent variables.
For example, it is known that the cost of a house depends on its size, but it can also depend on the square meters of construction, the age of the property. In this case, the dependent variable is the cost of the house, while the independent variables are the size, the square meters of construction and the age of the property.
For these cases, you can also apply regression, but because the procedure is similar, this topic will not be covered in this article.
Linear regression is a statistical technique that consists of finding the best straight line that describes the relationship between a dependent variable and one or more independent variables.
The regression line is the line that best describes the linear behavior between two variables and allows you to make predictions from it.
No, linear regression is a regression algorithm.
The residual (e) is the difference between the predicted value (y^) by the regression line and the actual value (y), that is, (e)=(y)-(y^).
When the scatter plot of your data has a linear behavior, you can use linear regression.
The method of analysing in order to determine or establish the strength of the relationship between two variables i.e the independent and dependent variables is called?
Bivariate Regression Analysis
The best diagram to use in regression analysis to visibly examine data to determine whether two variables are related linearly is a..............
Scatter plot
Speed and Distance covered can be said to have a..............
Linear regression
The line that best fits the trend as we observe a linear relationship between two variables in regression analysis is called?
Regression Line or Line of best fit
A situation where a scatter plot shows a pattern where one variable tends to increase as the other decreases is described as ____________
Negative association
___________ explains a situation where a scatter plot shows no pattern between the variables being compared or measured
Zero association
Already have an account? Log in
Open in AppThe first learning app that truly has everything you need to ace your exams in one place
Sign up to highlight and take notes. It’s 100% free.
Save explanations to your personalised space and access them anytime, anywhere!
Sign up with Email Sign up with AppleBy signing up, you agree to the Terms and Conditions and the Privacy Policy of StudySmarter.
Already have an account? Log in
Already have an account? Log in
The first learning app that truly has everything you need to ace your exams in one place
Already have an account? Log in