|
Bivariate Linear Regression in SPSS.
Regression analysis can have a couple of different purposes. Generally
regression is used as a means to predict values or scores on the outcome
variable using one or more predictor variables. However, regression is also
often used as a means of determining variable importance, meaning how are two or
more variables related in the context of a model. There are a vast number of
types and ways to conduct regression. This tutorial will focus exclusively on
ordinary least squares (OLS) linear regression. As with many of the tutorials on
this web site, this page should not be considered a replacement for a good
textbook, such as:
Pedhazur, E. J. (1997). Multiple regression in behavioral research:
Explanation and prediction (3rd ed.). New York: Harcourt Brace.
For the duration of this tutorial, we
will be using
RegData001.sav
Regression Assumptions: Regression is perhaps the most popular form of
statistical analysis. Unfortunately, regression also likely has the distinction of being the most
frequently abused statistical analysis, meaning it is often used incorrectly.
There are many assumptions of regression analysis. It is strongly urged that one
consult a good textbook (such as Pedhazur, 1997) to review all the assumptions
of regression. However, some of the more frequently violated
assumptions will be reviewed here briefly. First, regression works best
under the condition of proper model specification; essentially, you should have
all the important variables in the model and no un-important variables in the
model. Literature reviews on the theory and variables of interest pay big
dividends when conducting regression. Second, regression works best when there
is a lack of multicollinearity. Multicollinearity is a big fancy word for: your
predictor variables are too strongly related, which degrades regression's
ability to discern which variables are important to the model. Third, regression
is designed to work best with linear relationships. There are types of
regression specifically designed to deal with non-linear relationships (e.g.
exponential, cubic, quadratic, etc.); but standard multiple regression using
ordinary least squares works best with linear relationships. Fourth, regression
is designed to work with continuous or nearly continuous data. This one causes a
great deal of confusion, because 'nearly continuous' is a subjective judgment. A
9-point Likert response scale item is NOT a continuous, or even nearly
continuous, variable. Again, there are special types of regression to deal with
different types of data, for example, ordinal regression for dealing with an
ordinal outcome variable, logistic regression for dealing with a binary
dichotomous outcome, multinomial logistic regression for dealing with a
polytomous outcome variable, etc. Furthermore, if you have one or more
categorical predictor variables, you cannot simply enter them into the model.
Categorical predictors need to be coded using special strategies in order to be
included into a regression model and produce meaningful interpretive output. The
use of dummy coding, effects coding, orthogonal coding, or criterion coding is
appropriate for entering a categorical predictor variable into a standard
regression model. Again, a good textbook will review each of these
strategies--as each one lends itself to a particular purpose. Fifth, regression
works best when outliers are not present. Outliers can be very influential to
correlation and therefore, regression. Thorough initial data analysis should be
used to review the data, identify outliers (both univariate and multivariate),
and take appropriate action. A single, severe outlier can wreak havoc in a
multiple regression analysis; as an esteemed colleague is fond of saying...know
thy data!
Bivariate Regression. The simplest form of regression is
bivariate regression, in which one variable is the outcome and one is the
predictor. Very little information can be extracted from this type of analysis.
The most meaningful statistic is likely to be the correlation coefficient
squared (R²), which refers to
the amount of variance in one variable accounted for by the other.
Start by clicking on Analyze, Regression, Linear...
Next, highlight the y variable and use the top arrow button to move it to the
Dependent: box. Then, highlight the x1 variable and use the second arrow button
to move it to the Independent(s): box.
Next, click on the Statistics... button and select Confidence intervals and
Covariance matrix (Estimates & Model fit should be selected by default). Then,
click the Continue button.
Next, click on the Plots... button. Then, highlight *ZPRED and use the second
arrow button to move it to the X: box. Then highlight *ZRESID and use the top
arrow button to move it to the Y: box. Then, select Histogram and Normal
probability plot. Next, click the Next button (marked with a
red ellipse in the figure to the right). Finally,
click the Continue button.
Next, click the OK button to complete the regression analysis and produce
output similar to that displayed below.
Interpretation of the Model Summary Table shows us that the multiple
correlation coefficient (R; i.e. the correlation between the predictor
and the outcome variables) is .547, which when squared gives us .300 which can
be thought of as the amount of variance in the outcome variable that is
accounted for by the predictor variable. Generally expressed as: 30% of the
variance in y was accounted for by x1. However, R²
tends to be slightly optimistic and therefore, a more appropriate metric is
adjusted multiple correlation coefficient squared (adj. R² = .293). Next,
we see the ubiquitous ANOVA table which simply tests whether or not our model is
significantly better than just using the mean of x1 to predict new values of y.
Here, our model is significantly better. Another way of thinking about this
ANOVA table concerns whether or not the R² is significantly different
from zero. Next, we see our Coefficients table which gives the unstandardized
and standardized coefficients (for building a regression equation) as well as a
t test for each. So, if we wanted to predict new raw scores on the
outcome variable (y), we would use the following equation:
(1) y =
.779*x1 + 366.485
where the .779 is the
unstandardized coefficient for the predictor (often called the b-weight) and the
366.485 is the y-intercept term (often called a). The t test for the
constant or y-intercept has virtually no meaning. The t test for the predictor
coefficient is testing whether or not the coefficient is significantly different
than zero. The standardized coefficient (often called Beta and given the symbol
β) represent the correlation between the predictor and the outcome. As you can
see, in the case of only one predictor, it is the same as the multiple
correlation (R). If we were interested in predicting new standardized
scores of the outcome (y) then we would use the following regression equation:
(2) Zy
= .547*Zx1
where the .547 represent the
coefficient for the standardized predictor. There is obviously not an intercept
term when dealing with standardized scores because, the intercept is always zero
on both the x and y axis--commonly called the centroid.
The next table gives us the
correlation and covariance matrix for our coefficient(s). Then, we have the
residual descriptive statistics table which displays descriptive summary
statistics for the residuals, also called errors of prediction (y - yhat). This
table is followed by a histogram of the residuals, which we expect to be
normally distributed and finally a diagnostic plot showing the expected versus
observed probability values.
As shown in a previous tutorial,
we can get an informative scatterplot to represent our bivariate regression by
clicking no Graphs, Legacy dialogs, Scatter/Dot...
Next, click the Define button (the default
Simple Scatter is appropriate). Then highlight the y variable and use the top
arrow button to move it to the Y Axis: box. Then, highlight the x1 variable and
use the second arrow button to move it to the X Axis: box.
Next, click the OK button to create the simple
scatterplot.
Next, right click on the scatterplot in the
output and select Edit Content, In Separate Window to bring up the chart editor.
Using the Chart editor, right click on the
actual data points in the scatter plot (in the chart editor), at which point
they should turn a yellow color. Then, select Add Fit Line at Total.
Next, simply left click somewhere in the white
space of the output (outside the chart editor). You should now see something
similar to what is displayed below. Note, the y-intercept does not seem to match
with what is in the table above because, the scale of the x-axis begins at
approximately 80 rather than zero.
This concludes the bivariate regression section. The next section focuses on
Multiple Linear Regression.
|
