|
The following covers
a few of the SAS procedures for conducting component and factor analysis. Use the Import Wizard to import the
Example Data 4 file using the SPSS File (*.sav) source option and
member name example4. There should be 750 cases or observations with no missing
values and 16 variables.
Make sure the entire data set was successfully
imported to SAS with the following syntax:
PROC MEANS DATA=example4;
RUN;
Before we begin with the analysis syntax; let's take a moment to address and
hopefully clarify one of the most confusing and misarticulated
issues in statistical teaching and practice literature. An ambitious goal to be
sure.
First, Principal Components Analysis (PCA) is a variable reduction
technique which maximizes the amount of variance accounted for in the observed
variables by a smaller group of variables called COMPONENTS. As an example,
consider the following situation. Let's say, we have 500 questions on a survey
we designed to measure persistence. We want to reduce the number of questions
so that it does not take someone 3 hours to complete the survey. It would be appropriate
to use PCA to reduce the number of questions by identifying and removing
redundant questions. For instance, if question 122 and question 356 are
virtually identical (i.e. they ask the exact same thing but in different ways),
then one of them is not necessary. The PCA process allows us to reduce the
number of questions or variables down to their PRINCIPAL COMPONENTS.
PCA is commonly, but very confusingly, called exploratory factor analysis (EFA).
The use of the word factor in EFA is inappropriate and confusing because we are really
interested in COMPONENTS, not factors. This issue is made more confusing by some
software packages (e.g. PASW / SPSS) which list or use PCA under the heading
factor analysis.
Second, Factor Analysis (FA) is typically used to confirm the latent
factor structure for a group of measured variables. Latent factors are
unobserved variables which typically can not be directly measured; but, they are
assumed to cause the scores we observe on the measured or indicator variables.
FA is a model based technique. It is concerned with modeling the relationships
between measured variables, latent factors, and error.
As stated in O'Rourke, Hatcher, and Stepanski (2005): "Both (PCA & FA) are
methods that can be used to identify groups of observed variables that tend to
hang together empirically. Both procedures can also be performed with the SAS
FACTOR procedure and they generally tend to provide similar results.
Nonetheless, there are some important conceptual differences between principal
component analysis and factor analysis that should be understood at the outset.
Perhaps the most important deals with the assumption of an underlying causal
structure. Factor analysis assumes that the covariation in the observed
variables is due to the presence of one or more latent variables (factors) that
exert causal influence on these observed variables" (p. 436).
Final thoughts. Both PCA and FA can be used as exploratory analysis. But; PCA is predominantly used in an exploratory fashion and
almost never used in a
confirmatory fashion. FA can be used in an exploratory fashion, but most of the
time it is used in a confirmatory fashion because it is concerned with modeling
factor structure. The choice of which is used should be
driven by the goals of the analyst. If you are interested in reducing the
observed variables down to their principal components while maximizing the
variance accounted for in the variables by the components, then you should be using PCA. If you are concerned with
modeling the latent factors
(and their relationships)
which cause the scores on your observed variables, then you should be using FA.
### REFERENCE ###
O'Rourke, N., Hatcher, L., & Stepanski, E.J. (2005). A step-by-step approach to
using SAS for univariate and multivariate statistics, Second Edition. Cary, NC:
SAS Institute Inc.
##################
IX. Principal Components Analysis
So, here we go with the syntax. The generic syntax for Principal Components
Analysis with options is displayed below.
PROC FACTOR DATA=datasetname
SIMPLE
METHOD=PRIN
PRIORS=ONE
NFACT=
MINEIGEN=1
SCREE
ROTATE=
FLAG=.32
OUT=newdata;
VAR variable1 variable2 variable3...variableN;
RUN;
PROC FACTOR, as stated earlier, can be used for either principal components
analysis or factor analysis (you see why this can be confusing). The data step
should be familiar by now. The SIMPLE statement provides simple descriptive
statistics for each of the variables in the analysis (i.e. number of
cases/observations, means, standard deviations). The METHOD=PRIN specifies
the extraction method as principal components. The PRIORS=ONE specifies prior communality
estimates. When conducting principal components analysis, you should always use
ONE. The NFACT optional statement allows you to specify the number of retained
components (again, the use of fact or factor makes this confusing). The MINEIGEN=1
specifies the minimum acceptable (or critical) eigen value we want a component
to display in order for it to be retained. The SCREE simply specifies that we
want a scree plot to be displayed with the output. ROTATE= specifies a rotation
strategy. When components are correlated, we would choose an oblique rotation
strategy (e.g. PROMAX) and when components are not correlated, we would choose
an orthogonal rotation strategy (e.g. VARIMAX). The FLAG=.32 specifies that we want the output to flag (with an *) all
loadings greater than the number we specify. Here, 0.32 is specified because
when squared, it represents 10% of the variance in the variable accounted for by
the component. The OUT option specifies a name for a new data set which will
include the original variables and the retained component scores for each
observation. The OUT option can only be used when the input data is raw data
(as opposed to a correlation or covariance matrix) and the number of components
(NFACT) has been specified. The OUT option can be useful for determining whether
or not the components are correlated (e.g. running a PROC CORR on the newly
created data which includes the component scores). The VAR statement is used to
specify all the variables being subjected to the component analysis. It is
important to notice the presence of semi colons before and after the VAR
statement.
If the OUT option is used in the principal component analysis, then you will
likely want to explore the relationships between the components (named factor1
factor2...factorN by default) and the variables. In which case, the syntax below
provides the generic format for doing so. Again, the use of the term factor when
referring to components makes this stuff confusing.
PROC CORR DATA=newdata;
VAR factor1 factor2...factorN;
WITH variable1 variable2...variableN factor1 factor2...factorN;
RUN;
(1)
Now we can move on to a practical example. The current example uses Example Data
4 (example4) which contains 15 items or variables and 750 cases or observations.
For an initial components analysis, we specify no number of components to be
retained, no rotation strategy, and we are not interested in creating a new data
file.
PROC FACTOR DATA=example4
SIMPLE
METHOD=PRIN
PRIORS=ONE
MINEIGEN=1
SCREE
FLAG=.32;
VAR y1 y2 y3 y4 y5 y6 y7 y8 y9 y10 y11 y12 y13 y14 y15;
RUN;
We notice from the output, we have two items (y14 & y15) which do not load on
the first component (always the strongest component without rotation) but create
their own retained component (also with eigen value greater than 1). We know a
component should have, as a minimum, 3 items/variables; but let's reserve
deletion of items until we can discover whether or not our components are
related.
(2) Next, we re-run the PCA specifying NFACT = 5, which really means we
are specifying 5 components to be retained. We also specify the creation of a
new data set (ex4comp2) which will contain all the variables used in the PCA
and component scores for each observation. Also note, we removed the SIMPLE
option because the descriptive statistics were given with the previous PCA.
PROC FACTOR DATA=example4
METHOD=PRIN
PRIORS=ONE
NFACT=5
MINEIGEN=1
SCREE
FLAG=.32
OUT=ex4comp2;
VAR y1 y2 y3 y4 y5 y6 y7 y8 y9 y10 y11 y12 y13 y14 y15;
RUN;
The creation of the new data set allows us to determine if our components are
correlated.
PROC CORR DATA=ex4comp2;
VAR factor1 factor2 factor3 factor4 factor5;
WITH y1 y2 y3 y4 y5 y6 y7 y8 y9 y10 y11 y12 y13 y14 y15 factor1 factor2 factor4
factor5;
RUN
We see in this output that our components are not correlated, which indicates we
should use an orthogonal rotation.
(3) Now we can re-run the PCA with a VARIMAX rotation applied.
PROC FACTOR DATA=example4
METHOD=PRIN
PRIORS=ONE
NFACT=5
MINEIGEN=1
SCREE
ROTATE=VARIMAX
FLAG=.32
OUT=ex4comp3;
VAR y1 y2 y3 y4 y5 y6 y7 y8 y9 y10 y11 y12 y13 y14 y15;
RUN;
Here we see that the varimax rotation cleaned up the interpretation by
eliminating the global first component (see the Rotated Factor Pattern table).
And, because we created a new data file, we can verify the complete lack of
correlations between the components using the syntax below.
PROC CORR DATA=ex4comp3;
VAR factor1 factor2 factor3 factor4 factor5;
WITH y1 y2 y3 y4 y5 y6 y7 y8 y9 y10 y11 y12 y13 y14 y15 factor1 factor2 factor4
factor5;
RUN;
(4) Finally, we can eliminate the two items which (1) by themselves
create a component (components should have more than 2 items or variables) and
(2) do not load (at all) on the un-rotated or initial component 1.
PROC FACTOR DATA=example4
METHOD=PRIN
PRIORS=ONE
NFACT=4
MINEIGEN=1
SCREE
ROTATE=VARIMAX
FLAG=.32
OUT=ex4comp4;
VAR y1 y2 y3 y4 y5 y6 y7 y8 y9 y10 y11 y12 y13;
RUN;
PROC CORR DATA=ex4comp4;
VAR factor1 factor2 factor3 factor4;
WITH y1 y2 y3 y4 y5 y6 y7 y8 y9 y10 y11 y12 y13 factor1 factor2 factor4;
RUN;
To help clarify the purpose of PCA, consider reviewing the output for PCA (3)
with particular attention to the first page of that output (the page above the
scree plot). You will find there a table with the title "Eigenvalues of the
Correlation Matrix: Total = 15, Average = 1". The fourth column in that table is
called "Cumulative" and refers to the cumulative variance accounted for by the
components. Now focus on the fifth value from the top in that fourth column.
That value of .5503 tells us 55.03% of the variance in the items (specifically
the items' variance - covariance matrix) is accounted for by all 5 components.
As a comparison, and to highlight the purpose of PCA; look at the same table
only for PCA (4), which has the title "Eigenvalues of the Correlation
Matrix: Total = 13, Average = 1". Pay particular attention to the fourth value
in the fourth (cumulative) column. This value of .5517 tells us 55.17% of the
variance in the items (specifically the items' variance - covariance matrix) is
accounted for by all 4 components. So, we have reduced the number of items from
15 to 13, reduced the number of components, and yet have improved the amount of variance accounted for in the
items by
our principal components.
X. Factor Analysis
The generic syntax for Factor
Analysis (FA) with options is displayed below; however, the only real changes
are the extraction method and priors. Before we used principal for PCA while
here with FA, we will be using ML which refers to maximum likelihood extraction.
Some suggest using ULS which refers to un-weighted least squares extraction. The
other change is the use of SMC or squared multiple correlations in the priors
statement.
PROC FACTOR DATA=datasetname
SIMPLE
METHOD=ML or ULS
PRIORS=SMC
NFACT=
MINEIGEN=1
SCREE
ROTATE=
FLAG=.32
OUT=newdata;
VAR variable1 variable2 variable3...variableN;
RUN;
Continuing with the same data as was used above, we will submit our 15 initial
items to the Maximum likelihood FA with VARIMAX rotation, and SMC priors. We
leave out the SIMPLE option because we have already seen the descriptive
statistics for each item above. Will will leave out the OUT statement because we
do not need to use the factor scores for assessing the relationship between the
factors (we know from above they are not related). However, it is often useful
to save the factor scores for use in another analysis (SEM). We will leave out the MINEIGEN criteria so that we insure we get all 5 factors retained (often it is
the case that only one common factor is retain because only one factor displays
an eigen value greater than 1).
PROC FACTOR DATA=example4
METHOD=ML
PRIORS=SMC
NFACT= 5
SCREE
ROTATE=VARIMAX
FLAG=.32;
VAR y1 y2 y3 y4 y5 y6 y7 y8 y9 y10 y11 y12 y13 y14 y15;
RUN;
Look on the sixth page of the output, you will see a table titled "Rotated
Factor Pattern" in the middle of that page. This table displays the rotated
factor loadings for each item / variable on each factor retained. Notice that
Factor 5 has no items loading greater than 0.32 (* indicates loadings greater
than 0.32). Also notice that items y14 and y15 do not load on any factor greater
than 0.32. In fact, the greatest loading for y14 is with Factor 5 which is only
0.20; which when squared (.04) represents only 4% of the variance accounted for
in that item by factor 5. Furthermore, Factor 5 is only supported by two items
(y14 & y15) which themselves are not very good (indicated by the communalities).
For instance, if we look at the seventh page of the output, we find the majority
of a table titled "Final Communality Estimates and Variable Weights" which
displays the communalities for each item / variable. Communalities represent the
sum of the squared loadings for an item. They are interpreted as the amount of
variance in an item which is explained by all the retained factors after
rotation. So, we can see that both y14 and y15 display very low communalities
which indicates their variance is not explained by the combined factors. To be
more specific, y14 displays a communality of 0.042; which when interpreted
means: only 4.2% of the variance of item y14 is explained by all five factors
combined. The bottom line interpretation here is that Factor 5 and items y14 and
y15 can be removed.
PROC FACTOR DATA=example4
METHOD=ML
PRIORS=SMC
NFACT= 4
SCREE
ROTATE=VARIMAX
FLAG=.32;
VAR y1 y2 y3 y4 y5 y6 y7 y8 y9 y10 y11 y12 y13;
RUN;
Reviewing the last two pages of the most recent output, we see the "Rotated
Factor Pattern" table and the "Final Communality Estimates and Variable Weights"
table (which starts on the bottom of one page and continues on the last page of
the output). In the Rotated Factor Pattern table we see clear factor structure
displayed; meaning, each item loads predominantly on one factor. For instance,
the first four items load virtually exclusively on Factor 1. Furthermore, if we
look at the communalities we see that all the items displayed a communality of
0.32 or greater, with one exception. The exception is y4, which is a little
lower than we would like and given that Factor 1 has three other items which
load significantly on it, we may choose to remove item y4 from further analysis
or measurement in the future.
Finally; as an additional example; we can take a look at the same analysis but
with an oblique (PROMAX) rotation strategy.
PROC FACTOR DATA=example4
METHOD=ML
PRIORS=SMC
NFACT=4
SCREE
ROTATE=PROMAX
FLAG=.40
OUT=ex4comp5;
VAR y1 y2 y3 y4 y5 y6 y7 y8 y9 y10 y11 y12 y13;
RUN;
PROC CORR DATA=ex4comp5;
VAR factor1 factor2 factor3 factor4;
WITH y1 y2 y3 y4 y5 y6 y7 y8 y9 y10 y11 y12 y13 factor1 factor2 factor4;
RUN;
When interpreting the output of a run with oblique rotation, remember that the
oblique process is a two stage process. During the first stage, an orthogonal
rotation solution is produced. The current example provides output (on pages 6 &
7) which is identical to the previous VARIMAX rotated 4 factor and 13 item
solution from above. During the second stage, the factors are allowed to
correlate and the PROMAX rotation is then applied. Interpretation of the oblique
(PROMAX) solution begins on page 8 of the current output. The top of page 10
begins with the table named "Inter-Factor Correlations" but, directly below that
table one can find the "Rotated Factor Pattern (Standardized Regression
Coefficients)" table. Here is where the rotated loadings for the PROMAX rotation
are displayed. At the bottom of page 12 and continuing on to page 13 one will
find the communality estimates associated with the PROMAX solution.
XI. Internal Consistency Analysis (Cronbach's
Alpha Coefficient)
Often when one is conducting principal components analysis or factor analysis,
one will want to conduct an internal consistency analysis. Traditionally, reliability
analysis was used synonymously with internal consistency and/or Cronbach's Alpha
or Coefficient Alpha. However, Cronbach's Alpha is not a statistical measure of
reliability; it is a measure of internal consistency. Reliability generally
refers to whether or not a measurement device provides consistent data across
multiple administrations. Reliability can be assessed by correlating multiple
administrations of the measurement device given to the same population at
different times -- this is known as test-retest reliability. Internal consistency can be thought of as
the relationship between each item and each other item; and internal
consistency can be thought of as the relationship of each item to the collection
of items or total score. Internal consistency
is assessed using (1) the item to total score correlation and (2) Cronbach's
alpha coefficient. In SAS, the item to total score correlation and Cronbach's
alpha coefficient are provided using the PROC CORR procedure, but we would have
to specify the ALPHA option.
The generic form of the PROC CORR procedure for producing Cronbach's alpha
follows:
PROC CORR DATA=dataname ALPHA NOMISS;
VAR variable1 variable2 variable3...variableN;
RUN;
The PROC CORR for obtaining Cronbach's alpha uses the ALPHA option and can use
the NOMISS option to perform deletion of missing values.
As a practical example, consider the following syntax which will provide
Cronbach's alpha coefficient for the first four items of our data set (i.e.
Factor 1):
PROC CORR DATA=example4 ALPHA;
VAR y1 y2 y3 y4;
RUN;
The output of the ALPHA procedure contains 4 tables. The first table simply
reports the descriptive statistics for each item/variable entered in the ALPHA
procedure. The second table reports the raw and standardized versions of
Cronbach's alpha coefficient. The third table (often critically important)
reports the item-to-total score correlations and the alpha if item deleted for
each item in both raw and standardized forms. Keep in mind, we would expect a
good item to display a high correlation with the total score and a low alpha if
item deleted (i.e. if alpha drops when an item is deleted, then clearly that
item was important). |
