Research and Statistical Support

MODULE 7

The following covers a few of the SAS procedures for conducting component and factor analysis. Use the Import Wizard to import the Example Data 4 file using the SPSS File (*.sav) source option and member name example4. There should be 750 cases or observations with no missing values and 16 variables. Make sure the entire data set was successfully imported to SAS with the following syntax:

PROC MEANS DATA=example4;
RUN;

Before we begin with the analysis syntax; let's take a moment to address and hopefully clarify one of the most confusing and misarticulated issues in statistical teaching and practice literature. An ambitious goal to be sure.

First, Principal Components Analysis (PCA) is a variable reduction technique which maximizes the amount of variance accounted for in the observed variables by a smaller group of variables called COMPONENTS. As an example, consider the following situation. Let's say, we have 500 questions on a survey we designed to measure persistence. We want to reduce the number of questions so that it does not take someone 3 hours to complete the survey. It would be appropriate to use PCA to reduce the number of questions by identifying and removing redundant questions. For instance, if question 122 and question 356 are virtually identical (i.e. they ask the exact same thing but in different ways), then one of them is not necessary. The PCA process allows us to reduce the number of questions or variables down to their PRINCIPAL COMPONENTS.

PCA is commonly, but very confusingly, called exploratory factor analysis (EFA). The use of the word factor in EFA is inappropriate and confusing because we are really interested in COMPONENTS, not factors. This issue is made more confusing by some software packages (e.g. PASW / SPSS) which list or use PCA under the heading factor analysis.

Second, Factor Analysis (FA) is typically used to confirm the latent factor structure for a group of measured variables. Latent factors are unobserved variables which typically can not be directly measured; but, they are assumed to cause the scores we observe on the measured or indicator variables. FA is a model based technique. It is concerned with modeling the relationships between measured variables, latent factors, and error.

As stated in O'Rourke, Hatcher, and Stepanski (2005): "Both (PCA & FA) are methods that can be used to identify groups of observed variables that tend to hang together empirically. Both procedures can also be performed with the SAS FACTOR procedure and they generally tend to provide similar results. Nonetheless, there are some important conceptual differences between principal component analysis and factor analysis that should be understood at the outset. Perhaps the most important deals with the assumption of an underlying causal structure. Factor analysis assumes that the covariation in the observed variables is due to the presence of one or more latent variables (factors) that exert causal influence on these observed variables" (p. 436).

Final thoughts. Both PCA and FA can be used as exploratory analysis. But; PCA is predominantly used in an exploratory fashion and almost never used in a confirmatory fashion. FA can be used in an exploratory fashion, but most of the time it is used in a confirmatory fashion because it is concerned with modeling factor structure. The choice of which is used should be driven by the goals of the analyst. If you are interested in reducing the observed variables down to their principal components while maximizing the variance accounted for in the variables by the components, then you should be using PCA. If you are concerned with modeling the latent factors (and their relationships) which cause the scores on your observed variables, then you should be using FA.

### REFERENCE ###
O'Rourke, N., Hatcher, L., & Stepanski, E.J. (2005). A step-by-step approach to using SAS for univariate and multivariate statistics, Second Edition. Cary, NC: SAS Institute Inc.
##################

So, here we go with the syntax. The generic syntax for Principal Components Analysis with options is displayed below.

PROC FACTOR DATA=datasetname
SIMPLE
METHOD=PRIN
PRIORS=ONE
NFACT=
MINEIGEN=1
SCREE
ROTATE=
FLAG=.32
OUT=newdata;
VAR variable1 variable2 variable3...variableN;
RUN;

PROC FACTOR, as stated earlier, can be used for either principal components analysis or factor analysis (you see why this can be confusing). The data step should be familiar by now. The SIMPLE statement provides simple descriptive statistics for each of the variables in the analysis (i.e. number of cases/observations, means, standard deviations). The METHOD=PRIN specifies the extraction method as principal components. The PRIORS=ONE specifies prior communality estimates. When conducting principal components analysis, you should always use ONE. The NFACT optional statement allows you to specify the number of retained components (again, the use of fact or factor makes this confusing). The MINEIGEN=1 specifies the minimum acceptable (or critical) eigen value we want a component to display in order for it to be retained. The SCREE simply specifies that we want a scree plot to be displayed with the output. ROTATE= specifies a rotation strategy. When components are correlated, we would choose an oblique rotation strategy (e.g. PROMAX) and when components are not correlated, we would choose an orthogonal rotation strategy (e.g. VARIMAX). The FLAG=.32 specifies that we want the output to flag (with an *) all loadings greater than the number we specify. Here, 0.32 is specified because when squared, it represents 10% of the variance in the variable accounted for by the component. The OUT option specifies a name for a new data set which will include the original variables and the retained component scores for each observation. The OUT option can only be used when the input data is raw data (as opposed to a correlation or covariance matrix) and the number of components (NFACT) has been specified. The OUT option can be useful for determining whether or not the components are correlated (e.g. running a PROC CORR on the newly created data which includes the component scores). The VAR statement is used to specify all the variables being subjected to the component analysis. It is important to notice the presence of semi colons before and after the VAR statement.

If the OUT option is used in the principal component analysis, then you will likely want to explore the relationships between the components (named factor1 factor2...factorN by default) and the variables. In which case, the syntax below provides the generic format for doing so. Again, the use of the term factor when referring to components makes this stuff confusing.

PROC CORR DATA=newdata;
VAR factor1 factor2...factorN;
WITH variable1 variable2...variableN factor1 factor2...factorN;
RUN;

(1) Now we can move on to a practical example. The current example uses Example Data 4 (example4) which contains 15 items or variables and 750 cases or observations. For an initial components analysis, we specify no number of components to be retained, no rotation strategy, and we are not interested in creating a new data file.

PROC FACTOR DATA=example4
SIMPLE
METHOD=PRIN
PRIORS=ONE
MINEIGEN=1
SCREE
FLAG=.32;
VAR y1 y2 y3 y4 y5 y6 y7 y8 y9 y10 y11 y12 y13 y14 y15;
RUN;

We notice from the output, we have two items (y14 & y15) which do not load on the first component (always the strongest component without rotation) but create their own retained component (also with eigen value greater than 1). We know a component should have, as a minimum, 3 items/variables; but let's reserve deletion of items until we can discover whether or not our components are related.

(2) Next, we re-run the PCA specifying NFACT = 5, which really means we are specifying 5 components to be retained. We also specify the creation of a new data set (ex4comp2) which will contain all the variables used in the PCA and component scores for each observation. Also note, we removed the SIMPLE option because the descriptive statistics were given with the previous PCA.

PROC FACTOR DATA=example4
METHOD=PRIN
PRIORS=ONE
NFACT=5
MINEIGEN=1
SCREE
FLAG=.32
OUT=ex4comp2;
VAR y1 y2 y3 y4 y5 y6 y7 y8 y9 y10 y11 y12 y13 y14 y15;
RUN;

The creation of the new data set allows us to determine if our components are correlated.

PROC CORR DATA=ex4comp2;
VAR factor1 factor2 factor3 factor4 factor5;
WITH y1 y2 y3 y4 y5 y6 y7 y8 y9 y10 y11 y12 y13 y14 y15 factor1 factor2 factor4 factor5;
RUN

We see in this output that our components are not correlated, which indicates we should use an orthogonal rotation.

(3) Now we can re-run the PCA with a VARIMAX rotation applied.

PROC FACTOR DATA=example4
METHOD=PRIN
PRIORS=ONE
NFACT=5
MINEIGEN=1
SCREE
ROTATE=VARIMAX
FLAG=.32
OUT=ex4comp3;
VAR y1 y2 y3 y4 y5 y6 y7 y8 y9 y10 y11 y12 y13 y14 y15;
RUN;

Here we see that the varimax rotation cleaned up the interpretation by eliminating the global first component (see the Rotated Factor Pattern table). And, because we created a new data file, we can verify the complete lack of correlations between the components using the syntax below.

PROC CORR DATA=ex4comp3;
VAR factor1 factor2 factor3 factor4 factor5;
WITH y1 y2 y3 y4 y5 y6 y7 y8 y9 y10 y11 y12 y13 y14 y15 factor1 factor2 factor4 factor5;
RUN;

(4) Finally, we can eliminate the two items which (1) by themselves create a component (components should have more than 2 items or variables) and (2) do not load (at all) on the un-rotated or initial component 1.

PROC FACTOR DATA=example4
METHOD=PRIN
PRIORS=ONE
NFACT=4
MINEIGEN=1
SCREE
ROTATE=VARIMAX
FLAG=.32
OUT=ex4comp4;
VAR y1 y2 y3 y4 y5 y6 y7 y8 y9 y10 y11 y12 y13;
RUN;

PROC CORR DATA=ex4comp4;
VAR factor1 factor2 factor3 factor4;
WITH y1 y2 y3 y4 y5 y6 y7 y8 y9 y10 y11 y12 y13 factor1 factor2 factor4;
RUN;

To help clarify the purpose of PCA, consider reviewing the output for PCA (3) with particular attention to the first page of that output (the page above the scree plot). You will find there a table with the title "Eigenvalues of the Correlation Matrix: Total = 15, Average = 1". The fourth column in that table is called "Cumulative" and refers to the cumulative variance accounted for by the components. Now focus on the fifth value from the top in that fourth column. That value of .5503 tells us 55.03% of the variance in the items (specifically the items' variance - covariance matrix) is accounted for by all 5 components. As a comparison, and to highlight the purpose of PCA; look at the same table only for PCA (4), which has the title "Eigenvalues of the Correlation Matrix: Total = 13, Average = 1". Pay particular attention to the fourth value in the fourth (cumulative) column. This value of .5517 tells us 55.17% of the variance in the items (specifically the items' variance - covariance matrix) is accounted for by all 4 components. So, we have reduced the number of items from 15 to 13, reduced the number of components, and yet have improved the amount of variance accounted for in the items by our principal components.

The generic syntax for Factor Analysis (FA) with options is displayed below; however, the only real changes are the extraction method and priors. Before we used principal for PCA while here with FA, we will be using ML which refers to maximum likelihood extraction. Some suggest using ULS which refers to un-weighted least squares extraction. The other change is the use of SMC or squared multiple correlations in the priors statement.

PROC FACTOR DATA=datasetname
SIMPLE
METHOD=ML or ULS
PRIORS=SMC
NFACT=
MINEIGEN=1
SCREE
ROTATE=
FLAG=.32
OUT=newdata;
VAR variable1 variable2 variable3...variableN;
RUN;

Continuing with the same data as was used above, we will submit our 15 initial items to the Maximum likelihood FA with VARIMAX rotation, and SMC priors. We leave out the SIMPLE option because we have already seen the descriptive statistics for each item above. Will will leave out the OUT statement because we do not need to use the factor scores for assessing the relationship between the factors (we know from above they are not related). However, it is often useful to save the factor scores for use in another analysis (SEM). We will leave out the MINEIGEN criteria so that we insure we get all 5 factors retained (often it is the case that only one common factor is retain because only one factor displays an eigen value greater than 1).

PROC FACTOR DATA=example4
METHOD=ML
PRIORS=SMC
NFACT= 5
SCREE
ROTATE=VARIMAX
FLAG=.32;
VAR y1 y2 y3 y4 y5 y6 y7 y8 y9 y10 y11 y12 y13 y14 y15;
RUN;

PROC FACTOR DATA=example4
METHOD=ML
PRIORS=SMC
NFACT= 4
SCREE
ROTATE=VARIMAX
FLAG=.32;
VAR y1 y2 y3 y4 y5 y6 y7 y8 y9 y10 y11 y12 y13;
RUN;

Reviewing the last two pages of the most recent output, we see the "Rotated Factor Pattern" table and the "Final Communality Estimates and Variable Weights" table (which starts on the bottom of one page and continues on the last page of the output). In the Rotated Factor Pattern table we see clear factor structure displayed; meaning, each item loads predominantly on one factor. For instance, the first four items load virtually exclusively on Factor 1. Furthermore, if we look at the communalities we see that all the items displayed a communality of 0.32 or greater, with one exception. The exception is y4, which is a little lower than we would like and given that Factor 1 has three other items which load significantly on it, we may choose to remove item y4 from further analysis or measurement in the future.

Finally; as an additional example; we can take a look at the same analysis but with an oblique (PROMAX) rotation strategy.

PROC FACTOR DATA=example4
METHOD=ML
PRIORS=SMC
NFACT=4
SCREE
ROTATE=PROMAX
FLAG=.40
OUT=ex4comp5;
VAR y1 y2 y3 y4 y5 y6 y7 y8 y9 y10 y11 y12 y13;
RUN;

PROC CORR DATA=ex4comp5;
VAR factor1 factor2 factor3 factor4;
WITH y1 y2 y3 y4 y5 y6 y7 y8 y9 y10 y11 y12 y13 factor1 factor2 factor4;
RUN;

When interpreting the output of a run with oblique rotation, remember that the oblique process is a two stage process. During the first stage, an orthogonal rotation solution is produced. The current example provides output (on pages 6 & 7) which is identical to the previous VARIMAX rotated 4 factor and 13 item solution from above. During the second stage, the factors are allowed to correlate and the PROMAX rotation is then applied. Interpretation of the oblique (PROMAX) solution begins on page 8 of the current output. The top of page 10 begins with the table named "Inter-Factor Correlations" but, directly below that table one can find the "Rotated Factor Pattern (Standardized Regression Coefficients)" table. Here is where the rotated loadings for the PROMAX rotation are displayed. At the bottom of page 12 and continuing on to page 13 one will find the communality estimates associated with the PROMAX solution.

XI. Internal Consistency Analysis (Cronbach's Alpha Coefficient)

Often when one is conducting principal components analysis or factor analysis, one will want to conduct an internal consistency analysis. Traditionally, reliability analysis was used synonymously with internal consistency and/or Cronbach's Alpha or Coefficient Alpha. However, Cronbach's Alpha is not a statistical measure of reliability; it is a measure of internal consistency. Reliability generally refers to whether or not a measurement device provides consistent data across multiple administrations. Reliability can be assessed by correlating multiple administrations of the measurement device given to the same population at different times -- this is known as test-retest reliability. Internal consistency can be thought of as the relationship between each item and each other item; and internal consistency can be thought of as the relationship of each item to the collection of items or total score. Internal consistency is assessed using (1) the item to total score correlation and (2) Cronbach's alpha coefficient. In SAS, the item to total score correlation and Cronbach's alpha coefficient are provided using the PROC CORR procedure, but we would have to specify the ALPHA option.

The generic form of the PROC CORR procedure for producing Cronbach's alpha follows:

PROC CORR DATA=dataname ALPHA NOMISS;
VAR variable1 variable2 variable3...variableN;
RUN;

The PROC CORR for obtaining Cronbach's alpha uses the ALPHA option and can use the NOMISS option to perform deletion of missing values.

As a practical example, consider the following syntax which will provide Cronbach's alpha coefficient for the first four items of our data set (i.e. Factor 1):

PROC CORR DATA=example4 ALPHA;
VAR y1 y2 y3 y4;
RUN;

The output of the ALPHA procedure contains 4 tables. The first table simply reports the descriptive statistics for each item/variable entered in the ALPHA procedure. The second table reports the raw and standardized versions of Cronbach's alpha coefficient. The third table (often critically important) reports the item-to-total score correlations and the alpha if item deleted for each item in both raw and standardized forms. Keep in mind, we would expect a good item to display a high correlation with the total score and a low alpha if item deleted (i.e. if alpha drops when an item is deleted, then clearly that item was important).