+ The right answer to the wrong question The use of factor analysis - - PowerPoint PPT Presentation

+

The right answer to the wrong question

The use of factor analysis and principal component analysis in the social sciences

Jonathan Rose Research Fellow University of Nottingham Jonathan.Rose@Nottingham.ac.uk

+ Before we start

 A note of caution from the introductory section of a chapter

• n factor analysis and principal component analysis in The R

Book:

 These techniques are not recommended unless you know exactly

what you are doing, and exactly why you are doing it. Beginners are sometimes attracted to multivariate techniques because of the complexity of the output they produce, making the classic mistake

• f confusing the opaque for the profound. (Crawley, 2007: 731)

 This may somewhat be overstating the case, but is none the

less a healthy reminder. In extremis, people‟s lives are being staked on incorrect models (more on this later).

+ A fundamental conception of latent variables

 Latent structure: the possibility that the variance in the

• bserved variables (indicators) can be accounted for by a

smaller number of latent variables, which are conceivably of a more fundamental nature.

 These variables are „latent‟ in the sense that they are not

• bserved, and may well be unobservable.

 Think, for instance, of intelligence, trust, confidence, happiness,

etc.

 Almost everything we are really interested in measuring is a

latent variable, even if we don‟t use latent variable models.

+ What you want

 A method to analyze the structure of data

 Either by testing for a specific structure (confirmatory models), or

by attempting to discover a structure through various means (exploratory models)

 The understanding of which will tell you which of your

indicators are „like‟ the others, and which are „different‟: basically, what we can lump together and what we can‟t.

+ Why do you need that?

 More reliable measurement  Require fewer variables in an analysis  Avoid multicollinearity  Understand deep-seated processes that drive responses  Help with conceptualizing the world  Avoid spuriously high correlations caused by analyzing two

halves of the same whole as if they were in a cause and effect relationship

+ So what might you do?

 For many people, the first response would be this:

+ Then poke around the options…

 Now we‟re really getting somewhere

+ This is going surprisingly well!

 Now, we just move the variable names over. That big friendly

„OK‟ button looks so inviting. I bet if I press that I‟ll get my factor analysis…The defaults will be fine. What‟s the worst that can happen?

+ Result

 We have findings.  Yay! Science!  Now: interpret the numbers!

+ But what did we actually get here?

 Remember the method of analysis we chose?  And remember the title of the options box?  Any guesses?

+ That‟s right!

 We got a Principal Components Analysis (PCA).

 If you look carefully, there are clues that this is what you‟re getting;

but they don‟t make it anywhere near as explicit as it ought to be…

+ I‟ve heard of PCA – isn‟t it basically the same thing as a factor analysis?

 No, despite how they are usually treated.

 There are similarities, which we will discuss in a short while – but

the take-home message of the presentation is that PCA and FA are fundamentally different things, even if the results can be similar in some circumstances.

+ Terminological confusion

 Factor analysis has one of the most confused and

contradictory terminologies of any analytical method

 Confusion around principal components analysis and factor

analysis

 Confusion between various kinds of factor analysis  Confusion as to what you get out (e.g. factors, components,

principal components…)

 And that is without dealing with extraction system,

eigenvalues, factor retention criteria, loadings…

+ Perpetuating confusion

 One of the things that perpetuates confusion is the habit in

introductory texts to deliberately conflate FA and PCA.

 For example, in SPSS Survival Manual (2007, 3rd Ed.), Pallant

says, in the chapter called „Factor Analysis‟, “I have chosen to demonstrate principal components analysis in this chapter. If you would like to explore other approaches further, see Tabachnick and Fidell (2007)”.

 Judging by sales, and the number of copies in the library at

Nottingham, this book is clearly a popular way to learn about quantitative analysis using SPSS – but even in the FA chapter they don‟t discuss FA.

+ Perpetuating confusion

 You might have seen in research papers people saying

things like: “we employed a principal components factor analysis (PCF) to aggregate groups of attitudinal questions that reflect a common cluster”. Or “We performed a principal component factor analysis of all drug prescriptions during the entire course of the illness in a representative sample of naturalistically treated bipolar outpatients.” Or countless

• ther examples.

 „Principal components factor analysis‟ basically doesn‟t exist,

it is a conflation of PCA and FA – and it‟s difficult to know exactly what one gets when papers say that they did this.

+ But PCA and FA are similar, right?

 Somewhat. Indeed, sometimes people argue that “either that

there is almost no difference between principal components and factor analysis, or that PCA is preferable (Arrindell & van der Ende, 1985; Guadagnoli and Velicer, 1988; Schoenmann, 1990; Steiger, 1990; Velicer & Jackson, 1990).” (from Costello & Osborne, 2005, Best Practices in Exploratory Factor Analysis)

 However…

+ PCA vs Factor Analysis

 Whilst there are overlaps, and sometimes the solutions are

similar, they are fundamentally different procedures. They are different:

 Conceptually  Mathematically  Practically

 However, you should note that how different analyses will be

in practice is not easily specified before hand

+ Conceptual matters

 A very general latent variable model

 Applies to all kinds of latent variable models  Multiple causes of manifest items  But with an important shared cause  (note that this is slightly different from

how you might see such models elsewhere).

+ The factor analytic conceptual model

 Conceptually much like other latent variable models

 Unique components are included in the „error‟; they are

standardly lumped together because in reality you cannot separate them

+ The PCA conceptual model

 Notably different from the FA model, and from the conceptual

model of latent variables

+ PCA and causality

 It is also more difficult to interpret PCA as a causal model,

since PCA is aiming to give you a a number of linear combinations of the variables so as to capture the variance in the set of items as a whole, rather than an analysis of shared variance (as in FA). This breaks (standard) conceptual models of causality.

 There is no need for the relationship to be causal, and so it‟s

not such a big deal when people introduce items that are clearly not caused by an underlying factor.

+ Mathematics

 The equations underlying the procedures reflect this

difference in approaches.

 For factor analysis, the model is:  For PCA it is:

+ The mathematical differences between FA and PCA

 It‟s easy to see that the equations are different. One includes

error and unique variance, and the other does not. But this difference means that the analyses are not even conducted upon the same information.

+ The PCA matrix

+ The factor analysis matrix

+ Different matrices, different answers?

 So, we have seen that the mathematics are different, and that

means that we use different matrices for our analysis – but does that mean that we are likely to see radically different results when we perform analyses?

 According to Dunteman (1989) in the Sage green book on

PCA, “Both principal components analysis and factor analysis give similar results if the communalities of the variables are high and/or there are a large number of variables”

 That the communalities being high makes a difference is not

surprising, since it makes the diagonal increasingly close to 1 (which is how it is in PCA).

+ Practical matters

 If there were no practical implications of the choice between

FA and PCA, or only minor ones, there would be very little to worry about. Yes, one model might be formally inappropriate, but we use formally inappropriate models all the time: linear regression of dichotomous items, SEM of non- multivariate normal data, etc., etc.

 Unfortunately, FA and PCA are particularly susceptible to

small deviations – not really because of any mathematical quirk, because of you. FA and PCA, perhaps more than any

• ther method of analysis, require a significant degree of

interpretation and theoretical consideration. Coefficients never fully speak for themselves, but they do so even less in FA/PCA than we are used to.

+ A worked example

 Data on the psychological impact of Huntington‟s Disease  1803 cases  Dealing with:

 Depressed mood  Low self-esteem  Suicidal thoughts  Anxiety  Compulsions  Perseveration  Apathy

• Aggression
• Irritability
• Hallucinations
• Delusions

+ Research findings

 Of articles which analyze similar data, or older versions of

the same dataset, “[a]ll the studies have shown distinct factors for depression, executive functions and irritability.” (Rickards et al., 2011)

 This study finds 4 factors – depression, executive function,

irritability, and also psychosis.

 We might take issue with the idea of extracting 4 dimensions

anyway (more on this after the lunch break), but let‟s take it as read for the minute that there really are 4. Does the decision to use FA, rather than PCA, make a difference?

+ Again on terminology…

 Note that they call their article “Factor Analysis of…”, but

actually use PCA – as do most other people as best as I can tell (if you‟re going to do it, at least report what you actually do).

+ Two 4-D solutions: FA (left)/PCA (right)

+ Similar, but not identical results

 Compulsive behaviour, perseveration and apathy – or just

compulsive behaviour and perseveration ?

 Do hallucinations and delusions „go together‟?  Notice also the changes in the coefficients for aggression

(„dab‟) and irritability („ib‟)

+ Not just mathematical quirks

 This has real implications – the differences we see here are

not „massive‟, in a traditional sense, but they would have genuine consequences for how we interpret the world around us.

 In the article published using this research, they chose to

bold coefficients over 0.4 – on this criterion, apathy doesn‟t warrant inclusion in the FA model at all

 Yet the objective of the analysis is to develop rating scales for

use in actual day-to-day treatment. Errors here are potentially very serious.

+ If anybody is interested, a few seconds on rotation

+ Rotated solution

+ And finally, differences in software

 Whilst this may be a problem which is especially bad in SPSS,

the problem is far more general. Try looking through R packages that perform these types of analysis and try to figure out what you are actually getting from the routine (Normalization as standard? Rotation as standard? Which rotation? How many factors does it standardly extract?...)

 I‟ve been unable to easily replicate a PCA analysis between

SPSS and R (using prcomp or princomp) – although the code suggests I should be seeing essentially the same thing.

+ Recommendations

 We started with the advice that “[t]hese techniques are not

recommended unless you know exactly what you are doing, and exactly why you are doing it.”

 So what are practical ways to begin, to make sure you‟re

doing it right.

 Study the manuals, or if you can, the code itself. It is rarely obvious

what routines actually do from the main interface itself.

 Think carefully about what you actually want to find out before

you analyze the data you have – do you want a causal model? Do you care about „unique‟ variance?

 Test different options that you could have sensibly used to see

how big a difference it would make.

+ tl;dr

 Factor analysis is not the same as principal components

analysis.

 They can lead to different conclusions.  So be careful.

+

Q & A