Analytic Rotation PSY544 – Introduction to Factor Analysis Week 10 Introduction Introduction •Back in the early days, rotation was done by hand (!) using graphical methods – hence the name rotation • •Nowadays, analytical methods are employed using computers • •…however, it’s kinda funny to think about early factor analysts turning things around by hand on a big board (well, it was probably not them, but their assistants) [Most computations were done by “computers” – largely women, by the way] Introduction Introduction •When we formulate a factor analysis model, one of the things we do is interpreting the factors – giving them meaning •We rely on the structure of loadings to get a sense of what the factor might represent • •This endeavor is greatly simplified if most of the loadings corresponding to a manifest variable j are zero or close to zero (which means that the MV is influenced only by a small number of factors) Introduction Orthogonal rotations Orthogonal rotations Orthogonal rotations Thurstone’s “simple structure” Thurstone’s “simple structure” Thurstone’s “simple structure” •In general, Thurstone’s criteria suggest that each factor should be represented by relatively high loadings for a distinct subset of MVs and relatively low loadings for the remaining MVs. •In addition, these subsets defining different factors should not overlap too much. •Furthermore, each MV should only be influenced by some subset of the common factors. •The criteria do not imply that each MV should only be influenced by a single factor, which is a common misconception. Analytic rotation Analytic rotation Analytic rotation Quartimax Varimax Varimax •As simple structure improves, the squared loadings on factors become more variable (some loadings high, the rest low). Summing the variances of the squared loadings over all m factors provides a measure of simplicity. •The described criterion is known as raw Varimax because it is applied to the raw factor loadings. Kaiser found that it works well, but sometimes, in rows with small communalities, it does not. He therefore standardized rows of the factor matrix by dividing factor loadings by the square roots of communalities before rotation. This is usually called normal Varimax or Varimax with Kaiser normalization. • Varimax •Varimax tends to work well as an orthogonal rotation. • •However, Varimax almost monopolized the entire enterprise of orthogonal rotations in applied research (bluntly – everyone uses Varimax all the time) • •Let’s take a look at our example data, before and after a Varimax rotation. • Varimax •Unrotated factor loadings: • 1 2 3 WrdMean 0.68 0.53 -0.27 SntComp 0.72 0.38 -0.23 OddWrds 0.70 0.49 -0.16 MxdArit 0.90 -0.34 -0.03 Remndrs 0.84 -0.20 0.03 MissNum 0.86 -0.13 0.00 Gloves 0.42 0.09 0.43 Boots 0.48 0.25 0.54 Hatchts 0.48 0.30 0.67 Varimax •Rotated factor loadings: • • • • •(note the deviations from •simple structure) 1 2 3 WrdMean 0.15 0.87 0.22 SntComp 0.16 0.75 0.34 OddWrds 0.24 0.79 0.25 MxdArit 0.18 0.25 0.91 Remndrs 0.26 0.29 0.77 MissNum 0.26 0.36 0.75 Gloves 0.56 0.09 0.23 Boots 0.72 0.19 0.17 Hatchts 0.86 0.17 0.12 Analytic rotation •I suggest you perform rotations for various number of extracted factors when exploring the factor structure using EFA. This can also help you in determining the number of factors. •Under-factoring tends to result in multiple factors collapsed into one, which can manifest as a solution that heavily violates simple structure or that is not easily interpretable. •Over-factoring can result into a solution which has a column(s) of loadings with only a single non-zero element, or a column(s) of loadings with all elements very small. • • • • • Orthogonal rotation? • •As we know, orthogonal rotations require the rotated factors to be orthogonal. In other words, we impose the constraint that the transformation matrix T has to be an orthogonal matrix. • •Is this reasonable, though? With exploratory factor analysis, the goal is, after all, to explore the number and nature of the major common factors. How do we know a priori that the factors are uncorrelated? • • • • • Orthogonal rotation? • •In reality, this restriction is mostly uncalled for. In the domains we frequently use FA (mental abilities, attitudes, personality, consumer research, public health), we would on the contrary expect the factors to be a priori correlated. • •Orthogonal rotations are, however, still used very often in practice. Why is that? • • • • • Orthogonal rotation? • •It’s what everyone is doing, so I’ll do it, too. •It’s simple. •It’s the default setting in the program I use. •Lack of understanding of rotation. •Desire for the factors to be uncorrelated. •“Varimax” sounds cool. • • • • • Orthogonal rotation? • •Does any of that matter? Of course not. •Orthogonal rotations were made for times when computers were the size of a room and computations were slow. •We should be using oblique rotations instead. Imposing the constraint of uncorrelated factors is, by large, unjustified. •Moreover – if the best solution (in terms of simple structure) is a one with uncorrelated factors, oblique rotation will find it as such (with oblique rotation, factors are allowed to correlate, not required to) • • • • Orthogonal rotation? • •With oblique rotations, we can expect the solutions to be more easily interpretable with a simpler structure – just because we have accounted for the potential systematic relationships between the latent variables. • •It’s just more realistic. Keep it real, man. • • • • Oblique rotations Introduction Introduction •When the factors are correlated (oblique), the factor loadings can be interpreted as factor weights, representing the linear influence a factor has a particular MV, but no longer as simple correlation. •This sometimes confuses people when they see a rotated solution with correlated factors that contains factor loadings greater than +1 or smaller than -1. Oblique rotations Oblique rotations Crawford-Ferguson family Crawford-Ferguson family CF-Quartimax •Rotated factor loadings: • • • •Factor correlations: • 1 2 3 WrdMean -0.05 0.94 -0.03 SntComp 0.14 0.77 -0.03 OddWrds 0.00 0.83 0.08 MxdArit 1.01 -0.05 -0.04 Remndrs 0.81 0.04 0.06 MissNum 0.75 0.13 0.06 Gloves 0.17 -0.07 0.55 Boots 0.03 0.03 0.73 Hatchts -0.04 0.00 0.90 1 2 3 1 1 2 0.59 1 3 0.45 0.43 1 CF-Quartimax • • •As can be seen, the pattern of loadings is much simpler and easier to interpret. •Factors are substantially correlated. •Conducting oblique rotation is straightforward in most software. Use it! • Target rotation •There is one rotation that can be used in a more confirmatory manner – the target rotation. • •One can think of the target rotation as standing between exploratory and confirmatory factor analysis. It is useful when you already have some prior knowledge about the factor loading pattern, but not enough to warrant a fully confirmatory model. • •Can be oblique or orthogonal. Target rotation Target rotation •Target matrix, CEFA-style: • • •0 = loading expected to be small •? = unspecified, not small • •The sum of squares of loadings corresponding to the zeros is minimized • • • 1 2 3 WrdMean ? 0 0 SntComp ? 0 0 OddWrds ? 0 0 MxdArit 0 ? 0 Remndrs 0 ? 0 MissNum 0 ? 0 Gloves 0 0 ? Boots 0 0 ? Hatchts 0 0 ? Some final points •Use the CF family, and do oblique rotations. I really don’t see a lot of sense in performing orthogonal rotations. • •Try out multiple oblique rotations – CF-Quartimax, CF-Varimax… • •If you have a bit of an idea on what you expect, you might want to try using (oblique) target rotation. CEFA can do it, and this method is pretty under-utilized in applied research.