### Rmd

#### ANOVA (and R) - The ANOVA Controversy

ANOVA is a statistical process for analysing the amount of variance that is contributed to a sample by different factors. It was initially derived by R. A. Fisher in 1925, for the case of balanced data (equal numbers of observations for each level of a factor).

When data is unbalanced, there are different ways to calculate the sums of squares for ANOVA. There are at least 3 approaches, commonly called Type I, II and III sums of squares (this notation seems to have been introduced into the statistics world from the SAS package but is now widespread). Which type to use has led to an ongoing controversy in the field of statistics (for an overview, see Heer [2]). However, it essentially comes down to testing different hypotheses about the data. Type I, II and III Sums of Squares

Consider a model that includes two factors A and B; there are therefore two main effects, and an interaction, AB. The full model is represented by SS(A, B, AB).

Other models are represented similarly: SS(A, B) indicates the model with no interaction, SS(B, AB) indicates the model that does not account for effects from factor A, and so on.

The influence of particular factors (including interactions) can be tested by examining the differences between models. For example, to determine the presence of an interaction effect, an F-test of the models SS(A, B, AB) and the no-interaction model SS(A, B) would be carried out.

It is convenient to define incremental sums of squares to represent these differences. Let

SS(AB | A, B) = SS(A, B, AB) – SS(A, B)
SS(A | B, AB) = SS(A, B, AB) – SS(B, AB)
SS(B | A, AB) = SS(A, B, AB) – SS(A, AB)
SS(A | B) = SS(A, B) – SS(B)
SS(B | A) = SS(A, B) – SS(A)

The notation shows the incremental differences in sums of squares, for example SS(AB | A, B) represents “the sum of squares for interaction after the main effects”, and SS(A | B) is “the sum of squares for the A main effect after the B main effect and ignoring interactions” [1].

The different types of sums of squares then arise depending on the stage of model reduction at which they are carried out. In particular:

Type I, also called “sequential” sum of squares:

    SS(A) for factor A.

SS(B | A) for factor B.

SS(AB | B, A) for interaction AB.

This tests the main effect of factor A, followed by the main effect of factor B after the main effect of A, followed by the interaction effect AB after the main effects.

Because of the sequential nature and the fact that the two main factors are tested in a particular order, this type of sums of squares will give different results for unbalanced data depending on which main effect is considered first.

For unbalanced data, this approach tests for a difference in the weighted marginal means. In practical terms, this means that the results are dependent on the realized sample sizes, namely the proportions in the particular data set. In other words, it is testing the first factor without controlling for the other factor (for further discussion and a worked example, see Zahn [4]).

Note that this is often not the hypothesis that is of interest when dealing with unbalanced data. 

Type II:

    SS(A | B) for factor A.

SS(B | A) for factor B.

This type tests for each main effect after the other main effect.

Note that no significant interaction is assumed (in other words, you should test for interaction first (SS(AB | A, B)) and only if AB is not significant, continue with the analysis for main effects).

If there is indeed no interaction, then type II is statistically more powerful than type III (see Langsrud [3] for further details).

Computationally, this is equivalent to running a type I analysis with different orders of the factors, and taking the appropriate output (the second, where one main effect is run after the other, in the example above). 

Type III:

    SS(A | B, AB) for factor A.

SS(B | A, AB) for factor B.

This type tests for the presence of a main effect after the other main effect and interaction. This approach is therefore valid in the presence of significant interactions.

However, it is often not interesting to interpret a main effect if interactions are present (generally speaking, if a significant interaction is present, the main effects should not be further analysed).

If the interactions are not significant, type II gives a more powerful test. 

NOTE: when data is balanced, the factors are orthogonal, and types I, II and III all give the same results. Summary: Usually the hypothesis of interest is about the significance of one factor while controlling for the level of the other factors. This equates to using type II or III SS. In general, if there is no significant interaction effect, then type II is more powerful, and follows the principle of marginality. If interaction is present, then type II is inappropriate while type III can still be used, but results need to be interpreted with caution (in the presence of interactions, main effects are rarely interpretable).

#### The anova and aov Functions in R

The anova and aov functions in R implement a sequential sum of squares (type I). As indicated above, for unbalanced data, this rarely tests a hypothesis of interest, since essentially the effect of one factor is calculated based on the varying levels of the other factor. In a practical sense, this means that the results are interpretable only in relation to the particular levels of observations that occur in the (unbalanced) data set. Fortunately, based on the above discussion, it should be clear that it is relatively straightforward to obtain type II SS in R. Type II SS in R

Since type II SS tests each main effect after the other main effects, and assumes no interactions, the correct SS can be obtained using anova() and varying the order of the factors.

For example, consider a data frame (search) for which the response variable is the time that it takes users to find a relevant answer with an information retreival system (time). The user is assigned to one of two experimental search systems on which they run the test (sys). They are also assigned a number of different search queries (topic).

#### To obtain type I SS:

anova(lm(time ~ sys * topic, data=search))

If the data is unbalanced, you will obtain slightly different results if you instead use:

anova(lm(time ~ topic * sys, data=search))

The type II SS is obtained by using the second line of output from each of the above commands (since in type I SS, the second component will be the second factor, after the first factor). That is, you obtain the type II SS results for topic from the first command, and the results for sys from the second.

#### Type III SS in R

This is slightly more involved than the type II results. First, it is necessary to set the contrasts option in R. Because the multi-way ANOVA model is over-parameterised, it is necessary to choose a contrasts setting that sums to zero, otherwise the ANOVA analysis will give incorrect results with respect to the expected hypothesis. (The default contrasts type does not satisfy this requirement.)

options(contrasts = c(“contr.sum”,”contr.poly”))

Next, store the model:

model <- lm(time ~ topic * sys, data=search)

Finally, call the drop1 function on each model component:

drop1(model, .~., test=”F”)

The results give the type III SS, including the p-values from an F-test.

### Type II and III SS Using the car Package

A somewhat easier way to obtain type II and III SS is through the car package. This defines a new function, Anova(), which can calculate type II and III SS directly.

#### Type II, using the same data set defined above:

Anova(lm(time ~ topic * sys, data=search, type=2))

#### Type III:

Anova(lm(time ~ topic * sys, data=search, contrasts=list(topic=contr.sum, sys=contr.sum)), type=3))

NOTE: Again, due to the way in which the SS are calculated when incorporating the interaction effect, for type III you must specify the contrasts option to obtain sensible results (an explanation is given here).

Quelle

If there is a Type I SS and a Type II SS, it begs the question: Might there be more? You’re in luck! In fact, there are four types of sums of squares, each with their associated statistics. These four types, of course, are called Types I, II, III, and IV (Goodnight 1978). Though we are going to use only Type I and Type II SS during this course, here’s a brief description of all four.

#### Type I (sequential or incremental SS)

Type I sums of squares are determined by considering each source (factor) sequentially, in the order they are listed in the model. The Type I SS may not be particularly useful for analyses of unbalanced, multi-way structures but may be useful for balanced data and nested models. Type I SS are also useful for parsimonious polynomial models (i.e. regressions), allowing the simpler components (e.g. linear) to explain as much variation as possible before resorting to models of higher complexity (e.g. quadratic, cubic, etc.). Also, comparing Type I and other types of sums of squares provides some information regarding the magnitude of imbalance in the data. Types II and III SS are also know as partial sums of squares, in which each effect is adjusted for other effects.

#### Type III

Type III is also a partial SS approach, but it’s a little easier to explain than Type II; so we’ll start here. In this model, every effect is adjusted for all other effects. The Type III SS will produce the same SS as a Type I SS for a data set in which the missing data are replaced by the leastsquares estimates of the values. The Type III SS correspond to Yates’ weighted squares of means analysis. One use of this SS is in situations that require a comparison of main effects even in the presence of interactions (something the Type II SS does not do and something, incidentally, that many statisticians say should not be done anyway!). In particular, the main effects A and B are adjusted for the interaction A*B, as long as all these terms are in the model. If the model contains only main effects, then you will find that the Type II and Type III analyses are the same.

#### Type II

Type II partial SS are a little more difficult to understand. Generally, the Type II SS for an effect U, which may be a main effect or interaction, is adjusted for an effect V if and only if V does not contain U. Specifically, for a two-factor structure with interaction, the main effects A and B are not adjusted for the AB interaction because the interaction contains both A and B. Factor A is adjusted for B because the symbol B does not contain A. Similarly, B is adjusted for A. Finally, the AB interaction is adjusted for each of the two main effects because neither main effect contains both A and B. Put another way, the Type II SS are adjusted for all factors that do not contain the complete set of letters in the effect. In some ways, you could think of it as a sequential, partial SS; in that it allows lower-order terms explain as much variation as possible, adjusting for one another, before letting higher-order terms take a crack at it.

#### Type IV

The Type IV functions were designed primarily for situations where there are empty cells, also known as “radical” data loss. The principles underlying the Type IV sums of squares are quite involved and can be discussed only in a framework using the general construction of estimable functions. It should be noted that the Type IV functions are not necessarily unique when there are empty cells but are identical to those provided by Type III when there are no empty cells.

### References

[1] John Fox. “Applied Regression Analysis and Generlized Linear Models”, 2nd ed., Sage, 2008.

[2] David G. Herr. “On the History of ANOVA in Unbalanced, Factorial Designs: The First 30 Years”, The American Statistician, Vol. 40, No. 4, pp. 265-270, 1986.

[3] Oyvind Langsrud. “ANOVA for unbalanced data: Use Type II instead of Type III sums of squares”, Statistics and Computing, Volume 13, Number 2, pp. 163-167, 2003.

[4] Ista Zahn. “Working with unbalanced cell sizes in multiple regression with categorical predictors”, 2009. prometheus.scp.rochester.edu/zlab/sites/default/files/InteractionsAndTypesOfSS.pdf

### Beispiele, Übungen

#### Vollziehen Sie die oben gemachten Ausführungen zu Quadratsummentypen nach an Ihnen bekannten oder zugänglichen Datensätzen,

1. B. an den PACS-Daten oder den Stud-STAI Daten.