Experiments with More Than Two Groups
Experimental Psychology
Lecture, Chapter 11
Building the Experiment
lHow many IV’s? (one IV)
lHow many groups? (3 or more groups, or levels)
lIn a 3+ group design, there may or may not be a control group. For instance,
lYou may measure degree of learning (beyond simply reading the text) in terms of lectures, videos, and no methods (control).
lYou may measure degree of learning in terms of lectures, videos, and demonstrations.
Multiple Group Design
lIndependent Variable: method of treatment for smoking
Assignment to Groups
lHow are participants assigned to groups?
lRandom assignment occurs in independent groups designs, where each participant has equal chance of belonging to any group.
lGives more control over potential extraneous variables with large groups of participants
lCorrelated groups occurs when participants are grouped according to some common variable, then randomly assigned to treatments
lGives more control over potential extraneous variables with smaller groups of participants
lMatched sets (ex. sex, attitude, etc.)
lRepeated measures (generally not recommended with 3+ levels)
lNatural sets (ex. children in same family, rat littermates, etc.)
Multiple-Groups vs. Two Groups
lFor all groups: use the simplest design possible to answer the research question.
lTwo-group design will determine if a difference exists between treatment and no treatment; if yes, a multiple-group design might be the next step.
lIf either is appropriate, consider whether additional group(s) will add important information; if not, no need to complicate.
lThere is no real limit on the number of groups that can be used, but there is a practical limit.
lIn multiple groups designs, you must confirm that you have enough participants for all treatment levels (matched or natural sets) OR that your participants can withstand being tested at all the treatment levels repeated measures.
Comparing Multiple Group Designs
lControl issues
lIf you are concerned about an extraneous variable, and you don’t have enough participants for >10 per group, control through correlation may be needed to assure group equality and thus reduce error.
lEquality of groups.
lThe number of IV levels determines the # of participants needed in each matched or natural set. Since they must be equal, extra valuable participants may have to be eliminated.
l# participants available is an additional factor:
lIf enough to create groups of at least 10 with random assignment, use independent groups.
lIf not, additional control of correlated groups may be necessary.
lRepeated measures involve participants being measured at least 3 times, requiring additional time and multiple trips to the laboratory, leading to possible fatigue factor.
Variations of Multiple-Group Design
lIn comparing differing amounts of an IV, a researcher can examine placebo effect, or an effect caused by expectation.
lMultiple-Group designs can involve the use of already existing IV’s that are simply measured, or ex post facto designs.
Analyzing Multiple-Group Designs
lAnalyzing data from an experimental design with one IV that has 3+ levels involves a One-way ANOVA.
lCompletely randomized one-way ANOVAs are used to measure independently assigned groups.
lRepeated measures one-way ANOVAs are used to measure correlated groups.
lReminder: Each level of the IV must be defined in terms of the operations needed to produce them (ex. “hypnosis” may be defined three 30-minute sessions with a hypnotist).
Rationale of ANOVA
lVariability can be divided into two sources:
lBetween-groups variability is the variability in the DV scores that is due to the effects of the IV.
lWithin-groups variability, or error variability, is that which is due to factors such as individual differences, measurement error, and extraneous variation.
lThe ANOVA statistic is presented in terms of F to indicate the ratio of between groups variability divided by within groups variability.
F = between-groups variability
within-groups variability
One-Way ANOVA for Independent Samples
In a computer analysis, you will receive the following information:
lDescriptive statistics, including # participants, mean, SD, standard error, and confidence interval
lInferential statistics in the form of a source table, which divides or partitions sources (between and within groups) of variation.
lSum of squares – the sum of squared deviations around the mean, which indicates the variability in the DV attributable to that type of variation (again, either between groups or within-groups).
lMean squares – the averaged variability for each source, computed by dividing each source’s sum of squares by its degrees of freedom.
lBetween-groups df = # groups – 1
lWithin-groups df = # participants - #groups
lF ratio – derived by dividing between groups mean squares by within groups mean squares
lProbability level – should be .05 or below to be statistically significant
lPost hoc comparisons, which indicate where (between which groups) the statistically significant differences were found.
lConducts pairwise comparisons (between all sets of two means)
lIndicates level of statistical significance of the comparisons
Effect Size
lRather than the Cohen’s d, which is used for t-tests, the effect size for ANOVA is typically calculated using an “eta squared,” or ŋ2.
lYou can calculate an estimate of an eta squared by hand if you divide between-groups SS by total SS.
lNote that SPSS does provide effect sizes for ANOVAs!
One-Way ANOVA for Correlated Samples
lThe computer print-out will be very similar to that of independent samples with the following exceptions (assuming groups were correlated according to a relevant variable):
lDf will be smaller because individual differences are partitioned out into a separate category labeled “subjects.”
lBetween groups df = k – 1
l(k = # groups)
lSubjects df = n – 1
l(n = # participants per group)
lWithin groups df = (k – 1)(n – 1)
lThe F value will be larger due to reduced within group error.
lProbability for the statistic to be a result of chance only will be smaller.
lProportion of variability accounted for by the IV (ŋ2) will be larger.
lProbability for the post hoc comparisons to be a result of chance will be smaller.
Continuing Research Problem
lAfter preliminary experiment on 2 groups, we decided to test additional groups within the same one IV.
lDefine DV.
lDecide on assignment method.
lIf large # participants, randomly assign participants to treatment groups; analyze using one-way ANOVA for independent samples.
lIf small # participants, correlate participants into matched groups, repeated measures, or natural groups; analyze using one-way ANOVA for correlated groups.
lMake conclusions based on statistical results.