Internal and External Validity
Lecture, Chapter 8
What is Internal Validity?
ÚThe degree to which an IV actually created change in a DV indicates the amount of internal validity a project holds.
ÚInternal validity asks whether an IV is the only possible explanation of the results shown for a DV.
ÚWith adequate control techniques, an experiment should be free from confounding effect.
Threats to Internal Validity
ÚHistory – events occurring between repeated DV measurements.
ÚMaturation – changes in participants that occur over time during an experiment, including tiredness, boredom, hunger, etc.
ÚTesting – change in DV resulting from its being measured; practice effect.
ÚReactive measures – reactions to the methods of measurement that change the DV.
ÚNonreactive measures – methods that do not produce biased reactions that influence the DV; one-way mirrors, hidden cameras, naturalistic observation, deception, etc.
ÚInstrumentation – changes in measuring the DV due to equipment or human factors.
ÚStatistical regression – the natural tendency of very high or low scores to regress toward the mean during retest.
ÚSelection – choosing participants in such a way that our groups are not equal before the experiment, confusing whether the IV caused the change or not.
ÚMortality – participants’ dropping out; particular damaging when treatment and control groups experience different rates.
ÚInteractions with selection – systematic differences between or among selected treatment groups based on maturation, history, or instrumentation.
ÚDiffusion or imitation of treatment – one treatment group becoming familiar with the treatment of another group and copying that treatment, nullifying control over the IV.
Protecting Internal Validity
ÚImplement control procedures
ÚUse a standard procedure
Correlated groups design
Multiple group design
Ex post facto
What is External Validity?
ÚThe degree to which the experimental results can be applied to populations and situations different from those in the experiment indicates the amount of external validity a project holds.
ÚGeneralization refers to the act of applying results to a different population or situation.
ÚPopulation generalization involves applying results to a different, more encompassing group.
ÚEnvironmental generalization involves applying results to a different situation.
ÚTemporal generalization involves applying results to a different time period.
Methodological Threats to External Validity
ÚInteraction of testing and treatment – when a pretest sensitizes participants to the treatment yet to come.
ÚInteraction of selection and treatment – when a treatment effect is found only with a specific, unique sample of participants.
ÚReactive arrangements – when an experimental situation alters participants’ behavior, regardless of the IV.
Demand characteristics are features of an experiment that inadvertently lead participants to respond in a certain way by indirectly communicating the hypothesis to participants.
ÚMultiple treatment interference – when findings result only when participants experience multiple treatments in the same experiment
Participant Threats to External Validity
ÚComparative psychology – study of behavior in different species; when studying one species and generalizing to another.
ÚConvenience sampling – a researcher’s sampling method based on ease of locating participants; often does not involve true random selection.
ÚSampling only one sex and generalizing findings to both sexes.
ÚSampling only one ethnicity and generalizing findings to all ethnic groups.
ÚCross-cultural psychology – branch of psychology whose goal is to determine universality of research results; when studying one culture and generalizing to another.
ÚEthnocentricity – researchers’ perspsectives that other cultures are merely extensions of their own cultures.
Maximizing External Validity
ÚReplication – an additional scientific study that is conducted in exactly the same manner as the original research project, strengthening the ability to generalize.
ÚReplication with extension – an experiment that seeks to confirm a previous finding but does so in a different setting or with different participants or under different conditions.