Day 12 Table of Contents

• Controlling for Confounding Variables
• Choosing a Design
• Pre-Experimental Designs
• Quasi-Experimental Designs
• Experimental Designs
• Activity on Experimental Designs
• Single-Subject Design Ideas
• Closure

Controlling For Confounding Variables

"The validity of an experiment is a direct function of the degree to which extraneous variables are controlled." Gay, p 357.

Extraneous variables

• Also known as confounding variables
• May contribute to the results found during an experiment
• They sneak in due to the selection of our subjects, or the way the treatment is implemented
• Although we can't "control" these variables, we can "control for" them with good designs
  

General methods to control for confounding variables

• Use randomization. Although it does not guarantee equal groups, it increases the chances all groups will be similar:
  • Randomly select subjects from the population
  • Randomly assign subjects to groups
  • Randomly assign treatments to groups
    
• Hold constant environmental variables, such as time of exposure, materials, and experience of instructors
  
• Equate groups on critical variables
  • Match participants on variables related to dependent variable, then randomly assign to groups (exclude outliers)
  • Categorize subjects into homogeneous groups, then randomly assign
  • Use Analysis of Covariance--statistically "equates" scores

  
  

Choosing a Design

The design you choose will determine the statistics, threats to validity, and generalizability you can state.

In choosing a design, use the following steps.

1. Identify which is appropriate for your hypothesis
2. Identify which is feasible due to constraints in the environment
3. Identify which controls the most internal and external invalidity

Use the following dichotomous key to determine the type of experimental research. Start at the top, left side.

 
 

Pre-Experimental Designs

Pre-experimental designs are the least valuable because we have little control for confounding variables. Use only as a last resort (and probably not even then).

One-Shot Case Study

• X O (Treatment, then test).
  Almost all forms of invalidity are present (other than pretest threat).

One-Group Pretest-Posttest Design

• O X O (Pretest, treatment, posttest).
  Without a control group, can't tell if results are due to treatment. Only if dependent variable is not likely to change by itself.

Static-Group Comparison

• X₁ O
  X₂ O (Two groups; treatment, then test)

  Comparison group provides some information into effectiveness of treatment, but without random assignment or pretest, results are still questionable.

Quasi-Experimental Designs

Quasi-experimental designs do not use random assignment: Use only when true experimental designs can not be conducted. Try to ensure groups are as similar as possible.

Nonequivalent Control Group Design

• O X₁ O
  O X₂ O (Pretest, Treatment, Posttest)

  Often done in schools because of intact classes. Reduces reactive arrangements of students. Use analysis of covariance to equate scores.

Time-Series Design

• O O O X₁ O O O

O O O X₂ O O O (Multiple pretests, treatment, multiple posttests)

• Multiple tests necessitate that subjects are used to testing. The analysis looks for trends in scores.

Counterbalanced Design

X₁ O X₂ O X₃ O

X₂ O X₃ O X₁ O

X₃ O X₁ O X₂ O (Multiple treatments and tests for all groups)

The number of groups must equal the number of treatments. Employed when pretest is not possible, and intact groups must be used. Exposure to one treatment may influence subsequent treatments.

 

Experimental Designs

Experimental designs provide the best information because each of these use random assignment of subjects to treatment and control groups (and often random selection of subjects).

Pretest-Posttest Control Group

R O X₁ O

R O X₂ O (Random assignment, pretest, treatment, posttest)

• This design controlls for all forms of internal validity. Possible pretest interaction may reduce generalizability to other pretested groups. Gain score differences, or comparison of posttest, is generally used (use analysis of covariance to statistically equate groups).

  A delayed posttest looks for prolonged effects and reduces invalidity of treatment effect.

Posttest-Only Control Group

R X₁ O

R X₂ O (Random assignment, treatment, posttest)

• Used with short studies when subjects have no knowledge related to dependent variable. Mortality may be problem because of lack of knowledge before treatment.

Solomon Four-Group Study

R O X₁ O

R O X₂ O

R X₁ O

R X₂ O (Random assignment, treatment, posttest)

• Has the advantages of both Pretest-Posttest Control Group as well as Pretest-Only Control Group--it reduces all forms of both internal and external validity. Requires twice as many subjects. Not always needed.

Factorial Design

	Factor 2
Factor 1		Level 1	Level 2
	Level 1	Group 1	Group 2
	Level 2	Group 3	Group 4

Factorial design analyzes two or more separate independent variables (one is usually manipulated), and the interaction between those variables.

• The non-manipulated (control) variable is stratified to form the groups (ex. gender, IQ, age).
  
  Drawback is the increased number of subjects. Advantage is identifying potential interactions between the factors.

Now that you know the types of design, test your knowledge with this activity.

Single Subject Designs

Single subject designs are used when there is just one (or only a few) subjects available.

Constraints

• each subject serves as their own control (time series experiment)
• require multiple measurements during each phase
• commonly used in behavior modification research

Rationale

• not ethical to refuse treatment to control group
• size of a population may be too small for experimental research
• important in therapeutic, clinical settings; not for generalizability

Closure; Review and Assignments

Review questions: (To find the answers, click on the question mark icon)

• How do you determine which experimental design is best for your topic?
  Identify three general techniques to reduce threats to validity.
  Describe the main differences between pre-experimental, quasi-experimental, and experimental designs.
  Given a study description, determine the type of experimental design employed by the researchers.
  Under what conditions is single subject design appropriate?

Before next week:

• Read 469-491 in Chapter 13
• Complete Research Plan