Day11 Table of Contents

• Experimental research
• Designing Experimental Research
• Threats to Internal Validity
• Threats to External Validity
• Number Scales
• Closure

Definition and purpose: Experimental research

"In an experimental study, the researcher manipulates at least one independent variable, controls (other) relevant variables, and observes the effect on one or more dependent variables" Gay, p 342.

Independent variable

• The variable the researchers alters
• The cause for the difference
• The treatment
• Examples
  • Type of instruction
  • Type of learning materials
  • Size of group

Dependent variable

• The skill, knowledge, or attitudes which may have changed due to the experiment
• That which depends on the independent variable
• Must be measurable
• Examples of methods to measure the change
  • Comprehension of concepts in Ed690--Final exam test scores
  • Skills gained in EdTec 572--Instructional web pages produced
  • An appreciation for Educational Technology--Instructional attitudes survey

Other variables

• Controlled so as not to interfere with the independent variable

Practice: Identify the following variables as probably being used to measure either the dependent or independent variable

Dependent	Independent	Item
		Problem based learning
		Graduate Record Exam
		Attendance records
		Red Cross Lifesaving course results
		Time on computer in a classroom
		Number of siblings
		Pages of books read in two weeks
		Graphic organizers
		Video conferencing
		Role playing
		Final exam

Designing Experimental Research

Common sequence of events for an experimental research study:

1 State the research problem	Experimental research must be driven by at least one hypothesis that states an expected cause relationship between two variables.
2 Define the experimental group	Experimental group receives the treatment. You will directly manipulate the independent variable
3 Define the control group	Should be as similar as possible to the experimental group except for the independent variable (subject variables) Allows us to see how effective the independent variable really is
4 Administer treatment	Control all extraneous variables that might influence the groups (environmental variables)
5 Collect data on dependent variable	Collect data to determine the effect of the treatment
6 Analyze data	Determine if independent variable made a difference

Threats to Internal Validity

Threats to internal validity occur when collected results are due not to the experiment (manipulation of the independent variable), but may be due to a separate factor.

There are eight major threats to internal validity:

History	An event, external to the experiment, which causes a change in the results
Maturation	A change in results due to the maturation of the subjects
Testing	An improvement in scores due to a pretest
Instrumentation	Results are influenced due to an unreliable, or non consistent, instrument
Statistical regression	Extreme scores will tend to move towards the mean during a subsequent test
Mortality	Subjects who make it to the end of a treatment may be different than those who dropped out
Differential selection	Entire groups are seldom equivalent, and subsequent scores may reflect the initial difference
Selection-maturation, etc., interaction	If entire groups are used, some groups may be affected more than other groups by maturation or other threats

Threats to External Validity

External validity refers to problems of generalizing experimental results to groups who were not engaged in the experiment.

Pretest-Treatment Interaction	A pretest may sensitize subjects to upcoming knowledge, attitudes, or skills. These results will therefore be only generalizable to other groups who go through similar pretests.
Multiple-Treatment Interference	Due to previous treatments and a carry-over effect, groups may behave differently when given a new independent variable.
Selection-Treatment Interaction	Existing, intact groups may not represent the population, and therefore the changes attributed to the treatment may not be found if the population is studied.
Specificity of variables	Results are found to work with a specific population, with a specific treatment, in a specific way. Don't generalize broadly.
Treatment diffusion	This problem can occur when the two different groups are in proximity to one another, and share aspects of the independent variable.
Experimenter effects	We tend to retain our biases, and to relay our expectations to others. To reduce these effects, try not to run your own study.
Reactive Arrangements	The feelings of the subjects influence the results (Hawthorn, John Henry, placebo, and novelty effects are examples).

An easy practice activity identifying threats to validity.

A more difficult research example for your practice.

Number Scales

There are four main number scales used in statistics. Each contains previous characteristics, as well as specific qualities.

Type of Scale	Characteristic
Nominal	Scale is build on specific categories. All subjects can be in only one category.
Ordinal	Scale is built on an order in which all subjects fit, showing relation to other subjects.
Interval	Scale is built with not only order, but with all intervals being equal.
Ratio	Scale has intervals, and a true zero point.

Additional information on number scales, and a practice to see if you can classify examples.

Closure; Review and Assignments

Review questions: (To find the answers, click on the question mark icon)

• What is, and what is the purpose of, experimental research?
  Describe examples of threats to internal validity and explain the limitations if these threats exist.
  Describe examples of threats to external validity and explain the limitations if these threats exist.
  Define the four number scales and classify examples as to which scale they belong.

Before next week:

• Complete Chapter 11

 
 
 
   

    
   


   
   
 
 
    
x
    
 
 
 
 
 
 
 For comments about this page, please contact Julie Gallant