Day 13 Table of Contents

• Review topics from last week
• Inferential Statistics
• Standard Error of the Means
• Null Hypothesis and Tests of Significance
• Type I and Type II Errors
• Degrees of Freedom and Tails of a Test.
• Closure

Review

Experimental research is the only research which can test hypotheses for cause-and-effect relationships.

Equate the groups:	Randomization, matching, comparing homogeneous groups, and using analysis of covariance all help minimize differences.
Group designs:	Examine hypothesis to determine which designs are appropriate; then examine which are feasible; then examine which control for sources of invalidity
Pre-experimental:	These studies do little to isolate cause and effect from either no control group or no pretest results.
Quasi-experimental:	Although better than pre-experimental, they do not incorporate randomization to select subjects; conclusions not as strong as experimental.
Experimental:	The strongest results occur from experimental research; randomization of subjects is always included.
Single-subject designs:	Used in behavioral modification with the individual serving as both a control and experimental group.

Inferential Statistics

"Inferential statistics are concerned with determining how likely it is that results based on a sample or samples are the same results that would have been obtained for the entire population." Gay, p 466.

• We usually can't survey an entire population, so we select a sample
  
• A good sample "represents" the population characteristics. However, Not all sampling techniques do this well
  

Since no sample perfectly represents the population, inferential statistics identify:

• How likely it is that results we see in samples represent the results that would occur in the population?
  90%, 95%, or 99% sure?

  Inferential statistics give us probability statements that the results we see in samples would also be found in the population.

  Example 1: How sure are we that the mean age of the members of this class represent the mean age of members of our department?

  Example 2: How sure are we that the differences between two fourth-grade groups using graphic organizers and groups using lists of topics would represent 4th graders from around the country?

Standard Error of the Means

• If our population is defined as all 150 graduate EdTec students at SDSU, would a sample taken from our Educational Research (Ed 690) class have the same mean age as the population?
  
• Would a sample taken from EdTec 540 have the same median age as either our defined population or the Educational Research class?

To understand this, we need to examine sampling error and standard error of the means.

Sampling Error

Sampling error is the expected, chance variation among means. It is expected, and not the fault of the researcher.
When studies are conducted, differences may be
due to either the treatment or the sampling error.

Suppose our population is all graduate students at SDSU. If multiple, same size samples (let's say 18 students) are chosen from this population, a variety of means can be calculated. We would find that:

• These means will be normally distributed (they will vary some because of the sampling error).
• The mean of these sample means will approximate the population mean.
• The SD of these sample means can be calculated (this is called the Standard Error of the Means: SE_x).

Standard Error of the Means - SE_x

This is used to tell us how much we would expect means to differ if we chose a different sample from the same population.

It is possible to estimate the Standard Error with only one sample:

• SD (one sample)/ sq. root (N-1). This helps us conclude that:
  • the larger the N, the smaller the SE_x (another important concept)
  • the smaller SE_x the less the sampling error (which means we can better show treatment effects).

Example:

• Let's suppose we randomly selected a set of graduate students at SDSU and found their ages to be:

  25, 27, 30, 30, 32, 32, 33, 34, 34, 34, 35, 36, 38, 38, 40, 42, 43, 46

  N = 18; Mean = 34.9; SD = 5.5

  SE_x = 5.5/ 4.1= 1.3

  We can be 68% sure the mean age of all graduate students is 34.9 ± 1.3 or

  We can be 95% sure the mean age of all graduate students is 34.9 ± 2.6 or

  We can be 99% sure the mean age of all graduate students is 34.9 ± 3.9

Null Hypothesis

We know that differences between two experimental groups will be due to:

• Treatment (as stated in the research hypothesis), and/or
• The result of chance (sampling error)
  • The chance explanation for the difference is called the Null Hypothesis

How sure can we be that the results are probably due to the treatment?

This is determined, in part, by our setting a Level of Significance: An estimate of the probability that we are wrong when we say there is no difference between the two samples (that the results are due to chance--our null hypothesis).

Common levels of significance (alpha)

• .10 (10%) for an exploratory test
• .05 (5%) for many educational research tests
• .01 (1%) if you are very confident

Level is also influenced by the:

• scale of measurement;
• method of subject selection;
• number of groups; and
• number of independent variables.

Example: There is no difference between groups of 4th graders who use graphic organizers as compared to those who use list of topics in their study of ensuing content.

Alpha = .05 (establishes a 5% chance that if significant differences are found, those differences will be due to chance)

How do we determine a significant difference? That's next week's discussion.

Type I and Type II Errors

Our decision to keep or reject the Null Hypothesis, and the true results being due to chance or treatment, may agree or disagree.

If our decision and the truth match, we made the correct choice.

Type I error occurs when we say results are due to treatment, but the results are really due to chance;

• Null is true, but we reject it.
  

Type II error occurs when we say results are due to chance, but the results are really due to treatment;

• Null is false, but we say it is true.
  

This determination is due to where we set the significance, or probability level (alpha), of the Test of Significance. The alpha level is always set before the experiment.

If alpha set at .10, there is a higher chance of Type I error (it may set here if the consequences of a mistake are low).

Example: Trying to determine if a new technique makes a difference.

If alpha set at .01 or .001, there is a higher chance of Type II error (may set here if consequences of mistake are high).

Example: Trying to determine if a new, expensive program should be adopted.

The rejection of a null hypothesis does not prove the research hypothesis, it only supports the research hypothesis.

See the chocolate example of Type I and Type II errors

Tails of a Test; Degrees of Freedom

The null hypothesis says there "is no difference."

A two-tailed test is a research hypothesis that allows for differences by either group (in either direction).

Example: There is no difference in content acquisition between "discovery learning" and "direct instruction."

A one-tailed test says difference will be in one direction only. This provides an easier chance of finding a significant difference, but difficult to justify before study.

Example: Students who use "discovery learning" do not exhibit greater gains in content acquisition than students who use "direct instruction."

Degrees of Freedom are based on the number in the sample, minus the number of obligated points. All statistics will set degrees of freedom, which are used to calculate levels of statistical significance.

Closure; Review and Assignments

Review questions: (To find the answers, click on the question mark icon)

Explain the concept of standard error.
Describe how sample size affects standard error.
Define or describe the null hypothesis.
State the purpose of a test of significance.
Define or describe Type I and Type II errors.
Define or describe the concept of significance level (probability level).
Describe one-tailed and two-tailed tests.

Before next week:

Read Pages 476-485