This module provides a very broad overview of data, statistics, instruments for data collection. Statistics is
a powerful "foreign language." Keep the Handout "Statistical Family Tree" handy because it
will be your navigator in the "ocean" of statistics.
The rest of the modules will zoom-in on the major topics of this module and further illustrate how statistics is
used in research, especially in the field of educational technology. You will have opportunities to practice
"speaking" it in class activities and by working on your projects.
Data and Statistics
In summary, data are information and evidence collected in systematic ways. Data can be numerical, narrative, aural,
and visual. Statistics is used to analyze data "in such a way as to obtain a more efficient and comprehensive summary
of the overall results" (Coolidge, 2000, p. 5).
In general, statistics is about the differences between what you expect and what you observe. It compares chance
and random errors with systematic influence to determine the likelihood of an effect that has occurred due to
systematic influence. A difference or relationship is too large to occur by chance is called statistically significantly.
To toss a coin 100 times. How many times would you expect to see a head and how many times to see a tail? The answer
from common sense will be: 50. However, the results rarely turn out as an exact 50-50. How to explain this
variability? Is the variability due to chance or due to other factors (e.g., the skills of the coin tosser)? We
generate a hypothesis and then use statistical tests to test this hypothesis. We follow statistical rules to
determine if we should accept or reject the hypothesis.
The chance-to-influence test is also called signal-to-noise ratio, which is "borrowed from signal detection
theory where the effect of a treatment is considered the signal, and random variation in the numbers is considered
the noise" (Coolidge, 1998, p. 106).
Statistical tests therefore compare chance against other non-chance factors, and conclude if a difference is
statistically significant. It is therefore also a test of significance. For example, employees achieved higher
performance scores in web-based training than those trained in classroom setting. Is this difference due to
chance or due to the invention (the instructional strategies)? A t-test will test the signal-to-noise ratio.
If p(robability) level of the t-test is less than 0.05 (also known as alpha--the level of significance chosen
for this test), we'll conclude that the difference is NOT due to chance. Because the probability of chance effect
on the score differences is very small.
Types of Statistics
There are two types
of statistics: descriptive and inferential. The former are the building blocks for the latter.
Descriptive Statistics
"Descriptive Statistics involve measuring data using graphs, tables, and basic descriptions of numbers such as averages
or means. These universally accepted descriptions of numbers are called parameters" (Coolidge, 2000, p. 5-6).
It describes a sample’s characteristics through the measures of central tendency, variability, and relationship.
When and how to use graphs The "why" of using graphs is to make conclusions and arguments. Read the chapters about
frequency distribution in all textbooks. It is important that you know how to make a distribution graph both by hand
and with software.
Inferential Statistics
Making conclusions about the population (a large group of data) from the sample’s characteristics (a small group of data).
A general formula in using inferential statistics is Fly IDAIR--identify the problem, design the statistical test,
apply the method, infer from the test, and reporting the results.
| Parametric |
Nonparametric |
| Statistical technique used for group comparison when the characteristic being studied (e.g.,
learning outcomes) is normally distributed in the population, sample was randomly selected, and data being analyzed
are interval or ratio (e.g., test scores). |
Statistical techniques used for group comparison the characteristic being studied is not normally
distributed in the population, sample size is small and not randomly selected, and data being analyzed are ordinal
(rank) or nominal (categories). |
| t-test: for dependent samples (same group) and independent samples (two different groups)
Analysis of Variance (ANOVA)
ANCOVA: similar to ANOVA but for controlling the influence of an IV that may vary between groups before
the treatment is implemented.
MANOVA: multivariate ANOVA. Used when there is more than one IV. |
Wilcoxon matches pairs test (t-equivalent): used with dependent samples and ordinal data.
Mann-Whitney U Test (t-equivalent): used with two independent samples and ordinal data.
Friedman Two-Way Analysis of Variance: used with more than two dependent samples and ordinal data.
Kruskal-Wallis One Way Analysis of Variance: used with more than two independent samples and ordinal data.
Chi-Square (for categorical data): used to test the statistical independence of two variables (e.g.,
gender and learning styles). |
Note: t test and ANOVA are the foci of 690. T test is used to test the statistical significance of mean
differences of one or two groups.
ANOVA is similar to t test. But used when you compare more than two groups or have more than one independent
variable.
