Day 6 Table of Contents

• Review topics from last week
• Historical Research
• Qualitative Research
• Data Collection
• Analyzing the data
• Analysis and Interpretation; Cautions
• Closure

Review

Our knowledge of how to conduct educational and training practices comes from experiences, observations, and testing. Although testing provides best empirical results, there are a variety of items to consider.

Test standardization:	Allows for comparison of scores; administrative procedures, scoring, and norms.
Test validity:	The degree to which a test measures what it is suppose to measure: Content, concurrent, construct, and predictive
Test reliability:	How close obtained scores are estimates of true scores. Test-retest, equivalent forms, split-half, rational equivalence, and scorer/rater.
Choosing a test:	Is it valid, reliable, easy to administrate and costly?
Testing mistakes:	Generating test without knowing how, not following procedures, not establishing standards, trying too much.

Historical Research

"Systematic collection and evaluation of data . . . to describe causes, effects, or trends that may help to explain present events and anticipate future events." Gay, p 185.

Historical research not only can tell us about what worked, but what was ineffective or not feasible. Its purpose is to clarify, correct, or expand our existing knowledge.

Although you can't control variable, you can lend objectivity. Problem areas to be concerned with:

• External criticism--how authentic is the material you are examining? Was it really Einstein who said that?
  
• Internal criticism--how accurate is the material? Did he know what he was saying?
  • Was the author knowledgeable?
  • Did a large time elapse between event and recording?
  • Was the author biased?
  • Is there consistency in sources of data?

The report must be more than simply a listing, but an objective synthesis of knowledge developed through logic.

Qualitative Research

"The collection and analysis of extensive narrative date to gain insights into a situation of interest not possible using other types of data." Gay, p 208.

Attempts to ascertain the "why" of a situation

Five general characteristics of qualitative research

• Collected in a "naturalistic" setting
• Data are descriptive
• Emphasis is on a holistic approach
• Inductive analysis (generalization made from collecting multiple specific instances)
• Describe from the perspective of the research participants

Qualitative research data (field notes) focus on:

• Feelings
• Beliefs,
• Meanings people attach to the object under study

Data Collection

Qualitative research generally uses a multi-method approach, studying a few subjects over an extended period of time while collecting data through observations, interviews, and documentation.

Researchers look at events to generate global explanation (the why) to very general questions. Inductive logic.

Initial components of qualitative research:

• Literature review: not extensive so as not to bias the reporter.
• Sampling: small, very select sample for in-depth study
• Instrument: the researcher and the questions asked
  • Validity is how well the observations reflect reality and interviews reflect feelings and what was said.
  • Look for consistency between observations, documents, and interviews.
  • Keep in mind the "prime directive."
  • Observer bias--the ideas we come in with
  • Observer effect--the impact of having someone do a study.
    
• Data collection: Field notes include both what was seen and your reflections of what it all means. Provide a rich description. Not words like "happy."

 Analyzing the data

An iterative process to distill the information.

First get your data organized and check for completeness, then begin to analyze and interpret the data

Reading and memoing

• Reread notes to get an overall view and search for themes and threads

Describing

• Focus on painting a verbal picture of the context, processes, and view from the participant's perspective

Classifying

• Break out the main ideas, classify them, and eventually rebuild into theme and patterns

Interpreting

• Pulling together the important aspects; what is important, why is it important, and what can be learned from it?

Writing the report

• Chose theme of your report, then include those items you've found insightful and appropriate.

Analysis and Interpretation

Qualitative research evolves as it is conducted

• Start the analysis right from the beginning of data collection
• Tweak the instrument as you go to obtain the data you need
• Look for categories or themes
• Include many direct quotes to capture feelings, attitudes, and beliefs

The process is very subjective, and no attempt is usually made to generalize to a larger population.

Cautions:

• Results are difficult to analyze
• The process is very costly in the time spent to do it properly
• Conclusions are tentative, and can't be generalized
• Hard to pull it off and obtain a worthwhile results

Closure; Review and Assignments

Review questions: (To find the answers, click on the question mark icon)

• What is the purpose of historical research?
  When is qualitative research better than quantitative research?
  Which method of data collection for qualitative research do you feel would be best? Why?
  Explain some of the differences between qualitative and quantitative in the research process.

Before next week:

• Read Chapter 8 and pages 431-443 in book