Interviewing is one of the strongest tools in the evaluator’s “tool kit” but it’s a data collection format that’s extremely difficult to master.

The interview allows for the gathering of LARGE amounts of data from a relatively small sample of participants; however, if the interview is NOT WELL-DESIGNED or if the interviewer is UNSKILLED, the data can be biased and/or difficult to interpret.

Why interview?

Interviews provide texture and an almost palpable sense of program reality in which to ground a study.

Interviews offer respondents the chance to …

• speak to a good listener

• reflect on things

• reminisce about past experiences

• be stimulated intellectually by thinking about issues they don’t often consider, and

• express attitudes candidly without fear of the consequences.

When the interviewer does his or her job well, the experience is a pleasant break in the respondent’s daily routine.

Formal v. informal interviews

Informal: Conducted up-front, they help orient the evaluator to the study and to the situation; conducted post hoc, they help the evaluator better understand information gleaned from other sources or other methods.

Formal interviews represent a systematic method for obtaining data.

Formal—open-ended

Open ended interviews start with a list of topics or issues to be covered and a clear sense of what kinds/types of information are being solicited.

The interviewer then segues to the topics in which s/he is interested and, with more or less guidance, allows the respondent to “tell the story.”

S/he may follow up with neutral probes (“… and then what happened?”) until the subject data seem complete.

In essence, each person answers in his own terms and offers information that’s personally salient.

Formal—structured

Here, interview questions are carefully formed and tested, and the instrument is administered in EXACTLY the same way to everyone.

The evaluator chooses from a myriad of question types:

• some questions are short-answer (age, marital status, education level)

• some are closed-ended (people choose from provided sets of response options)

• some are more open narratives.

Interviews (Patton)

Patton uses a three-level classification system:

• the informal conversational interview

• the general interview guide approach

• the standardized protocol

The informal interview relies entirely on the spontaneous generation of questions in the natural flow of an interaction. Typically it occurs as part of ongoing observational fieldwork.

The general guide (what I used with the MIDP evaluation—which I’ll explain in class) really consists of a list of questions or issues to be explored in the course of an interview; it’s topic or subject oriented.

• Issues are identified but not ordered

• Actual wording of questions is not determined in advance

• It’s more open, less formal

• Interviewer has considerable latitude

• Data can be difficult to summarize/interpret

The standardized protocol consists of questions carefully worded and arranged for the purpose of taking each respondent through the same sequence; it is a highly structured approach.

What to ask: a broad range of question “types” are at your disposal…

• Experience[1]

• Opinion-belief questions

• Feeling questions

• Knowledge questions

• Sensory questions

• Background/demographic questions

Interviews: tips

• Training is key to good interviewing

• Multiple interviews must be consistently “administered”/conducted

• Responses must be faithfully recorded

• The interviewer MUST AVOID expressing his or her own opinions to the respondent

• The interviewer must build rapport, the goal being to build confidence, understanding, trust

• The interviewer is honor-bound to assure (and ensure!) confidentiality

• The wise interviewer develops a “schema” for expressing appreciation as people respond to the questions

• The savvy interviewer avoids the appearance of cutting someone off … though he or she may need to “refocus” the direction of the conversation

• It’s good practice to reiterate that there are no right or wrong answers

• Its best to use probes judiciously (and paraphrase to ensure comprehension)

• The overarching idea is to help each respondent see that participation benefits him/her

• It’s important that the interviewer not “tip his hand” … not to suggest that one response is better than another

As your time to collect data draws closer, we can discuss the advantages and disadvantages of interviews conducted by phone, email, live chat, or threaded discussion.

Phone interviews work well if: respondents are scattered across a wide geographic area AND if phone numbers are readily available (and respondents don’t feel that their own confidentiality has been compromised).

Of all the options noted above, it is the least “satisfying.” People must be able to attend well aurally … once distracted, it’s hard to get the conversation back on track!

Weiss (1998) offers a comprehensive set of guidelines you might find useful. She also covers critical components in interviewer training, and introduces the notion of CODING responses (pp. 166-168) -- something we’ll cover a lot on an individual basis.

Interviews: general characteristics

An interview has a beginning, a middle, and an end – and can (unfortunately) “feature” elements or actions that do not facilitate communication:

• advising

• sympathizing

• probing excessively

• withdrawing

• moralizing

• comparing (to others)

• judging

Verbal actions that do facilitate communication include noncommittal “feedback” (in essence, confirmation that ideas have been heard) and segues, motivations, and transitions that move the conversation along (without sounding or appearing overly abrupt).

Nonverbal actions include direct eye contact, facial expressions, and posture/body movements.

Interviews: when to use

• When it’s not necessary that respondents read and write

• When respondents are readily accessible

• When the budget allows (time and money)

• When probing is important (or branching to subquestions)

• When it’s likely respondents will have questions, or when the material under discussion is complex

• When respondents need to be motivated

Interviews: steps

• Develop question matrix

• Determine an appropriate interview structure

• Draft: instructions, questions, anticipated responses

• Field test

• Review

• Conduct interviews

• Follow-up as necessary and appropriate

Interviews: preparing to facilitate

• Introduction must cover key points (not unlike the cover letter (or explanatory text) that accompanies a questionnaire

• Logistics must be determined (length, medium, location)

• Sample size (and selection criteria) must be determined

• Interviewers must be trained

• An interview schedule must be established

• A system for recording responses must be set up

• Strategies for “reinforcing” respondents must be determined – a “gray” area

• A system for coding responses must be in place

• Follow-up procedures must be in place

• Directions must be clear and thorough

• A branching scheme must be in place (so you know where to go next when people don’t respond as “expected”)

FYI: A good interviewer knows how to keep respondents on task, but still allow them to report any other information they deem pertinent.

Interviews: final thoughts...

It’s a bit “old hat” … but Flanagan’s critical incident technique is worth knowing. Here, interviews are “reports” of incidents...generally exemplary and/or substandard performances.

Respondents are experts … and data are collected over time.

Incidents are categorized, and the researcher looks for patterns and trends.

Focus groups … variation on the interview theme/process

Here, a number of topics are explored in depth with somewhere (optimally … and depending on the situation/setting) between 6 to 10 people

Demographic optimals: people do not know each other, but share similar social and economic backgrounds; viewpoints are diverse

Tools: use of an interview guide

Time optimal: 30 to 90 minutes

Setting optimal: comfortable; designed to foster interaction

We follow fairly strict procedures in Ed791. First, the facilitator is always accompanied by a nonparticpatory scribe (whose sole role is to capture data that he or she would otherwise miss). Second, to ensure compliance with Human Subjects/IRB guidelines, we don’t accommodate/allow “observers” who just want to watch (supervisors/managers, (for example). Third, we’re always in control of the raw data (which means no transcripts are provided to the client). And fourth, in a pinch we’re willing to conduct such sessions electronically (I’ll show you examples) to avoid extreme inconvenience.

Focus group interviews are growing in popularity. Your texts have good information regarding the following tenets:

• How to select participants

• How to structure the event

• How to record responses

• How to analyze responses

• How to interpret your “findings”

Many of this semester’s 791BC students plan to conduct focus groups; attend their final presentations (in December) to learn more about their experiences!

---

[1] Questions about what a person does or has done … questions aimed at eliciting descriptions of experiences, behaviors, actions, and activities that would have been observable had the evaluator been present.