Validity Activity #2

Background

Read the following debriefing of a research study, then answer the questions. Take your time, it's a bit tough.

Sankar, C. S., Kramer, S. W., & Hinogorani, K. (1995-96). Teaching real-world issues: Comparison of written versus annotated still image case study. Journal of Educational Technology Systems, 24(1) 31-53.

Abstract:

A well-accepted method of bringing real-world issues to students is the use of case studies. This method was modified in this experiment by bringing a manager to the classroom electronically using picture phone systems. The project manager presented current issues from the project site with the help of still images. Students interacted with the manager through speakerphones. This article compares the effectiveness of the two methods, written case studies and electronically supported case studies, for teaching project management issues. The effectiveness of both methods were evaluated on the four constructs of useful, challenging, clear, and attractive. A survey instrument was administered at the end of each case study and responses were analyzed. The method employing the picture phone was found to be more useful, attractive, and challenging than the written case studies. The students preferred the new method regardless of their GPA or work experience. Written comments, in addition to statistical results, suggest that this method exposed the students to important issues in project management.

Description:

Two methods of bringing real-world issues into the classroom were compared. One method was developed by faculty at the Harvard Business School. It involved the use of written case studies (WCS). The other method took advantage of advances in information technology. The technology case studies (TCS) involved the use of full motion video and picture phones linking the classroom with a project manager in the field.

Hypothesis:

Because there had been no pervious research on this problem the researchers used the following hypothesis:

H1: WCS and TCS would be equally effective in teaching a typical issue faced by a project manager.

H1a: WCS and TCS would be equally useful in understanding a typical issue faced by a project manager.

H1b: TCS would be more challenging than WCS in understanding a typical issue faced by a project manager.

H1c: WCS would be more clear than TCS in teaching a typical issue faced by a project manager.

H1d: TCS would be more attractive than WCS in teaching a typical issue faced by a project manager.

H2: The students GPA (GPA > 3 and GPA < 3) and method (WCS vs. TCS) do interact to affect the means of useful, challenging, clear, and attractive constructs.

H3: The students' work experience (yes and no) and method (WCS vs. TCS) do interact to affect the means of useful, challenging, clear, and attractive constructs.

Procedure:

• Forty-three students participated.
  
• Twenty-five students had a cumulative GPA greater than 3.0, the rest of the students had GPA's lower than 3.0.
  
• Twenty students had related work experience.
  
• Four specific industry problems were brought into class using WCS, four other problems were brought into class using TCS.
  
• A questionnaire was designed to obtain the student's responses after each case discussion. The questionnaire evaluated the four constructs of useful, challenging, clear, and attractive on a 5-point Likert scale.

Analysis Method:

The TCS and WCS were compared using the four constructs of useful, challenging, clear, and attractive. Since the four constructs could be correlated, it was appropriate to use a multivariate test. A multivariate analysis of variance- MANOVA was used to reduce the overall type I error as compared to a series of four univariate tests. MANOVA procedures were run for all the four constructs against the following independent variables:

• the case method used (WCS vs. TCS),
• the interaction between the case method and students' GPA
• the interaction between the case method and students' work experience.

The results from the MANOVA and ANOVA were used to test the Hypotheses 1 through 3.

Activity

Directions:

Read the research summary then answer the following questions.

Variables:

1. Experimental research is the only type of research that can truly test hypotheses concerning cause-and-effect relationships. Briefly describe the cause-and-effect relationship of the research summary.

2. A number of things characterize experimental research; the researcher manipulates at least one variable, controls other relevant variables, and observes the effect in one or more dependent variables.

• Describe the independent variable (including how it was manipulated). Does the independent variable take the form of presence versus absence, presence in varying degrees, or presence of one kind versus presence of another kind?

• Describe the dependent variable(s).

• Describe the variables which were controlled. Did the researcher take into account both subject variables and environmental variables?

   

3. What methods were used to control for differences between the groups?

 

4. Based on the brief description of the experimental design, comment on the researcher's efforts at minimizing possible threats to internal and external validity.

 

5. Given unlimited resources of time and money, suggest some ways to improve the research design.