Performance Analysis and Training Needs Assessment


Arguably the most crucial phase of any project, performance analysis and training needs assessment are also among the most ignored or short changed. The reason? Hints lie in the misguided sentiment, "We don't want to study the problem, we want to solve it!" Add this desire to implement a solution quickly to the endemic shortage of resources in any organization and performance analysis and training needs assessment seem like a waste of time and money. 

Such reasoning could not be more false. Training and human resource people in all types of organizations recite epic tales of training programs that didn't solve the problem, instructional products that solved non-problems, or programs that resulted in entirely unexpected and often unpleasant outcomes. The danger of not conducting performance analysis and training needs assessment, for us as educational technologists, is that we are the ones held accountable for both success and failure of our programs and products. Aside from our innate desire for the satisfaction of working on wonderful, effective projects, we have a direct stake in whether people see our work as useful and productive or as a waste of their time and energy.

Performance analysis and training needs assessment help you assure that you design appropriate solutions to real problems and opportunities. They help you avoid creating unnecessary instruction and help you position the instruction you do create in a supportive context.

This chapter is an adaptation and synthesis of many ideas, most notably those of Rossett (1987, 1999) and Mager (1984). You might want to refer to their books for more detailed information on how to conduct performance analysis and needs assessment.


Connect

In this module:
Performance analysis
The Individual Performance Model
The solution system
Performance analysis sources
Performance analysis tools
Performance analysis report
From Performance Analysis to Training Needs Assessment

Performance Analysis
Performance analysts, such as Bob Mager, have developed a variety of heuristics over the years for conducting performance analysis. A heuristic is a set of guidelines, rather than a set procedure. Let's take a closer look at one system and explain the steps.

Describe the opportunity or problem
The first thing that usually happens--the event that triggers the performance analysis--is that the sponsor (a client or someone in your organization) describes an opportunity or problem that they think merits attention. The school superintendent tells you she wants to get all the classrooms "online" as soon as possible. Your boss wants to get TQM (Total Quality Management) going in your organization. Your commanding officer is getting dinged for safety violations, and wants a new safety workshop right away. Your client wants a new multimedia training series for air conditioning technicians.

Notice that these descriptions come in a variety of forms. Some are general descriptions of opportunities or problems, like getting "online" or putting TQM in place. This is a good place to start, because it implies that you will first need to figure out what is needed (performance analysis), recommend an array of interventions or initiatives, and then implement them. If the request contains words like "workshop" or "training," however, it indicates that the sponsor has already decided on the solution. You need to find a way to get them to back up a step or two and analyze the opportunity or problem carefully before spending their money, or political capital, on expensive training.

The first step in analyzing performance problems or opportunities is to find out how the sponsor would like things to be, and then compare that with the way things are now.

Optimals vs. Actuals
There are two ways to approach the comparison of the ideal and current situations. If the sponsor is wanting to take advantage of a new opportunity, install a new system, initiate a new program, it makes sense to begin by getting them to describe what that would look like, ideally. Rossett (1987) calls the characteristics of such an ideal situation the "optimals." If the sponsor wants to fix an existing problem, such as high accident rates or low standardized test scores, however, it often works better to quantify the problem first. In this case, we focus on what Rossett calls the "actuals" first.

Whichever approach you take, eventually you want to get the other side of the story as well. Let's take the example mentioned above, the school superintendent asking you to help her get all the classrooms "online."

You could start by describing the optimals--finding out what "getting all the classrooms online" means. Do they mean the teachers or the students? Are they thinking of one connection per classroom or several? Do they envision modems with phone lines or more "direct" Internet connections? If you're really astute in this type of situation, you'll probe beneath the surface a little. Why do they want to get "online" to begin with? What do they think getting online will accomplish in terms of educational outcomes? Who wants it? Parents? Students? Teachers? The school board? The state? What other school districts are already online? What do they look like? Ask as many questions as you need to try to define as clearly as you can what it is they expect you to make happen.

Because the superintendent is approaching this as a new opportunity, you started by asking them to help you define their vision. At some point, however, you will also want to find out what the current situation is with respect to this vision--the actuals. How many classes are online already? How are teachers and students using their Internet access? Have there been some success stories? Are there problems? What exists now in terms of cabling? How many teachers know how to use the Internet? How many classrooms have computers? These and other questions will help you sketch a picture of where the district is at in terms of getting online.

In the case of the commanding officer who is getting reprimanded for too many safety violations, you would naturally begin by asking her to help you define the actuals. How many violations are we talking about? When did they start going up? What does she think is behind the increase? What are the potential consequences of continued violations? Is this a problem everyone is having, or is it just in this unit? These and other questions will help you sketch a clear picture of the current state of affairs.

But now you want to know exactly what things would look like when you solve the problem. Is there an acceptable rate of safety violations, and, if so, what is it? What does a "safe" unit look like, ideally? Are there some other, similar units with exemplary safety records? The answers to these questions help you define the optimals.


Table 4.1: Information we seek when we do Performance Analysis (according to Rossett, 1999)

Kind of information Description Find it where?
Optimals
(ideal situation)

What we want them to know and do and think about.
Customer/Subject Matter Expert (SME) opinion
Strategic documents
Literature
Observation of model performers
Actuals
(current situation)
What they now do: current performance and efforts. Work products
Feedback from customers
Records of results
Gaps The difference between what we want (optimals) and what we have (actuals). Comparison of optimals and actuals
Prioritization of gaps


When you have a handle on both the current and ideal situations with respect to the opportunity or problem, you're ready to compare them. That's what you do when you determine the gaps.

Describe the difference between the optimals and actuals (gaps)
To the extent that you can quantify the ideal and current situations, take advantage of the numbers. If the superintendent indicates that all 423 district classrooms should have a minimum of 2 direct Internet connections, and you discover that 81 classrooms already have a single connection, you can state that the district's goal is to connect the remaining 342 classrooms and install a minimum of 765 new connections, by such and such a date. That will help people get a handle on the magnitude of the initiative.

In other cases, arithmetic may not be as helpful, or you simply won't know what the numbers are. How will you know when TQM is in place in an organization? You could count a variety of parameters, like the numbers of "quality" interactions, or even look at measures of productivity, but there are probably some factors that elude quantification or are too much trouble to count, such as employee attitudes. In these cases, just state the difference as clearly as you can: "More employees will feel a sense of loyalty to the company product."

Figure 4.1: Finding the gaps (according to Rossett, 1999)


Decide whether or not to do anything
If the difference between the ideal and the current situations is really pretty insignificant or unimportant, you may report that back to the sponsor and recommend that you do nothing at all. Let's say that in investigating your client's request for multimedia training for air conditioning technicians you find that the technicians are already working near the top of any reasonable scale of productivity. The sponsor would gain very little in this regard by providing new training. (There may be other reasons for wanting a new program, of course, such as an anticipated expansion of the air conditioning operation, but let's assume that is not the case.) By pointing up the small difference that exists between the ideal and current state of affairs in the air conditioning operation, you may help the client decide that other initiatives take greater priority.

If there is a significant difference, however, between the ideal and current situations, you still don't know how to bridge the gap. The next stage in the performance analysis heuristic is to pinpoint the performance "barriers" or "drivers," as they are sometimes called.


The Individual Performance Model
Until recently, whenever anyone noticed a problem or wanted to install a new system they usually assumed that the main thing standing in the way was their people's skills and knowledge. They jumped to the conclusion that enhancing or instilling new skills and knowledge was the door to success, and reflexively prescribed training and education as the key to open that door.

Experience taught us, however, that this approach was either far too simplistic or just dead wrong. As educational technologists, we found ourselves hard pressed to help people improve their performance merely by teaching them to understand or do something new. People who know how to do something don't always do it. Why not? There are three additional factors that predict whether people will actually put their skills and knowledge to work. We need to ask, "Are they motivated to do it?" "Does their environment support them doing it?" and "Does their organization support them doing it?" Figure 4.2 shows one way to model these four factors.

Figure 4.2 The Individual Performance Model, showing four factors that promote successful performance.

Motivation
Starting from the center of the model, and, figuratively speaking, from the center of the individual, we're interested in whether people have the motivation to perform. You can probably name a lot of skills you have that you don't do because you don't want to do them. I'm very experienced at mowing lawns, but you won't catch me doing it on a Saturday morning because I don't want to do it.

One way to think about motivation is to use the formula Motivation = Value x Confidence.

If either value or confidence are low, motivation flags. In the example just cited, I'm quite confident about my ability to mow grass, I simply don't value it. I don't get any satisfaction from walking around behind a noisy, polluting internal combustion engine cutting off the tips of little plants. Nor do I value the end result. I prefer a more variegated, natural landscape, full of flowers, trees, and shrubs. Grass doesn't figure in at all. It's a weed, frankly. Confidence is high, but value is low, in my case. You could make me take the best course in lawn care, given by Arnold Palmer himself, and come to my house the next week, you'll still not see me mowing. I'm not motivated.

Similarly, think of an example in your own experience where you value a performance but aren't very confident about being able to do it. Perhaps you dream of buying a home in the country. You dream of log cabins, tall trees, rushing streams, starlit nights, and logs crackling in a fireplace. But you have this wonderful career going in the city, and you know that, for the time being, at least, it's impractical to live very far out. You value rural living highly, but you have very little confidence that you can do anything about it anytime soon. Under these circumstances, your motivation for looking for property in the country is almost nonexistent.

What can you do to help people feel motivated? You can help them value the performance or feel more confident about doing it. Your learners don't think they can really master that multimedia authoring system? Give them a few short exercises they can master quickly to build their confidence. Your students don't care about being able to write complete sentences? Show them some comparisons of life earnings for people who do and do not complete high school. Or, more directly, give them an internship in a career in which they're interested and which involves writing complete sentences.

Skills and knowledge
Moving our attention outward to the next ring of the individual performance model (Figure 4.2), we encounter our old friend skills and knowledge. This is a good time to remind ourselves that, while skills and knowledge aren't the only factors that drive performance, they are certainly vital to the mix. Human beings have been motivated to fly for thousands of years, we just didn't know how until recently. Once we humans understood the basic dynamics of atmospheric lift and developed the skill to take advantage of it, we could make our machines perform every which way--forward, up, down, roll, loop, hover, and so forth.

But even knowing that skills and knowledge are vital to human performance doesn't necessarily indicate training. Training is relatively expensive, when you consider the time it takes to develop instruction, the need to distribute media or--even more costly--send out an instructor, the time away from the job that learners need to master new skills, and so forth. So it pays to find out whether people need training, or merely performance support, to get the job done.

What's the difference between training and performance support? Education and training mean helping performers store new skills and knowledge in long term memory. Information support means providing the appropriate information when and where the performer needs it. Storing stuff in long term memory involves more work--constructing and strengthening cognitive links through strategies like varied representation, multiple examples, and repeated practice. Information support may be as simple as listing the steps of a procedure. Examples of information support include job aids and Electronic Performance Support Systems (EPSS).

Some time ago a student in our educational technology course came to us from a community college where she supervised a photo lab. She was frustrated because the lab users were not following appropriate procedures when using an expensive piece of equipment. It would have been very expensive to train everyone to use it properly--a lot of people used this piece of equipment. It might also have been futile, since any given individual might only use that particular machine once or twice a semester, and, at the time they needed the procedure, have forgotten most of it. Not doing anything about it would have been expensive, too, since improper use was likely to damage the equipment.

Her solution was a simple job aid with some brief instructions and a couple labeled diagrams. She posted the job aid on the machine so that each user would see it when they used that piece of equipment. No one needed to carry the procedure around in their head, because the performance support was available when and where they needed it.

How will you know when information support will do and when training is indicated? Rossett and Gautier-Downes (1991), in their excellent book on job aids, provide a litany of considerations, of which we will mention three of the most important. First, do performers use the knowledge or skill frequently? If they do, then training might be cost effective. If they don't, a job aid may suffice. For example, if you buy a new VCR and need to set the clock only at first and after occasional power failures, you probably don't need training. You can look it up in an informational job aid such as a manual or "quick reference" card. By contrast, if you need to repair transmissions on a daily basis, and get them done in a reasonable amount of time, you might want to get some training.

A second consideration is the importance of automaticity. Jet fighter pilots with an enemy aircraft on their tail can't take time to refer to a job aid on evasive maneuvers. "Let's see, Hector, on page 6 it says to pull up on the stick and…." No. Intensive training, to the point of automaticity, is vital to their survival. On the other hand, when the crew is sitting in their plane on the tarmac performing a preflight check, a job aid makes more sense. Automaticity almost works against them here, since their lives depend on paying conscious attention to each system in the plane, in contrast to the reflexive response needed to evade attackers.

The third consideration has to do with the magnitude of the consequences of the performance. Sending a rocket into space with astronauts aboard has great consequences, both human and monetary. It is unwise to trust such a complex, important procedure to memory. A checklist--in fact a number of checklists--help technicians pay attention to every detail leading up to and immediately following the launch of a space vehicle.

These three factors, and the others mentioned by Rossett and Gautier-Downes (1991), should be considered together. For example, the above-mentioned fighter pilots' lives depend both on the preflight check and on the reflexive evasive maneuvers. On the criteria of magnitude of consequences, both seem to be good candidates for performance support. But in light of the need for automaticity, it's clear that a job aid is more appropriate for the preflight check and training is strongly indicated for the evasive maneuvers.

Environment
Referring back to the individual performance model (Figure 4.2), the next ring out after motivation and skills and knowledge is the performance environment. Suppose we know that our performers are strongly motivated and highly skilled, but still aren't doing the job. We need to ask whether they have the time, tools, space, and processes in place to enable them to perform.

A classic situation occurs frequently in public school settings. Teachers are typically very enthusiastic about new ways to help kids learn. They're very motivated, for example, to use cooperative learning in their classrooms. Some forward-thinking administrator arranges a workshop on cooperative learning and the teachers love it. They leave the workshop fired up and ready to start using cooperative learning as soon as possible. Often, however, they find they don't have the time it takes to carefully plan the new teaching strategy. Moreover, in older schools, classroom chairs are bolted down in rows so the children can't rearrange them in groups. They may or may not have classroom aides or parent volunteers to assist in the process.

These are all environmental issues. The teachers are motivated, they know what to do (following the workshop), but they find themselves in an environment that doesn't lend itself to actually using the new approach.

Administrators can come to the rescue with several interventions in this situation. They can provide release time for teachers to plan the new curriculum. They could coordinate physical plant to get the desks unbolted or provide new chairs and tables. They could help teachers identify and organize parent volunteers, or whatever else it takes to reengineer the classroom management process to support teachers using cooperative learning.

The same interventions work in companies and military units, where job redesign, scheduling, new hardware and software tools, ergonomic and space design considerations, and process reengineering can adjust the work environment to support change.

Incentives
Your manager asks you to help get TQM (Total Quality Management) happening in your organization. On the basis of a quick survey you discover, to your delight, that people are really motivated to do TQM. You get some TQM workshops going, and people get it right away. You work with various managers and supervisors to make sure everyone has the new schedules and that all the environmental supports are in place to make it happen. You're so successful that the Rochester plant asks you to come up for six weeks to do the same thing to them. When you return, however, you discover that TQM has gradually fizzled at your own plant. What went awry?

The organization didn't support the change. You find out that the top managers didn't participate in the TQM workshops and really don't know what it's all about. They didn't talk it up and pat people on the back. In fact, when some of the TQM groups came up with good suggestions, they ignored them. People who kept quiet and just did their job the way they were told were rewarded.

What could the managers have done? They could have participated in the workshops, talked up TQM, rewarded people for actively participating, and stopped rewarding the old ways. The same holds true in schools. Administrators at the highest levels need to learn about cooperative learning themselves, talk it up at staff meetings, celebrate successes in newsletters, send innovative teachers to present their work at conferences, and so forth.

There are several questions you can ask to gauge organizational support. The first one is simply, "Do people know they're supposed to?" That means, has the top level of management actually said, "Yes, we really want you to start doing cooperative learning" or TQM or whatever it may be. If managers and administrators don't say it and keep on saying it, people may feel they're kind of doing this on their own, and that the people "up there" could care less. They won't feel the organization is behind them.

Once you've established that it's behind the effort, you need to ask whether the organization is actually providing support. There are three potential roadblocks here. If managers or administrators are punishing performance, rewarding nonperformance, or not rewarding performance, the performance will gradually (or even suddenly) disappear.

Examples abound. You want teachers to help children develop higher level thinking skills but when they reallocate time away from basic skills instruction you punish them for their kids' lower scores on standardized tests. You want the sales people to use the new laptop computers but this punishes them because it takes time to learn, and during that learning period they'll be able to make fewer sales calls per day, resulting in fewer commissions. Punishing performance is usually inadvertent--that's why it's so difficult to spot.

What about rewarding nonperformance? You want teachers to turn in daily lesson plans to the front office. They make lesson plans anyway, and it benefits the teachers by fostering home-school communication, but for some reason they're still not turning them in. You ask around and discover that they get an extra quarter hour per day by not taking the time to word process the beasts and get them up to the office. To a teacher, that's fifteen minutes they can be talking with parents, working with kids, or doing something they see as really beneficial. It's their reward for not turning in lesson plans.

Not rewarding performance is equally subversive. The mail room clerks increase throughput by 23%, but no one notices, pats them on the back, or mentions them in the company newsletter, let alone gives them a bonus. A teacher tries out a new computer program with her kids. No one drops by the classroom to see how it's going, asks her to relate her experience in staff meeting, or offers to see if they can get her an second computer.

Together, these rewards and punishments make up organizational support. To provide appropriate support may require changing organization policies, revising individual performance appraisal guidelines, or overhauling recognition programs. It may call for shared goal-setting (in schools, site-based management) so that everyone knows what they're supposed to be doing and why, and it almost certainly calls for management development, to get top administrators and managers on board and knowledgeable about the changes.


The solution system

We've presented these four individual performance factors beginning with the individuals own feelings of motivation and their skills and knowledge, and moving out to look at the environment in which they work, and the organizational support they receive. In real life, as an educational technologist, you would want to look at all four factors simultaneously. These drivers are all at work, for better or worse, in every initiative or fix you attempt. Ignoring them is presumptuous and a bad gamble.

One roadblock that often surfaces for experienced human resources development (HRD) folks is that they don't have any control over some of these factors, particularly the environment and organizational support. There are several ways to deal with this. First, you can record your performance analysis data and report it to those who can do something about the environment and organizational support. Second, you can at least make note of the need to address those factors in your performance analysis report so that, later, when the training by itself has "failed" and the managers want to try again, you're in a better position to explain to them why it failed and what they can do about it.

What you strive to provide in your performance analysis report is a comprehensive system of solutions to solve the existing problem or implement the new system or initiative. To get the data you need to describe the ideal and current situations, and the recommendations for providing the performance factors, you need input from a variety of sources.


Performance analysis sources

Sources of performance analysis data in companies usually include managers, supervisors, job incumbents, subject matter experts, customers, and existing records. Educational organizations have the same sources, but they might be called district and site administrators, teachers, and parents and students, in addition to subject matter experts and extant data.

Which of these sources you use for a given performance analysis depends on the situation (compare Table 4.1). Two considerations guide your choice of sources. First, you want sources of data on the ideal situation or goals, on the current situation, and on specific factors that drive performance in this situation. Second, you want multiple sources for each type of data, to "triangulate" or verify the information.

Let's consider a company that designs and manufactures electronic controls for all kinds of industrial and consumer products. The technology is changing very quickly, as is the knowledge about how humans interact with machines. The competition is heating up, and management wants to implement a tighter, more customer-oriented process that integrates sales, design, and fabrication. You've been called in to help design and implement the new system. You immediately recognize the need to define the vision for the new system, and you want to know the current state of affairs as well. Where can you get the data you need?

For the vision, clearly you need to talk with several kinds of people. First, you want to talk with the highest level of management to which you have access. You want to ask them why they think a change is important, whether they have some specific vision of what it should look like in their heads, and perhaps where they got the idea that such a change would benefit the organization. You might also turn to subject matter experts, both within and without the organization, to get a sense of the "state of the art" of this type of process. You can "benchmark" some exemplary organizations and examine the literature on the topic to find out what others are doing (extant data). Customers may give you an idea of what they would look for in an ideal relationship.

To describe the current situation, you may want to get perspective from supervisors and job incumbents on how they currently interact with customers, and perhaps dig out extant data in the form of sales and design records to get a sense of current practice. You may want get customers perspectives on the current state of affairs as well.

To scan the horizon for the best ways to accomplish better client-company interaction, you would again consult experts and managers, but perhaps listen to what the job incumbents and their supervisors have to say as well. You're using more than one source of data to help you get a handle on the current and ideal situations and what to do about the difference between them.

Suppose your school district wants to get the teachers and students using computers for teaching and learning. You're asked to help get the ball rolling. You decide you need to know a little more about what technology in schools should look like, and what it looks like in the district right now.

You probably want to talk with the district superintendent or someone as high up in the district administration as you can access, to get some specifics on what "technology for teaching and learning" means to this district at this time. Then you'll want to read the literature (extant data), talk with school technology and curriculum experts, site administrators, and technology-using teachers to further refine your thinking.

To find out the current state of affairs, you'll want input from teachers, parents, and students. You might want to look at extant data in the form of hardware and software inventories, computer lab use records, and data having to do with program outcomes like grade records and standardized test scores.

Armed with these two visions, you can develop some recommendations about how to proceed based on information from school technology experts, the literature on school technology integration, and the teachers themselves. Again, you've used a variety of sources to triangulate your data.

Performance analysis tools
There are several useful tools when conducting performance analysis. The most common is the simple interview. Talk to people, ask them questions, find out what they know and perhaps what they need to know. Interviews are particularly useful when you're asking open-ended questions and seeking in-depth answers to questions from managers, subject matter experts, and others.

When you've pretty well established the terrain, you may want to turn to surveys. While impersonal and not well suited to obtaining in-depth information, surveys can help confirm what you already know from interviews, and provide hard numbers to help convince others. However, often you don't have the time to engage in lengthy surveys. One way to speed up the process is to distribute your survey by email or to have respondents fill out a form on the web.

Depending on the performance you're attempting to establish or improve, direct observation may be useful. Watching company sales clerks interact with customers, observing teachers, or watching forklift operators unload cargo may give you valuable data you can't get by talking with people.


Performance analysis report

Finally, you're ready to draft your performance analysis report. Whether in a school or a business organization, the report should be concise and to the point. If you want to include additional information, put it in an appendix where an interested reader can find it. Keep the body of the report short and readable. Use graphs, diagrams and bullets to summarize data. Use direct quotes to give it life. The sections of the performance analysis report parallel the process outlined earlier.

Introduction: In the introduction of the report, describe the original opportunity or problem you're charged with assessing. Define the scope and purpose as clearly as possible.

Next, define the vision and the current situation, starting with the former when the desired performance is a new system or initiative, and with the latter in the case of an ongoing problem.

Vision: Set out the goals of the initiative or the fix as your data reveals them. Note your sources, and quote them when appropriate. Try to state them in observable terms (see Chapter 5, Goal Analysis).

Current situation: Report your data on where things stand at present, once again citing data and sources where possible. If things are bad, you may need to protect some of your sources by aggregating the data or keeping sources anonymous.

The differences between the vision and current situation: Don't assume your readers will infer the differences between the vision and the current situation spell them out. In other words, describe and define the gap. Use numbers where possible, and concise, bulleted lists for easy reading and reference.

Factors driving successful performance: Given the current situation, the vision, and the differences between them, report what the experts, literature, managers, and others feel will turn the situation around or make it take root and grow. Address each of the differences in the previous section. Be wary if you find yourself describing only a single driver, such as training. Almost all performance situations hinge on multiple drivers.

Recommendations: Finally, for each performance driver, recommend a range of solutions. Make your suggestions realistic, but don't restrict yourself to the ones over which you have control. If you think the organization needs a new performance appraisal system, put it diplomatically, but say so. Giving decision-makers a range of solutions allows them to pick and choose based on their priorities, instead of having to make a yes or no decision.

Conclusions: Here's where you summarize your recommendations and put them in context. You can point out that, while you will be moving forward to put the requested training program in place, the organization will get greater effectiveness by paying attention to the recommendations as a system, rather than as a collection of isolated interventions.

Appendix: Summarize all your data here, in the form of graphs, diagrams, tables, or lists of answers to specific questions. Include anything a skeptical reader will want to examine.  Better to present the bulk of the data in an appendix and write the body of your report in a way that drives the reader to this additional information.


From Performance Analysis to Training Needs Assessment

Once we know what causes the pain, we are better able to address the problems. After we've conducted a thorough performance analysis, we should know if training would help us with the specific problem that is under examination. Said another way, performance analysis will identify whether lack of skills/knowledge is a root cause of the challenge-at-hand.  If so, what should that training look like? Training needs assessment will help us figure that out.

In many respects, training needs assessment is pretty similar to performance analysis. The major differences are that we only do it when we know that training is part of the solution system and that it is much more elaborated and in-depth. In other words, we should only engage in an extensive training needs assessment when performance analysis gives us the justification for doing so.

Performance analysis helps us determine the causes of the problem or the opportunity. It provides us with answers to questions such as "Where are we going and what is in our way?" The outcomes of performance analysis are a description of the plan of action as well as the data which led us to these conclusions.

When we conduct a training needs assessment we want to know the best ways to accomplish our goals. Our emphasis is now on the optimals and how to reach them. Possible outcomes of a training needs assessment are detailed during the creation of instructional plans or job aids. These items will be brought up in a later module.

Figure 4.3 below describes the relationship between performance analysis and training needs assessment.

Figure 4.3: The relationship between Performance Analysis and Training Needs Assessment
(Note: T&D = Training & Development. CBT/WBT = Computer-based training / Web-based training. Solution partners are colleagues in the organization working in Human Resources, Organizational Development, etc. who need to be considered when the solution system entails more than training).


Apply

Activity 1: Komodo Case Study 

Go to the Resources area in Blackboard and click on the "Komodo Case Study".  Listen to perspectives from the different stakeholders and contemplate what action you would take.  Is training the answer here?

Activity 2: Buffy's Global Gym
This website presents a more in-depth description of the relationship between performance analysis and training needs assessment as well as a worked example. Spend some time with Buffy to become familiar with this subject matter. The website will open in a new window.

Reflect

I've posted a "Name Your Topic" thread in our Discussion area. I encourage those who desire feedback to post a short (3-4 sentences max) description of your topic here. You will want to vet your topic using the Selecting Your Semester Topic job aid first. It's posted under the Assignments section of the course website. While it is not a requirement for you to clear your topic with me before beginning your project, I highly recommend doing so.

Extend

Overview of this section:
People in action
Main points
Next step
For more information

People in action

Barbara has noticed that students coming to her class each year have problems reading directions. This could cause serious problems during science experiments. To get a better handle on the problem, she decided to do a performance analysis.

The sources she contacted to gather information included the science and reading teachers for her students from last year, the principal in the elementary school, three students (a high, middle, and low achiever), a high school science teacher, and two parents. Extant data included reading scores obtained through standardized tests and aptitude test scores for incoming middle school students.

During questioning, she found that all the adults agreed that reading was an important skill, and that at a minimum, all students entering her class should be reading at grade level, with age-level comprehension skills. Test results showed that over 85% of her incoming students were at this level or higher. Teachers from last year said that students were reading at an adequate level when they left fifth grade, yet acknowledged that these students tend to be highly gregarious, impatient, and often don't take the time to think processes through before acting. The students interviewed said that although reading is an important skill, social interactions with their friends is a higher priority. They also said that some of the directions were hard to follow, it is easy to mistake some of the flasks and jars, and that trial and error usually works fine anyway.

The barriers and recommendations that Barbara identified follow.

Barrier to performance Recommendation
Lab directions in the text book are sometimes too lengthy.

Create job aids to walk students through each section of an experiment
or
Provide short instructional sequence for finding critical components in a laboratory exercise.

There is a lack of incentive for students to read carefully. Reward students who complete two experiments without errors by a party after school.
Some equipment is confused if not carefully inspected. Color code equipment that can be easily mistaken and reflect this in the job aid.
Students don't realize the importance of reading carefully. Increase students' motivation, by increasing their perceived value, to read carefully.


Last week Roberto was called in by his manager. "Look at this" she says as she dumps a pile of performance rating forms on the desk. "We've hired some of the best managers and staff in the business, yet they're not filling out these forms correctly. You've got to set up some sort of training program that we can implement throughout the organization to correct these errors. We've got to get our staff to develop a better sense of responsibility."


Main points
1. Although the most critical phase, performance analysis and training needs assessment are the most ignored or short changed.
2. Without a solid performance analysis, solutions may solve non-problems or create programs that are irrelevant.
3. Performance analysis help assure that appropriate solutions are created for real problems and opportunities.
4. The performance analysis will find out how the sponsor would like things to be, and compare that to how things are now.
5. Don't jump to a solution that indicates training or any other intervention until the performance analysis is completed.
6. Start the performance analysis by having the sponsor either describe an ideal setting (optimals), or quantify an existing problem.
7. Next, identify the current conditions (actuals) within the organization.
8. Third, describe the difference (gap) between the ideal and current situation, quantifying the difference if possible.
9. If the difference is not insignificant, the next step is to identify the barriers causing the lack of performance.
10. The barrier is not always lack of skills or knowledge: Three other barriers are lack of motivation, lack of environmental support, and lack of organizational support (incentives).
11. Motivation can be conceived as being equal to the value being placed on the skill or knowledge by the individual, times their confidence in accomplishing the task.
12. Even if skills or knowledge are identified as the barrier, training should not be assumed to be the solution because of the high cost for this activity.
13. Whereas training helps learners store skills and knowledge in long term memory, information support (such as a job aid) provides information to the performer when they need it.
14. Job aids may be an appropriate solution if the task is done infrequently, the performance does not have to be done automatically, or if the consequence of error are high.
15. If a performer's motivation is high, and they have the skills and knowledge yet are not doing the job, examine if they have the necessary environmental support (such as tools, time, space, or processes).
16. The final barrier to successful performance is lack of organizational support; often caused by the administrators punishing performance, rewarding nonperformance, or not rewarding performance.
17. The performance analysis report should suggest recommendations to solve the performance problem by matching solutions to barriers.
18. To get information on the ideal and current situation, and the barriers, you want to talk to a variety of stake holders and triangulate the information to verify the data.
19. In addition to people, existing or extant data may provide information as to the idea and current situation analysis.
20. The performance analysis report should be crisp and concise, and contain an introduction, vision, current situation, identification of differences between vision and current situation, recommendations for a solution, and conclusion.
21. Recommendations which present a range of solutions allow decision makers to select which they feel is most appropriate.
22. The main differences between a performance analysis and a training needs assessment are that you do the latter only when the performance analysis has established training as being part of the solution system and that a training needs assessment is more in-depth and elaborate.
23. With performance analysis you focus on the causes whereas training needs assessment clarifies how to reach the optimals.

Next step
You can see why performance analysis and training needs assessment are so important to educational technologists. Without seeing the whole context, any education and training you develop is doomed to at least partial failure.

In most cases, one of the recommendations you propose to solve a given problem will have something to do with helping people improve their skills and knowledge. If that is the case, then you as an instructional designer are in business. In the next chapter, we'll look at some of the different types of analyses that performance analysis and training needs assessment involve.  Lots of analysis you say?  Perhaps, but as Abraham Lincoln once said, "If I had six hours to chop down a tree, I'd spend the first four hours sharpening the axe."  Analysis is our opportunity to sharpen the solution axe.

For more information
Mager, R. (1984). Analyzing performance problems. Belmont, CA: Lake Publishing Company.

Rossett, A. (1987). Training needs assessment. Englewood Cliffs, NJ: Educational Technology Publications, Inc.

Rossett, A. (1999). First things fast. San Francisco, CA: Jossey/Bass- Pfeiffer.

Rossett, A., & Gautier- Downes, J. (1991). A handbook of job aids. San Diego: Pfeiffer.

 

Introduction
Connect
Apply
Reflect
Extend

Page authors: Bob Hoffman & Donn Ritchie & James Marshall Last updated: Spring, 2006

All rights reserved