Electronic Performance Support Systems

By Susan Mullen

What is an EPSS? An electronic performance support system (EPSS) may also be known as electronic performance support, automated job aids, electronic performance support tools, electronic coaches, intelligent online help, performance centered design, embedded support, or enriched learning environments. Performance support systems such as job aids have been a matter of importance to performance technology for a long time. The availability of computer technology, the increasing complexity of the workplace, and the need for increased workplace performance to enable global corporate competition are three factors that have lead to an increased concern about how to most effectively design and develop performance support tools in an electronic environment. The purpose of EPSS is to enable workers to perform a task or set of tasks better by providing information and procedures when they're needed, in the worker's natural environment. Providing performance support "just-in-time," ( JIT) obviates the need for the worker to store knowledge; they can access it as needed, and within context. This is not only more convenient than out-of-context training, but may allow better transfer of learning, because workers often learn better in authentic performance settings. An EPSS can be simple, such as a system that provides a template for a person giving telephone rate quotes, or it may be complex, such as a system that supports operators of military communications systems based on expert systems and artificial intelligence. Another example of an EPSS is a software package that lets a homeowner design a backyard deck, then generates a materials list. Gery (1991, in Gustafson, 2000) defined an EPSS as including four elements: an information base, some form of advisor, tutorials, and tools to assist the user. As Gustafson (2000) points out, information bases may now included multiple knowledge sources, expert systems or artificial intelligence may serve as advisors, tutorials may be extensive and contain multimedia, and support tools have become more sophisticated. When to use an EPSS An EPSS boasts a reduced need for training, integration of software into performance, and a change in focus from knowing to performing (Cole, Fischer, and Saltzman, 1997, in Stolovich and Keeps, 1999). An EPSS adapts to the pace of the learner. Because users come and go, an EPSS must accommodate the needs of the novice as well as the more experienced user, without frustrating either audience. Users are coming to expect shallow learning curves and high performance, even among novices (Gery, 1997, in Stolovich and Keeps, 1999). As Gustafson proposes, the relevant question is "what are the respective roles of job design, training, and EPSS and how do they complement each other in promoting high quality performance?" The EPSS must be considered in relation to other interventions, such as training, supervision, and mentoring. Documentation is still scarce about how to create an EPSS, partly due to intellectual capital concerns in the marketplace, and partly because EPSS technology is still immature -- many creative EPSSs are not based on empirical evidence. A review of literature by Reeves (n.d.) reveals some questions to ask when considering whether an EPSS is an appropriate solution to a performance problem: Do performers have easy (better yet, constant) access to computing? Is the task/job that requires support complex enough to warrant an EPSS? Generally, the more infrequently a task is performed, the higher the percent of errors per transaction and relative task completion time (Miller). See Figure 1. How stable is the task/job? (If it changes often, an EPSS may be more appropriate than other approaches that are more difficult to maintain, change, and disseminate.) How critical is the task? What is the cost of non-compliance or poor performance? Is time available for support? (Some tasks are so time-critical that the notion of consulting an electronic performance support system is ludicrous.) Do performers possess the necessary characteristics to use an EPSS in terms of literacy, computer expertise, or motivation? Is turnover among potential users of the EPSS high? (High turnover may often justify an EPSS because an EPSS is often more readily accessible than many forms of training.) Are the logistics of getting people to other approaches (e.g., leader-led training) so complicated or expensive that an EPSS is a more efficient solution? Will the EPSS be used for empowerment of performers or to assure that they comply with specified standards of performance? Is the task frequently repeated? If not, an EPSS may be a sound strategy. Are complex decisions involved in the tasks? Can an EPSS be supported/maintained? How will it be updated? Figure 1: Task frequency error rates and performance time Designing or selecting an EPSS Gustafson outlines seven different design considerations when selecting or creating an EPSS: Black box/glass box objective: do you want the user to know the algorithms underlying the decision support, or simply give them the outcome? Part-task/whole task support: Some tasks are very complex. How much of a series of tasks do you want to support? Embedded/linked/external connection: Designers must decide how support elements will be connected to the job task faced by the user. Self-contained/networked and shared workspace: advancing technology provides the promise of supporting teams through shared workspace, which has unique design considerations. User controlled/system controlled: How and when the support system operates is another design decision. Wizards are an example of user control. Tax preparation software generally uses both types of control. Users can ask for more information, but in some places the application will not advance until correct data is entered by the user. User/Organization modified: who tailors the system? Static/dynamic system: Support systems can "learn" about the user and be modified, but this can have drawbacks for the user as well. The costs and benefits must be weighed. Figure 2: Support for a complex decision algorithm (Miller). How well does an EPSS work? An EPSS is not a magic bullet, and measuring ROI for an EPSS is as important as for other interventions. The cost of an EPSS is primarily incurred during the initial development. While delivery costs may be lower than that of traditional training, it is more difficult to track them. Performance support occurs at the same time as the employee's performance, in contrast to traditional training, which is normally finite and easier to isolate from the cost of performing the work itself. Because most of the cost of an EPSS is up-front (during implementation), the ROI for an EPSS increases the longer the system is used. An EPSS can also eliminate the costs associated with traditional training, such as travel expense and classroom space. Hawkins, Gustafson, and Nielsen (1998) created a web-based EPSS ROI calculator for use by the U.S. Department of Veterans Affairs. Some benefits of an EPSS tracked in an example web-based ROI calculator were: New employees: reduced time required to learn the task reduced supervision hours (answering questions, training the employee, organizing work) reduced help from coworkers reduced calls to help line reduced down time (waiting for help, consulting manuals) Continuing employees: fewer mistakes fewer employees needed reduced overtime Quality benefits: reduced employee turnover reduced grievances reduced absenteeism/tardiness (morale improvements) The sample calculations with data are provided here. Unfortunately, the data are only samples, presumably because the author cannot provide actual government data. The sample numbers are impressive, but not verifiable. How doesn't an EPSS work? The success of an EPSS depends on both technical and organizational factors. The number of users, the availability of domain experts and EPSS implementation specialists, and the appropriateness of the tasks themselves to EPSS support are important factors. To successfully complete an EPSS, skills are needed in the areas of instructional analysis and design, technical analysis, online writing, programming, and graphic art, as well as subject matter expertise. An interesting case study provides many examples of unfortunate unintended consequences of implementing an EPSS, in this case in a hospital setting (Mauldin, 1996). The vendor who created the system did not understand the environment the system would be used in, and they expected the workers to modify their behavior to fit the system, rather than supporting workers in their normal context. Multiple hardware incompatibility issues arose, requiring the hospital to upgrade their hardware and train the users on the new operating system (Windows NT). Clinicians could not go to the computer and wait for test results when urgent medical issues faced them. In addition, the "customizable" system was set up so only one person in the IT department could modify the system. The workload was much bigger than anticipated, so the hospital ended up hiring staff to maintain the new EPSS. A poor interface design resulted in the inability of users to correct errors in data entry. This required support staff to spend a large amount of time checking for errors. Some user groups refused to participate in adopting the system. This meant that, in one example, the software could not create the intended graphs and charts because one department's staff would not supply the missing data that enabled reporting of grouped information. These unfortunate circumstances lead the authors to make the following recommendations when implementing an EPSS: Understand the what the worker is trying to do (what is the workers's job)? Understand the environment in which the system will be used Determine whether a person's behavior has to be radically altered to use the system. Understand the various sources of information entering the system, and enable verification of accurate data, and database maintenance. Carefully consider the maintenance, speed, and archiving of the database, the system's primary engine. Manage expectations about how and when the EPSS will be implemented and the levels of performance that will be available through the various stages of implementation.

References

Author Note

Back to top | Back to Menu