Read an Excerpt

The Trainer's Handbook

AMACOM Books

Chapter One

TRAINING is the business of bringing about change. To know whether you have achieved change, you must be able to evaluate the effects of your instruction. You've defined your objectives and determined what the change should be. You've assessed the present status and have a program for changing it to meet those objectives. Now you need a means of measuring the success you've achieved. This last step is evaluation.

There are three levels of change in performance that must be monitored and evaluated. Level one is the measurement of how well the trainees can perform the skills you have been communicating to them. It takes place during and upon completion of training.

Level two is the observation of the trainees' performance when they return to the job. It is a measure of what theorists call the "transfer" of learning to the workplace. You know from level one evaluation how well each of the trainees is able to perform the skills you've taught. Now you must follow up and observe whether or not they are actually transferring those skills to their jobs. If they are not doing so, you will need to perform a narrow-focused needs analysis in order to troubleshoot the problem, diagnose it, and propose a solution.

Level three is a measurement of the impact of the training on the operations of the department for which you have performed the training or, indeed, on the entire organization. It is a measurement of the dollar return on the money investment by management in training. Level three is the bottom-line evaluation. It answers the question "Are we getting our money's worth out of training?"

As you can see, evaluation is an important ongoing function for the trainer. It is also a vital function for the trainees. If you remember, one of Thorndike's principles of learning is the law of effect: nothing succeeds like success. Trainees must get constant feedback to develop the motivation to continue. Constant evaluation not only lets you know where you are, but it also does the same for your trainees. For convenience, the evaluation function has been divided into several operations, but there is considerable overlap and much mixing of technique. It is likely that you will be using several methods at the same time.

Level One: Short-Term Evaluations

Level one evaluation consists mainly of short-term projects with which we are most familiar. Homework assignments, class projects, term papers, and tests are the forms of evaluation we all remember from school. This level of evaluation usually consists of some challenging task set by the instructor and performed by the student. How well the student performs the task is the measure of his or her success at learning it. The task provides the learner with necessary feedback on how well he or she is learning. At the same time, it provides the instructor with feedback on how well the learner is mastering the skills being taught and what coaching, if any, will be required. Incidentally, it also provides feedback on the instructor's success at teaching the material.

Exams and Tests

Many people dread exams because, when they were in school, the results were associated with passing or failing. Since exams were the basis for vital judgments affecting our future, it isn't surprising that exams are thought of as almost punitive to some of us. You will have to reposition the purpose and function of exams to show your trainees how tests let them know how they are doing. Let exams be a service to them, a diagnostic tool to point out strengths and weaknesses. The fact that you can also use them to evaluate yourself is really immaterial. Tests exist solely for them, to provide important feedback. We'll talk about how to structure those tests later in this chapter.

Alternative Types of Evaluation

There are, however, several other types of short-term evaluations. They range from observational techniques, such as eye contact, to various types of projects and reviews. Let's take a quick look at them, then discuss in detail how to create and use each one as an evaluation tool.

Socratic questioning. In Chapter 2, we established the power and teaching value of asking questions. But one of the other major benefits of asking questions is that it lets you monitor the state of mind of the learners and assess the degree of learning taking place. To develop your questioning technique, see Chapter 3.

Methods for Short-Term Evaluations

1. Examinations

2. Socratic questioning

3. Eye contact and observations

4. Spot quizzes and reviews

5. Project Sessions

6. Case histories

7. Practice sessions

8. formal and informal assessment sessions

Eye contact. Also in Chapter 3, we mentioned the importance of eye contact as positive nonverbal communication between instructor and learner. As with most communication, eye contact is a two-way exchange. Contact is initiated and maintained by the instructor, but the learner sends back a message as well. What do your trainees' eyes tell you? Eyes that stare or glare at you are challenging you or disagree with what you say. Eyes that frown are expressing challenge and doubt. Eyes that are glassy and expressionless have had enough. It's time to change the subject. Eyes that shine are challenged and interested, while eyes that droop are sleepy. Ask questions or change the topic. Eyes that blink rapidly or wander about are nervous; the person may be holding something back.

Interpreting Eye Contact

Glare or stare-challenge or disagreement

Frown-doubt or deep thought

Glassy or blank-had enough

Shining eyes-challenged and interested

Droopy or sleepy-tuned out or bored

Blinking or wandering-nervousness or hiding something

Spot quizzes and reviews. By reviewing a topic using a question format, in either a written or oral quiz, you can take the pulse of the group and find out how well they understand the material. Formal testing and how to structure questions for tests are discussed later in this chapter. Project sessions. Assigning work to be done in class allows you to circulate and check their understanding as they work. In a project session, you are looking for how well the trainees can use what you've taught them. You also have the opportunity later on to respond in writing to their projects, expressing your evaluation of them. I usually follow up by discussing the project with the group, using several of their efforts as examples. Case histories. The case history is a more involved, practical project. It challenges the learners to use what they've learned, while it allows you to see how well they are doing. Case histories are discussed in greater detail in Chapter 7. Practice sessions. Hands-on practice serves to lock in learning. It also provides you with an excellent opportunity for evaluation and correction. Assessment sessions. Assessments usually take place at the end of a program. When technical training has been involved, it might be a troubleshooting session in which equipment has been intentionally maladjusted. Trainees are evaluated on their speed and accuracy in correcting the situation. In "soft" skill areas, you can create hypothetical crises and assign members of the class to respond appropriately. Role plays are useful simulations.

Creating and Using Short-Term Evaluation Tools

Let's look more closely at the evaluative tools just cited. Asking questions and using eye contact have been covered in Chapters 3 and 6, so let's begin with quizzes and reviews.

Oral Reviews and Spot Quizzes. Oral reviews are a series of well-framed questions. You want to get the trainees talking about what they've just learned. Don't recite what you've covered-ask them to. I like to ask my classes to highlight what they remember from the previous session. This forces them to review the material and allows me to expand upon key areas, correct misunderstandings, and include material that was inadvertently missed.

Structure the review questions around your objectives. This creates strong redundant patterns. Also, never answer your own questions because that allows the class to stop thinking, and it defeats the purpose of the review.

Don't review large blocks of work at once. Break them down into segments and review each shortly after you finish it. You can always test overall knowledge later. Likewise, keep your reviews short. If you find, however, that the group doesn't know the material, you may have to cover it again by prompting Socratically. Look at it as an effective redundant loop. Lastly, be consistent. Review regularly, and don't miss a chance to review. Reviews not only provide feedback and evaluation of what trainees have learned but give them a perspective on what they will be learning. One way to ensure regular reviews is to make reviewing previously covered material a part of the preparation step of each new unit of instruction. I find that this keeps me in the habit of reviewing regularly.

Written quizzes are similar to oral reviews, but remember that the level of understanding you are testing is simple recall. The quiz should (1) discover what the trainees have grasped, and (2) provide a redundant loop to help lock in learning. Questions should be straightforward, not tricky.

Keep the quiz short. Ten questions are more than enough. Fill-in-the-blank questions or questions calling for one- or two-word answers usually work best. Avoid true-false, multiple-choice, and essay questions; save them for full exams.

Since the real value of a quiz is to let the learners see how they're doing, have them self-score the quiz. Do ask, however, to look over those with more than a couple of wrong answers. You may be able to correct a problem before it escalates.

Alternate quizzes with oral reviews, but be sure to cover each segment of your course with one or the other. In addition, consider using a pretest-posttest format to measure growth and mastery. Administer a quiz at the beginning of the class; most will do poorly. Upon completion of training, give them the identical test; their responses will show a marked improvement.

Project Sessions. As mentioned earlier, project sessions are in-class assignments. For them to work, make the tasks real. There are several ways to do this, but my favorite is to ask participants to bring a critical incident to the training seminar. (A critical incident is an event they recently experienced that was crucial to or had a significant effect on the performance of their job.) If the group is with you for several days, collect the incidents and choose the most germane. Assign them to be worked on the following day, then discuss them.

Developing and Evaluating Project Sessions

1. Make them real.

º Critical incident º Worst-case scenario

º Actual case

º Routine problem

2. Fit the project to the group and the content of the course. 3. Have the correct answers ready, based on the material taught. 4. Make the difficulty realistic for the time allowed. Allow seminar time for at least part of the project. Be available to help and explain. 5. Alternate assignments between individual and group projects. 6. Give each project a written evaluation, with both positive and constructive feedback. 7. Set clear objectives for critiquing and remain consistent. 8. Take up the project with the whole group. Give them feedback.

Project sessions could be based on a real-life situation with which you are familiar or the worst situation you can imagine. A third way is to simply provide routine problems the trainees would handle every day. But whatever project you use, make your sessions fit the group. If they are not training in crisis management, don't give them a crisis to manage. If they are learning routines, give them routines to perform. Have correct answers ready if there is a chance there may be some doubt about the outcome. Use the material you've taught to verify their answers. (This repetition also locks in learning.)

Assign a project that can be completed in the time you allow. The projects should be challenging (nothing is more boring than an easy project), but not unreasonably so. Allow seminar time for at least part of the project to be completed. Homework is good, but practice time at the learning site (with the trainer available) is better.

Projects can be either individual or group; in fact, give them both. An individual project ensures that everyone participates and gets a chance for feedback and evaluation. A group project builds teamwork, and reflects the more realistic working environment. Group activities set up the personal interactions that all of us must cope with every day.

Evaluate each project individually. This is usually done most easily in the evening after the day's training session is over. Give each individual attention and indicate that you have seen the work by making marginal notes, corrections, and responses. Give both positive and negative feedback. If you allow only the negative, you will discourage the learners. If you give only the positive, they will not correct bad habits. I prefer to begin with the positive specifics of what I like about their work and then address the problem areas. (See Figure 5-1.)

Remember also that you are not grading! Make no comparisons. You are providing feedback on how each trainee or group has done. Set in your mind the specific things you will look for, and remain consistent. Usually these things relate directly back to your objectives, which will help you target your criticism. Don't hesitate to correct the work if it appears they don't understand, but be wary of doing it for them. If you feel they should be able to respond, challenge them with your critique. Make them rethink their work and correct the errors. After all, they will be expected to do that in their jobs.

Discuss the projects with the whole group. Explain what you were looking for and show examples of those who did it correctly (use different people each time if possible). Select one or two that weren't up to par and ask the group to explain how they could have been done better (again, not always the same people). This is not as hard on them as it may seem. It provides them with shared constructive feedback, helps to create a climate that allows for errors (see Chapter 1), and reenforces a strong leadership message (see Chapter 3).

Case Histories. A case history is an enlarged project session. Rather than addressing an isolated incident, it encompasses many separate events in realistic complexity.

Continues...

---

Excerpted from The Trainer's Handbook by Garry Mitchell Copyright © 1998 by AMACOM. Excerpted by permission.
All rights reserved. No part of this excerpt may be reproduced or reprinted without permission in writing from the publisher.
Excerpts are provided by Dial-A-Book Inc. solely for the personal use of visitors to this web site.

---