• CtrGenPathCurriculum

CtrGenPathCurriculum

Dear Faculty,

The Center for Genomic Pathology (CGP) is now an official nonprofit educational foundation (501c3). We decided that you, the faculty, should have separate governance, the Academy of Genomic Pathology (AGP). Therefore, you are founding members of the Academy of Genomic Pathology. This will free you of the operational details of the foundation but will require that you to make some important organizational decisions. At the present time, we number 20 and can keep things relatively informal. However, additional members will be recruited necessitating consideration of type of organization we want. Do we want to exclusive or inclusive?

The CGP Board of Directors at the first annual meeting on September 24th, asked that I develop a curriculum outline before we request funding. Attached please find an outline. *You are asked to:*

*1) **Comment on the outline. *

*2) **Make suggestions. All suggestions are most welcome. Be creative.*

*3) **Provide educational materials. Several of you have already provided materials. If you have formal PowerPoint Presentations that you are willing to share, we can use them to implement the curriculum. Steve Barthold, Steve Griffey, Rod Bronson and I have committed our lecture presentations. We may have to ask you to reorganize and annotate your .ppt. At this time, we need to identify what materials we have and what will need to be developed.*

*4) **Send links to any and all web sites that have useful educational content. Cory and Jerry have extensive, useful web sites. Please share yours.*

Edit the text below
(Please note the some of the formating did not come out correctly so I am linking the original word document as well.)
Curriculum.doc

• CENTER FOR GENOMIC PATHOLOGY

CURRICULUM

The Center for Genomic Pathology offers an online curriculum designed to serve the needs of the scientific community using general laboratory mice and genetically engineered mice. Our goal is to meet the enormous need for trained manpower expected from the tremendous expansion of mice. We offer training courses for three categories of trainees.

Tier 1: Technical Training: Suitable for all levels but the emphasis is on the training of technical personal who need to understand the animals they are handling and how to help the scientific staff characterize the mice.

The succesfull student should expect to be able to understand and practice good laboratory practice animal hubandry, perform and accurately describe meaningful and thorough necropsies, prepare and process tissues for microscopic examination and correlate microscopic findings to the clinical and gross observations. This includes accurate and thorough record keeping.

Tier 2: Parapathology Training: Designed for the advanced trainee with graduate degrees with exposure to but without professional training in comparative pathology. Ideally, the Tier 2 candidate has an advanced degree in a biological science (Ph.D., M.D. , D.V.M.). The training will prepare the student to recognize and interpret the pathophysiology of the animals that occur in their labororatories or facilities.

The successul student should expect to be able to recognize and interpret the significance of the clinical, gross and microscopic findings in their mice. They should be able to compare and contrast the diseases found in their mice to those found other mice or animals such as humans. They should be able to read, understand, interpret and use the literature on selected subjects. They will be able to recognize when the lesions presented are unusual.

Tier 3: Genomic Pathology: Designed for the advanced Health Sciences student with board eligability or certification in either human or veterinary pathology. The student will learn advanced up-to-date concepts and techniques in mouse biology that will allow them to analyze, evaluate and interpret the usual and the unusual pathological lesions that develop in genetically engineered mice. The Tier 3 student will have a command of the Tier 1 and Tier 2 information and techniques.

The successful student will have the opportunity to attend professional online seminars on advanced subjects, to participate in focused discussion groups, to consult with expert pathologists and to develop specific areas of expertise. The trainees are expected to organize and synthesize professional presentations.

The goals and objective of our curriculum will be accomplished by online lectures, selected reading assignments, audio-visual aids, live and online demonstrations, discussion groups, practice by doing, teaching others. Each task is individualized to meet the trainee’s level of competency, need and background.

The curriculum is scaled so that the Tier 1 assignments are primarily completed through online lectures, reading and demonstrations that can be accessed, studied and reviewed at will. Proficiency assessments are based on written online tests involving identification of and explanation of basic pathological lesions. The trainee is encouraged to interact with the instructors through interactive synchonous and asynchronous internet applications. Online “office hours” will be scheduled. The trainees are encouraged to attend scheduled Tier 2 discussion groups.

The Tier 2 Trainee will be responsible for the broad knowledge foundation of Tier 1. However, the focus will be on the understanding and interpretation of the pathobiology of the mouse. The training will involve more detailed analysis and interpretation of microscopic pathology with correlation with clinical and gross observations. This is accomplished through a series of case-based studies that can be tailored to the specific interests of the trainee. The trainee will be required to participate in discussion groups and live interactive discussions with assigned mentors. Certification will be based on completion of study sets and participation in Group Discussions.

The Tier 3 Trainee will be responsible for the broad Tier 1 knowledge base and have demonstrated proficiency in Tier 2 case studies. Case studies that are more complex will be given to the Tier 2 trainee for consideration and discussion. They will have access to online lectures on advanced subjects in Genomic Pathology by our Academy of Genomic faculty. They will be expected to learn and demonstrate the details of genomic pathology in their field of interest. They will be expected to develop and present a professional level project integrating the genomics and pathology under the guidance of one of our experts.

Feedback and continuing education: Our inline education is designed to be interactive while each task is carried out a task offering hints, feedback, modeling, reminders, etc... The goal is to help the student attain proficiency through iteration. Since education is a life-long adventure, continuing education classes will be offered to our graduates.

Curriculum Topics:

A. Animal Husbandry, Handling, Methods and Procedures. (Tier 1)

1. The origins and evolution of the laboratory mouse
2. Design and Management of
   1. Breeding Facilities
      1. Mouse Reproductive Biology
   2. Research Facilities
3. Mouse Handling and Regulations
   1. GLP
   2. ALAAS
   3. IACUUC
4. Nutrition and environment
5. Health monitoring and Quality Assurance
6. Biomedical and surgical techniques
   1. Small Animal Imaging
   2. Transplantation
   3. Genetic Engineering
   4. Mutagenesis
7. Biostatistics and experimental design
   1. Statistics
   2. Record keeping
   3. Controls
      1. Gender
      2. Age
      3. Parity
      4. Use of historical controls
8. Informatics in mouse biology and research
   1. Basic Computer skills
   2. Computer resources
   3. Using the Web

B. Mouse Biology

1. Biology of mouse strains
   1. Mouse Strain and Genetic Nomenclature
   2. Unique strains
2. Fundamental Mouse Genetics
3. The Mouse Genome
4. Mouse Embryology
   1. Methods for handling
   2. 1. Mouse Immunology
5. Mouse Physiology
6. Mouse Endocrinology
7. Mouse Anatomy
   1. Muscle-skeletal
   2. Internal organs
   3. Special organs
      1. Extraocular lachrymal gland
      2. Harderian gland
      3. Bulbourethral gland
      4. Preputial gland
      5. Zymbal’s gland
   4. Image interpretation
8. Mouse Histology
   1. Recognition of normal organs and cytological variations
   2. Fixation and processing of tissues
   3. Evaluation of quality
      1. Trouble shooting
      2. Correcting

   4. Standard stains H&E

   5. Special stains
   6. IHC
   7. ISH
   8. Confocal
   9. Electron Microscopy
   10. Whole Slide Imaging
   11. Image Analysis
9. Necropsy
   1. Techniques
      1. Necropsy
      2. Sampling and blocking
      3. Infectious disease
   2. Recognition of disease and record keeping
10. Introduction to mouse pathology
    1. Recognition of Gross Lesions
       1. Diffuse
          1. Inflammatory
          2. Growth
          3. Cardiovascular (congestion)
       2. Focal
          1. Inflammatory (abscess, granuloma)
          2. Neoplastic (benign, malignant)
          3. Developmental
          4. Vascular (Infarct)
    2. Recognition of Microscopic Lesions
       1. Diffuse
          1. Inflammatory (Acute, Chronic, Granulomatous)
          2. Vascular (congestion)
          3. Neoplastic (leukemia, lymphoma)
       2. Focal
          1. inflammatory (abscess, granuloma)
          2. Neoplastic (benign, malignant)
          3. Developmental
          4. Vascular (Infarct)
11. Proficiency Demonstrations (Examples):
    1. Name the regulatory agencies.
    2. What is standard chow?
    3. What are the most common mouse strains?
    4. What are the coat colors of 129, FVB?
    5. Identify a granuloma.
    6. What are the characteristics of the three types of inflammation?
    7. Compare and contrast inflammation and repair.
    8. What are the steps in a necropsy?
    9. Explain statistical “Power Analysis”.

C. Mouse General Pathology

1. General Pathology
   1. Background Pathology
   2. Behavior and Gross appearance
   3. Special Pathobiology of the common strains
   4. Intercurrent infections
   5. Nutritional and metabolic disorders
   6. Environment-related disease
   7. Behavioral Disorders
   8. Aging and degenerative disorders
   9. Neoplasms
2. Clinical Pathology
   1. Chemistry-Biochemistry
   2. Hematology
   3. Molecular diagnostics
3. Microbiology and Infectious Diseases
   1. Effects on research
   2. DNA Viruses
   3. RNA Viruses
   4. Retroelements and Retroviruses
   5. Bacterial
   6. Mycoses
   7. Parasitic
4. Advanced Necropsy
   1. Techniques
      1. Necropsy
      2. Sampling and blocking
      3. Infectious disease
   2. Recognition and recording of Disease (What the pathologist needs from you)
   3. Detailed of gross lesion pathology
5. Advanced Microscopic pathology
   1. Histology
   2. Recognition of lesions
   3. Classification of lesions
      1. Disease Ontologies
      2. Disease Classifications
      3. Mapping to human
   4. Description of lesions
6. Toxicology
   1. Genetic toxicology: design and methods
   2. Study design for general toxicology
   3. Study design for specific organ systems
   4. Analytical techniques
   5. Statistical analysis
   6. Pharmacology
   7. Legislative and Regulatory
7. Pharmacology
   1. Principles of Pharm study design
      1. Route of administration
      2. Biotransformation/metabolism
      3. End point
      4. Toxicology
   2. Dose Response
   3. Drug kinetics:PK/DK
8. Proficiency Demonstrations (Examples)
   1. Fill out an Animal Use Protocol form
   2. Explain why the pituitaries are harvested in FVB older than 18 months old.
   3. Given a slide, name the microbial agent
   4. Discuss how retroviruses may influence your interpretation.
   5. Explain the use of Aristolian Ontologies
   6. Compare and contrast granulomas associated with fungi and silicosis
   7. Describe the renal lesions that might be considered evidence of toxicity.
   8. Annotate whole slide image and suggest possible etiologies.
   9. View case study and answer study questions

D. Systems Pathology (Primary Tumor Biology) (Tier 3)

1. Disease Classifications, Ontologies and Controlled Vocabularies (Life in the information age)
2. Nervous System
   1. Central
   2. Peripheral
3. Musculoskeletal
   1. Bones
   2. Joints and synovia
   3. Skeletal Muscle
4. Cardiovascular
   1. Heart
   2. Vessels
5. Oral: Teeth, tongue and salivary glands
6. Gastrointestinal
   1. Stomach
   2. Small intestine
   3. Colon
   4. Liver and biliary system
7. Respiratory
   1. Lung
   2. Airways
   3. Pleura
8. Genital
   1. Male
   2. Female
9. Urinary
   1. Renal
   2. Ureter, bladder and urethra (male/female)
10. Mammary
11. Special Sebaceous glands: Preputial, zymbal, clitoral, anal
12. Skin and Subcutis
13. Hematopoietic
    1. Marrow/spleen
    2. Spleen
    3. Thymus
    4. Lymph Nodes
14. Endocrine
    1. Pituitary
    2. Thyroid
    3. Parathyroid
    4. Adrenal
    5. Pancreas
15. Sensory
    1. Eye
    2. Ear
    3. Vibrissa
16. Proficiency Demonstrations (Examples):
    1. Annotate whole slide images for Liver (or other organ)
    2. Suggest what additional studies should be performed
    3. Compare whole slide image and case history with similar examples in the literature.
    4. Discuss the significance of a given disease with your group
    5. Provide demonstration that places a given case into relationship of pathobiology of the disease.
    6. Provide Prower Point Lecture on Disease of your Choice
    7. Provide a critique of lecture or demonstration during course.

Requirements

1. B.S. or M.S. special interest in pathology.
2. Ph.D. with special qualifications in Comparative Pathology
3. M.D. or Ph.D. Board Qualified or Certified in Veterinary or Medical Pathology

Educational Resources:

Lectures: Annotated or Voice Over Power Point Presentations

Reading: Assigned and suggested

Audio-Visual: On line programs and applications

Virtual necropsy: http://tvmouse.compmed.ucdavis.edu/virtualNecropsy/

ICIARC, ICMIAD:

http://medocs.ucdavis.edu/PMD/WWWICIARC/WWW-ICIARC.html

Demonstration: Expert demonstrates case study

Discussion Group: Trainees meet and discuss case study and other assignments

Practice by Doing: Trainee annotates Whole Slide Images and provides explanation to instructor. The Trainee annotates and demonstrates their own cases.

Teach Others: Trainee gives seminar (thesis defense).

Competencies:

Background Information-Related Knowledge

Vocabulary

• Medical Histology/Anatomy Pathology Mouse nomenclature Infectious Disease

Skills

• Identification of normal organs and tissues Identification of diseased organs and tissues Identification of Pathological Processes Identification of Microorganisms

Conceptual Knowledge

• Provide a diagnostic term for the pathological process Discuss the relation of the process to the diagnostic term Discuss the etiology of the disease process

Discuss the relationship of the disease with other diseases in the current case or other cases.

Procedural Knowledge

• Perform a necropsy Sample and trim samples

Describe and annotate a slide, naming organs, primary disease and other secondary conditions.

• Find the literature concerning a disease entity Compare and contrast the histopathology with other cases. Provide a discussion of the significance of the lesion.

Simulated Reality

• Correctly annotate a series of slides. Propose potential follow up stains and experiments Prepare a presentation of your findings

Identical Elements

Educational Procedures:

• Reference Materials

Cognitive Apprenticeship

Cognitive Apprenticeship is a method of teaching aimed primarily at teaching the processes that experts use to handle complex tasks. The focus of this learning-through-guided-experience is on cognitive and metacognitive skills, rather than on the physical skills and processes of traditional apprenticeships. Applying apprenticeship methods to largely cognitive skills requires the externalization of processes that are usually carried out internally. Observing the processes by which an expert listener or reader thinks and practices these skills can teach students to learn on their own more skillfully (Collins, Brown, Newman, 1989, p. 457-548). This method includes:

1. Modeling -- involves an expert's carrying out a task so that student can observe and build a conceptual model of the processes that are required to accomplish the task. For example, a teacher might model the reading process by reading aloud in one voice, while verbalizing her thought processes (summarize what she just read, what she thinks might happen next) in another voice.
2. Coaching - consists of observing students while they carry out a task and offering hints, feedback, modeling, reminders, etc.
3. Articulation - includes any method of getting students to articulate their knowledge, reasoning, or problem-solving processes.
4. Reflection - enables students to compare their own problem-solving processes with those of an expert or another student.
5. Exploration - involves pushing students into a mode of problem solving on their own. Forcing them to do exploration is critical, if they are to learn how to frame questions or problems that are interesting and that they can solve (Collins, Brown, Newman, 1989, 481-482).

Lecture, Reading, audio-visual, demonstration, discussion group, practice by doing, teach others.

For more information:

• Bloom's Taxonomy

Model Questions and Key Words

Based on Bloom's Taxonomy, Developed and Expanded by John Maynard

||<width="46px"> ||<width="259px">implicit in the statement is ||<width="260px">

• ||

(1414) (Close this window to return to the Learning Resources Page

Classroom training, job aids,

e-learning, mentoring, just-in-time

training, after-work educational

opportunities, knowledge manage-

ment, and so forth

Performance Capacity

Recruitment programs, retention

programs, resource allocations,

workforce planning, new computer

technologies, and so forth

Expectations & Feedback

Communication opportunities at

retreats and roundtables, perform-

ance reviews, balanced scorecards,

participation in strategic planning,

and so forth

Tools, Environment, &

Processes

Computer systems, workplace

redesign, process reengineering,

ergonomics review, communications,

and so forth

Rewards, Recognitions, &

Incentives

Awards program, communications,

monetary incentives, performance

reviews, balanced scorecards, and

so forth

Motivation & Self-Concept

Mentoring, career counseling, moti-

vation workshops, team-building pro-

grams, performance appraisals, and

so forth

Source: Figure is based on similar graphic in Watkins, 2007; Watkins & Wedman, 2003; Wedman & Graham, 1998.

14

www.ispi.org

•

DOI: 10.1002/pfi

•

FEBRUARY 2007

technologies. Share your identification and evaluation

processes, along with the criteria established through the

performance objectives, with each of your partners. Next,

encourage them to champion aspects of the initiative that

affect their division or organization. They can then help

guide the individual interventions to the achievement of

useful results.

Finally, before you begin to design, develop, and imple-

ment any performance technologies,you should again con-

firm that the selected mix of performance technologies is

adequately aligned with the desired results of your organi-

zation and its partners. Verify that each performance tech-

nology is clearly linked to the results of the performance

analysis and the performance objectives established at the

individual or team, organizational, and societal levels. You

can use a simple table to make sure that each selected per-

formance technology will make a valuable contribution to

the performance improvement system (see Table 3).

PERFORMANCE IMPROVEMENT

SYSTEMS

Performance objectives define the results you wish to

accomplish based on your analysis of the strategic ambi-

tions of your organization and its partners. Accomplish-

ing these results is then left to the set of performance

technologies that you select for implementation. This set

may be a combination of performance appraisals and

coaching for some performance objectives and a combi-

nation of training, new equipment, and incentives for

others. No matter which performance technologies are

included in your final set of solutions, it is your responsi-

bility to ensure they work together to achieve the desired

results of your organization and its partners.

The development of a set of performance improve-

ment technologies involves not only the selection of

appropriate single interventions but also the development

of a comprehensive system. As you move ahead with the

design, development, and implementation of the multiple

performance technologies in your system, you should

work to avoid suboptimization (that is, the improvement

of performance in one subsystem at the expense of per-

formance in other subsystems or in the system as a

whole). At each step on the path toward improved per-

formance, verify that the decisions you make are develop-

ing a comprehensive system that will accomplish desired

results.

Performance improvement systems are only as strong

as their weakest performance technology. You would not,

for instance, want to develop costly training and perform-

ance appraisal technologies only to learn later that reach-

ing the desired objectives also requires an intervention

aimed at performance capacity (such as an updated

recruitment program or new resource allocation plan).

Developing a complete performance system requires that

you continually review your decisions from multiple per-

spectives and apply systemic tools for selecting appropri-

ate technology solutions.

NEXT STEPS TO SUCCESS

The performance technologies selected for the perform-

ance improvement system in your organization must be

aligned with those factors known to contribute to

achievement of useful results. The performance analysis

provides a systematic process that can both define the

necessary results and guide your search for potential per-

formance technologies. From balanced scorecards and

TABLE 3

VERIFY SELECTED PERFORMANCE TECHNOLOGIES ALIGN WITH

PERFORMANCE OBJECTIVES

Objective A

Competence: knowledge and skill

a. E-learning

b. Job aids

c. After-work educational programs

Expectations and feedback

a. Balanced scorecard initiative

b. Quarterly performance reviews

Motivation and self-concept

a. Mentoring

b. Career counseling

PERFORMANCE OBJECTIVES

CONTRIBUTING FACTORS

TO SUCCESS

SELECTED PERFORMANCE

TECHNOLOGIES

Source: Based on Watkins, 2007.

Performance Improvement

•

Volume 46

•

Number 2

•

DOI: 10.1002/pfi

15

coaching programs to training and job aids, the perform-

ance technologies likely to produce useful results can then

be evaluated, compared, and prioritized using grounded

criteria and standards established for each performance

objective. This systematic process for identifying options

and evaluating their value is an essential step to the

achievement of useful results.

The selection of performance technologies based on

assumptions, organizational politics, or someone’s

favorite solutions is never advisable. These and other

nonsystematic processes reduce your chances of accom-

plishing desired results on a consistent basis. In contrast,

the findings of a systematic selection process can align

technologies with performance objectives, justify your

decisions, and improve performance (Watkins, 2007).

Although selecting the appropriate performance tech-

nologies is a critical step toward accomplishing desired

results, the selected technologies still have to be designed,

developed, tested, and implemented for desired results to

be achieved.You will also want to use systematic process-

es for designing and developing each of the performance

technologies in your system. Managing the development

of a performance system, therefore, requires skillful coor-

dination and leadership to create a synergistic system

where desired results are achieved effectively and effi-

ciently. In the third, and final, article in this series on

designing for performance, you will find guides for pro-

ducing performance technologies that accomplish results

in your organization.

References

Gilbert, T., & Gilbert, M. (1989). Performance engineering:

Making human productivity a science. Performance and

Instruction, 28(1), 3–9.

Harless, J.H. (1975). An ounce of analysis is worth a pound of

objectives. Newnan, GA: Harless Performance Guild.

Watkins, R. (2007). Performance by design: The systematic selec-

tion, design, and development of performance technologies that

accomplish useful results. Amherst, MA: HRD Press.

Watkins, R., & Leigh, D. (2001). Performance improvement:

More than bettering the here and now. Performance

Improvement, 40(8), 10–15.

Watkins, R., & Wedman, J. (2003). A process for aligning per-

formance improvement resources and strategies. Performance

Improvement, 42(7), 9–17.

Wedman, J.F., & Graham, S.W. (1998). Introducing the concept

of performance support using the performance pyramid.

Journal of Continuing Higher Education, 46(3), 8–20.

Related Readings

Jonassen, D.H., Tessmer, M., & Hannum, W.H. (1999). Task

analysis methods for instructional design. Mahwah, NJ:

Erlbaum.

Kaufman, R. (2006). Change, choices, and consequences: A guide

to mega thinking and planning. Amherst, MA: HRD Press.

Kaufman, R., Oakley-Brown, H., Watkins, R., & Leigh, D.

(2003). Strategic planning for success: Aligning people, perform-

ance, and payoffs. San Francisco: Jossey-Bass.

Mager, B., & Pipe, R. (1997). Analyzing performance problems

(3rd ed.). Atlanta, GA: The Center for Effective Performance.

Rossett, A. (1999). First things fast. San Francisco: Jossey-Bass.

Stolovitch, H. (2002). Front-end analysis, implementation

planning, and evaluation: Breaking out of the Pamela syn-

drome. Performance Improvement, 41(4), 5–7.

Stolovitch, H. (2006). Three performance principles to keep

you on target. Workforce Performance Solutions, 2(5), 16.

Watkins, R. (2006). Aligning performance technologies with

organizational strategic plans. In J. Pershing (Ed.), The hand-

book of human performance technology (3rd ed., pp. 191–207).

San Francisco: Jossey-Bass/Pfeiffer.

Wedman, J.F., & Diggs, L.L. (2001). Identifying barriers to

technology-enhanced learning environments in teacher educa-

tion. Computers in Human Behavior, 17(4), 421–430.

RYAN WATKINS, PhD, author of Performance by Design: The Systematic Selection, Design, and

Development of Performance Technologies That Accomplish Useful Results, is associate professor of

educational technology at the George Washington University in Washington, DC. Ryan is also the

author of four other books, including the best-selling 75 E-Learning Activities: Making Online Courses

More Interactive, and the E-Learning Companion: A Student’s Guide to Online Success. In addition,

he is the author of more than 60 articles and chapters. Recently Ryan was a visiting scientist with

National Science Foundation, and he has been an active member of ISPI for the past decade. Ryan

may be reached at www.ryanwatkins.com or email him at rwatkins@gwu.edu .

42

Performance Improvement, vol. 46, no. 4, April 2007

©

2007 International Society for Performance Improvement

Published online in Wiley InterScience (www.interscience.wiley.com)

•

DOI: 10.1002/pfi.124

DESIGNING FOR PERFORMANCE, PART 3:

DESIGN, DEVELOP, AND IMPROVE

Ryan Watkins

In this final of three articles on how to design for performance, you will find practical

steps and useful tools for developing the performance technologies that make up a

performance improvement system. Use these systemic and systematic processes as the

starting place in creating performance technologies that achieve the strategic performance

objectives of your organization and its partners. Then improve on these processes by

customizing them for the specialized Human Performance Technology (HPT) solutions you

have selected for your performance improvement efforts.

THE ACCOMPLISHMENT OF VALUABLE results rarely

occurs by chance. Rather, desired results are best achieved

through the systemic and systematic design and develop-

ment of comprehensive performance improvement sys-

tems. These systems use multiple performance

technologies (such as Six Sigma, balanced scorecards,

retention programs, coaching, training, motivational

seminars, strategic planning, reengineering, and elec-

tronic performance support) to accomplish desired

results. Going beyond quick fixes or narrowly focused

efforts, the design of performance systems enables indi-

viduals and organizations to achieve strategic ambitions

through processes that are both systemic and systematic.

The first article of this three-part series,“Designing for

Performance: Aligning Your HPT Decisions from Top to

Bottom” (Watkins, 2007a), provided guidance for identi-

fying the strategic objectives of your organization and its

partners as the starting place for defining what results

should be achieved through any performance improve-

ment efforts. These strategic ambitions were then trans-

lated into explicit performance objectives that could be

used to guide decision making throughout the selection,

design, and development of an improvement system. In

the second article, “Designing for Performance: Selecting

Your Performance Technologies” (Watkins, 2007b),

processes were offered for developing performance assess-

ments that could be used to evaluate the alternative per-

formance technologies available to today’s organizations.

In addition, alternative performance technologies were

identified for seven interdependent components of

a comprehensive system for improving performance

(Watkins, 2007b, 2007c). Guidance was provided for

selecting the right combination of performance technolo-

gies for accomplishing strategic objectives.

This final article in the series focuses on the design,

development, and implementation of the performance

technologies that have been evaluated and selected to

improve performance. Building on the foundational

products of the first and second articles, this article offers

processes, tools, and techniques for producing perform-

ance technologies that are capable of accomplishing valu-

able organizational results.

DESIGNING PERFORMANCE

TECHNOLOGIES

The selection of performance technologies such as mentor-

ing, recruitment programs, e-learning, leadership retreats,

coaching, and workplace redesign provides a blueprint for

implementing improvements within your organization.

Although this blueprint can provide an essential map of

the relationships between selected technologies and the

desired results, you have to design and develop, and later

implement and improve, performance interventions for

useful results to be achieved. From executive coaching and

process reengineering to job aids and performance

appraisals,the multiple performance technologies included

in your performance improvement system will require

varying design and development processes. For instance,

some technologies may require the creation of software

support tools,while others may rely on the approval of new

Performance Improvement

•

Volume 46

•

Number 4

•

DOI: 10.1002/pfi

43

organizational policies. Thus, the creation of each per-

formance technology will involve a different series of steps

as it moves from design and development to implementa-

tion and continual improvement.

Just as no single solution will improve performance in

all organizations, no one set of performance technologies

is the right choice for all organizations. For that reason, as

a Human Performance Technology (HPT) professional,

you should use a mix of performance technologies that

address the complex performance discrepancies unique

to your organization. These should also take into account

the current processes of your organization, the contribut-

ing factors, the organization’s culture, the people who will

be asked to change, and the input of external partners and

clients.

Most of us enter any performance improvement process

with primary experience and skills related to the design

and development of a specific performance technology.

Whether they are instructional design processes for pro-

ducing training interventions, multimedia development

procedures for creating electronic job aids, or steps for

establishing a comprehensive mentoring program, the

majority of us are most familiar and experienced with the

design and development processes of one or two perform-

ance technologies at most (and these may or may not be

the selected performance technologies for this improve-

ment effort). Generalizing these systemic and systematic

processes is, therefore, a valuable first step toward success.

Many analogous processes can be applied across the

design and development efforts of various performance

technologies.Comparable team structures,equivalent roles

and responsibilities, corresponding timelines, and similar

formative evaluation requirements can help support the

systematic creation of a complete performance improve-

ment system. In addition, many generalized project man-

agement principles can help guide you toward success.

The tasks associated with designing and developing

performance solutions build on the results of the previous

processes. Findings from your needs assessment and

SWOT analysis, for instance, will help guide you in mak-

ing useful performance improvement decisions through-

out the design and development process.The performance

objectives and assessments that you identified prior to

selecting performance technologies will also provide

guidance throughout implementation and formative eval-

uations. You can improve efficiencies within your per-

formance improvement system by capitalizing on the

common components found among diverse performance

technologies (see Table 1). For example, if each of three

performance technologies requires the input of expert

performers from the sales division, then all three design

teams can work together to share data from their inter-

views and performance observations with these experts.

Planning for the coordination of the multiple per-

formance technology development processes within your

set of solutions is a priority in generating a comprehen-

sive and synergistic performance improvement initiative.

Maintain a systemic perspective throughout the design

and development of selected performance technologies.

This coordination of performance technologies ensures

that all strategic performance objectives are addressed, all

contributing factors are responded to, and all technology

interventions capitalize on the efforts of other improve-

ment efforts to maximize efficiency and effectiveness

(Watkins, 2007c).

Work to avoid “random acts of improvement” when

designing and developing the performance technologies.

The sustained success of a performance improvement ini-

tiative hinges on the capacity of distinct performance

technologies to achieve desired results while supporting

the accomplishments of other performance technologies

in the performance system. Therefore, use systemic and

systematic processes to develop each technology interven-

tion, producing consistent and sustainable results that are

aligned with strategic performance objectives.

MANAGING THE DEVELOPMENT

PROCESSES

Most interventions selected to improve performance

will necessitate changes for both the organization and

those who work within it (see Lick and Kaufman, 2000).

As a consequence, develop a management plan that

aligns information collected during the planning stages

of the improvement initiative to the implementation

requirements of the individuals, small groups, teams,

divisions, and organizations tasked with accomplishing

useful results. This comprehensive management plan

can then address the integration of the collaborative

performance technologies, the procedures for develop-

ing each distinct technology, and the successful imple-

mentation of the technologies in your organization.

Successful performance improvement initiatives rely on

the support and commitment of key individuals within an

organization as well as the valuable backing of external

partners. Organize and develop the support you require

for accomplishing useful results as an ongoing responsibil-

ity (see Table 2). This includes not only developing appro-

priate performance technologies but also creating the

infrastructure capable of sustaining (and continually

improving) the performance initiative. Leading change in

your organization is a role you must take on when you

develop a performance improvement system.

The design and development of performance technolo-

gies routinely requires specific procedures and processes

that are unique to each organization. For instance, in one

organization, the development of an interactive e-learning

course may require a task analysis, while another organiza-

tion may find that earlier task analyses provide the neces-

sary information for its development process.Yet while the

specific steps in creating performance technologies may

require distinctive applications within an organization,

there are generalizable development processes that can be

applied across multiple technology development projects.

For example, the ADDIE (analyze, design, develop, imple-

ment, and evaluate) process that is applied in many

instructional design projects can also be applied in its

generic form to the development of many other perform-

ance technologies (see Table 3).

Create a design and development plan for each perform-

ance technology selected for implementation. It is often

helpful to involve both internal and external partners in all

steps of creating the plans for each performance technology

to learn from their experience, expand their knowledge of

the performance improvement system, and gain their sup-

port for the specific performance technology.

FORMATIVE EVALUATIONS

Include in the development of all performance technolo-

gies time and resources for formative evaluations (see

Table 4). Formative evaluations provide multiple oppor-

tunities for an intervention’s deliverables to be examined

by future users, demonstrated for experts, and pilot-

tested in the performance environment. It is, therefore,

valuable to plan for ongoing formative evaluations

throughout the creation of each performance technology.

Going beyond the interim revisions that are characteristic

of most any project, formative evaluations are done to

identify specific design and development considerations.

These considerations can add to the quality of perform-

ance technologies and ensure that the performance

improvement initiatives focus on accomplishments.

Be sure to incorporate formative evaluations into your

performance interventions prior to implementation. Key

roles for formative evaluations in any performance

improvement initiative include receiving feedback, docu-

44

www.ispi.org

•

DOI: 10.1002/pfi

•

APRIL 2007

STEPS IN PLANNING FOR THE DESIGN AND DEVELOPMENT OF

TABLE 1

PERFORMANCE TECHNOLOGIES

STEPS TOWARD

SUCCESS

BRIEF DESCRIPTION

Step 1

Begin by creating a visual representation (or map) of multiple performance interventions and their associated

Verify alignment of

performance objectives. Use this to verify that the selected performance interventions will work together to

selected performance

produce the desired results and that you can create synergy among the design processes for each perform-

technologies

ance technology included in the system.

Step 2

For both the overall performance improvement initiative and each of the individual performance technologies

Define roles,

you have selected for implementation, identify the key roles that internal and external organizational partners

responsibilities,

can play in the effort. In addition to the general roles (including leader, advocate, and manager) of almost

and partnerships

any performance technology project, there are commonly unique technical aspects to each performance tech-

nology that require specialized staff to be assigned to the individual development project. For each role that

you identify as a necessary component for designing and developing the included performance technologies,

define the associated responsibilities (see Table 2).

Step 3

Apply a generic development process to guide the development of various performance technologies. Although

Design and develop

a generic process will not provide you with the details for managing all aspects in the creation of any single

performance

performance technology (for which there are likely many books available), as a framework it can offer guide-

technologies

lines and structure that are useful in developing a more detailed plan. Examples of how a generalized process

can be applied to specific performance technologies are included in Table 3 (based on Watkins, 2007c).

Step 4

Within your performance improvement initiative, there will be multiple performance technologies selected to

Align common and

accomplish valued performance objectives. During the design and development of these performance technolo-

unique processes

gies, there will also be objectives, tasks, and resources that are shared among the projects. Therefore, capital-

ize on the similarities, maximize the benefits of the unique aspects, and ensure the alignment of all perform-

ance interventions. Throughout the design and development process, maintain your systemic perspective. You

can then avoid both suboptimization and the development of performance technologies that do not accomplish

desired results. Monitor the accomplishments and tasks of each development project to assess where improve-

ments to the development processes can be made.

Source: Based on Watkins (2007c).

Performance Improvement

•

Volume 46

•

Number 4

•

DOI: 10.1002/pfi

45

menting recommendations and changes, and assessing

performance. In its initial role, feedback from formative

evaluations offers essential input into the revision

process that should be part of the development life cycle

for any performance technology project. From specific

information on which processes are not taking place in

the correct sequence to general concerns regarding

employee attitudes about policy changes, let feedback

from formative evaluations guide the revision of project

deliverables.

In your formative evaluations, pull together perform-

ance data using a variety of data collection techniques

(see Table 5). This helps ensure the quality of your evalu-

ations and provides sound justifications for changing

your current design and development processes. For

example, use a combination of data that is externally

verifiable to supplement data that may represent the

unverifiable perspectives of employees, clients, or other

community partners. In addition, collect data that are

numerical along with data represented in text, prose,

audio, or illustrations. By combining data from numerous

sources, you can gain a fuller view of how performance

technology contributes to desired results.

Within a formative evaluation, each data collection

technique has unique advantages and disadvantages that

should be considered in determining which is most

appropriate for meeting your objectives. For instance, if

you are conducting a formative evaluation of a new

employee recruitment program that focuses on inter-

viewing, you would likely want to include survey data

from potential employees who were interviewed (provid-

ing soft quantitative data), focus groups with interviewers

(providing hard qualitative data), and performance data

related to the performance objectives achieved by the

recruitment program (providing hard quantitative data).

The triangulation of data from each of these sources can

provide valuable information for making decisions about

how the technology can be improved (Watkins, 2007c).

Collect formative evaluation data for each of the per-

formance technologies in your system, and then work

with your development teams to identify systematic steps

for improving effectiveness and efficiency. Sometimes

achieving desired performance objectives requires only a

few tweaks to a performance technology; at other times,

you may have to go back to performance objectives to

redesign an entire performance technology. Since the lat-

Source: Based on Watkins (2007c).

SOME COMMON ROLES AND RESPONSIBILITIES ASSOCIATED WITH

TABLE 2

PERFORMANCE IMPROVEMENT SYSTEMS

COMMON

ROLES

RELATED RESPONSIBILITIES

Performance

• Oversees the design and development of multiple performance technology projects within a performance

improvement

improvement initiative

initiative leader

• Develops plans for the coordination of multiple performance technology projects

• Assembles and manages the necessary partners (internal and external to the organization)

• Communicates among the internal and external partners to ensure initiative success

• Responsible for the successful accomplishments of the performance improvement initiative

Initiative

• Communicate the benefits of the performance improvement initiative to internal and external partners

advocates

• Work with partners to ensure alignment of strategic directions

(internal and

• Ensure that the initiative and related projects receive the support required for success

external)

• Serve as a change agent within the organization

Performance

• Manages the design and development of one or more specific performance technology interventions

intervention

• Works with the initiative leader to ensure the alignment of performance technologies

project manager

• Leads the technical development team

• Responsible for the accomplishments of specific performance objectives

Technical

• Provides the design and development support necessary for creating performance technologies

development

• Offers a range of professional backgrounds (for example, information technology, human resources, instructional

team

design, computer interfaces) necessary for the design and development of distinct performance technologies

• Creates draft products and obtains feedback through formative evaluations

• Revises design and development products as necessary

ter scenario can be costly, routinely conduct formative

evaluations throughout the design and development

process, as well as when the performance technology is

nearing completion.

While you may be tempted during the formative

evaluations to focus solely on the operational details of

single performance technologies (for example, improved

communication strategies for promoting a new incentive

system or more interactive elements in an e-learning

course), you can risk losing sight of the long-term strate-

gic objectives that are to be achieved. Balance the recom-

mendations for specific revisions to a performance

technology with those related to the accomplishment of

identified strategic objectives.

Plan for and make continual improvements through-

out the implementation of the multiple performance

technologies in your improvement system. Use your per-

formance objectives to establish performance standards

and criteria. From small changes that can improve process

efficiencies of a single technology to performance reviews

that ensure that the system of performance solutions is

accomplishing necessary results, continual improvement

processes lead to sustainable success.

Furthermore, continually assess your performance

improvement processes. The procedures, steps, tech-

niques, and processes that you are using to select, design,

and develop a system of performance technologies should

be continually assessed and improved to make future per-

formance improvement efforts even more effective and

efficient.

CONCLUSION

The desire to improve individual or organizational per-

formance is an admirable ambition and one that HPT

professionals can achieve through the application of sys-

temic and systematic processes. In this three-article series,

the performance-by-design framework provided system-

atic steps for creating performance improvement systems

that accomplish desired results (see Figure 1). Guided by

the strategic objectives of your organization and its part-

ners, you can apply this framework to most organiza-

tional settings and not be tied to any one performance

solution such as training, electronic performance support,

e-learning, balanced scorecards, mentoring, or coaching.

The framework provides a systematic guide for selecting,

designing, and developing a system of complementary

performance technologies that can be aligned with your

strategic goals and objectives.

Use the framework as a starting place when planning

your next performance improvement effort, realizing

that it provides an initial guide rather than a restrictive

46

www.ispi.org

•

DOI: 10.1002/pfi

•

APRIL 2007

Source: Based on Watkins (2007c).

1

For more information on balanced scorecards, see Kaplan and Norton (1993).

A GENERALIZABLE DEVELOPMENT PROCESS APPLIED TO SAMPLE

TABLE 3

PERFORMANCE TECHNOLOGIES

Electronic Performance

1. Analyze performance requirements

Support System

2. Complete a task and performance analysis

3. Define system specifications

4. Identify integrated performance assessments

5. Select performance support requirements

6. Define media and software requirements

7. Create rapid prototype of support system and do formative evaluation

8. Review and revise based on formative evaluations

9. Complete development of performance support system, and do formative evaluation

10. Review and revise as required

Balanced Scorecard

1

1. Analyze performance requirements

2. Identify critical success factors

3. Identify and define appropriate performance measures for financial, external, internal, and innovation

perspectives

4. For each measure, collect baseline data

5. For each measure, identify desired performance standards based on objectives at the societal,

organizational, and individual and team levels

6. Review and revise performance measures as necessary

7. Identify performance gaps or measures

8. Define implications of performance gaps

9. Create action plans for addressing performance gaps

10. Monitor action plan implementations

11. Review, revise, and repeat process, as necessary

Performance Improvement

•

Volume 46

•

Number 4

•

DOI: 10.1002/pfi

47

Source: Based on Watkins (2007c).

STEPS IN THE FORMATIVE EVALUATION AND CONTINUAL IMPROVEMENT OF

TABLE 4

YOUR PERFORMANCE SYSTEM

STEPS TOWARD

SUCCESS

BRIEF DESCRIPTION

Step 1

Formative evaluations can be conducted using a variety of techniques and a range of participant groups.

Define objectives

Therefore, begin by defining the results you want to achieve through the evaluations. Specify evaluation

of formative

objectives for each development project independently to ensure that you collect all the useful data for making

evaluations

improvement. You may, however, be able to capitalize on similar objectives for the formative evaluation of each

of the performance technologies focused on the achievement of shared performance objectives.

Step 2

Select a combination of data collection techniques that ensure the attainment of adequate and useful information

Select formative

for making future decisions. Use a combination of one or more evaluation techniques to ensure that you have

evaluation techniques

data from several perspectives. For example, you can combine focus group reviews of the performance techno-

ogy with both a one-on-one review by an expert performer and a pilot test of the technology with future users.

Step 3

Plan for the formative evaluation of each performance technology early in the design and development process.

Manage formative

Let the desired results that are expected from the formative evaluations guide your decisions, and consider com-

evaluations

binations of multiple evaluation techniques. Consider the sequencing and timing of formative evaluations within

the development process to ensure that adequate time and resources are available for making necessary revisions.

Step 4

Begin by taking an inventory of the findings from the formative evaluations of each performance technology,

Apply results

and compare the results from each of the evaluation techniques. This gives you the opportunity to assess and

analyze the feedback from the multiple participants and perspectives. In the analysis, look for similarities,

contradictions, recommendations for improvements, and opportunities to capitalize on related performance

improvement activities.

Prioritize the recommendations based on the potential value (for example, effectiveness, efficiencies) they add to

performance and the potential costs (for example, time, financial, resource) associated with implementing the

recommendations. Not all recommendations for improving a performance technology will lead to specific alter-

ations to the current development processes. Some recommendations may be catalogued for use in subsequent

continual improvement efforts after initial implementation, while others may be shared with other performance

technology projects. All reasonable recommendations should, however, be documented, and an action should be

assigned to each.

Step 5

While formative evaluations provide for the improvement of performance technologies before they are imple-

Continually assess

mented, systematic and continual improvement efforts are necessary for making improvements during and

and improve

throughout implementation. No performance technology will accomplish all of its desired results in the most

performance

effective and efficient manner when it is first implemented. Therefore, plan for the continual improvement of all

performance technologies during their design and development.

STEP TOWARD

SUCCESS

TABLE 5

EXAMPLES OF DATA COLLECTION TECHNIQUES

HARD

SOFT

(EXTERNALLY VERIFIABLE DATA)

(NOT EXTERNALLY VERIFIABLE DATA)

Quantitative

• Performance data

• Surveys with a Likert-type scale that quantifies

(numerical expressions

• Budget analysis

perceptions

of a variable)

• Performance ratings

Qualitative

• Focus groups

• Open-ended opinion surveys

(nonnumerical expressions

• Analysis of posting to a listserv

• Individual interviews

of a variable)

• Document review

• Single-source performance observations

• Multisource performance observations

Source: Based on Watkins (2007c).

48

www.ispi.org

•

DOI: 10.1002/pfi

•

APRIL 2007

procedural set of rules. In application, the steps in the

framework will regularly overlap and be completed in

various sequences. So while the framework may appear

to be rigid, in use it is quite fluid and dynamic. When

starting out, however, you may want to follow the

process closely. As you build expertise, use the frame-

work as a heuristic that can be adapted to many situa-

tions. Later, the framework may become an integral part

of your approach to improving performance, offering a

unique and systemic perspective on selecting, designing,

and developing performance technologies.

Nevertheless, no matter how you apply the steps of the

framework, always maintain a focus on results. Use the

strategic objectives of your organization and its external

partners to guide your decisions, continually differentiat-

ing between the results to be achieved (performance) and

the processes, tools, and resources used by individuals and

organizations (performing). Appling this performance-

focused approach to your improvement efforts will lead

to valuable performance outcomes for you and the organ-

izations you serve.

References

Kaplan, R., & Norton, P. (1993, September–October). Putting the

balanced scorecard to work. Harvard Business Review, 134–147.

Lick, D., & Kaufman, R. (2000). Change creation: The rest of

the planning story. In J. Boettcher, M. Doyle, & R. Jensen

(Eds.), Technology-driven planning: Principles to practice. Ann

Arbor, MI: Society for College and University Planning.

Watkins, R. (2007a). Designing for performance: Aligning your

HPT decisions from top to bottom (Part 1 of a 3-part series).

Performance Improvement, 46(1), 7–13. (DOI: 10.1002/

pfi.033).

Watkins, R. (2007b). Designing for performance: Selecting

your performance technologies (Part 2 of a 3 part series).

Performance Improvement, 46(2), 9–15. (DOI: 10.1002/

pfi.102).

Watkins, R. (2007c). Performance by design: The systematic

selection, design, and development of performance technologies

that accomplish useful results. Amherst, MA: HRD Press, and

Silver Spring, MD: International Society for Performance

Improvement.

Related Readings

Gilbert, T., & Gilbert, M. (1989). Performance engineering:

Making human productivity a science. Performance and

Instruction, 28(1), 3–9. (DOI: 10.1002/pfi.4170280103).

Kaufman, R. (2006). Change, choices, and consequences: A guide

to mega thinking. Amherst, MA: HRD Press.

Kaufman, R., Oakley-Brown, H., Watkins, R., & Leigh, D.

(2003). Strategic planning for success: Aligning people, perform-

ance, and payoffs. San Francisco: Jossey-Bass.

Watkins, R. (2006). Aligning performance technologies with

organizational strategic plans. In J. Pershing (Ed.), The hand-

book of human performance technology. (3rd ed., pp. 191–207)

San Francisco: Jossey-Bass/Pfeiffer.

Watkins, R., & Leigh, D. (2001). Performance improvement:

More than bettering the here and now. Performance

Improvement, 40(8), 10–15. (DOI: 10.1002/pfi.4140400805).

Watkins, R., & Wedman, J. (2003). A process for aligning per-

formance improvement resources and strategies. Performance

http://www.educationalconcepts.com/pres/design.swf