Consider a past (or current) performance issue you’ve observed in an organization in which you’ve worked. Describe the situation (while maintaining confid

PM – Performance Assessments Biagi

Performance Management With Shannon Biagi, MS, BCBA (she/her)

PM – Performance Assessments Biagi

Assessments in Performance Management:

Part 1

PM – Performance Assessments Biagi

Objectives

The goals of this lesson are to: • Determine why assessments have been

historically neglected in performance management

• Select the justification(s) for implementing pre- intervention assessments in the workplace • Determine which records/documents to review

during a historical assessment, given a potential intervention target

• Given a scenario, select the area of the Performance Diagnostic Checklist (PDC) that may indicate an effective intervention

PM – Performance Assessments Biagi

Objectives

The goals of this lesson are to: • Identify the factors taken into consideration in

ABC/Consequence analysis

• Determine the most powerful consequences to integrate into an intervention, depending on the desired behavior change • Identify the different descriptive assessments, given

a scenario

• Determine the conditions under which a descriptive assessment is appropriate (or inappropriate)

• Select reasons why experimental analysis is less common in PM practice than clinical ABA

PM – Performance Assessments Biagi

Foundations The Basics of Performance Assessment in the Workplace

PM – Performance Assessments Biagi

Assessment in PM The process of conducting a _______________ assessment of staff performance is important in

performance management practice. Using an assessment, just like in a clinical context, will allow the practitioner to correctly identify the environmental variables (antecedents and consequences) that influence

employee performance. (Wilder, Lipschultz, King, Driscoll, & Sigurdsson, 2018)

We’re going to explore the different types of assessments and tools available to performance management professionals, the origins of those tools, and why, historically, assessments have been neglected in this

area of behavior analysis.

PM – Performance Assessments Biagi

In the beginning…

Mager and Pipe • Published Analyzing Performance Problems (or You

Really Oughta Wanna) in 1970, one of the earliest approaches to conducting performance analysis of staff behavior

• This book is/was included in a seminal series written for instructional designers, with the objective to encourage folks to assess whether __________ is the most appropriate intervention

• The model presented therein is cited at the root of nearly every assessment tool popularized in PM today

PM – Performance Assessments Biagi

Adapted from Mager & Pipe (1984)

PM – Performance Assessments Biagi

Houmanfar, Alavosius, Binder, & Johnson (2019)

Thomas Gilbert • Considered the father of “human performance

technology”

• Published Human Competence: Engineering Worthy Performance in 1978, which introduced his approach to analyzing performance issues in organizations, known as the Behavioral Engineering Model (BEM)

• In contrast to previous approaches, Gilbert emphasized a focus on valuable __________, or accomplishments, rather than an exclusive focus on behavior alone

• These analyses were supplemented with interviews and verbal reports to produce the most effective and efficient interventions possible

PM – Performance Assessments Biagi

Adapted from Gilbert (1978)

Information Instrumentation Motivation

Environmental Variables Data Resources Incentives

Individual Variables Knowledge Capacity Motives

PM – Performance Assessments Biagi

Why Have PM Assessments Been Neglected in Practice, Historically?

PM professionals have historically selected effective interventions in the absence of assessments (although perhaps they would have been faster/ more effective with them!)

PM has been Successful ________________ Behavior at work is heavily

rule-governed, rather than being exclusively shaped by direct-acting contingencies, and an assessment analyzing the rules that influence employee behavior has yet to be developed.

_______________ Behavior

Many of the behaviors PM professionals analyze are behaviors that are not occurring (as opposed to the behavioral excesses analyzed in clinical ABA). How do we analyze behavior when it isn’t happening?

PM Focuses on ________________

Austin, Carr, & Agnew (1998)

PM – Performance Assessments Biagi

Without assessments, a practitioner may or may not

correctly identify the key variables for influencing employee performance.

Assessments allow us to “generate quicker-acting and ____________ effects” of our interventions, and should be

considered best practice (just like in clinical behavior

analysis).

Austin, Carr, & Agnew (1998)

PM – Performance Assessments Biagi

Types of Performance Assessments 1. Historical assessments 2. Indirect/Informant assessments 3. Direct/Descriptive assessments 4. Experimental analysis

Overview

PM – Performance Assessments Biagi

Which of the following is false? PM practitioners have neglected

conducting assessments in the past because…

Checking in:

A. There aren’t sufficient assessments that account for rule-governed behavior

B. Assessments are mostly used for behavioral excesses in behavior analysis

C. Assessments are resource-intensive and questionably effective

D. Practitioners have had good outcomes without utilizing them systematically

PM – Performance Assessments Biagi

Why should a PM practitioner conduct an assessment prior to selecting an

intervention?

Checking in:

A. Assessments allow us to save time and resources

B. Functionally-appropriate interventions are more effective

C. A functionally-appropriate intervention is more likely to sustain long-term

D. Assessment is a cornerstone of behavior-analytic practice

E. All of these

PM – Performance Assessments Biagi

Historical Assessments

Analyzing the Past to Influence Future Interventions

PM – Performance Assessments Biagi

Historical Assessments A lot of data already exists in organizations, including information on employee behaviors, quality and

quantity of outputs/accomplishments/permanent products, and long-term business results (productivity, expenses, revenue, satisfaction, etc.).

Gathering and reviewing _______________ can help determine the current state of the organization, the true extent of the issues reported, and even past interventions that weren’t successful.

Between 2000-2015, ___% of empirical studies published in the Journal of Organizational Behavior Management reported implementing formal historical assessments (Wilder, et al. 2018).

PM – Performance Assessments Biagi

Rodriguez, Sundberg, & Biagi (2017)

Types of Documents to Analyze in a Historical Assessment • ALL THE (relevant) DATA • Planning documents (strategic plans/priorities) • ______________________ • Industry benchmarks and standards • Competitor information and data • Product or service standards • Workplace conditions • Client lists • Handbooks/Policies • _______________________ • Other ”loose information”

PM – Performance Assessments Biagi

Rodriguez, Sundberg, & Biagi (2017) Some Important Questions to Answer • How does the current performance compare to the industry

or competition?

• Are we targeting behaviors that differentiate the organization in the marketplace? How will our intervention potentially impact services/products received by the clients?

• How does the organization communicate, both internally and externally?

• Do policies/mission/values match what the organization does?

• What are the leadership behaviors currently like, and what is the potential that they will impact the sustainability/ maintenance of an intervention?

• What technology is in use, and is it up-to-date?

PM – Performance Assessments Biagi

Historical assessments and records review processes are often not universally structured due to the wide variety of information available

to analyze and the wide array of potential targets for intervention.

Additionally, historical assessments are not recommended for use as the _______

assessment strategy but provide a useful starting point for targeting and focusing your

“true” functional assessment (Gravina, Nastasi, & Austin, 2021).

PM – Performance Assessments Biagi

An organization wants to determine if their communication expectations (within 24 hours) are realistic. Which of the following would be

good to review during your historical assessment?

Checking in:

A. Current communication-related policies in the employee handbook

B. Past communication latency, retrieved from the email system

C. The systems used to communicate (text, email, phones, other apps)

D. Onboarding training

E. All of these

PM – Performance Assessments Biagi

Indirect/Informant Assessments

PM – Performance Assessments Biagi

Indirect/Informant Assessments An indirect or informant assessment is conducted through either structured or unstructured interviews with

various stakeholders involved in a particular performance. These should include the employees (or the potential subject of the intervention), as well as managers and other key organizational representatives.

These types of assessments are (comparatively) very popular in the PM literature, with ___% of empirical studies published in the Journal of Organizational Behavior Management between 2000-2015 reporting the

use of such indirect assessment tools (Wilder, et al. 2018)

PM – Performance Assessments Biagi

Gravina, Nastasi, & Austin (2021)

Informant/Indirect Assessments • Surveys • Rating Scales • Interviews

• Advantages: • Easy • Quick • Require ______________ • Get information from various sources • Yield desirable intervention outcomes

PM – Performance Assessments Biagi

Two Popular Indirect/Informant Assessments in Performance Management:

Performance Diagnostic

Checklist (PDC, and variations thereof)

ABC/Consequence Analysis

(PIC/NIC, E-TIP, etc.)

PM – Performance Assessments Biagi

Gravina, Nastasi, & Austin (2021)

The Performance Diagnostic Checklist (PDC) • A result of Dr. John Austin’s doctoral thesis, which was designed to create an easy-to-use

tool that covered the components of ________’s Behavioral Engineering Model • Consists of a list of questions to use as an interview with performers, and to further guide

descriptive data collection, in 4 categories: • ___________ and Information • Equipment and ___________ • Knowledge and Skills • ____________

• Variations on the assessment have also been developed to meet specific industry needs, including the PDC-Human Services (Carr et al. 2013), PDC-Safety (Martinez-Onstott, Wilder, & Sigurdsson, 2016), and PDC-Parents (Hodges et al. 2020)

PM – Performance Assessments Biagi

The Performance Diagnostic Checklist (PDC)

PM – Performance Assessments Biagi

The Performance Diagnostic Checklist (PDC)

PM – Performance Assessments Biagi

The Performance Diagnostic Checklist (PDC)

PM – Performance Assessments Biagi

The Performance Diagnostic Checklist (PDC)

PM – Performance Assessments Biagi

The PDC: Results Example (Pampino et al, 2003)

• Conducted research to determine why coffee shop staff were not completing post-shift cleaning tasks

• Interviewed four employees using Austin’s PDC

PM – Performance Assessments Biagi

Rodriguez, Sundberg, & Biagi (2017)

The Performance Diagnostic Checklist (PDC) • Practitioner tips for using the PDC:

• Deciding who to interview – This is based on time required, availability of performers, consistency of responses as the interviews are conducted.

• Preparation of the tool – Determine which questions you’re going to ask and the _________ you will use.

• Conducting the interview – You can ask follow-up questions during the interview, just be mindful that it will increase the ____ required. Skills to conduct good general interviews are critical in this process.

• After the interview – Analyze the data, and determine what interventions are best indicated by the results.

PM – Performance Assessments Biagi

You conduct a PDC, and an employee reports that they have no idea where to

find the checklists developed for their job tasks. What category of the PDC would

this issue fall into?

Checking in:

A. Antecedents and Information B. Equipment and Processes C. Knowledge and Skills D. Consequences

PM – Performance Assessments Biagi

When describing the onboarding process, an employee mentions that they

did practice the tasks for their job, but only a few times. What category of the

PDC would this issue fall into?

Checking in:

A. Antecedents and Information B. Equipment and Processes C. Knowledge and Skills D. Consequences

PM – Performance Assessments Biagi

Staff report that feedback in the organization is delivered inconsistently,

and is often not objective or focuses only on mistakes. What category of the PDC

would this issue fall into?

Checking in:

A. Antecedents and Information B. Equipment and Processes C. Knowledge and Skills D. Consequences

1. performance management practice Using an assessment just like in a clinical context will allow the:
2. encourage folks to assess whether:
3. or:
4. Behavior:
5. PM professionals have:
6. Many of the behaviors PM:
7. interventions and should be:
8. true extent of the issues reported and even past interventions that werent successful:
9. Management reported implementing formal historical assessments Wilder et al 2018:
10. undefined:
11. undefined_2:
12. assessment strategy but provide a useful:
13. of empirical:
14. Require:
15. tool that covered the components of:
16. undefined_3:
17. Equipment and:
18. undefined_4:
19. questions youre going to ask and the:
20. will increase the: