Validated Assessment Model for Pilot Selection Benjamin Tan Senior - - PowerPoint PPT Presentation

Validated Assessment Model for Pilot Selection

Benjamin Tan

Senior Consultant | Org. Psychologist

About us Types of solutions Who will be successful? How can we use our insights? Why use these assessment models?

Agenda

About us

World leader in psychometric innovations We focus on:

▪ Creating client centric solutions ▪ Better user experience ▪ Enabling better talent decisions ▪ Delivering clear use cases ▪ Innovation & science

2018

2002

Robust psychometrics People analytics Talent advisory

Who

are we?

What

do we offer?

What

makes us unique?

cut-e in numbers

Over

30 million

assessments carried out each year More than70

• nline assessment tools

Online assessment available in

40

languages

2.000

client references

400

employees worldwide

35

• ffices

worldwide More than

300

integrations with HRMS Candidates from more than

90

countries

Process efficiencies Best in class UX Quality of hire The science behind your success

cut-e solutions are designed against guidelines by key professional bodies including: Reduced interviews by

40% 42%

more profit generated

by high scorers than low scorers

98%

completion rate More… More… More…

cut-e solutions deliver

Our aviation clients

A snapshot

… and many more!

Types of solutions

+1 perfect prediction

+0.58

Structured interviews and unstructured interviews

Schmidt, Oh & Shaffer (2016)

0 random chance

+0.36

Assessment centres

+0.00

Graphology and astrology

+0.65

Cognitive ability tests

+0.46

Integrity tests

+0.35

Biographical data

+0.10

Years of education

Selection context

Pre-application assessment

Selection

Realistic Job Preview Personality questionnaire, Situational Judgement Questionnaire, aptitude assessments Online assessments Video interview Assessment centre One or two way video interviews Individual & group exercises, interviews Technical evaluation Assessment of technical skills

How can we identify pilots …

▪ who will be successful in the future …

… when we have so many to hire and train … in a timely and cost-efficient manner?

Online Test Modules

Considerations for online tests

▪ No special input devices ▪ No sound ▪ Self-explanatory ▪ Forgery-proof

What tools will tell us who will be successful?

What do we need to measure in pilots?

Across the following roles

▪ Captain ▪ First Officer ▪ Cadet/Trainee Pilot

▪ Aptitudes critical at early stages – predicts training success ▪ Behaviour becomes more critical than aptitude for more experienced pilots (Captains) Role fit

• Org. fit

Aptitude Behaviour Pilot-specific assessment

The measures – cut-e pilot suite

The ability to…

time sharing shift back and forth between two or more activities or sources of information (such as speech, sounds, touch, or

• ther sources)

spatial orientation know your location in relation to the environment or to know where other objects are in relation to you perceptual speed quickly and accurately compare similarities and differences among sets of letters, numbers, objects, pictures,

• r patterns which may be presented at the same time or one after the other

control precision quickly and repeatedly adjust the controls of a machine or a vehicle to exact positions speed of closure quickly make sense of, combine, and organize information into meaningful patterns

• ral expression

communicate information and ideas in speaking so others will understand category flexibility generate or use different sets of rules for combining or grouping things in different ways information ordering arrange things or actions in a certain order or pattern according to a specific rule or set of rules (e.g., patterns of numbers, letters, words, pictures, mathematical operations) inductive reasoning combine pieces of information to form general rules or conclusions (includes finding a relationship among seemingly unrelated events) number facility add, subtract, multiply, or divide quickly and correctly selective attention concentrate on a task over a period of time without being distracted deductive reasoning apply general rules to specific problems to produce answers that make sense

Cognitive assessments

Pilot-specific abilities tests

… measures the ability to perform several tasks simultaneously

Example: multitasking

Behavioural Research

Stress resistance Co-operation Communication Decision making Teamwork Leadership Situational awareness Stress management Identify risk Manage error Communication skills Co-operation Assertiveness Leadership Self discipline Self-critical attitude Stress management Self organisation Professional aspiration level

F-JAS - Goeters et al. (1997) IATA (2012) CRM - Flin (2010) TEM - Maurino (2005)

Competency Cadet First Officer Captain

• perational

Safety orientation

✓ ✓ ✓

Decision-making

✓ ✓ ✓

Customer & commercial orientation

✓ ✓ ✓

Planning & organising

✓

interactive Teamwork

✓ ✓ ✓

Communication style

✓ ✓ ✓

Managing the crew

✓

motivational Resilience

✓ ✓ ✓

Motivation

✓ ✓ ✓

Self-development

✓ ✓ ✓

Pilot NOTECH model

What is required?

Predicting simulator performance

… reduces fail rate in SIM by 12%

▪ Combination of tests assessing complex control, multi-tasking, spatial

• rientation and situational judgement …

Knowledge Decision making Handling Management Communication Procedures Application

• f

procedures Situational Awareness

How can we use our insights?

Candidates tell us about their preferred work style

cut-e recommended pilot process

This allows us to be smarter about how we interview them We make predictions about their future behaviour from this

Pilot Report - Inputs

shapes – 18 aspects of preferred work style squares – capacity to resist aspects of the situation that may cause unsafe behaviour

Cognitive assessments

Sample report

Area

NOTECHS

poor limited fair good excellent

• perational

Safety Orientation

prioritises safety; takes responsibility for safety; follows rules and regulations; enforces rule following where necessary; acts as a role model for safety; speaks up when notices a potential safety issue or error

Decision-Making

quickly draws accurate conclusions; takes into account relevant information when making decisions; accurately anticipates the outcome

• f situations; adapts decision-making in changing circumstances;

identifies risks

Customer & Commercial Orientation

recognises the importance of customer satisfaction as a business goal; understands the link between own behaviour and business performance; thinks in terms of revenue and cost; behaves positively towards customers Data quality The analysis of the response pattern does not show any irregular or unusual aspects. Thus, data quality is sound and the interpretation of the results is reliable and accurate.

Detailed Profile

Dealing with emotions

Strong

First Officer Selma Sample is even-tempered and confident. She is resilient even in the case of failure and remains calm under pressure. At times she considers how she is perceived by others, but is hardly affected by this. Thus, she can focus attention on the tasks and duties of a First Officer. This means that First Officer Selma Sample is likely assess situations accurately and also when under pressure. Her decision-making is likely to be based on relevant information only. At this stage it seems that there are no major issues affecting First Officer Selma Sample’s health and well-being on the long run.

Organising work and self

Risk

First Officer Selma Sample responds to sudden change in a spontaneous and intuitive manner and feels constrained by exact planning. She sees the big picture rather than the details and is distracted easily. Moreover, she is quick to question rules and regulations. Moreover, colleagues might perceive her as unreliable and undisciplined. This means that First Officer Selma Sample is likely to struggle with adhering to rules and regulations and to have difficulty maintaining attention during routine tasks. Her colleagues can rely on her only to a limited extent and this may impair their planning and lead to interpersonal conflict.

Sample report

Detailed Profile

Why assess using these instruments?

Why?

Ability tests

▪

Predict flying performance during training and the actual flight NOTECHS

▪

Predict training and job performance

▪

Are important for preventing accidents (CRM, TEM)

▪

Can be trained only to a limited extent, therefore need to be assessed before hire Assessing ability and NOTECHS at an early stage in the selection process

▪

Increases pass rates in AC and simulator

▪

Makes the process timely and cost-efficient

www.cut-e.com go

Make better talent decisions!