Safety Culture Assessment: What are we Trying to Achieve? Joelle - - PowerPoint PPT Presentation

▶
safety culture assessment what are we trying to achieve
SMART_READER_LITE
LIVE PREVIEW

Safety Culture Assessment: What are we Trying to Achieve? Joelle - - PowerPoint PPT Presentation

Mar 04, 2024 370 likes •769 views

Safety Culture Assessment: What are we Trying to Achieve? Joelle Mitchell Safety Psychology Conference Sydney, October 2012 Outline What is safety culture? Theory and research Are perception surveys useful? Assumptions

slide-1
SLIDE 1

Safety Culture Assessment: What are we Trying to Achieve?

Joelle Mitchell

Safety Psychology Conference Sydney, October 2012

slide-2
SLIDE 2

Outline

  • What is safety culture?

– Theory and research

  • Are perception surveys useful?

– Assumptions – Evidence – Challenges

  • Is safety culture a valuable concept?
slide-3
SLIDE 3

Culture: Theory and Research

slide-4
SLIDE 4

Culture Theory

  • Safety culture is understood as a component of
  • rganisational culture
  • Schein (1985) is commonly referenced

4 A248801 29/10/2012

slide-5
SLIDE 5

Schein’s Model

Artefacts Espoused Values Basic Assumptions

5 A248801 29/10/2012

slide-6
SLIDE 6

How is culture built?

6 A248801 29/10/2012

slide-7
SLIDE 7

How Is Culture Built?

  • A definable group with a shared history
  • Experience and solve challenges
  • Solutions work well enough to be considered

valid

  • Taught to new members as the correct way to

perceive, think, and feel in relation to the problem

  • Become basic assumptions
  • Informs future behaviour

7 A248801 29/10/2012

slide-8
SLIDE 8

Key Defining Features

  • Abstract construct (not a concrete phenomenon)
  • Relatively stable and enduring
  • Multiple dimensionality
  • Shared group norms
  • Socially constructed
  • Provides meaning about observed things

8 A248801 29/10/2012

slide-9
SLIDE 9

Safety Culture: A Definition

  • “The product of individual and group values,

attitudes, perceptions, competencies, and patterns of behaviour that determine the commitment to, and the style and proficiency of, an organisation’s health and safety management.”

– International Nuclear Safety Advisory Group (1991)

  • The way we do things around here
  • Why we do things the way we do them around

here

9 A248801 29/10/2012

slide-10
SLIDE 10

Safety Culture Research

  • Perception surveys
  • Normative sets of factors
  • Many, many questionnaires
  • Many, many factors
  • Factor structures difficult to replicate
  • A few factors are common to most studies

10 A248801 29/10/2012

slide-11
SLIDE 11

Criticisms

  • “We have been too focused on a traditional

research paradigm… that has produced very reliable results about very unimportant things.

  • …we have lost touch with some of the

important phenomena that occur in

  • rganizations… simply because they were too

difficult to study by the traditional methods available.”

– Schein (1993)

11 A248801 29/10/2012

slide-12
SLIDE 12

Criticisms

  • “As applied by safety researchers, the culture

concept is deprived of much of its depth and subtlety, and is morphed into a grab bag of behavioural and other visible characteristics, without reference to the meaning these characteristics might actually have, and often infused with normative overtones.”

– Guldenmund, 2010

12 A248801 29/10/2012

slide-13
SLIDE 13

Perception Surveys

slide-14
SLIDE 14

Why Perception Surveys?

  • We want a leading indicator

– To intervene before something goes wrong – To prevent major events

  • We like to benchmark and measure change

– Between-groups comparisons – Trends

  • Easy to analyse
  • Lots of nice graphs
  • Relatively cheap
  • Relatively minimal time “off-tools”

14 A248801 29/10/2012

slide-15
SLIDE 15

Problems with Perceptions

  • Perceptions are subjective

– Difficult to interpret meaningfully

  • Perceptions are volatile
  • Self-report issues

– Honesty – Selective Memory

  • Construct Validity

– Mono-method bias – What are we actually measuring?

  • Predictive Validity

15 A248801 29/10/2012

slide-16
SLIDE 16

Core Assumption

  • Safety Culture has a causal impact on safety

performance

– Safety culture can be reliably measured with perception surveys – Therefore, perception surveys provide a leading indicator

  • f safety performance
  • Is this assumption accurate?

16 A248801 29/10/2012

slide-17
SLIDE 17

Core Assumption Part 1

  • Can safety culture be reliably measured

with perception surveys?

17 A248801 29/10/2012

slide-18
SLIDE 18

Is a Normative Approach Appropriate?

  • How does a culture develop?
  • Are all organisations likely to hold the same basic

assumptions?

  • Are basic assumptions likely to impact
  • rganisational outcomes in uniform ways?

18 A248801 29/10/2012

slide-19
SLIDE 19

Do perception surveys measure safety culture?

19 A248801 29/10/2012

Artefacts

Espoused Values

Basic Assumptions

slide-20
SLIDE 20

So What?

  • What cultural issues are identified after major

incidents?

– High level decision-making – Top-level leadership practices – Initiatives, priorities and targets – Allocation of resources – Routine violations

  • Unlikely to be detected by perception surveys

20 A248801 29/10/2012

slide-21
SLIDE 21

Core Assumption Part 2

  • Safety culture is not reliably measured

with perception surveys

  • But…
  • Do perception surveys provide a leading

indicator of safety performance?

21 A248801 29/10/2012

slide-22
SLIDE 22

What has research found?

  • Weak to moderate correlations between

perception survey scores and injury frequency rates (e.g. TRIFR)

  • Used to demonstrate tool validity

22 A248801 29/10/2012

slide-23
SLIDE 23

Common Interpretation

Safety Perceptions Safety Behaviours Injury Rate

23 A248801 29/10/2012

slide-24
SLIDE 24

Another possibility…

24 A248801 29/10/2012

Safety Behaviours Injury Rate Safety Perceptions

slide-25
SLIDE 25

…and another…

Injury Rate Safety Behaviours Safety Perceptions

25 A248801 29/10/2012

slide-26
SLIDE 26

…and another!

Variable ‘X’

Safety Perceptions Injury Rate

26 A248801 29/10/2012

slide-27
SLIDE 27

Is it a Useful Correlation?

27 A248801 29/10/2012

  • Can we have poor safety outcomes and a

low TRIFR?

  • Longford Gas Plant 1998
  • Texas City Refinery 2005
  • Deepwater Horizon 2010
slide-28
SLIDE 28

Predictive Value

  • Perception survey scores correlate with
  • ccupational injury rates
  • HOWEVER…
  • Low TRIFR ≠ Safe operations
  • Do perception surveys predict major incidents?

28 A248801 29/10/2012

slide-29
SLIDE 29

Case Study

  • Snorre Alpha, Norway

– Antonsen, 2009

  • Positive perception survey results
  • Gas blowout 12 months later
  • No lives lost – luck not design
  • No significant changes to the organisation or

facility over that 12 months

29 A248801 29/10/2012

slide-30
SLIDE 30

Contradictory Findings

30 A248801 29/10/2012

Perception Survey results Safety a priority Rigorous risk assessments High levels of compliance Good safety communication Incidents reported Learning and change from incidents Incident investigation findings Production targets prioritised Lack of risk assessments and poor understanding Non-compliance was normal Poor communication climate Not all incidents reported Limited use of own and others’ incident information

slide-31
SLIDE 31

Key Question

  • Are perception surveys the best way to spend

limited safety resources?

  • Perception surveys do not predict major events

– some correlation with occupational injuries

  • There are other lead indicators for occupational

injury

– accuracy – cost-effectiveness – time-efficiency

31 A248801 29/10/2012

slide-32
SLIDE 32

Safety Culture is a Valuable Concept

slide-33
SLIDE 33

Qualifiers

  • Don’t try to ‘measure’ culture
  • OUTCOMES:

– understand the meaning of observable things – uncover basic assumptions – develop effective improvement strategies – identify potential change barriers

  • NOT:

– as a benchmarking exercise – as a replacement for other technical and systems processes

33 A248801 29/10/2012

slide-34
SLIDE 34

How?

  • Triangulate your methodology!
  • Use qualitative methods

– Interviews – Observations – Focus Groups – Action Research

34 A248801 29/10/2012

slide-35
SLIDE 35

But we want to benchmark!

  • Use existing measurable data

– Reporting rates – Inspection findings – Workplace observations – Audit findings – Quality Assurance – Incidents – Near misses – Outcomes and lessons learnt

  • Changing themes
  • Follow-up on previous findings

35 A248801 29/10/2012

slide-36
SLIDE 36

What about Perception Surveys?

  • Are the questions the right ones?
  • How was the survey developed and validated?

– Industry – Company – Links to safety outcomes

  • Use data to drive qualitative inquiry

– Open-ended questions – Interviews – Focus groups

  • Remember the limitations of perception data

36 A248801 29/10/2012

slide-37
SLIDE 37

Final Thoughts

  • The ultimate goal is to keep people safe
  • Safety resources are not unlimited
  • Target the tools and strategies that are most

likely to lead to improved safety outcomes

  • Take an evidence-based approach

37 A248801 29/10/2012

slide-38
SLIDE 38

Thank you! Questions?

slide-39
SLIDE 39

References

  • Antonsen, S. (2009). Safety Culture: Theory, Method and
  • Improvement. Surrey: Ashgate Publishing Group.
  • Guldenmund, F. W. (2010). (Mis)understanding safety

culture and its relationship to safety management. Risk Analysis, 30, 1466-1480.

  • International Nuclear Safety Advisory Group (1991).

Safety Culture. Vienna: International Atomic Energy Agency.

  • Schein, E. H. (1985). Organizational Culture and
  • Leadership. San Francisco: Jossey-Bass.
  • Schein, E H. (1993). Legitimating clinical research in the

study of organizational culture. Journal of Counseling and Development, 71, 703-708.

29/10/2012 A248801 39