Survey Questions on Opioid-Related Impairment Stephanie Willson - - PowerPoint PPT Presentation

National Center for Health Statistics

The Role of Respondent Experience in Answering Survey Questions on Opioid-Related Impairment

Stephanie Willson Collaborating Center for Question Design and Evaluation Research

DC-AAPOR/WSS Summer Preview/Review Conference July 12, 2019

Study Background

• Cognitive interview evaluation of the plausibility of asking
• pioid-related questions for household surveys

– Use – Impairment – Misuse – Disorder

Study background

• 180 interviews (40 in Spanish, 140 in English) in summer of

2018

• Eight different geographical regions in the United States
• DC/Maryland
• Kentucky
• Alabama
• Washington
• Massachusetts
• Kansas
• Texas/New Mexico
• Illinois

Impairment (measured as side effects)

 Drowsiness  Dizziness  Confusion  Calm  Carefree  Lack of concentration  Blurred vision  Off balance  Irrational  Paranoid

When taking opioids, do you feel any of the following side effects? [Check all that apply]

Side effects question is important - serves as a filter for other impairment questions

• During the past 30 days, have you gone to work at a paid job

while experiencing those side effects?

• During the past 30 days, have you driven a car while

experiencing those side effects?

• During the past 30 days, have you had an injury or hurt

yourself because of the opioids or the side effects?

Two principle problems with asking about side effects

• 1. Understanding the intended definition of a side effect
• 2. Knowing what causes a feeling state

• 1. Question intent: What are “side effects”?
• Medical perspective

– Side effects are not therapeutic effects (i.e., pain relief) – Side effects are considered secondary effects

• Unintended
• Sometimes beneficial, but mostly unpleasant or even

harmful

Unintended interpretation of the “side effects” question

• Not all respondents understood the question as asking about

side effects in accordance with the medical use of the term

• Many respondents were thinking about the impact of the

therapeutic effect of the opioids

Interpretations were not random but instead often associated with different groups of respondents

• The likelihood of answering the question as intended was

associated with respondents’ experience with opioids, specifically length of and reason for use – Temporary injury or condition/surgery (short-term users) – Chronic pain management (long-term users)

Different interpretations of “side effects” is associated with different experiences with opioids

• Short-term users (surgery; temporary injuries)

– More likely to answer the question in terms of the secondary and unpleasant effects of opioids

Examples: short-term users, unpleasant effects

You don’t feel…well. That’s how I have to put it. There’s something wrong, but you don’t know what it is. If I got up and walked around, it would make me really dizzy. [My wife] made me stay in bed [for safety reasons]. Yeah, I couldn’t have a decent conversation. Because I wasn’t thinking right. I’d have people call me on the phone and I’d tell them, ‘Yeah, you know, I’m not up for it. Call me back in a few weeks.’ Because I didn’t know what I was going to say! I tried not to do any business or anything.

Common choices among short-term users

• Often chose many side effects

– To convey the overall unpleasant feeling – Difficult to describe how they felt

Different interpretations of “side effects” is associated with different experiences with opioids

• Long-term users (chronic pain management)

– More likely to answer the question in terms of the therapeutic effect of the opioids

Common choices among long-term users

• Drowsy
• Calm
• Carefree

Meaning of ‘drowsy’, ‘calm’, and ‘carefree’, for long- term users

• Not negative or unpleasant

– ‘Calm’ & ‘carefree’ = absence of pain; allows respondents to function in daily life

• Ability to work
• Ability to drive

– ‘Drowsy’ allows respondents to sleep

Examples: Long-term users, therapeutic effect

Drowsy: The pain already wore you out. And when the pain subsides because of the pain medication [exhales loudly], I’m going to take a nap. Because I’ve been struggling through all this discomfort and pain. Calm: You hurt and you take pain reliever – gonna give you that relief, and that makes me calm. Allows you to take a deep breath in. Carefree: It was being able to run with my kids and move again. Because when you are strapped in this much pain all the time, being carefree is not something you can be.

Why does this matter?

• Question functions mostly as intended for short-term users

– Intent of question = inability to function = short term users BUT…

• Question misses and misunderstands the experiences of long-

term users – Unintended meaning = ability to function = long-term users

• 2. Side effects and determining causality
• Respondents not always able to ascertain the cause of the

feeling states asked about in the question

Other possible causes (besides opioids)

• Age
• Personal characteristic or condition
• Surgery
• Lack of sleep
• Pain
• Multiple medications

Examples: Difficulties in determining causality

Age: Once in a while I have trouble hearing and seeing – blurred vision and like this. But I don’t know if that’s from, like, getting old or [the opioid]. I had cataract surgery about five years ago, so I don’t know if that had anything to do with my age or the pills. Personal characteristic or condition: I’m pretty calm in general, so I don’t know if that’s a side effect. Surgery: I assumed those were kind of regular with after-surgery stuff. Or whether it could have been a side effect of that [opioid]? It’s kind of hard to say.

Examples: Determining causality

Lack of sleep: I do get drowsy. But whether it’s attributed to this [oxycodone] or not, I don’t know. I’m sleepy a good deal of the time. Now whether it’s this, I don’t know. Pain: I don’t know if I was confused because of the pills or the pain. Multiple medications: I don’t get sleepy. I don’t get dizzy. But my vision is blurry a little bit. [Respondent on 3 medications, all of which list blurred vision as a side effect.]

Different groups = different level of difficulty

• Short-term users more likely to connect the feeling with the
• pioid
• Long-term users less likely to identify causality

– Too long to remember – The positive outweighs any negative they associate with

• pioid use

Why does this matter?

• Difficult question to answer

– Increased respondent burden – Decreased validity of respondent answers (we don’t know whether they are reporting an actual opioid side effect) – Misunderstand long-term users

Conclusion

• Question design principle: questions that align with

respondent experiences capture the most valid data

• Opioid use: different experiences with opioids affect

knowledge, understandings and ability to answer survey questions as intended

Conclusion

• Opioid impairment: current survey question is more likely to

perform as intended among short-term users than among long-term users

• Opioid use survey questions: should be designed with

different groups of users in mind in order to capture the intended information

Thank you!

Contact information: swillson@cdc.gov Cognitive interview final reports: https://wwwn.cdc.gov/qbank/