Causal Analyses of Electronic Health Record Data for Assessing the - - PowerPoint PPT Presentation

▶
causal analyses of electronic health record data for
SMART_READER_LITE
LIVE PREVIEW

Causal Analyses of Electronic Health Record Data for Assessing the - - PowerPoint PPT Presentation

Mar 26, 2023 257 likes •569 views

Causal Analyses of Electronic Health Record Data for Assessing the Comparative Effectiveness of Treatment Regimens Romain Neugebauer, PhD Research Scientist II, Division of Research, Kaiser Permanente Northern California Romain Neugebauer

slide-1
SLIDE 1

Causal Analyses of Electronic Health Record Data for Assessing the Comparative Effectiveness of Treatment Regimens

Romain Neugebauer, PhD

Research Scientist II, Division of Research, Kaiser Permanente Northern California

slide-2
SLIDE 2

2

Romain Neugebauer

Disclosures

Rela latio ionship ip Company ny(ies es) Speakers Bureau Advisory Committee Board Membership Consultancy Review Panel PCORI Funding ME-1403-12506; ME-2018C1-10942 Honorarium Ownership Interests

slide-3
SLIDE 3

3

CER for Effective Management of Chronic Conditions

  • Long-term care requires frequent re-evaluation of treatment decisions
  • Chronic care model emphasizes personalization of care based on patient’s needs
  • Evidence most suited to inform real-world care involves the comparison of dynamic

treatment plans (as opposed to static treatment plans)

  • Example in diabetes care:
  • To avoid complications, clinicians aim to control blood glucose levels
  • Over time, glycemia tends to deteriorate prompting treatment intensification (TI)
  • TI timing is best informed by comparing dynamic treatment plans such as: “Intensify

therapy when the patient’s A1c reaches 7% versus 7.5%” instead of static plans “Intensify therapy 3 versus 6 months from now”

slide-4
SLIDE 4

4

Evaluation of Dynamic Treatment Plans

  • Ideally, using a trial design:
  • Randomize patients to one of several dynamic treatment plans
  • Contrast average outcomes between any two arms (e.g. survival curves)
  • Alternatively, by emulating trial inference using observational data:
  • Using a cohort study design and Causal Inference methods, such as:
  • Inverse probability weighting (IPW) estimation
  • Targeted minimum loss based estimation (TMLE)
  • Both IPW and TMLE methods can address time-dependent confounding
  • Originally, methods applied in studies with regular clinic visits (e.g., every 6 months)
  • More recently, methods applied to Electronic Health Record (EHR) data
slide-5
SLIDE 5

5

Evaluation of Dynamic Treatment Plans

  • IPW estimation is a propensity score (PS) method
  • Estimate the probability of exposure conditional on confounders over time
  • Use these estimated PS to construct weights
  • Compute weighted average of outcomes in each exposure group

 Correct inference relies on estimating the PS correctly

  • TMLE can provide more precise effect estimates and is doubly robust
  • Implemented by a sequence of (weighted) outcome regressions
  • PS are used to fit each regression
  • Compute average predicted values from the last regression in each exposure group

 Relies on either estimating the PS correctly or the outcome regressions

slide-6
SLIDE 6

6

Illustration with “Treatment Intensification” (TI) Study

  • Current recommendations specify a target A1c of <7% for most patients
  • Conflicting supporting evidence if patient on 2+ oral medications or basal insulin
  • To avoid or delay kidney disease, when should patient start an intensified therapy?
  • A retrospective cohort study using EHR of ≈ 51,000 adults from 7 US regions
  • Median follow-up of 2.5 years starting at first A1c≥7% between 2001 and 2009
  • Contrasted onset or progression of albuminuria between 4 dynamic treatment plans:

dθ: “Patient initiates TI the first time a newly observed A1c≥ θ% and continues the intensified therapy thereafter” with θ=7; 7.5; 8; 8.5%

  • Original results indicated strong evidence of risk reduction for TI at lower A1c
  • Frequency of A1c monitoring was ignored in the analyses
slide-7
SLIDE 7

7

Impact of A1c Monitoring on CER Evidence

  • In a trial to evaluate dynamic treatment plans, the

intervention protocol would specify an A1c monitoring schedule.

  • A patient randomized to treatment strategy d7.5 will

initiate TI when a physician first detects A1c ≥ 7.5%.

slide-8
SLIDE 8

8

Impact of A1c Monitoring on CER Evidence

  • In a trial to evaluate dynamic treatment plans, the

intervention protocol would specify an A1c monitoring schedule.

  • A patient randomized to treatment strategy d7.5 will

initiate TI when a physician first detects A1c ≥ 7.5%.

  • First time when A1c≥ 7.5%: just before quarter 9
slide-9
SLIDE 9

9

Impact of A1c Monitoring on CER Evidence

  • In a trial to evaluate dynamic treatment plans, the

intervention protocol would specify an A1c monitoring schedule.

  • A patient randomized to treatment strategy d7.5 will

initiate TI when a physician first detects A1c ≥ 7.5%.

  • First time when A1c≥ 7.5%: just before quarter 9
  • Detection time when A1c’s separated by
  • 1 quarter: quarter 9
slide-10
SLIDE 10

10

Impact of A1c Monitoring on CER Evidence

  • In a trial to evaluate dynamic treatment plans, the

intervention protocol would specify an A1c monitoring schedule.

  • A patient randomized to treatment strategy d7.5 will

initiate TI when a physician first detects A1c ≥ 7.5%.

  • First time when A1c≥ 7.5%: just before quarter 9
  • Detection time when A1c’s separated by
  • 1 quarter: quarter 9
  • 2 quarters: quarter 10
slide-11
SLIDE 11

11

Impact of A1c Monitoring on CER Evidence

  • In a trial to evaluate dynamic treatment plans, the

intervention protocol would specify an A1c monitoring schedule.

  • A patient randomized to treatment strategy d7.5 will

initiate TI when a physician first detects A1c ≥ 7.5%.

  • First time when A1c≥ 7.5%: just before quarter 9
  • Detection time when A1c’s separated by
  • 1 quarter: quarter 9
  • 2 quarters: quarter 10
  • 3 quarters: quarter 12
slide-12
SLIDE 12

12

Impact of A1c Monitoring on CER Evidence

  • In a trial to evaluate dynamic treatment plans, the

intervention protocol would specify an A1c monitoring schedule.

  • A patient randomized to treatment strategy d7.5 will initiate

TI when a physician first detects A1c ≥ 7.5%

  • First time when A1c≥ 7.5%: just before quarter 9
  • Detection time when A1c’s separated by
  • 1 quarter: quarter 9
  • 2 quarters: quarter 10
  • 3 quarters: quarter 12
  • Infrequent A1c testing leads to delayed TI for this patient.
slide-13
SLIDE 13

13

Impact of A1c Monitoring on CER Evidence

  • Trial inference about the comparative effectiveness of two identical dynamic plans is thus a function of

the A1c monitoring schedule chosen.

  • Similarly, CER evidence from observational data is also specific to the A1c monitoring frequency in the

cohort.

  • Methodological challenge and opportunity:
  • Generalizability problem: differences in monitoring protocols between two populations limits the

extrapolation of CER findings in one to the other

  • Generate new CER evidence to inform clinical monitoring decisions

We developed methods that can exploit the monitoring variability in EHR data to evaluate how monitoring and treatment decisions interact to impact health.

slide-14
SLIDE 14

14

Shortcomings of Existing Methods

  • Using EHR data from the TI study, we aimed to emulate a trial where participants would

be randomized to one of 6 arms:

dθ: “Patient initiates TI the first time a newly observed A1c≥ θ% and continues the intensified therapy thereafter” with θ=7.5; 8; 8.5% AND nX: A1c tests are separated by X quarters with X=1; 3.

  • Standard causal inference methods (IPW and TMLE) were used to evaluate

the effects of these 6 joint dynamic treatment and static monitoring interventions.

slide-15
SLIDE 15

15

Shortcomings of Existing Methods

  • When A1c’s are separated by one

quarter:

  • Effect of TI at lower A1c is mostly

protective

  • Wider confidence intervals

compared to original analyses without monitoring intervention

slide-16
SLIDE 16

16

Shortcomings of Existing Methods

  • When A1c’s are separated by 3

quarters:

  • Inconsistent and weak evidence
  • f a protective effect of TI at

lower A1c

slide-17
SLIDE 17

17

Shortcomings of Existing Methods

  • What explains the poor performance of standard IPW and TMLE?
  • Many patients follow the 3 dynamic treatment plans over two years.
  • Only patients with regular A1c tests contribute an outcome to standard analyses.
  • Very few patients exactly follow such rigid testing schedules over 2 years. Many

have more frequent or irregular A1c tests.

  • Example: ≈25,000 patients followed the dynamic treatment d8.5 through 1.5

years into the study but ≈1,000 did so while also having A1c tests collected every other quarter.

  • Small “sample sizes” in each exposure group explain the relative poor estimation

performance of standard analyses.

slide-18
SLIDE 18

18

Our Research to Address these Shortcomings

  • A five-pronged approach:
  • Develop theory to estimate the effect of a joint dynamic treatment and monitoring

intervention under a No Direct Effect assumption (NDE)

  • Empirically evaluate and illustrate resulting NDE-based IPW and TMLE estimators

with EHR data from TI study

  • Develop software to disseminate the novel estimation approaches
  • Validate theoretical findings and software implementation with simulated data
  • Seek guidance from stakeholder partners to improve practical relevance
slide-19
SLIDE 19

19

Results

  • Novel analytic tools can provide more precise estimates of the effects of

joint dynamic treatment and monitoring interventions.

  • Practical relevance:
  • Improve the generalizability of evidence about the comparative

effectiveness of dynamic treatment plans to population with different monitoring standards

  • Optimize clinical monitoring decisions
slide-20
SLIDE 20

20

Illustration with TI Study

  • No Direct Effect (NDE) assumption:
  • “A1c monitoring decisions can only impact outcomes and other covariates through

treatment decisions”

  • Potential violation: A1c testing increases subsequent health-seeking behaviors that

decrease the risk of albuminuria onset/progression.

  • If the assumption holds, we can estimate a joint dynamic treatment and static

monitoring intervention using data from patients whose A1c is monitored more frequently than what the monitoring intervention requires.

slide-21
SLIDE 21

21

Illustration with TI Study

  • When A1c’s are separated by one

quarter:

  • Without NDE (left):
  • Effect of TI at lower A1c is

mostly protective

  • Wide confidence intervals
slide-22
SLIDE 22

22

Illustration with TI Study

  • When A1c’s are separated by one

quarter:

  • Without NDE (left):
  • Effect of TI at lower A1c is

mostly protective

  • Wide confidence intervals
  • With NDE (right):
  • Effect of TI at lower A1c is

always protective

  • Tight confidence intervals
slide-23
SLIDE 23

23

Illustration with TI Study

  • When A1c’s are separated by three

quarters:

  • Without NDE (left):
  • Inconsistent evidence of TI

protection at lower A1c

  • Wide confidence intervals
slide-24
SLIDE 24

24

Illustration with TI Study

  • When A1c’s are separated by three

quarters:

  • Without NDE (left):
  • Inconsistent evidence of TI

protection at lower A1c

  • Wide confidence intervals
  • With NDE (right):
  • Effect of TI at lower A1c is

mostly protective

  • Tight confidence intervals
slide-25
SLIDE 25

25

Illustration with TI Study

  • If patients initiate TI at first A1c ≥

8%:

  • Without NDE (left):
  • Testing A1c more frequently

is protective

  • Wide confidence intervals
slide-26
SLIDE 26

26

Illustration with TI Study

  • If patients initiate TI at first A1c ≥

8%:

  • Without NDE (left):
  • Testing A1c more frequently

is protective

  • Wide confidence intervals
  • With NDE (right):
  • Testing A1c more frequently

is protective

  • Tight confidence intervals
slide-27
SLIDE 27

27

Limitations

  • Analytic methods developed rely on the strong assumption of “no

unmeasured confounding” (NUC) for both the effect of treatment decisions and the effect of monitoring decisions on outcome.

slide-28
SLIDE 28

28

Acknowledgments

Investigators Alyce Adams • Richard Grant • Julie Schmittdiel • Oleg Sofrygin • Mark van der Laan TI Study Investigators Connie Trinacty • Kristi Reynolds • Denise Boudreau • Marsha Raebel • Gregory Nichols • Patrick O’Connor • Joe Selby Patient, Provider, & Healthcare System Stakeholders Arjun Varma • Patrick O’Connor • Edward Yu • Richard Brand • Juanita Thomas • Marc Jaffe • Michael Chae Visiting Fellow Noémi Kreif

slide-29
SLIDE 29

29

Learn More

  • Manuscripts
  • Neugebauer, R, Schmittdiel, JA, Adams, AS, Grant, RW, van der Laan, MJ (2017). Identification of the

joint effect of a dynamic treatment intervention and a stochastic monitoring intervention under the no direct effect assumption. J Causal Inference, 5, 1.

  • Kreif, N, Sofrygin, O, Schmittdiel, JA, Adams, AS, Grant, RW, Zhu, Z, van der Laan, MJ, Neugebauer, R

(2018). Evaluation of adaptive treatment strategies in an observational study where time-varying covariates are not monitored systematically. In review. https://arxiv.org/abs/1806.11153

  • Software
  • Sofrygin O, van der Laan MJ, Neugebauer R (2016). stremr R package.

https://github.com/os9ofr/stremr.