SLIDE 1
Rethinking Evidence Synthesis
Prof Enrico Coiera Director, Centre for Health Informatics Australian Institute of Health Innovation Macquarie University Sydney, Australia
SLIDE 2 Variation in care is high
- Caretrack study found 57% of Australians receive care in
line with level 1 evidence or consensus guidelines
(Med J Aust 2012; 197 (2): 100-105.)
- Causes for practice variation include:
- Patient specific needs e.g. co-morbidity
- Patient preferences
- Clinician preferences
- Working with out of date evidence
SLIDE 3 Evidence synthesis is slow
In Australia we don’t always deliver the care that guidelines and experts agree
22794056
Systematic reviews can take years to complete and are extremely resource-intensive, so many are
- ut of date – some as soon as they are published.
20644625
3
Systematic reviews could be updated as soon as a new study results are available (this means we need to do the right trials).
17638714
SLIDE 4 Clinical evidence is often biased
Due to biases in the design, undertaking, reporting, and synthesis in clinical research, about 85% of it is wasted.
24411643
Trials that are funded by industry are less likely to be published within 2 years, and when they are, they are more likely to have favourable results.
20679560
When trials are published, some outcomes are incompletely reported or not reported at all. Safety
- utcomes are affected more than efficacy outcomes.
25285542
When reviewers and systematic reviewers synthesise the results from many clinical studies, those with financial conflicts of interest are more likely to report favourably.
23861749
4
SLIDE 5 RCTs and guidelines have limitations
- They do not represent real-world populations:
- Co-morbidities are excluded
- May be highly geographically localized introducing biases
- Often are too small to detect small effect sizes and too
short to detect long-term effects.
- Patients have their own preferences once benefits and
harms are explained.
SLIDE 6 Panel
6
- Dr. Guy Tsafnat “The automation of evidence
summarisation”
- Dr. Julian Elliott “Combining human effort and machines”
- Dr. Adam Dunn “When biases in evidence synthesis lead to
harm or waste”
- Dr. Blanca Gallego-Luxan “Learning from ‘patients like
mine’”
MQ | AIHI I CHI
Leads the Computable Evidence Lab which is dedicated to automation and optimisation of evidence based medicine Head of Clinical Research at Alfred Hospital and Monash University and Sr Researcher at Australasian Cochrane Centre Leads the Computational Epidemiology Lab, monitoring biases in the design, reporting, & synthesis of clinical trials Leads the Health Analytics Lab, designing, analysing and developing models derived from complex empirical data
SLIDE 7 Panel
7
- Dr. Guy Tsafnat “The automation of evidence
summarisation”
- Dr. Julian Elliott “Combining human effort and machines”
- Dr. Adam Dunn “When biases in evidence synthesis lead to
harm or waste”
- Dr. Blanca Gallego-Luxan “Learning from ‘patients like
mine’”
MQ | AIHI I CHI
Leads the Computable Evidence Lab which is dedicated to automation and optimisation of evidence based medicine Leads the Computational Epidemiology Lab, monitoring biases in the design, reporting, & synthesis of clinical trials Leads the Health Analytics Lab, designing, analysing and developing models derived from complex empirical data Head of Clinical Research at Alfred Hospital and Monash University and Sr Researcher at Australasian Cochrane Centre
SLIDE 8 8 AIHI I CHI I CEL
Systematic Reviews
A robust model for evidence based medicine
* Tsafnat, Glasziou, Choong, et al. Sys Rev 3:74 2014
SLIDE 9
Preparation Appraisal Synthesis Meta-Analysis
The Manual Process
* Tsafnat, Glasziou, Dunn, Coiera The BMJ, 346:f139, 2013
Retrieval Write-up
SLIDE 10
Automation
* Tsafnat, Glasziou, Dunn, Coiera The BMJ, 346:f139, 2013
Preparation Appraisal Synthesis Meta-Analysis Retrieval Write-up
SLIDE 11
Automation
* Tsafnat, Glasziou, Dunn, Coiera The BMJ, 346:f139, 2013
SLIDE 12 12 AIHI I CHI I CEL
Guidelines Clinical Queries
SLIDE 13 Search Automation
13
Saved strategies
AIHI I CHI I CEL
SLIDE 14 Citation Networks
14
*Robinson, Dunn, Tsafnat, Glasziou, Journal of Clinical Epidemiology 67(7) 2014
AIHI I CHI I CEL
SLIDE 15 Information Extraction from Trials
15
* Kiritchenko et al., BMC Med Inform Decis Mak , 10, 2010 (text from Kawamura et al. Dev med child neuro 49, 2007)
AIHI I CHI I CEL
(from Kawamura et al. Dev med child neuro 49, 2007)
This study compare the effects of low and high doses of botulinum toxin A
(BTX-A) to improve upper extremity function. Thirty-nine children (22 males, 17 females) with a mean age of 6 years 2 months (SD 2y 9mo) diagnosed with spastic hemiplegia or triplegia were enrolled into this double-blind, randomized controlled trial. The high-dose group received BTX-A in the following doses: biceps 2U/kg, brachioradialis 1.5U/kg, common flexor origin 3U/kg, pronator teres 1.5 U/kg, and adductor/opponens pollicis 0.6U/kg to a maximum of 20U. The low-dose group received 50% of this dosage. Outcomes were measured at baseline and at 1 and 3 months after injection, and results were analyzed with a repeated-measures analysis of variance.
SLIDE 16
SLIDE 17
SLIDE 18 Panel
18
- Dr. Guy Tsafnat “The automation of evidence
summarisation”
- Dr. Julian Elliott “Combining human effort and machines”
- Dr. Adam Dunn “When biases in evidence synthesis lead to
harm or waste”
- Dr. Blanca Gallego-Luxan “Learning from ‘patients like
mine’”
MQ | AIHI I CHI
Leads the Computable Evidence Lab which is dedicated to automation and optimisation of evidence based medicine Leads the Computational Epidemiology Lab, monitoring biases in the design, reporting, & synthesis of clinical trials Leads the Health Analytics Lab, designing, analysing and developing models derived from complex empirical data Head of Clinical Research at Alfred Hospital and Monash University and Sr Researcher at Australasian Cochrane Centre
SLIDE 19 Julian’s video goes here
19
SLIDE 20 Panel
20
- Dr. Guy Tsafnat “The automation of evidence
summarisation”
- Dr. Julian Elliott “Combining human effort and machines”
- Dr. Adam Dunn “When biases in evidence synthesis lead to
harm or waste”
- Dr. Blanca Gallego-Luxan “Learning from ‘patients like
mine’”
MQ | AIHI I CHI
Leads the Computable Evidence Lab which is dedicated to automation and optimisation of evidence based medicine Leads the Computational Epidemiology Lab, monitoring biases in the design, reporting, & synthesis of clinical trials Leads the Health Analytics Lab, designing, analysing and developing models derived from complex empirical data Head of Clinical Research at Alfred Hospital and Monash University and Sr Researcher at Australasian Cochrane Centre
SLIDE 21 The evidence-practice disconnect
21 CENTRE FOR HEALTH INFORMATICS | AUSTRALIAN INSTITUTE OF HEALTH INNOVATION
Systematic reviews are fundamentally limited by the quality and transparency of the primary evidence on which they are based…
Solutions: (a) improve the quality and transparency of the studies that can be included in reviews, or (b) create new forms of evidence synthesis that do not rely on the current ways that clinical studies are reported.
Registering clinical trials (2003) 10.1001/jama.290.4.516
SLIDE 22 22 CENTRE FOR HEALTH INFORMATICS | AUSTRALIAN INSTITUTE OF HEALTH INNOVATION
Synthesis biases: when reviews include evidence selectively or when results and conclusions don’t match. Publication bias: when clinical studies are never published, or published after a long delay. Reporting bias: when reports of clinical studies miss or misrepresent parts of what was measured. Design bias: when clinical studies are not designed to answer the right questions at the right times.
40–62% of studies had ≥1 primary outcome changed, introduced, omitted. 10.1371/journal.pone.0066844 66% of trials had published results 10.7326/0003-4819-153-3-201008030-00006 Systematic reviews with COIs produced more favourable conclusions 10.7326/m14-0933 Industry statin trials used more surrogate outcomes, fewer safety outcomes, were faster. 10.1038/clpt.2011.279
SLIDE 23 23 CENTRE FOR HEALTH INFORMATICS | AUSTRALIAN INSTITUTE OF HEALTH INNOVATION
Sharing of patient-level data: The third movement in the push for completeness and transparency, with pressure
- n funders/companies – and the technologies they need.
Linking trial design to practice: Making (post-approval) clinical trials match practice to properly address safety and effectiveness – and the technologies they need. Bigger, better studies using EHRs: Connecting research and practice to fix enrolment and make trials much more efficient – and the technologies they need.
Right answers, wrong questions in clinical research. 10.1126/scitranslmed.3007649 A new architecture for connecting clinical research to patients through EHRs. 10.1136/amiajnl-2014-002727 A new future for clinical research through data sharing (YODA Project). 10.1001/jama.2013.1299
SLIDE 24 24 CENTRE FOR HEALTH INFORMATICS | AUSTRALIAN INSTITUTE OF HEALTH INNOVATION
www.researchintegrityjournal.com
www.biomedcentral.com
Editors-in-Chief: Stephanie Harriman (UK), Maria Kowalczuk (UK) Iveta Simera (UK), Elizabeth Wager (UK)
- Focuses on research into peer
review and research integrity
- High visibility – permanent,
unrestricted, free online access
- Highly-respected editorial board
- Rapid and thorough peer review
SLIDE 25 Panel
25
- Dr. Guy Tsafnat “The automation of evidence
summarisation”
- Dr. Julian Elliott “Combining human effort and machines”
- Dr. Adam Dunn “When biases in evidence synthesis lead to
harm or waste”
- Dr. Blanca Gallego-Luxan “Learning from ‘patients like
mine’”
MQ | AIHI I CHI
Leads the Computable Evidence Lab which is dedicated to automation and optimisation of evidence based medicine Leads the Computational Epidemiology Lab, monitoring biases in the design, reporting, & synthesis of clinical trials Leads the Health Analytics Lab, designing, analysing and developing models derived from complex empirical data Head of Clinical Research at Alfred Hospital and Monash University and Sr Researcher at Australasian Cochrane Centre
SLIDE 26 Context
Towards a Learning Health Care System Traditional Health Care System Learning Health Care System
- Patient care is integrated with medical research
FACILITATING Clinical practice continuously monitored, updated and improved
- Medical research is integrated with patient care
FACILITATING Research continuously informed and guided by clinical practice
New Methods
SLIDE 27 Our Research
Building Models to Support Decision Making at the point-of-care
Will my patient develop diabetes in the next 2 years? Is it safe to give Ibuprofen to my patient?
Is it likely that my patient will remain hospitalised in the next 5 days?
What treatments did patients like mine had?
Decision support at the point-
Experimental Evidence: Point-of-care RCT Observational Evidence: Predictive Models/ Cohort Studies at the point-of-care
SLIDE 28 Forecasting patient trajectories
Will a patient be: in hospital, at home or dead in the next week?
We simultaneously predict the probability of discharge, readmission and death for each of the next 7 days, throughout the patient’s hospitalisation. Average AUC per day per outcome class=0.8 (Death AUC=0.9)
ED
87 years old male Arrives by ambulance Triage: Urgent
ED
4 hours in hospital 8 panels of tests High Bilirubin Low Albumin Low Sodium Low Chloride High Creatinine Low eGFR High CRP High APTT
ICU
7 hours in hospital 12 panel tests Low RBC Low Haemoglobin Low Haematocrit Low Platelets High WBC High Neutrophils High Creatinine Low eGFR
Geriatrics
52 hours in hospital 16 panel tests Low RBC Low Haemoglobin Low Haematrocrit High WBC High Creatinine
Patient arrives to ED New information = Updated prediction
SLIDE 29 Forecasting patient trajectories
Will a patient be: in hospital, at home or dead in the next week?
0.2 0.4 0.6 0.8 1 Day 1 Day 2 Day 3 Day 4 Day 5 Day 6 Day 7 Probability
Home Hospital Death Discharge Cai et al, JAMIA 2015 (accepted)
SLIDE 30 Forecasting patient trajectories
Will a patient be: in hospital, at home or dead in the next week?
0.2 0.4 0.6 0.8 Day 1 Day 2 Day 3 Day 4 Day 5 Day 6 Day 7 Probability
Home Hospital Death Death Cai et al, JAMIA 2015 (accepted)
SLIDE 31
Bringing cohort studies to the bedside
Framework for a ‘green button’ to support clinical decision-making
Longhurst et al, Health Affairs, Vol 3 (7)1229-35,2014
Capability in the EHR system that resolves the tension between ‘evidence-based medicine’ AND ‘practice-based evidence’
Green Button
SLIDE 32 Bringing cohort studies to the bedside
Framework for a ‘green button’ to support clinical decision-making
From: http://shahlab.stanford.edu/greenbutton
Gallego et al, J. Comparative Effectiveness Research 4(3) 191-197, 2015
SLIDE 33 Panel Discussion
33
Dr Guy Tsafnat “The automation of evidence summarisation” Dr Julian Elliott “Combining human effort and machines” Dr Adam Dunn “When biases in evidence synthesis lead to harm or waste” Dr Blanca Gallego-Luxan “Learning from ‘patients like mine’”
MQ | AIHI I CHI
Leads the Computable Evidence Lab which is dedicated to automation and optimisation of evidence based medicine Head of Clinical Research at Alfred Hospital and Monash University and Sr Researcher at Australasian Cochrane Centre Leads the Computational Epidemiology Lab, monitoring biases in the design, reporting, & synthesis of clinical trials Leads the Health Analytics Lab, designing, analysing and developing models derived from complex empirical data
Prof Enrico Coiera “Rethinking evidence synthesis”
Director of the Centre for Health Informatics; information and communication technologies to support health service delivery
