No data left behind making the case with datasets or evidence - - PowerPoint PPT Presentation

The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no [115303], resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

No data left behind – making the case with datasets or “evidence synthesis for learning about real world effectiveness”

17th June 2016 Matthias Egger, University of Bern Chrissie Fletcher, Amgen

The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no [115303], resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

Disclaimer (Chrissie Fletcher)

• The views expressed herein represent those of the presenter

and do not necessarily represent the views or practices of Amgen or the views of the general Pharmaceutical Industry.

2 2

The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no [115303], resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

Agenda

• Key questions we’re addressing
• Latest thinking on methods and guidance for evidence

synthesis and predicting effectiveness

• Case study illustrations

– Combining Randomised Controlled Trials (RCTs) and Observational Data: Schizophrenia – Predicting effectiveness from efficacy: RA

• Software & tools to aid implementation
• Information and materials for stakeholders
• How you can help and provide feedback

3

The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no [115303], resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

Key questions we’re addressing

4

The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no [115303], resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

5 Egger, Fletcher, Moons. JRSM 2016

Questions Outcomes Applicability Data sources Evidence synthesis Conditions 1) How efficacious and safe is this drug? Efficacy, safety Typical patients included in clinical trials Phase II/III randomised clinical trials Clinical trials, standard meta- analysis Study conditions 2) How efficacious and safe is this drug compared to alternative therapies? Relative efficacy, relative safety Typical patients included in clinical trials Phase II/III randomised clinical trials Network meta- analysis Study conditions 3) How effective and safe is this drug compared to alternative therapies, in the patients who will likely receive it post- launch? Relative effectiveness, relative safety in predicted study populations Patients predicted to receive the drug post-launch Phase II/III randomised clinical trials, clinical databases and registries Individual patient data (IPD) network meta-analysis and meta-regression Study conditions 4) How effective and safe is this drug compared to alternative therapies, in the patients who will likely receive it in the real world of a health care system? Relative effectiveness, relative safety in predicted real world populations Patients predicted to receive the drug post-launch in a given health care system Phase II/III randomised clinical trials, clinical databases and registries, expert opinion, patient preferences Mathematical modelling Real world conditions

Egger, Fletcher, Moons. JRSM 2016 5

The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no [115303], resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

Latest thinking on methods and guidance for evidence synthesis and predicting effectiveness

6

The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no [115303], resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

We performed three systematic reviews on methods for:  network meta-analysis (NMA)  individual participant data (IPD) meta-analysis  mathematical modelling to predict real-world effectiveness based on evidence from randomized controlled trials (RCTs) Our aim was to identify and describe state-of-the-art methods in these three research areas, to summarize methodological challenges and limitations and to give recommendations on the use of the discussed methods. All three reviews were accepted for publication in the Research Synthesis Methods journal

7

The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no [115303], resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

• 1. Get Real in network meta-analysis: a

review of the methodology

Orestis Efthimiou, , Thomas P. A. Debray, Gert van Valkenhoef, Sven Trelle, Klea Panayidou, Karel G. M. Moons, Johannes B. Reitsma, Aijing Shang and Georgia Salanti

8

The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no [115303], resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

• Guidance:

 Presented the advantages and limitations of alternative approaches.  Discussed in depth methods to assess the validity of the underlying assumptions  Provided technical details regarding a series of special issues: network meta- regression, accounting for the risk of bias, multiple outcomes and repeated measures, defining the number of nodes, planning future studies, etc.  Listed software tools for fitting NMA and for assessing its assumptions.

• Our review constitutes the most comprehensive collection of methods for NMA to

date and can be a valuable tool for both experienced researchers as well as researchers taking their first steps in NMA

Efthimiou, O., Debray, T. P. A., van Valkenhoef, G., Trelle, S., Panayidou, K., Moons, K. G. M.,Reitsma, J. B., Shang, A., Salanti, G., GetReal in network meta-analysis: a review of the methodology. Res. Syn. Meth.

9

The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no [115303], resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

Assumption in NMA

Transitivity, congruence, consistency….

The treatments we compare are in principle jointly randomizable The groups of studies that compare them do not differ with respect to the distribution of effect modifiers

They have the same indication, I can imagine a mega-trial with all treatments being compared etc

You can test this assumption if you have enough studies per comparison

Direct and indirect treatment effects are in statistical agreement

Various statistical tests

In the outset When you find the studies When you extract the

• utcomes

Cipriani A et al. Conceptual and Technical Challenges in Network Meta-analysis Annals of Internal Medicine 2013

10

The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no [115303], resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

• 2. GetReal in meta-analysis of individual

participant data: a review of the methodology

Thomas P. A. Debray, Karel G. M. Moons, Gert van Valkenhoef, Orestis Efthimiou, Noemi Hummel, Rolf H. H. Groenwold, Johannes

• B. Reitsma

11

The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no [115303], resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

Guidance

• Advantages and limitation of existing approaches for IPD-MA
• Description of statistical methods and underlying assumptions

– Investigating heterogeneity of treatment effect – Combining IPD and published AD – Dealing with missing participant data – Modelling different types of outcomes – Including evidence from non-randomized studies

• Overview of existing software tools
• Example code in the R software package

12

The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no [115303], resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

Considerations

• Implementation of IPD-MA requires additional effort and statistical

expertise

• IPD-MA should not be conducted without systematic review
• IPD-MA is no panacea against poorly designed and conducted primary

research

Recommendations

• Before undertaking an IPD-MA, it may be helpful to perform a meta-

analysis of aggregate data (AD)

• Researchers should carefully assess whether the potential advantages of

IPD outweigh the extra effort involved

13

The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no [115303], resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

• 3. GetReal in mathematical modelling: a

review of studies predicting drug effectiveness in the real world

Klea Panayidou, Sandro Gsteiger, Matthias Egger, Gablu Kilcher, Maximo Carreras, Orestis Efthimiou, Thomas P. A. Debray, Sven Trelle, Noemi Hummel

14

The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no [115303], resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

Findings

• We searched the literature for methods used to predict real-world effectiveness
• f drugs from randomized controlled trial (RCT) efficacy data
• We identified four approaches used in only 12 articles: multi-state models,

discrete event simulation models, physiology-based models, and survival and generalized linear models.

• Outcomes were predicted over time, for new patient populations and drug doses.
• Most studies included sensitivity analyses, but external validation was done in
• nly three studies.
• Methods predicting real-world effectiveness are not widely used at present, and

are not well validated.

• The articles are included in a publicly available, online database

15

The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no [115303], resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

Case study illustrations

16

The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no [115303], resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

Case Study Applications

Based on the findings from our three systematic reviews, we employed the following case studies:

• Case study depression: to explore methods for the meta-analysis
• f individual patient-level data.
• Case study schizophrenia: to extend methods for a joint network

meta-analysis of RCTs and observational data.

• Case study rheumatoid arthritis: to explore methods on

predictive modelling using RCT and observational data.

17

The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no [115303], resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

Schizophrenia case study: insights and recommendations

The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no [115303], resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

Aim of the case study

• To assess existing methodology and develop new methods for

combining evidence on relative treatment effects coming from RCTs as well as non-randomized studies in a network meta-analysis (NMA).

Case study: schizophrenia

• 168 RCTs. Study-level data were available.
• 1 large cohort study, SOHO: 11.000 patients, 5 cohorts. Patient-

level data available.

Network of 15 antipsychotic drugs in schizophrenia

` `

ZOT ZIP SER RIS QUE PBO PAL OLA LURA ILO HAL CPZ CLO ASE ARI AMI

The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no [115303], resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

Estimating agreement between sources of evidence

4 3 2 1 5 4 3 2 1 5

Network of RCTs Network of NRSs

• Direct randomized
• Indirect randomized (via drug 2)
• Direct non-randomized
• Indirect non-randomized (via drug 1)

The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no [115303], resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

For each treatment comparison there may be up to 4 different types of evidence

Estimating agreement between sources of evidence

• .4
• .2

.2 .4

4vs15

Direct randοmized Indirect randomized Direct observational

Direct ≠ Indirect

• Direct observational
• Indirect observational
• Direct randomized
• Indirect randomized
• .3
• .2
• .1

.1

4vs6

RCT ≠ RWE

The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no [115303], resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

Synthesis of RCTs and RWE

Differences in the credibility of RWE can be encompassed in

• 1. Design-adjusted analysis
• 2. Informative priors from RWE
• 3. A three-level hierarchical model

The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no [115303], resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

Synthesis of RCTs and RWE

Higher risk of bias and large precision?

• .2
• .1

.1 .2 .3

The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no [115303], resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

Synthesis of RCTs and RWE

• .2
• .1

.1 .2 .3

β bias correction Higher risk of bias and large precision? w precision correction

The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no [115303], resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

We reduce the weight

• f the RWE by dividing the

variance by w

• .2
• .1

.1 .2 .3

β bias correction

Synthesis of RCTs and RWE

Higher risk of bias and large precision? w precision correction

The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no [115303], resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

Design-adjusted analysis

• Adjust each study separately

– For bias (add β to the summary effect) – Decrease the weight it carries in the summary effect by w – w = 1 : RWE taken at face value – w = 0 : ignore RWE

• Pinpointing exact values for β and w may be a difficult task

– Needs expert opinion – Sensitivity analyses are necessary

By changing the value of w researchers can control the amount of confidence they want to place to the RWE

The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no [115303], resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

Design-adjusted analysis: Results

• .1

.1 .2

SMD

w=0

4v6

w=0.2 w=0.5 w=0.8 w=1 RCTs only Naive pooling

Results for the other comparisons are even less sensitive to the amount of confidence placed in RWE No bias adjustment (β=0), a single w parameter (only one non-randomized study)

Less confidence to RWE More confidence to RWE

The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no [115303], resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

Using non-randomized evidence as prior information

• Observational studies can be viewed as prior-knowledge which when

combined with the observed data gives a posterior summary effect

• Adjust for bias and downweight the prior distribution to address concerns
• f bias and over-precision

Prior: RWE Likelihood: RCT Posterior

+

Dividing the variance of the prior distribution by w ≈ raise the likelihood function to power α

The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no [115303], resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

• Estimates of relative treatment effects obtained from NRS need to be adjusted at

the study level, using valid statistical methods. IPD is necessary for that.

• Before combining evidence from different sources one needs to check their

compatibility

• Several approaches can be used to jointly synthesize randomized and non-

randomized evidence. The choice between them can be driven by considerations related to data availability and also the resources and the technical expertise available in the research team

• The quality of the evidence needs to be assessed whatever method used
• Sensitivity analyses should always take place.

Combining randomized and non-randomized evidence in a network meta-analysis Efthimiou O, Debray TPA, Samara M, Leucht S, Belger M, Mavridis D and Salanti G, submitted

Key points

The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no [115303], resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

Learning and Predicting Real-World Treatment Effect based on RCTs and Observational Data: A Case Study in Rheumatoid Arthritis

31

The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no [115303], resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

Motivation

• Obvious gap in treatment oucome

Conventional DMARDs Biologic treatment

Change in DAS28 after 6 months

The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no [115303], resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

Research Question

Set up a mathematical model that allows to predict the real-world effect

• f a new biologic treatment in patients with

Rheumatoid Arthritis (RA) if…

• nly RCT data on the new treatment and …
• no observational data on the new treatment, but …
• bservational data on an existing similar treatment …

are available?

33

The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no [115303], resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

Graphical Model Representation

• Directed acyclic graph visualizing RCT conditions
• Directed acyclic graph visualizing real-world conditions

Covariates (C)  Confounders Treatment (Trt) Outcome (Y) Covariates (V)  Non-Confounders Covariates (B) Covariates (X) Treatment (Trt) Outcome (Y)

C V B

34

EyM EyM

The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no [115303], resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

Variable Classification and Selection

Outcome: Change in RCT DATA EyM RWE Covariates B Covariates V Confounders C

DAS28 age calendar year BMI/obesity age HAQ disease duration hospital (y/n) gender disease duration EQ5D BMI/obesity socio-economics steroid intake seropositivity ACR seropositivity …… # [concomitant DMARDs] baseline DAS28 CDAI gender baseline HAQ # [previous anti- TNF agents] RADAI # [previous anti- TNF agents] type of concomitant DMARDs …. # [previous DMARDs]

….. ….. …… smoking comorbidities

……

Confounders (C) Treatment Outcome (Y) Covariates (V) Covariates (B)

E x p e r t (RA)

Stats (Cross- valid.) Not selec- ted

DAS28 – Disease activity score (28 joints) HAQ – Health assessment questionnaire DMARD – Disease modifying anti-rheumatic drug TNF – Tumor necrosis factor BMI – Body mass index

35

The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no [115303], resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

• 1. Develop a mathematical model, informed by …
• bservational evidence on treatment decision
• RCT(s) on the efficacy of the new treatment, and on all significant effect modifiers

and prognostic factors

• 2. Predict real-world treatment effect

for the RCT population(s)

• Predict treatment decision based on RWE
• Predict treatment outcome, using evidence from the available RCT(s)
• 3. Predict treatment effect for a real-world patient population,

using evidence from the available RCT(s)

Modelling Concept

36

The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no [115303], resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

Predicted vs. Observed

Findings – RCT population:

• Predicted effectiveness is lower than
• bserved efficacy
• nly 99 out of 1214 trial participants

would receive the biologic agent

• Predicted effectiveness is higher than

effectiveness observed in real-world

• strict RCT inclusion criteria

Conventional DMARDs New biologic treatment

Change in DAS28 after 6 months

The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no [115303], resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

Findings – RCT population:

• Predicted effectiveness is lower than
• bserved efficacy

 only 99 out of 1214 trial participants .

. would receive the biologic agent

• Predicted effectiveness is higher than

effectiveness observed in real-world

Conventional DMARDs New biologic treatment

Change in DAS28 after 6 months

Findings – real-world population:

• Predicted real-world effect of the

new biol. treatment exceeds the

• bserved real-world effect of the

existing similar agent

Predicted vs. Observed

The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no [115303], resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

Predicted Effectiveness vs. Observed Efficacy

Findings – RCT population:

• Predicted effectiveness is lower than
• bserved efficacy

 only 99 out of 1214 trial participants .

. would receive the biologic agent

• Predicted effectiveness is higher than

effectiveness observed in real-world  strict RCT inclusion criteria

Conventional DMARDs New biologic treatment

Change in DAS28 after 6 months

Findings – real-world population:

• Predicted real-world effect of the

new biol. treatment exceeds the

• bserved real-world effect of the

existing similar agent

• Predicted and observed effects of the

conventional DMARDs differ notably

• unconsidered effect modifiers

and prognostic factors?

• insufficient prior information?

Effectiveness Efficacy/ Effectiveness

The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no [115303], resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

Discussion

Deliverable

Bayesian inference framework to connect information from various sources

• Prediction of real-world treatment effect
• Assessment of the efficacy-effectiveness gap
• Main concerns: Predictive and external validity
• Work in progress:
• Inclusion of results from network meta-analyses to predict relative drug

effectiveness

• Consideration of dynamic treatment regimes with time-varying confounders

and censoring information

RCTs RWE Individual participant data Aggregate data

40

The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no [115303], resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

Software & tools to aid implementation

41

The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no [115303], resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

ADDIS – GetReal software platform

42

The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no [115303], resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

ADDIS – web based user interface

43

• Make methods

accessible to a broader audience

• Promote best practice in

the use of methods

• Prevent coding errors

through automation

• Eliminate duplication of

effort in data extraction

The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no [115303], resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

ADDIS – analysis code

• All analyses built as R packages

– Open source – peer review

• Models based on established best practice

– NICE DSU guidance (Dias et al.)

44

Other analysis code

• R packages used in the case studies will be available in each of

the case study publications

• Some (but not all) of the approaches investigated in the case

studies have been incorporated into ADDIS

The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no [115303], resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

Information and materials for stakeholders

45

The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no [115303], resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

A variety of materials have been developed to present and illustrate the analytical approaches

• Methodological guidance and best practice recommendations

for (aggregate and IPD) evidence synthesis and predictive modelling of effectiveness

• Software tools for conducting evidence synthesis and

predictive modelling of effectiveness

• Education and training materials
• Publications, conference presentations and posters
• Communications and discussions with multi-stakeholders e.g.

workshops

46

The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no [115303], resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

How you can help and provide feedback

47

The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no [115303], resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

We need your input and perspectives on:

48

1. What do you think about these methods using RCT and RWE for evidence synthesis and predictive modelling? Do you have any concerns and how could these be overcome? 2. Will these methods be useful for your relative effectiveness projects? How would you use these methods? 3. Will these methods and the results be acceptable to address questions relating to relative effectiveness? 4. Are there any key issues that have not been addressed / considered? 5. Would you consider using the ADDIS software platform / analytic tools to conduct relative effectiveness projects? Would you need any additional support to use these tools?