Prof. Dr. phil. Gabriele Meyer Martin Luther University - - PowerPoint PPT Presentation

Intervention Studies:

Principles, Opportunities and Pitfalls

• Prof. Dr. phil. Gabriele Meyer

Martin Luther University Halle-Wittenberg, Germany

• 20 English language nursing journals with highest IF (1.221-

2.103) Inclusion: 223 studies from 21 EU-European countries Results: 34% report on nursing interventions 45% observational studies 39% qualitative 12% experimental 4% randomised controlled trials

Confirmed by: Mantzoukas Int J Nurs Stud 2009; Forbes Int J Nurs Stud 2009; Yarcheski et al. Int J Nurs Stud 2012

LINKING EVIDENCE TO ACTION

• Researchers in nursing should design, undertake, and report fewer

descriptive studies and more experimental research into the effectiveness of nursing interventions to ensure a more balanced proportion of intervention and descriptive research in nursing.

• Researchers should structure their studies to explicitly link the

development, testing, evaluation, and implementation of nursing interventions in coherent programs of research activity rather than as stand-alone projects.

LINKING EVIDENCE TO ACTION

• Nursing researchers should consider using the UK Medical Research

Council’s “Complex Interventions Research Framework” to organize studies that will deliver an increased evidence base for nursing interventions.

• Doctoral education programs for nurses should encourage students

to undertake experimental work into the efficacy and effectiveness

• f nursing interventions.

(Craig et al. 2012; IJNS)

Examples

• Organisation

– Stroke unit

• Health professions

– Guideline implementation

• Defined Groups

– School based programmes for smoking cessation

• Individuals

– Lifestyle intervention in diabetes

Terms used for complex interventions

• Multicomponent
• Multifaceted
• Multifactorial

Introduction of such a system did not significantly reduce the incidence of our study outcomes. Possible explanations for our findings are that the MET system is an ineffective intervention; the MET is potentially effective but was inadequately implemented in our study; we studied the wrong outcomes; control hospitals were contaminated as a result of being in the study; the hospitals we studied were unrepresentative; or our study did not have adequate statistical power to detect important treatment effects.

Interventional study designs

• Also called „experimental study designs“
• Where the researcher intervenes at some point throughout

the study

• To evaluate study questions related to either treatment

(prophylactic agents, treatments, surgical approaches, diagnostic tests) or prevention (protective equipment, engineering controls, management, policy or any element that should be evaluated as to a potential cause of disease or injury)

• Strongest design: randomised controlled trial
• Other: Pre-post study design, non-randomised controlled

trials, quasi-experiments

Most RCTs aim to determine whether one intervention is superior to another. Equivalence trials aim to determine whether one (typically new) intervention is therapeutically similar to another, usually an existing treatment. A non-inferiority trial seeks to determine whether a new treatment is no worse than a reference treatment. Proof of exact equality is impossible, therefore, a pre-stated margin

• f non-inferiority for the treatment effect in a primary patient outcome is defined.

Heal et al. BMJ 2006; 332

Explanatory trial Pragmatic trial Benefit of a treatment under ideal conditions (efficacy) Benefit of a treatment in routine clinical practice (effectiveness) Homogeneous population as possible; aims primarily to further scientific knowledge Variations between patients as in real clinical practice; aims to inform choices between treatments Standardised intervention Protocol based administration of different treatment to IG Intermediate outcomes often used Full range of health gains

Bias

„Any process at any stage of inference

tending to produce results that differ systematically from true values.“

(Murphy 1976)

Internal validity

Threated by

Selection bias Performance bias Detection bias Attrition bias

Blinding …

• minimizes
• cointervention bias (differential use of cointerventions),
• attrition bias (differential patient dropout)
• response bias (differential reporting of symptoms)
• ensures a similar degree of placebo effects in compared groups
• protects against observer/detection/ascertainment bias

Trials that are not double blind exaggerate treatment effects (odds ratios) by 13%, and by 23% when outcomes are subjective

(Savović J et al. Ann Intern Med 2012)

Schulz KF & Grimes DA Lancet 2002

Conclusions: The risk of selection bias could not be ascertained for most trials due to poor

• reporting. Many trials which did provide details on the randomisation procedure

were at risk of selection bias due to a poorly chosen randomisation methods. Techniques to reduce the risk of selection bias should be more widely implemented.

n=475 RCTs with mITT

Modified ITT (Anraha et al. BMJ 2010)

• Treatment: „mITT consisted of patients who

received at least six doses of study drug.“

• Baseline assesment: „mITT included patients

with at least one baseline observation.“

• Target condition: „The mITT population

consisted of those patients who were randomly assigned to study treatment minus those who were not H pylori positive.“

Modified ITT (Anraha et al. BMJ 2010)

• Post-baseline assessment: „mITT includes all

randomized patients who have . . . at least one post-baseline measurement.”

• Follow-up: “All participants who completed

follow-up were analyzed as a part of the group to which they were randomized. This was not a strict intent-to-treat analysis as some study participants were lost to follow-up.”

Time for a short exercise

Stages of waste in the production and reporting

• f research evidence

• Regulation bodies are not in charge (e.g. FDA, EMA)
• Prospective regulation voluntary; recommendations by International Committee
• f Medical Journal Editors, WHO Statement on Public Disclosure of Clinical Trial

Results, World Medical Association’s Declaration of Helsinki, Reporting Guidelines (e.g. CONSORT, SPIRIT), AllTrials campaign, United Nations  non-regulated RCTs less often registered as regulated RCTs, although 40% of all published RCTs

• Survey, 220 RCTs from clinical geriatric journals
• Published RCT registered in a publicly accessible register?
• Prospectively registered before participants‘ recruitment?
• Agreement between registration, published study protocol

and published report of results?

• 140/220 RCTs registered.
• Registration in only 15% of RCTs prospectively.
• Half of RCTs report on results of registered outcomes or at

least refer to registered outcomes.

• Time of enrolment of participants remains unclear in one third
• f registered trials and half of non-registered studies.

Sample: 133 trials with 137 interventions Outcome instrument: 8-item checklist Results: 53/137 (39%) were adequately described; after contact to authors: n=81 (59%) according to 63 answers of 88 contacted authors.

Transparency in reporting of complex interventions (Möhler et al. 2011) No studies (out of 5) Intervention … Theoretical basis

3

Piloting Costs Education… Description of participants

3

Curriculum

2

Access to education material Process evaluation Description standard care Implementation of study protocol Staff turnover

2

CReDECI

• Designed to integrate relevant aspects of the complete

research process of development and evaluation of a complex intervention

• 13 Items in 3 sections:

– development (n=4 items) – feasibility and piloting (n=1) – introduction of the intervention and evaluation (n=8)

• Not focussing on a particular study design

What are cluster randomised controlled trials?

cRTCs are experiments in which (interacting) social units rather than individuals are randomly allocated to study groups: communities, schools, families, hospitals, nursing homes …

Reasons for cRCT?

• How many levels are involved?

– General practice - patient – Nursing home - ward - resident – Single person (with limbs, teeth, eyes)

Trials with one cluster per arm?

• Minimum number of clusters per arm to

ensure a valid analysis should be at least four

(Hayes & Moulton, 2008)

Challenges of cRCT

• Outcome for each participant cannot be assumed to

be independent of that for any other participant since those within the same cluster are more likely to have similar outcomes (clustering effect).

• The reduction in effective sample size depends on

average cluster size and the degree of correlation within clusters, ρ, also known as the intracluster (or intraclass) correlation coefficient (ICC).

Challenges of cRCT

• Standard sample size formulas will lead to

underpowered studies  larger sample sizes are required

• Cluster adjustment has to be taken into

account for statistical analysis to avoid unit of analysis bias

Sample size calculation cRCT - example

• Incidence: 60 hip fractures/ 1000 residents/ year
• Effect size: Relative risk reduction 50%
• Observation time: 18 months
• β = 80%,  = 5%
• For cluster design: Number and size of clusters (eligible for

recruitment and participation, patients/residents per cluster)

Participant randomisation

n = 384

Cluster randomisation Design effect = 1+(m-1)p=1.24

n = 384 x 1.24 = 477

Sample size calculation cRCT - example

Unfeasable trial

Provided an ICCC of 0.04 and a mean cluster size of 200, more than 900 clusters with a total of 186,000 patients were required to demonstrate a relative risk reduction of 33%.

A football stadium holds about 40 000 fans. Our trial requires 4.7 times the number of participants shown.

The problem of cluster baseline imbalance

• Many cRCTs do not have enough clusters to create

a reasonable expectation for cluster-level balance.

• Ivers et al. (BMJ 2011); n=300 cRCTs, published

between 2000 and 2008

– Median number of clusters 21 – 25% had fewer than 12 clusters – 14% had less than 4 clusters

• Different techniques of restricted randomisation

have been discussed to reduce baseline imbalance: stratification, pair-wise matching, etc.

Example: Köpke et al. JAMA 2012

• Computer-generated randomization lists were

used for allocation of clusters in blocks of 4, 6, and 8 nursing homes. Randomization was stratified by region, ie, Hamburg and Witten.

Example: Köpke et al. JAMA 2012

Nursing homes were eligible if they had a self-reported rate of at least 20%

• f residents with physical restraints, assessed by a short questionnaire

completed by the head nurses.

Example: Köpke et

• al. JAMA

2012

Other challenges

• Poor design of cluster trials risks bias in

selection of participants  post- randomisation recruitment bias

– Clusters are randomly allocated and participants are consecutively recruited – Risk of biased application of eligibility and recruitment criteria  selective inclusion

• How to solve this problem?

Solution

• Ideally participants should be identified

before the cluster is randomised

• When this is not possible recruitment should

be by someone masked to the cluster allocation

How could a user of a publication on a cRCT detect a possible recruitment bias?

• More people recruited when in intervention group
• Differential refusal to consent rate between the conditions
• Baseline characteristics’ imbalance

Attrition bias

Why worse in cRCTs than in iRCTs?

Cluster attrition

(data from Puffer et al. BMJ 2003)

Study No of clusters No of participants Description Clusters lost after randomisation Flottorp 2002 142 12 369 To improve general practice management of sore throat and urinary tract infections 22 King 2002 116 410 Behavioural therapy to treat patients with depression 32 Morrison 2001 221 689 Infertility guidelines for general practitioners 7 Olivarius 2001 311 1470 Structured personal care

• f type 2 diabetes

mellitus 10

How to avoid attrition bias or reduce ist impact?

1 nursing home in the control group denied to collect data

• n accidental falls (secondary
• utcome: fall events and hip

protector use during fall) after randomisation. Primary outcome: hip fracture

Other challenge of cRCTs: Blinding

• Participant blindness (nurses, physicans,

patients/residents) to intervention assignment (Cave: proven criterion of internal validity) is difficult or impossible to maintain in cRCTs.

• What could be done?
• Outcome assessors should be blinded.

Please discuss with your neighbour how blinding of

• utcome assessor
• could be guaranteed
• controlled for whether successfully conducted

Example: Köpke et al. JAMA 2012

Intervention component Rationale

Declaration Proven strategy in own previous studies Structured 90-minute information program for all nursing staff Cochrane review; Theory of planned behavior External structured 1-day intensive training workshop for nominated key nurses Proven strategy in previous studies Structured support for key nurses Proven strategy in previous studies Practice guideline Guideline implementation recommendations Printed supportive material for nurses, relatives, legal guardians Guideline implementation recommendations Publications on involvement of significant others Other supportive material Guideline implementation recommendations

To check for effective blinding, during the second measurement point at 3 months, external researchers were asked about their perception of the visited cluster’s group allocation using a short questionnaire. If raters visited the cluster at 2 or all 3 time points, the questionnaire was completed at the last visit. During 69 visits at the 3-month follow-up, 37 ratings (53.6%; 95% CI, 41.2%- 65.7%) were correct in identifying clusters as an intervention group or control group cluster. Example: Köpke et al. JAMA 2012

Stepped wedge design

Randomisation in terms

• f the period for receipt
• f the intervention 

type of cluster crossover trial if the unit of randomisation is a cluster.

Thank you very much for your attention!