A New Reporting Guideline for Trials of Social

A New Reporting Guideline for Trials of Social

A New Reporting Guideline for Trials of Social and Psychological Interventions: CONSORT-SPI ESRC Bath 2016 Prof Paul Montgomery- Oxford Dr Jane Dennis- Bristol Project funded by the UK Economic and Social Research Council Applied Behavioral/Social Scientists Live in Exciting Times UK What Works Network - Create, share, and use high-quality evidence on policy programs

and practices which combined receive public spending of more than 200 billion - Public health, social care, education, crime reduction, and economic growth US Social and Behavioral Science Team - Assists federal agencies in applying behavioural science insights to policies and operations - Improve public welfare, programme outcomes, and cost effectiveness Obamas Executive Order in Sept 2015 https://www.whitehouse.gov/the-press-office/2015/09/15/executive-order-using-behavioral-science-insights-better-serve-american Most Published Research May be False We have problems reproducing psychological science (Open Science

Collaboration, 2015) - Replications of 100 experimental and correlational studies - 97% of original studies vs 36% of replications had significant results Mean effect size was half the magnitude of the mean effect size of the original effects There is still more work to do to verify whether we know what we think we know Why Most Published Research Findings are False A research finding is less likely to be true (Ioannidis 2005): - when the studies are smaller - when effect sizes are smaller - when there is a greater number and lesser pre-selection of tested relationships

- where there is greater flexibility in designs, definitions, outcomes, and analytical modes - when there is greater financial and other interest and prejudice - when more teams are involved in chase of statistical significance 85% of Biomedical Research Funding ($210 Billion) Is Being Avoidably Wasted Several stages of research production may lead to waste (Moher 2015) How to Make More Published Research True Some research practices that may help increase the proportion of true research findings (Ioannidis 2014): - Large-scale collaborative research

Adoption of replication culture and reproducibility practices Registration (studies, protocols, analysis codes, datasets, raw data) Sharing (data, protocols, materials, software, and other tools) Containment of conflicted sponsors and authors More appropriate statistical methods Standardisation of definitions and analyses More stringent thresholds for claiming discoveries or successes Improvement of study design standards Improvements in peer review, reporting, and dissemination of research Better training of scientific workforce in methods and statistical literacy Research Transparency Can Increase Value, Reduce Waste The Lancet REWARD (REduce research Waste And Reward Diligence) Campaign Center for Open Science to improve openness and integrity of scientific

practices - Open Science Framework for transparent, cloud-based management of scientific projects - Transparency and Openness Promotion (TOP) Guidelines (Nosek 2015) Berkeley Initiative for Transparency in the Social Sciences (BITSS) Data Access and Research Transparency (DA-RT) Statement for social scientists Meta-Research Innovation Center at Stanford (METRICS) Laura and John Arnold Foundation (LJAF) Research Integrity Grants (over $80 million since 2012) Enhancing the Quality and Transparency of Health Research (EQUATOR) Network reporting guidelines Objectives Social and psychological intervention RCTs

Reporting Guidelines & CONSORT Developing CONSORT-SPI The CONSORT-SPI Checklist Project Publications Mayo-Wilson et al. (2013). Developing a reporting guideline for social and psychological intervention trials. Trials, 14, 242. Grant et al. (2013). Reporting quality of social and psychological intervention trials: a systematic review of reporting guidelines and trial publications. PLoS One, 8(5), e65442 Montgomery et al. (2013). Protocol for CONSORT-SPI: An Extension for Social and Psychological Interventions. Implementation Science, 8, 99. Editorials Grant et al. (2012). Development of a CONSORT extension for

interventions in public health and related disciplines. The Lancet, 380(Supp. 3). S14. Spreckelsen et al. (2012). Letters: Additional requirements for complex interventions. BMJ, 345, e8003. Grant et al. (2013). Letter to the Editor: New guidelines are needed to improve the reporting of trials in addiction sciences. Addiction, 108, 1687-1688. Gardner et al. (2013). Editorial Perspective: New guidelines are needed to improve the reporting of trials in child and adolescent mental health. Journal of Child Psychology and Psychiatry, 54(7), 810-812. Social and Psychological Interventions Social/Psychological mechanisms For various health/social issues Behavioural/psychological/structural techniques

Naturalistic, hard to control settings Complex Interventions: MRC Framework Multiple Intervention Components Behaviours of providers/recipients Various levels of

intervention/outcome Flexibility/tailoring intervention Example: Multisystemic Therapy Intensive intervention for chronic juvenile offenders Therapists, caseworkers, psychologists, psychiatrists Work with individual, family, peers, and neighbourhood Settings: home, school, community Services may focus on cognition and behaviour change, communication skills, parenting skills, family relations, peer relations, school performance, or social networks Tailored to the specific needs of the youth and family

Systematic Reviews of RCTs Overall intervention effectiveness Key intervention components Target populations/recipients/settings Important implementation factors Good RCT Reporting Includes Participant and setting characteristics Interventions and their implementation Outcome assessment Theories informing the study Trial design The Problem: Poor Reporting

Reporting Guidelines Minimum set of items on article content Reflect issues related to bias Based on evidence and consensus Our case: The CONSORT Statement CONSORT Extensions CONSORT-SPI Project Official CONSORT Extension Rigorous consensus development Multi-pronged dissemination strategy Phase 1: Lit Reviews

Phase 3: Phase 2: Consensus Delphi Process Meeting Phase 4: Write-up Phase 5: Disseminate and Implement

Project Executive Paul Montgomery, University of Oxford Evan Mayo-Wilson, Johns Hopkins University Sean Grant, University of Oxford Geraldine Macdonald, Queens University Belfast Sally Hopewell, University of Oxford Susan Michie, University College London

David Moher, Ottawa Health Research Institute International Advisory Group J Lawrence Aber Chris Bonell David Clark Frances Gardner

Steve Hollon Jim McCambridge Laurence Moore Mark Petticrew Steve Pilling Lawrence Sherman James Thomas

Elizabeth Waters David Weisburd Jo Yaffe Phase 1: Do we need a new guideline? 1. Examine previous guidance What published standards already exist? - E.g. WIDER; JARS; Non-Pharm. Is current guidance sufficiently rigorous?

Phase 1: Do we need a new guideline? 2. Assess current reporting quality of social and psychological intervention RCTs - Is reporting adequate across disciplines? - Single set of standards that can apply across disciplines? 1: Review of Previous Guidelines Systematic literature search - Multiple electronic databases - Expert databases of guidelines Assessed according to recommended techniques for guideline development (Moher 2010)

What makes a good guideline? (Moher et al. 2010) Preliminary Activities - Literature Review Document Development - Consensus Methods (Delphi Exercise, Consensus Meeting) Publication Strategy - Extension and Elaboration Document Dissemination - Journal endorsement, Open Access Average percentage of recommended techniques used to develop reporting guidelines Guideline Development Stage

CONSORT Documents (n = 6) Preliminary Activities 91.7% 70.8% 67.9% Document Development 75.0%

44.4% 31.0% Publication Strategy 66.7% 5.5% 23.8% Dissemination 76.7%

10.0% 37.1% Median Citations per year (Range) 73.7 (43.3 535.5) 9.9 (0.2 480.2) 4.4 (1.0 65.0)

Non-CONSORT Medical Documents (n = 6) Social Science Documents (n = 7) Findings 89 new/modified reporting items not in CONSORT No current guidance sufficiently incorporates all key standards Social/behavioural science guidelines could be more rigorously developed and disseminated

Social/behavioural science researchers have not participated in previous CONSORT guidelines 2: Review of Reporting Quality Systematic literature search - Multiple electronic databases - Expert databases of guidelines Assessed according to recommended techniques for guideline development (Moher 2010) 2: Review of Reporting Quality Reporting items from guideline review Divided into 3 domains

- Internal validity - External validity - Study details Number of RCTs varied per journal - 11 had one RCT to 35 (JCCP) & 39 (C&E) Average compliance with reporting items per study Reporting Area Clinical Psychology Criminology Education Social

Work All Disciplines Internal Validity 47.0% 27.4% 30.0% 41.2% 37.6%

External Validity 48.4% 42.2% 47.7% 41.8% 46.8% Details 43.2%

20.3% 27.2% 38.3% 33.9% Standards 47.2% 34.6% 39.5%

41.1% 42.2% Study All RCTs per discipline: Clinical Psychology99, Criminology31, Education89, Social Work20 Number of reporting standards: Internal Validity40, External Validity83, Study Details24 Cochrane Risk of Bias Reporting Area Clinical Psychology

Criminology Education Social Work All Disciplines Randomisation Sequence 30.1% 11.3%

18.0% 28.8% 23.0% Allocation Concealment 26.3% 17.2% 3.4%

28.3% 16.7% Blinding 20.2% 4.3% 11.2% 18.3% 14.6%

Participant Flow 55.4% 14.5% 20.4% 37.5% 35.6% Data Analysis 50.0%

31.8% 36.0% 44.6% 41.9% Outcomes 67.2% 54.8% 53.7%

56.3% 59.6% Protocols 29.9% 11.6% 14.6% 27.0% 21.6%

RCTs per discipline: Clinical Psychology99, Criminology31, Education89, Social Work 20 RS = 4 Items, AC = 3 Items, B = 3 Items, PF = 8 Items, DA = 13 Items, O = 4 Items, P = 5 Items Implementation Data Reporting Area Clinical Psychology Criminology Education Social Work All Disciplines

Intervention: Design 74.1% 69.7% 79.7% 80.0% 76.1% Intervention: Delivery

43.8% 35.5% 44.8% 37.9% 42.6% Intervention: Uptake 27.8%

17.1% 26.5% 20.7% 25.3% Control: Design 60.5% 62.1% 70.9%

43.1% 63.1% Control: Delivery 32.3% 31.5% 41.4% 16.2% 34.2%

Control: Uptake 21.0% 18.8% 25.8% 5.8% 21.3% I-Des = 10 Items, I-Del = 12 Items, I-Up = 7 Items, C-Des = 8 Items, C-Del = 12 Items, CUP = 6 Items Average compliance of RCTs with key reporting standards

Ref: Grant et al (2013). PLoS One, 8(5), e65442 Phase 1: Lit Reviews Social/behavioural science guidelines developed/disseminated with less rigour 89 new/modified reporting standards compared to CONSORT guidelines 239 RCTs report <50% of standards on average Ref: Grant et al (2013). PLoS One, 8(5), e65442 Phase 2: Delphi Process Phase 2: Delphi Process N = 384 (32 countries total) 58 items recommended for inclusion

- All but 1 of CONSORT 2010 checklist items (registration) Substantive qualitative feedback for consensus meeting and E&E Items for Round 1 Possible checklist items were generated from reporting standards found in systematic review (Grant 2013) - CONSORT Statement and relevant Extensions - Guidelines from behavioural/social sciences IAG reviewed draft of Delphi Round 1 and requested changes prior to Delphi launch 77 Items for Delphi Round 1 PICOT

Blinding/Masking Theory of Change Recruitment Secular Events Baseline Data Results Service Environment Implementation Intervention: Design

Generalisability Intervention: Delivery Implications Intervention: Uptake Other papers about this trial Outcome Measures Conflicts of Interest Data Collection

Ethical Approval Data Analysis Intervention Development Participants for Delphi Process Various SPI stakeholders - Trialists and systematic reviewers Methodologists Editors & peer-reviewers Funding Representatives Policy- Makers

Practitioners Phase 2: Delphi Round 1 Inclusion criteria - Researchers: publish at least one SPI publication Journal editor: editorial board of journal publishes SPI RCTs Practitioner: Provide SPI services Funder: Current position involves funding SPI RCTs Policy-maker: SPI related public or advisory post Consumer Rep: current position/membership with organisation Phase 2: Delphi Round 1 Recruitment -

Journal editorial boards Intervention literature Signed up on website Research society/org IAG Recommendation Conference contact Delphi participant recommendation Phase 2: Delphi Round 1 Procedures - Emailed invitations (and reminders) with survey link - *Survey active for ~4 weeks *Government shut-down - Ranked items on 1-10 Likert scale - Free text comment box for each item

- Analysed and categorised according to median rank, dispersion of scores, and free-text comments - Items produced for Round 2 Phase 2: Delphi Round 2 Procedures - Emailed Round 1 participants with link to survey - Survey active for ~10 weeks (due to winter break) - Ranked Round 1 Indeterminate items as Include, Exclude, or Unsure - Free text comment box for each manuscript section - Analysed and categorised according to percentage rankings and freetext comments - Items produced for consensus meeting Phase 2: Delphi Process N = 384 in Round 1 (32 countries total)

- N = 321 in Round 2 (no sig differences) Gender: 48% Female, 51% Male Age: 14% <35, 24%: 35-44, 24%: 45-54, 28%: 55-64, 7% >65 Stakeholder roles: 92% Academic/Researcher, 29% Practitioner, 34% Journal Editor, 12% Funder, 9% Policy-maker, 6% Consumer Rep Delphi Process Participants Delphi Process: Academics/Researchers Academic/Researcher N

% of total sample Trialist 178 46% Systematic Reviewer 211 55% Statistician

45 12% Methodologist 155 40% Other 75 20%

Total academics/researchers (% of sample) 355 92% Phase 2: Delphi Process Items - 77 items in Round 1 Included (n = 36) Indeterminate (n = 41 in Round 1; split to n = 61 for Round 2) New (n = 5) - 66 items in Round 2 Included (n = 22), indeterminate (n = 44) - 58 items recommended for inclusion in CONSORT-SPI

Median > 8 and low dispersion in Round 1; or >80% ranking of Inclusion in Round 2 Phase 3: Consensus Meeting 31 participants from Delphi Process Phase 1: Lit Reviews Phase 3: Phase 2: Consensus Delphi Process Meeting Phase 4:

Write-up Phase 5: Disseminate and Promote Uptake Structured, face-to-face meeting to discuss and vote on items for CONSORT-SPI Checklist Consensus Meeting Participants Doug Altman Peter Kaufmann

Kamaldeep Bhui Spyros Konstantopoulos Andrew Booth Kenneth McLeroy Peter Craig Brian Mittman Manuel Eisner Arthur Nezu

Mark Fraser Edmund Sonuga-Barke Larry Hedges Gary VandenBos Robert Kaplan Robert West Phase 3: Consensus Meeting Consensus Meeting: Day 1

Agenda Introduction to reporting guidelines Review of reporting guidelines and trial reporting quality Group discussion: trials in your area(s) Specific issues for SPI trials CONSORT-SPI Delphi process Group discussion: thoughts on Delphi Consensus Meeting: Day 2 Agenda Discuss and vote on items for CONSORT-SPI Checklist - Introduction section - Methods - Results - Discussion section - Other important information

Consensus Meeting: Day 2 Agenda - Ample time to allow sufficient time for thorough discussion Want to prevent hasty decision-making - Structured discussions of items proposed for the checklist from the Delphi process - Care taken to ensure: All views are expressed All ideas are considered - Voting is confidential using anonymous ballots to promote honest answers and allow participants to rethink their position if a re-vote is needed

Options on the Table What is in CONSORT, is in CONSORT Making decisions about items for CONSORT-SPI checklist Some material will go into E&E guidance (i.e., the users' manual) Looking to provide minimum standards and concise items from the entire process - Delphi items longer than desired to make sure Delphi participants were clear on concepts Some materials may be future work building from this meeting! New/Adapted Items Intervention theory of change

Eligibility criteria for settings and providers Intervention/comparator delivery and uptake Intervention materials (e.g., manual, website) How missing data were handled

Number approached, screened, and eligible New/Adapted Items Socioeconomic baseline variables Availability of trial data Other potential interests than funder

Involvement of the intervention developer Other stakeholder involvement Incentives offered as part of the trial Phase 4: Write-Up Draft official guideline extension Tailored E&E documents to disciplines

- Rationale for each item - Examples of good reporting Phase 5: Dissemination Simultaneous co-publication Journal endorsement and adherence Presentations at conferences/meetings Editorials and newsletters Training and education CONSORT-SPI Checklist Title and abstract Item # Standard CONSORT Description

1a Identification as a randomised trial in the title 1b Structured summary of trial design, methods, results, and conclusions (for specific guidance see CONSORT for abstracts) Extension for CONSORT-SPI indicates that an extension item for cluster trials exists CONSORT-SPI Checklist Introduction: Background and Objectives

Item # 2a 2b Standard CONSORT Description Extension for CONSORT-SPI Scientific background and explanation of rationale Specific objectives or hypotheses If pre-specified, how the intervention was hypothesised to work

CONSORT-SPI Checklist Methods: Trial Design Item # Standard CONSORT Description 3a Description of trial design (such as parallel, factorial) including allocation ratio 3b Important changes to methods after trial

commencement (such as eligibility criteria), with reasons Extension for CONSORT-SPI If the unit of random assignment is not the individual, please refer to CONSORT for Cluster Randomised Trials CONSORT-SPI Checklist Methods: Participants Item # Standard CONSORT Description

4a Eligibility criteria for participants 4b Settings and locations where the data were collected Extension for CONSORT-SPI When applicable, eligibility criteria for settings and those delivering the interventions

CONSORT-SPI Checklist Methods: Interventions Item # 5 Standard CONSORT Description The interventions for each group with sufficient details to allow replication, including how and when they were actually administered Extension for CONSORT-SPI Extent to which

interventions were delivered and taken up as planned, including what they actually involved *Where other informational materials about delivering the intervention can be accessed When applicable, how intervention providers were assigned to each group *Indicates item may move to another section CONSORT-SPI Checklist Methods: Outcomes

Item # Standard CONSORT Description 6a Completely defined pre-specified primary and secondary outcome measures, including how and when they were assessed 6b Any changes to trial outcomes after the trial commenced, with reasons Extension for CONSORT-SPI

CONSORT-SPI Checklist Methods: Sample Size Item # Standard CONSORT Description 7a How sample size was determined 7b When applicable, explanation of any interim analyses and stopping guidelines

Extension for CONSORT-SPI CONSORT-SPI Checklist Methods: Randomisation Item # Standard CONSORT Description 8a Method used to generate the random allocation sequence 8b Type of randomisation; details of any restriction

(such as blocking and block size) 9 Mechanism used to implement the random allocation sequence (such as sequentially numbered containers), describing any steps taken to conceal the sequence until interventions were assigned 10 Where applicable, who generated the random allocation sequence, who enrolled participants, and who assigned participants to interventions

Extension for CONSORT-SPI CONSORT-SPI Checklist Methods: Awareness of Assignment Item # Standard CONSORT Description 11a Who was aware after assignment to interventions (for example, participants, providers, those assessing outcomes), and how any masking was done 11b

If relevant, description of the similarity of interventions Extension for CONSORT-SPI CONSORT-SPI Checklist Methods: Awareness of Assignment Item # Standard CONSORT Description 12a Statistical methods used to compare groups for primary and secondary outcomes

12b Methods for additional analyses, such as subgroup analyses and adjusted analyses Extension for CONSORT-SPI How missing data were handled (e.g., complete case analysis, simple imputation, multiple imputation), with details of any imputation method CONSORT-SPI Checklist

Results: Participant Flow Item # 13a 13b Standard CONSORT Description Extension for CONSORT-SPI Where possible, the For each group, the numbers randomly assigned, number approached, received intended treatment, and analysed for screened, and eligible prior to random assignment,

the primary outcome with reasons for dropout For each group, losses and exclusions after randomization, together with reasons Enrolment Approached (n= ) Screened/assessed for eligibility (n= ) Excluded (n= ) Not meeting inclusion criteria (n= ) Declined to participate (n= ) Other reasons (n= ) Randomised (n= )

Allocation Allocated to intervention (n= ) Received allocated intervention (n= ) Did not receive allocated intervention (give reasons) (n= ) Providers/organisations/areas (n= ) Number of participants by provider/organisation/area (median = ... [IQR, min, max]) Allocated to intervention (n= ) Received allocated intervention (n= ) Did not receive allocated intervention (give reasons) (n= ) Providers/organisations/areas (n= ) Number of participants by provider/organisation/area

(median = ... [IQR, min, max]) Follow-Up Lost to follow-up (give reasons) (n= ) Discontinued intervention (give reasons) (n= ) Lost to follow-up (give reasons) (n= ) Discontinued intervention (give reasons) (n= ) Analysis Analysed (n= ) Excluded from analysis (give reasons) (n= ) Analysed (n= ) Excluded from analysis (give reasons) (n= )

CONSORT-SPI Checklist Results: Recruitment Item # Standard CONSORT Description 14a Dates defining the periods of recruitment and follow-up 14b Why the trial ended or was stopped Extension for CONSORT-SPI

CONSORT-SPI Checklist Results: Baseline data and numbers Item # Standard CONSORT Description Extension for CONSORT-SPI 15 A table showing baseline characteristics for each Including socioeconomic variables where applicable group 16

For each group, number included in each analysis and whether the analysis was by original assigned groups CONSORT-SPI Checklist Results: Outcomes and Estimation Item # 17a 17b 18 19

Standard CONSORT Description Extension for CONSORT-SPI For each primary and secondary outcome, results for each group, and the estimated effect *Indicate availability of trial size and its precision (such as 95% confidence data interval) For binary outcomes, presentation of both absolute and relative effect sizes is recommended Results of any other analyses performed, including subgroup analyses and adjusted analyses, distinguishing pre-specified from exploratory

All important harms or unintended effects in each group (for specific guidance see CONSORT for harms) CONSORT-SPI Checklist Discussion Item # Standard CONSORT Description 20 Trial limitations, addressing sources of potential bias, imprecision, and, if relevant, multiplicity of analyses

21 Generalisability (external validity, applicability) of the trial findings 22 Interpretation consistent with results, balancing benefits and harms, and considering other relevant evidence Extension for CONSORT-SPI CONSORT-SPI Checklist Important Information Item #

Standard CONSORT Description 23 Registration number and name of trial registry 24 Where the full trial protocol can be accessed, if available 25 Sources of funding and other support, role of funders

Extension for CONSORT-SPI Declaration of any other potential interests CONSORT-SPI Checklist Stakeholder Involvement Item # New Item Standard CONSORT Description Extension for CONSORT-SPI

*Any involvement of the intervention developer in the design, conduct, analysis, and reporting of the trial *Other stakeholder involvement in trial design, conduct, and/or analyses *Incentives offered as part of the trial Applying CONSORT-SPI See Example Paper 1a: Identification of Randomised Trial in Title Article Title - A Pilot Randomised Control Trial: Testing a Transitional Care Model for Acute Psychiatric Conditions

1b: Structured Summary Abstract - Objective - Method - Results - Conclusions 2a: Scientific Background & Rationale Background 2b: Specific Objectives or Hypothesis The article reports findings from research that examined the feasibility and effectiveness of the Naylor

TCM for individuals with SMI and comorbid health conditions with the aim of reducing hospital readmissions, reducing emergency department (ED) use, improving continuity of care, and improving health quality of life following a hospitalisation for an acute psychiatric condition. 3a: Description of Trial Design Such as parallel, factorial, including allocation ratio 3b: Important Changes to Methods After Trial Commencement Such as eligibility criteria with reasons None noted

4a: Eligibility Criteria Hospital patients were eligible if they: a) were aged 18 to 64 b) had a diagnosis of an SMI such as schizophrenia, bipolar & major depression c) had a diagnosis of major medical condition such as diabetes, asthma, or cancer d) were English speaking e) resided in the city of Philadelphia 4b: Settings & Locations Where Data Collected From an inpatient unit of a 515 bed acute care general hospital in a large northeastern urban city. Baseline in hospital and follow up in community

5: Interventions for Each Group Intervention Procedure & Protocol described Usual care meant that a case manager was assigned & psychiatrist provided medication management 6a: Outcomes Completely specified - Health related quality of life - Continuity of care - Service utilisation No changes in outcomes 7a & b: Sample Size

Power not done as pilot study Sample size of 40 No interim analyses 8: Randomisation Sequence generation A computerised randomisation schedule was based on a double parallel method using a sample size of 60 (assuming a single group size of 30 with attrition to 20 participants) with block sizes of 6 participants. This means that for every block 3 were assigned to the intervention and 3 assigned to control. There were a total of 10 blocks in this design. This design chosen to keep the psychiatric NP caseload manageable as we only had one nursing providing intervention.

9: Allocation Concealment Mechanism Once experimental participants were consented by the research assistant the NP was notified of the patient participant 10: Implementation Randomisation done by computer Research Assistants enrolled participants Research assistants then contacted PI for next allocation assignment 11: Awareness of Assignment NP aware of experimental intervention participants Research Assistants not blinded as often need assistance in locating participants for follow up

interviews Noted that experimental intervention also received usual care 12: Awareness of Assignment Statistical methods to compare groups Chi square or t test of differences between groups conducted 13: Participant Flow CONSORT flow chart included Losses & exclusions after randomization all noted in flow chart Reasons for losses noted 14: Recruitment

Dates March 2011 to August 2011 Follow ups at 6 & 12 weeks from baseline 15: Baseline Data Table 2 characteristics of participants by groups Table 3 health related quality of life by group at baseline (and at 12 week follow up) 16: Numbers Analysed Participants analysed by assigned group - Analysed outcomes of intervention group = 18 and control = 17 (baseline both groups 20) 17: Outcomes and Estimation

Each outcome for each group Tables 3, 4, & 5 Effect sizes not noted outcomes not statistically significant no difference from zero except one in wrong direction 18: Ancillary Analyses Other analyses conducted, e.g., subgroup - none conducted 19: Harms No harms noted 20: Limitations Last paragraph indicates limitations titled limitations

21: Generalisability Applicability of trial findings noted in conclusion of article 22: Interpretation Discussion section 23: Registration None noted not registered 24: Protocol Where protocol could be accessed

- Not noted - Basically articles covers protocol of study - Referenced to another article from the same study 25: Declaration of Interests End of article - Declaration of conflicting Interests Authors declared no potential conflicts of interest with respect to research, authorship, and/or publication of this article. Funding from Robert Wood Johnson Foundation Interdisciplinary Nursing Quality Research Initiative A New Evidence Grading System for Complex Interventions

GRADE-CI https://www.spi.ox.ac.uk/research/details/grade-extension-for-complex-social-inter.html Project Executive Dr Erik von Elm Institut Universitaire de Mdecine Sociale et Preventive (IUMSP), Lausanne, Switzerland Dr Eva Rehfuess Institute of Medical Informatics, Biometry and Epidemiology Ludwig-Maximilians University, Munich, Germany Prof Geraldine Macdonald University of Bristol, Bristol, UK Dr Jane Dennis Research Synthesis Ltd, Bristol, UK Prof Paul Montgomery Centre for Evidence-Based Social Intervention, University of Oxford Dr Sean Grant RAND Corporation, Santa Monica, USA Dr Susan Norris Guideline Review Committee Secretariat,

WHO International Steering Committee Gordon Guyatt Mark Petticrew Holger Schunemann

Steven Hollon Peter Tugwell Bonnie Spring Ian Shemilt

Frances Gardner Stephanie Chang Julia Littell Andrew Booth

James Thomas Philip Davies Sandra Wilson Birte Snilstveit

Manual Eisner Matthew Morton The GRADE Approach The GRADE approach offers a transparent and structured process for developing and presenting (effectiveness) evidence summaries for systematic reviews and for carrying out steps involved in developing recommendations: It specifies an approach to: Framing questions for systematic reviews and guidelines Choosing outcomes of interest and rating their importance Assessing and rating the quality of a body of evidence

Incorporating effectiveness evidence with other considerations to arrive at recommendations (DECIDE) The GRADE Methodology and Process 1 2 3 Question formulation (PICO) Comparative effectiveness Search & retrieval of relevant studies Evidence synthesis

Meta-analysis of RCTs; narrative summary 4 Rating the quality of evidence for each outcome RCTs high, Observational Low Downgrading - Risk of bias - Inconsistency - Imprecision - Indirectness - Publication bias 5 6

Upgrading - Large effect - Dose-response - All plausible residual confounding Quality of evidence is defined as the extent of our confidence that the estimates of the effect are correct Rating the overall quality of evidence Grading recommendations for practice - Problem priority - Acceptability

- Benefits/harms - Preferences/values Quality of evidence - Resource use - Feasibility - Equity DECIDE Thank You! Please email with questions/comments: [email protected] Visit our website: http://tinyurl.com/CONSORT-study

Recently Viewed Presentations

  • Live Fire Day Live Fire Practical Evolutions Live

    Live Fire Day Live Fire Practical Evolutions Live

    The Commission adopts NFPA 1403 with the following changes; 2) Use of appropriate amounts of combustible fuel to start fire Pallets Straw Test Your Fuel Fuel Test Prepare Room Pre-Attack Briefing Attack Overhaul Post-Attack Briefing Set up pallets and straw.
  • Materials - Duxbury

    Materials - Duxbury

    Stones are natural Paper is man-made Wool is natural Plastic is man-made Glass is man-made Wood is natural Can you make a list of natural and man-made materials? Natural Man-made Well done! You now know what is natural and what...
  • Ethics - ASEE Brookings 2005 - NDSU

    Ethics - ASEE Brookings 2005 - NDSU

    The Graduate Course in Electromagnetics: Integrating the Past, Present, and Future. David A. Rogers & Benjamin D. Braaten . Electrical and Computer Engineering
  • Diuretic Agents Carbonic Anhydrase Inhibitors Carbonic anhydrase is

    Diuretic Agents Carbonic Anhydrase Inhibitors Carbonic anhydrase is

    A new class of drugs, the vaptans, are ADH antagonists. Diuretic Combinations. Loop Agents & Thiazides. Since these agents have a short half-life (2-6 hr.), refractoriness may be due to an excessive interval between doses.
  • HR STRATEGY FORMULATION PROCESS Facilitators: Basanta Raj Sigdel

    HR STRATEGY FORMULATION PROCESS Facilitators: Basanta Raj Sigdel

    An analysis of internal organizational factors which reviews and investigates the prevailing processes, resources and performance of organization. ... Assess feasibility. Determine desirability . Determine goals. Decide means of achieving goals.
  • THE INVERTEBRATES The Worms GENERAL CHARACTERISTICS soft-bodied, long,

    THE INVERTEBRATES The Worms GENERAL CHARACTERISTICS soft-bodied, long,

    anterior region has 2 cross-eyed eyespots on the dorsal surface that are sensitive to light. Planarians. Planarians. ... Suckers and hooks on scolex attach worm to host. pork Tapeworms (cont.) body is divided into proglottids that contains both testis and...
  • Seeing the Good in People: What the Holocaust and People&#x27;s ...

    Seeing the Good in People: What the Holocaust and People's ...

    Read each question carefully and make sure to answer completely. When you are finished, place your test in the basket on the table at the front of the room. Reading Selection: Wiesel's Message After reading this selection, write a short...
  • CEDEL2 Corpus Escrito del Espaol L2 http://www.uam.es/woslac/cedel2.htm Cristbal

    CEDEL2 Corpus Escrito del Espaol L2 http://www.uam.es/woslac/cedel2.htm Cristbal

    "Selective deficits at the syntax-discourse interface: Evidence from the CEDEL2 corpus". Eds. Y.-I. Leung, N. Snape y M. Sharwood-Smith. Representational Deficits in SLA. Amsterdam: John Benjamins. Lozano, C. y Mendikoetxea, A. en prensa. "Postverbal subjects at the interfaces in Spanish...