Study protocol for developing deprescribing clinical practice guidelines: evidence-based GRADE methodology and a Delphi consensus method | BMC Geriatrics

Study design

The development of this clinical practice guideline consists of evidence-deriving and consensus-building processes. For the first part, evidence will be derived using a systematic review and meta-analysis of the literature and a rigorous assessment of the certainty of the evidence using the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) framework. Systematic reviews and meta-analyses are considered the gold standard in evidence synthesis [35] whereas the GRADE framework is increasingly seen as the preferred approach for summarising findings in systematic reviews and rating the certainty of a body of evidence [36]. For the second part about the consensus-building process where evidence is insufficient or lacking, a modified Delphi approach will be used as it is well-suited for gathering input from individuals across diverse professions and specialties, especially when there are expected differences in opinions [37, 38]. The traditional Delphi method involves generating qualitative data in the first round of data collection to guide the development of statements for the subsequent rounds. A modified method will be used that omits the first qualitative round and begins the series of rounds with a set of carefully selected statements derived from the literature, previous research, or existing clinical practice guidelines for treatment [39]. The modification was carefully considered to expedite the process, minimise participant fatigue and increase overall engagement throughout the subsequent iterative rounds without compromising the integrity of the consensus-building process [40]. We acknowledge the potential biases inherent in the Delphi approach. Nevertheless, the Delphi technique provides advantages particularly through the anonymity of responses during the survey rounds. This anonymity helps mitigate potential dominance effects, halo effects, and groupthink commonly encountered in other group settings such as a focus group [41]. Additionally, the iterative process allows for controlled feedback, ultimately facilitating the achievement of group consensus. We plan to include a diverse group of Delphi panel members and establish a predetermined cut-off response rate to mitigate potential selection and response biases.

The development of the clinical practice guideline follows the principle of robust clinical guideline development standards as outlined in the WHO Handbook for Guideline Development, Australian National Health and Medical Research Council (NHMRC) Guideline Development Methodology and the Appraisal of Guidelines for Research & Evaluation (AGREE) II Instrument and User’s Manual [42,43,44]. In Australia, guidelines developed by groups external to NHMRC may be approved by NHMRC. While it is not mandatory for guidelines to obtain approvals, NHMRC offers step-by-step guidance to produce high-quality guidelines. Additionally, multiple tools have been developed for guideline appraisal among which the AGREE II instrument is the most commonly used and forms part of the process suggested by NHMRC [36]. Adhering to the processes detailed in the WHO handbook, NHMRC guidance and AGREE II instrument will ensure the guidelines meet the requirements for methodological rigour in the development and reporting of guidelines. However, developing a new guideline is typically a costly and time-consuming process, which may pose a challenge to resource-constrained settings [45]. We opted for pragmatic design adaptations to suit the purpose of the guideline, and available resources, in consideration of the urgent need for recommendations to support clinical decision-making.

Purpose of the guideline

The guideline will provide guidance on the key aspects for deprescribing considerations in people aged 65 years and over, which are to determine when, how, and for whom a medicine should be deprescribed, as well as identify monitoring requirements during deprescribing and the ongoing treatment needs as applicable.

Scope of the guideline

This guideline prioritises providing recommendations for medicines commonly prescribed and dispensed to older people as it is likely to have the largest impact on clinical practice. We leveraged data from the Australian Pharmaceutical Benefits Scheme (PBS) to identify the top 100 medicines as priorities for future deprescribing efforts. The PBS is routine administrative data that provides an accurate representation of common medicines used by the population. The Australian PBS subsidises the cost of most medicines in Australia for eligible residents, with over half (54%) of PBS-subsidised medicines dispensed to people aged 65 and over [46]. This guideline will be limited to medicines intended for regular use. Hence, medicines prescribed for short-term, intermittent, as required, or acute use only (e.g. systemic or topical antibiotics) will not be included.

The guideline steering committee analysed the PBS data for people aged 65 or over who were dispensed PBS-listed medicines in 2023 to identify the top 100 medicines with the highest dispensing volume or the largest number of unique persons dispensed, excluding non-regular medicines. The volume-based metric represents the total number of dispensing in a calendar year, while the person-based metric refers to the number of people who received the medicine in a calendar year. The person-based metric is included to account for medicines with less frequent dosing, such as denosumab, which is typically administered every six months. This methodology was previously adopted in a study investigating the inclusion of information about medication withdrawal and medicine use in older people [47]. While the primary focus of this guideline will be common PBS-listed medicines, the guideline development group will review and consider on a case-by-case basis the inclusion of additional medicines where evidence for deprescribing is identified in the systematic search. The rationale is to not exclude medicines simply because they are not listed on the PBS but to consider the potential risks of inappropriate use and the impact of deprescribing.

Stakeholder involvement

Guideline steering committee

The guideline steering committee (authors of the current protocol) has a primary role of guiding and overseeing the overall development of the guideline. Their responsibilities are to establish a guideline development group, propose the topic and scope of the guidelines to members of the guideline development group, refine the key clinical questions, as well as plan and lead the development of high-quality, credible recommendations and good practice statements. The steering committee will support the implementation of the guideline in clinical practice and actively take part in the dissemination process.

Guideline development group

The guideline steering committee will establish a guideline development group with members from each of the following categories: (1) physicians including general practitioners, geriatricians, clinical pharmacologists, and geriatric psychiatrists, (2) nurse practitioners (3) pharmacists, (4) statisticians, (5) policymakers, (6) allied health professionals (optometry, dental, podiatry, physiotherapy, physiotherapist), (7) methodologist with experience in guideline development, methodology or systematic reviews, (8) expert in implementation science or behavioural science, (9) pharmacoepidemiologist, (10) health economist, and (11) patients or carers with lived experience. Clinicians and pharmacists must be practising in the field of geriatric care or pharmacotherapy relevant to people aged 65 and over meeting one or more of the following selection criteria:

1. 1.

   Demonstrable clinical experience in the field of geriatric and gerontology or specialised in providing pharmaceutical care for older people (e.g. practice-based experts who are practising or having practised in the field for at least five years).

2. 2.

   Recognised as an expert in the field by peers (e.g. invitation to a relevant symposium, conference or other academic events as a speaker or presenter, or membership in an association or research group).

3. 3.

   Recent publications as a first or last author on the relevant topic in peer-reviewed journals within the past five years.

4. 4.

   Post-graduate qualification or current credential relevant to geriatric pharmacotherapeutics (e.g. a geriatrician or a pharmacist credentialed with a certificate in geriatric pharmacy).

At least one member, regardless of profession, will be practising in each hospital, residential aged care facility and private practice settings and at least one member will be practising in a rural or remote area. Individual members may fulfil multiple criteria, such as a general practitioner practising in a hospital in a rural area.

All members of the guideline development group and steering committee will be required to declare any perceived or actual financial or non-financial competing interests. The guideline steering committee will record and manage potential conflicts of interest relevant to the guideline development. Members of the guideline development group will be identified through professional networks and snowball sampling. If there is a lack of relevant content expertise for a specific therapeutic area, the guideline steering committee will be responsible for recruiting additional clinical experts with relevant expertise and credentials based on their existing professional networks.

At least four members of the guideline development group will represent a specific consumer advisory group consisting of at least one layperson, patient, and carer with lived experience. These individuals will be identified through the Western Australian Health Translation Network Consumer and Community Involvement Program. These individuals will be invited to provide critical insights into the challenges and needs that are often overlooked in clinical practice, provide their input on draft deprescribing recommendations, and ensure that at every stage the views of patients and carers are prioritised. Layperson, patients and carers will be reimbursed for their time. By integrating their perspectives, we aim to create guidelines that resonate with the actual experiences and preferences of patients and the wider public, ensuring greater relevance and uptake in real-world settings.

All members of the guideline development group will contribute by reviewing draft recommendations and taking part in the modified Delphi study to establish consensus-based recommendations for common drug classes without evidence from the literature.

External experts

External experts will be individuals with the expertise and experience relevant to the methodology or the content of the guideline who have indicated a preference to provide external expert feedback independent from the guideline development group.

Rigour of guideline development

The development of the guideline comprises two main phases (Fig. 1). Phase 1 involves synthesising evidence using a systematic review and meta-analysis approach. Phase 2 will involve presenting draft recommendations to the guideline development group to determine consensus using a systematic modified Delphi method.

Phase 1: synthesising evidence

Identifying relevant evidence

The guideline steering committee updated a 2016 systematic review and meta-analysis [48] assessing the effects of deprescribing in older people to capture new evidence that emerged since the original publication [49]. This systematic review and meta-analysis summarised comprehensive findings on the effects of deprescribing on mortality, physical health, cognitive function, quality of life, effect on medication regimen, and adverse drug withdrawal events. The process follows the Cochrane Handbook for Systematic Reviews of Interventions, which consists of an updated literature search, screening for study inclusion, data extraction, quality appraisal, data analysis and synthesis (meta-analysis), and interpretation of findings [50]. The methods have previously been described in a published protocol [51]. The updated systematic review and meta-analysis has since been published [49]. Briefly, the review had broad selection criteria without limitations on study settings, patient sub-groups, or types of medicines targeted, aiming to capture all relevant studies related to deprescribing in older people. The Population, Intervention, Comparator and Outcomes (PICO) framework guiding the evidence review as well as the predetermined inclusion and exclusion criteria for screening relevant studies are shown in Table 1.

Electronic database searches were conducted in CINAHL, Medline, Embase, Scopus, Web of Science, and ProQuest (Dissertations and Theses Global) to identify relevant published studies up to April 2024 in which older people (aged 65 years and older) had at least one medicine deprescribed. Both experimental (randomised or non-randomised controlled trials) and observational studies with or without concurrent control groups (before-and-after, case-control or cohort studies) were included. Studies were grouped by study designs and targeted medicines for data analysis. The risk of bias was assessed using the Cochrane tool and the Newcastle-Ottawa tool. Odds ratios or mean differences were calculated as the effect measures using either the Mantel–Haenszel or generic inverse-variance method with fixed- or random-effects meta-analyses.

Formulating draft recommendations

The certainty of the evidence in the systematic review and meta-analysis will be rated using the GRADE approach. This structured and transparent approach will enable recommendations to be synthesised based on the evidence and its certainty, while also considering overall benefits and harms, patient values and preferences, resource implications, and the feasibility of implementation [52]. The GRADE approach has been adopted by national and international organisations as a preferred approach to rate the certainty of evidence in systematic reviews to develop and determine the strength of recommendations in clinical practice guidelines [53]. The key steps in developing recommendations using the GRADE approach are: 1) selecting and rating the importance of outcomes, 2) summarising the evidence, 3) determining the quality of evidence, and 4) moving from evidence to recommendations.

It is acknowledged that the importance of an outcome may only become known once evidence is reviewed, or the analyses were carried out (e.g. serious adverse effect). The search strategy for our systematic review and meta-analysis has thus included broad outcomes of mortality, physical health, cognitive function, quality of life, effect on medication regimen, and adverse drug withdrawal events. Initially, the guideline steering committee will organise the evidence of outcomes identified from the systematic review and meta-analysis by drug classes. Each outcome will be rated on its relative importance by the guideline development group for decision-making: critical, important but not critical, or low importance [54]. Outcomes rated as critical and important will be used to produce the GRADE evidence profile and GRADE summary of findings table which will bear on guideline recommendations.

Two researchers trained in the GRADE approach will independently assess the certainty of the evidence for each outcome by considering eight GRADE criteria (risk of bias, directness of evidence, consistency and precision of results, risk of publication bias, magnitude of the effect, dose-response gradient, and influence of residual plausible confounding). Outcomes will be rated as high, moderate, low, or very low certainty (Table 2), with discrepancies between the researchers resolved through discussion and consensus. These outcomes along with the certainty of the evidence will be included in the GRADE evidence profile and subsequently GRADE summary of findings table. The latter is intended to be a quick summary and will not contain details of the judgments about the certainty of the evidence.

The GRADE Evidence-to-Decision framework provides a structured and transparent framework to develop recommendations based on the relative importance and certainty of the evidence, while also considering the overall benefits and harms, patient values and preferences, implications for resource utilisation, equity, acceptability and feasibility of deprescribing [52]. The guideline steering committee will follow the GRADE Evidence-to-Decision framework to draft recommendations. Specifically, when considering the overall balance of benefits and harms, best estimates of the magnitude of effects on desirable and undesirable outcomes and the relative importance of outcomes based on estimated values and preferences will be considered. Patient values and preferences for each drug class will be investigated based on consultations with the layperson, patient, or carer representatives in the guideline development group as well as non-systematic reviews of the available literature, or clinicians’ experience of interactions with their patients. Investigations on the implications of resources, equity, acceptability and feasibility of deprescribing will rely on non-systematic reviews of the available literature, expert opinions or individual experiences of the guideline development group members.

Draft recommendations will be presented to all members of the guideline development group along with the evidence for review presented in tables and as narrative reviews, and where appropriate including statistical data such as meta-analysis results. The group members will be briefed on the guideline development methodology and the GRADE framework so each member can independently apply their judgement in a consistent and systematic way. The guideline development group will ultimately determine the type of recommendation categorised based on the type and source of evidence that supports them (Table 3). For evidence-based recommendations, the strength (strong or weak) and direction (for or against) of recommendations will also be determined. The guideline steering committee will be responsible for revising the draft recommendations based on feedback from the development group ensuring all appropriate viewpoints are considered. Where the guideline development group identifies there is insufficient quality evidence or lack of evidence, they may choose not to make a recommendation. Alternatively, consensus methods (Phase 2) will be used to develop recommendations or good practice statements grounded in expert opinions and individual patient or carer experiences.

Phase 2: consensus-building process using a modified Delphi method

While peer-reviewed evidence has long been considered the gold standard for developing guideline recommendations [56], research is not always available to inform guideline recommendations. As deprescribing is a relatively new field, there is sparse evidence for patient-important outcomes, as shown in previous systematic reviews [48, 49]. In situations where there is insufficient information, consensus methods provide another means of synthesising information grounded in expert opinions and experiences.

For the second phase of guideline development, the guideline steering committee will use surveys to elicit opinions from leading experts and patients or carers to reach a consensus on core recommendations for deprescribing clinical practice guidelines to fill the knowledge gap (Fig. 1).

Selection, identification and recruitment of the panel

All members of the guideline development group involved in Phase 1 will take part in the modified Delphi study in Phase 2 to establish consensus-based recommendations for common drug classes with insufficient or a lack of evidence from the literature review and where appropriate, good practice statements. As the guideline steering committee drafted the recommendations, they will not be involved in the decisions made by the consensus panel. Guideline development group members will be included if they are willing to participate in all Delphi rounds and declare ongoing conflicts of interest. While literature commonly suggests a panel size of 10 to 18 for a Delphi study, we plan to include a minimum of 20 participants to account for potential attrition [57].

Delphi method

The online surveys will be administered using the Qualtrics software (Qualtrics, Provo, UT, USA. [58]. The content of the survey will include proposed recommendations or statements that are specific to drug classes and will be organised into the pre-defined aspects of deprescribing, as described above.

In each round, an online survey will be disseminated via email that prompts the panel members to review the statements provided based on their best judgment and experience. The response to each proposed recommendation or statement in the survey will be binary, either agree or disagree, with an optional free-text comment section at the end of each drug class section. This is so we can capture any valuable insights that are not in the statements provided. We will not consider a round valid if the response rate falls below 70%, as it may give rise to non-response bias [59]. Survey responses will remain confidential and accessible only to the guideline steering committee responsible for data analysis. Participants will complete the survey independently, without direct interaction with other survey respondents.

The participants will initially be given 14 days to complete the survey. The survey will be designed to automatically save responses, enabling participants to resume and complete the survey at a later time, even if they exit the survey before submitting. If participants are unable to complete the survey by the original deadline, individual extensions will be granted for a reasonable duration to accommodate their schedules. To further maximise the retention rate, we will provide an email update to all participants about the study progress, including the anticipated date for the next survey round, so the participants are prepared [60]. After each round is concluded, we will summarise and anonymise the feedback. We will share a brief feedback report that summarises the response percentages for each question to the panel members who participated, and they will be thanked for their contribution to the study.

Definition of consensus

In theory, consensus is achieved when all panel members agree or disagree on the items. However, a full agreement is rarely achieved and likely not feasible in a Delphi study. The goal of consensus is to reach a mutually acceptable level of agreement. Although consensus is fundamental to Delphi studies, a systematic review revealed that it is often poorly defined and rarely reported [61].It is recommended to pre-specify a threshold percentage for agreement [61].In this study, we will define 75% or greater agreement as consensus, as this percentage of agreement is generally considered acceptable in literature [40, 61, 62].

Delphi rounds

We will conduct at least one survey round to allow feedback and revision of responses. The most common number of rounds to reach consensus in practice is typically two to three [63]. Statements where consensus has been achieved will be incorporated into clinical practice guidelines as consensus-based recommendations. If consensus has not been achieved, the statements will be presented to the panel members in the next round. The steering committee will modify the survey in the next round to include refined statements to capture the evolving consensus. If consensus is still not achieved after a reasonable number of rounds, we will identify these items in the clinical practice guideline under ‘Areas of Debate’, highlighting a lack of consensus for future research. In this regard, no recommendation will be made.

Data analysis

We will collect information about the participants’ demographics, including name, email address, age, gender, and where relevant, the geographic location of their current primary work location, job title and number of years of experience. These data will be analysed descriptively [64]. As the focus of this study is on quantitative data collection and analyses for binary responses, the participants’ responses for each statement will be aggregated and analysed in percentages to determine consensus. We anticipate optional free-text comments may include main insights, reasons for agreement or disagreement, and any suggestions for statement revisions. Qualitative data collected from the free-text comments will be analysed thematically to identify common themes and topics that emerge from the participants’ responses.

Patient and public involvement

Independent review

Following Phases 1 and 2, we will invite at least two independent expert peer-reviewers who are not part of the guideline development group to review the guideline using the AGREE-II instrument. The independent review stage helps identify areas for improvement before the guidelines are finalised by assessing the methodological quality of guidelines against the AGREE II instrument. The independent reviewers will be identified by the guideline development group through existing networks. They will have the expertise and experience relevant to the content of the guideline as well as an understanding of the context in which the guideline will be implemented.

Public consultation

Prior to finalising the overall guideline, we will also conduct a public consultation process to seek input from the wider community on the draft recommendations. This will ensure that the guideline recommendations are aligned with the community’s values and expectations. The public includes individuals, patient organisations, policymakers, and professional organisations that will be involved in, or affected by, the implementation of the clinical recommendations of the guideline. We will notify the public of the opportunity to review the draft and share their written feedback via newspaper, emails, social media, website notices, or directly emailing relevant stakeholders. As part of the public consultation process, we will make the draft guideline available for a minimum period of 30 days on an online platform for public access. An extension of the consultation period may be considered if reasonable requests have been made from the public. Following the conclusion of the public consultation, we will prepare a public consultation report with a summary of the process and the changes made to the guidelines as a result. All stakeholders who have made a submission will be formally acknowledged in the guideline, provided they have given consent to do so.