Protocol for the development of a transdiagnostic core outcome set for mental health disorders in adults: the Patient Important Outcomes in Psychiatry (PIO-Psych) Initiative
Protocol for the development of a transdiagnostic core outcome set for mental health disorders in adults: the Patient Important Outcomes in Psychiatry (PIO-Psych) Initiative
Mental health problems are important causes of disability and economic costs worldwide. Randomised clinical trials examining the treatment of mental health disorders measure heterogeneous outcomes, causing difficulties in data synthesis, interpretation and translation into clinical practice. The aim of the Patient Important Outcomes in Psychiatry (PIO-Psych) Initiative is to develop an overarching, transdiagnostic research-based and consensus-based core outcome set for adult mental health disorders.
The development of the PIO-Psych transdiagnostic core outcome set will include three phases: (1) a systematic scoping review of the literature to develop the initial list of outcomes for the Delphi study; (2) a Delphi study in three rounds including people with lived experience of mental health disorders and their relatives, clinicians, researchers and others (administrators, mental healthcare policymakers, philosophers); (3) a hybrid consensus meeting to agree on the final overarching, transdiagnostic core outcome set and corresponding time points of assessment of each outcome.
Ethical approval is not applicable to this study according to the Research Ethics Committee of the Capital Region of Denmark, as it is not an interventional study. All data will be reported anonymously, and it will not be possible to identify study participants. Results will be disseminated via stakeholder and research networks and peer-reviewed publications.
http://creativecommons.org/licenses/by-nc/4.0/
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
Decision-making in healthcare should always have the patient at the centre. Therefore, outcome selection in randomised clinical trials and measurements in the clinic should always reflect what is important to patients and society. Patient important outcomes are outcomes that are intrinsically valuable to patients, typically including outcomes such as all-cause mortality, quality of life, adverse effects and functional status.1 Ideally, treatments should also produce socially relevant outcomes such as improved work capacity or reduced anti-social behaviour. However, many psychiatric and psychological interventions are currently recommended based on their efficacy/effectiveness defined commonly as the ability to affect a score on a questionnaire or observer-based rating scale assessing symptom severity,2–8 and social outcomes, in particular, are under-represented.9 Symptom severity may be important, but symptom suppression may not necessarily lead to increased function or improved quality of life. Furthermore, symptom scores do not always factor in important aspects such as adverse effects or other negative consequences of the received intervention. For example, a sedative may improve short-term symptom scores but may not enhance functioning and may impair it in the long term. Additionally, antidepressants may alleviate depressive symptoms in the short term; however, they can also be associated with adverse effects that persist into the long term, such as sexual dysfunction and emotional blunting.10 11
A solution to the proposed problems is the development of core sets of outcomes. Core outcome sets can be developed to establish a minimum list of patient important outcomes for a given patient population. The development and use of a core outcome set ensures that researchers measure and report those outcomes that are most likely to be relevant to patients and other people making decisions about healthcare, and to ensure that they do so in a uniform way across trials.12 In summary, there are a number of important benefits from a core outcome set:
Core outcome sets are developed in a process involving relevant stakeholders, including patients, families, researchers, healthcare providers, funders and policymakers in an international consensus-based Delphi approach, recommending selection and measurement of the most important patient outcomes. Initiatives have been created to facilitate this process, including The Core Outcome Measures in Effectiveness Trials (COMET) and The International Consortium for Health Outcome Measurement.14–16 On 2 February 2024, we searched both databases for core outcome set-projects in ‘Mental health’. A list of finalised core outcome sets identified in the two databases for adults with mental health disorders can be found in online supplemental appendix 1.
Core outcome sets have been increasingly developed and used in other medical specialties. The uptake of core outcome sets in subsequent randomised clinical trials has been reviewed showing that uptake rates reported for randomised clinical trials varied from 0% of trials (gout) to 82% of trials (rheumatoid arthritis) measuring the complete core outcome set.17 Suggested barriers to uptake included lack of validated measures, lack of patient and other key stakeholder involvement in core outcome set development and lack of awareness of the core outcome set.17
Previous core outcome sets for adult mental health disorders have primarily recommended symptom rating scales (online supplemental appendix 1). It is evident that assessing mental health symptoms is highly relevant in mental health trials, but it is unclear if symptom reduction results in increased functioning or improved quality of life or decreased risks of adverse events. However, focusing only on symptom scales is problematic because of the absence of clearly determined minimal clinically important differences for most scales, problems with missing data and the subjective nature of symptom measures.18 Additionally, patients with mental health disorders often have multiple co-occurring mental health disorders, making it redundant to only measure symptom severity for one diagnosis, and it is likely that some outcomes are important to patients regardless of their diagnosis (eg, risk of suicide, psychiatric hospitalisations, inability to work or quality of life). Moreover, new diagnoses can occur (eg, complex post-traumatic stress disorder has been added to the 11th revision of the International Classification of Diseases, ICD-11) and diagnostic symptoms of existing diagnoses can change over time (eg, the categorical classification of personality disorders has been changed to a dimensional classification in ICD-11) making reliance solely on disorder-specific symptom scales problematic. Recent advantages in psychotherapy research have focused on developing transdiagnostic interventions for mental health disorders,19–21 including the Unified Protocol, which is a transdiagnostic cognitive behavioural therapy.21–23 Additionally, transdiagnostic and dimensional approaches to psychopathology have been developed, for example, The Hierarchical Taxonomy Of Psychopathology model.24 Hence, transdiagnostic psychopathological assessments and treatments cannot be assessed using diagnosis-specific symptom scales and must rely on global measures of distress and impairment.
We believe that there is a need for a minimum list of overarching, transdiagnostic (regardless of mental disorder) core outcomes to be used across randomised clinical trials of all common adult mental health disorders. No transdiagnostic core outcome set recommendations are available for common mental health disorders.
The objective is to develop an overarching core outcome set for adult mental health disorders. The overarching core outcome set will contain only the most critical and minimal domains (ie, what to measure) and the final core outcome set may then be used for all adult mental health disorders. The core outcome set can be used concurrently with disorder-specific core outcome sets and/or other assessments. The core outcome set will provide feasible and generalisable recommendations for a minimum list of patient important outcomes to be used in clinical trials, clinical practice and clinical studies within each adult mental health disorder.
The core outcome set development methodology will follow the recommendations from the COMET Initiative12 25 by adhering to the Core Outcome Set-STAndards for Development guideline. The project has been prospectively registered in the COMET database25 on 17 May 2024 (https://www.comet-initiative.org/Studies/Details/3125). The project will not integrate other core outcome set frameworks beyond COMET. The project commenced on 1 June 2024 and will be completed by 31 December 2029. Participants will be recruited from September 2025.
The Patient Important Outcomes in Psychiatry (PIO-Psych) Initiative will be structured in three phases as depicted in figure 1.
The outcomes used in current clinical research will be identified by reviewing recent randomised clinical trials to extract primary and secondary transdiagnostic outcomes (as defined below). We will include trials published within the last 2 years to review current practice.26
Two review authors will independently screen titles and abstracts. We will retrieve all relevant full-text study reports/publications. Two review authors will independently screen the full texts and will record reasons for the exclusion of the ineligible studies. The same two review authors will resolve any disagreements through discussion, or if required, they will consult a third author (JCJ).
We will include randomised clinical trials irrespective of trial design and setting (eg, inpatient, outpatient or primary care settings). We will not include prevention trials of healthy participants or trials including primarily somatic patients.
We will include trials assessing any mental health disorder or transdiagnostic symptoms (eg, self-harm) in adults, for example, major depressive disorder, bipolar disorder, anxiety disorders, obsessive-compulsive disorder, eating disorders, personality disorders, post-traumatic stress disorder and schizophrenia.
We will include any randomised clinical trial irrespective of intervention and comparator groups.
We will extract transdiagnostic primary or secondary outcomes used in the trials. Transdiagnostic outcomes will be defined as any clinical outcome that is important to patients and not tied to a particular mental health disorder. Examples of transdiagnostic outcomes are suicide, suicide attempts, psychiatric hospitalisation, quality of life and level of functioning. Each component of composite outcomes will be handled as single outcomes. We will not extract biomarkers as they are usually unvalidated surrogate outcomes.
Two review authors will independently extract data from included trials in a predefined form. Disagreements will be resolved by discussion with a third author (JCJ). Each trial will be named after the first author and year of the primary publication, and all eventual secondary publications will be classified under that name.
Outcomes identified from the scoping review will be pruned into a smaller list of generalisable outcomes resulting in the ‘scoping-review-generated outcomes’ for the Delphi process.27 First, duplicate verbatim outcomes will be removed after homogenising spelling (eg, suicide and suicides, rates and rate and so on). Second, outcomes, meaning the same, will be rewritten by two reviewers independently to develop non-verbatim outcomes. The rewriting process will be documented. Finally, outcomes remaining after the removal of duplicate non-verbatim outcomes will define the list of unique outcomes.
PIO-Psych organisation
The PIO-Psych organisation consists of a steering group and two advisory groups: (1) the people with lived experiences advisory group, and (2) the advisory group experienced in core outcome set development. The steering group will meet with the advisory groups between every round in the Delphi process.
We will include the following four stakeholder groups (from now on referred to as participants) in our Delphi survey:
Others (eg, administrators, mental health policymakers, funders).
Participants voting in the Delphi process will be identified through the author lists of trials included in the systematic scoping review, the list of potential stakeholders provided in online supplemental appendix 2, through snowball sampling, or the steering committee will appoint them. People with lived experiences will be identified through contact with national and international patient and relatives’ organisations. We will contact all stakeholders who participated in previous rounds to invite them to participate in subsequent rounds. The participants will be included irrespective of sex, gender, background, setting, age, profession and nationality, if they are able to communicate in English. In cases of low engagement, we will contact additional stakeholders (eg, additional patient organisations, mental health clinics, administrators) from additional English-speaking countries or invite additional members into the PIO-Psych steering group to aid with stakeholder recruitment in their respective countries.
The consensus process will be organised according to the COMET Handbook V.1.0 and V.2.0 guidelines26 28 and following recommendations for the Delphi process.29–31 The Delphi process will run in three rounds, which are visualised in figure 2.
Round 1: All potential outcomes that have been identified in the systematic scoping review of mental health trials (see Step 1) will be sent to all four stakeholder groups, namely (1) people with lived experience and relatives, (2) clinicians, (3) researchers and (4) others. Prior to completing round 1, all participants will be requested to watch a short educational video on Delphi processes and then answer five comprehension questions (mandatory and with unlimited number of attempts). Each participant will then rate the importance of each outcome on a scale from 1 to 3 (not important), 4 to 6 (important) and 7 to 9 (critically important). In round 1 of the Delphi study, participants can suggest new items to be included in the second round. Outcomes that are rated between 7 and 9 on the rating scale by ≥50% of all participant groups combined will move to round 2 of the Delphi process. Furthermore, new outcomes suggested by ≥15% of all participants will be added to the list of outcomes that move to round 2. These thresholds have been suggested by the COMET Handbook.26
Round 2: In round 2, each participant, irrespective of their participation in round 1, will be provided with the list of outcomes that moved forward from round 1. They will also be shown the number of respondents and the distribution of scores for each outcome for all stakeholder groups separately, together with their own score from round 1. Any additional outcomes suggested in round 1 will be scored in round 2. Along with the quantitative scores, blinded qualitative explanations for the scores will be provided to all participants. At this point, each participant will again rate the outcomes on the scale from 1 to 9. Besides rating outcomes, each participant will also be asked to rate the most important time point of assessment of each outcome. Time points will be categorised as: 0–3 months, 3–6 months, 6–12 months, >12 months. Consensus will be defined a priori as requiring a score between 7 and 9 by at least 70% of the participants. This stringent consensus criterion will help to narrow down the list of core outcome set to outcomes that are meaningful from both clinician/researcher and people with lived experience perspectives.
Round 3: In round 3, the outcomes and time points that met the consensus criteria in round 2 will be ranked. The top five ranked outcomes for each stakeholder group will be retained and an average rank will be calculated. For example, an outcome that was ranked as 1 by the people with lived experience and relatives group and the researcher group but 3 by the clinician group and the other group will have an average rank of 2 ((1+1+3+3)/4). The outcomes will then be ranked based on their averaged ranks. Time points will be ranked in primary and secondary time points for each outcome.
The Delphi process will be conducted online and managed using DelphiManager software developed by the COMET Initiative.32 The anonymised data from both rounds of the Delphi process, itemised by stakeholder group, will be published as supplementary material together with the publication of the results.
All participants will, as a final step, be invited to a hybrid meeting (both physical and online attendance available) in Copenhagen, where the final core outcome set will be presented and discussed. The final outcomes and corresponding time points after this meeting will constitute the core outcome set. The final aim of the Delphi technique is that after several rounds, consensus will be reached on a set of outcomes and corresponding time points of assessment that all participants agree are important. This process is illustrated in figure 2.
Patients or the public were not involved in the design of this protocol, but will be involved in the future conduct, reporting and dissemination plans of the PIO-Psych Initiative. People with lived experiences will be included as important stakeholders in the Delphi process. Additionally, we will contact national and international patient organisations to involve them in our reporting and dissemination plans, once the data collection is finalised.
Ethical approval is not applicable to this study as it is not an interventional study, as assessed by the Research Ethics Committee of the Capital Region of Denmark (application ID: F-24054116). The study will be approved by the Danish Data Protection Agency before commencement of the Delphi process. All data will be reported anonymously, and it will not be possible to identify study participants.
We will disseminate our results in international peer-reviewed scientific journals and the COMET Initiative10 and COMET database.23 Furthermore, we will participate in national and international adult mental health conferences. In that way, we will reach our target audience, which primarily is other researchers who are planning to launch future randomised clinical trials for adults with mental health disorders. We will also write and publish a national and an international press release with the results of each core outcome set project. Furthermore, we will develop a website and social media accounts (LinkedIn, Facebook), so that key stakeholders can be updated about recent advances in the project. Finally, we will contact Sundhedsstyrelsen (the Danish Health Authority), the Canadian Mood and Anxiety Guideline developers, the National Institute for Health and Care Excellence, and other relevant guideline developers for adult mental health disorders to inform them that a transdiagnostic core outcome set has been developed, so that the recommendations for outcome selection can be included in future national and international clinical guidelines.
Assessing outcomes that are of the most importance to mental health patients is of utmost importance both from a patient perspective and from a societal perspective. Developing an overarching, transdiagnostic core outcome set to identify the most patient important outcomes is essential to answer adult patient relevant clinical questions and to optimise the data quality of randomised clinical trials. No finalised transdiagnostic core outcome set recommendations are available for prevalent adult mental health disorders. The PIO-Psych Initiative aims to fill in this gap.
This project has several strengths. It is based on strict methodological guidelines for core outcome set development provided by the COMET Initiative,12 and the project was pre-registered in the COMET database25 before data collection begins. This procedure eliminates the risk of selective reporting of results, and it increases the reliability of the results.
This project also has limitations. First, while we plan to reach as many stakeholders as possible (eg, through online surveys which will be distributed nationally and internationally through social media and via email by selected international stakeholder groups), we may run the risk that the included stakeholders are not representative of all stakeholders due to language barriers and self-selection/volunteer bias. Second, the flexible nature of Delphi studies, including voting over multiple rounds, can lead to drop-outs and introduce bias towards majority opinions.29 To mitigate potential drop-out, we allow registered participants to vote in any round, regardless of their participation in previous rounds, and we will take into account the potential bias caused by the risk of majority views influencing individual responses when interpreting the results. Third, we will not include recommendations on how to measure each outcome in the finalised core outcome set (ie, which specific interviews or questionnaires). However, when the transdiagnostic core outcome set is finalised, we will give examples of relevant outcome measurement instruments or registries to assess each outcome, but the Delphi process will not include recommendations for outcome measurement instruments. Fourth, we may find some overlap in our transdiagnostic core outcome set with disorder-specific core outcome sets. For example, weight loss/loss of appetite can be a transdiagnostic symptom, but it can also be a symptom related to depressive disorders or eating disorders. However, we will make sure to include people with lived experience from multiple mental health disorder groups in the Delphi process. Hence, if a particular outcome is included in the final core outcome set, we will ensure it has a transdiagnostic value. We will consider these limitations when presenting the core outcome set results.
There is currently no overarching transdiagnostic core outcome set for adult mental health disorders. This transdiagnostic core outcome set will ensure that a minimum list of relevant outcomes is measured in clinical trials, clinical practice and clinical studies for adult patients with mental health disorders concurrently with disorder-specific outcomes.
Not applicable.
We thank student assistant Louise Le for graphical assistance with the figures.