Involving the Community in Long COVID Research: Methods, Results, and Suggestions from the Long COVID and Episodic Disability Study

Community engagement in research is critical to ensure relevant, meaningful and impactful research that will advance clinical practice and policy for persons living with complex chronic conditions. Community engagement, also known as Patient and Public Involvement (PPI), is described as research done ‘with’ or ‘by’ people with lived experience, rather than ‘to’, ‘about’ or ‘for’ them [1, 2], or ‘meaningful and active collaboration’ [3]. People with lived experiences are critical to the research process as they bring specialised knowledge of health conditions, and their input improves the quality and relevance of research [4]. Actively involving people with lived experiences of health conditions in the design and management of health policy, services and research has increased since the 1990s [4]. This growth was largely grounded within the activism of people with lived experiences of disability and illness, and community principles of participation, of ‘nothing about us without us’ [5, 6, 7]. For example, persons living with HIV developed position statements asserting their right to be involved in decisions about them, such as the Greater Involvement of People Living with HIV/AIDS (GIPA) and Meaningful Engagement of People Living with HIV/AIDS (MEPA) [8, 9, 10, 11]. Investigators should engage patients in all stages of research, from design and conduct, through to interpretation, dissemination and implementation [12]. Guidance documents exist to advise researchers embarking on patient engagement in research [13]. Some national funding organisations have established guidance and resources for fostering meaningful community engagement, such as the Canadian Institutes of Health Research (CIHR), Strategy for Patient‐Oriented Research (SPOR) and National Institute for Health Research (NIHR) PPI in research. These funders require investigators seeking their funding to follow this guidance [2, 3]. Despite the importance of community engagement in research and existing guidance, community engagement has yet to be fully embedded in all aspects of health research [14].

While a range of terminology exists to characterise PPI in research, we use ‘community engagement’ as the concept of inquiry and refer to team members with lived experiences as ‘community partners’ in this manuscript. We chose this terminology to reflect leadership among members with lived experiences on the team within the wider community of people with Long Covid.

The Covid‐19 pandemic highlighted the need for community advocacy and representation throughout the research process. Community engagement was particularly critical in the case of this newly emerging disease condition, and one which was overlooked by research and health authorities in the early days of its emergence. The voices of those with lived experiences with Long Covid have been instrumental in driving research on this condition. Early in the Covid‐19 pandemic, many were noticing that they were experiencing symptoms of Covid‐19 for much longer than the acute period and recognised these symptoms as a unique manifestation of Covid‐19. As early as April 2020, people with lingering illness from SARS‐CoV‐2 had self‐organised into online groups and were sharing their experiences [15, 16]. The term Long Covid was coined by a prominent patient advocate and taken up by the community [15]. A group of people with Long Covid founded the Patient‐Led Research Collaborative (PLRC) and were the first to document the multiple symptoms associated with Long Covid on 6 May 2020 [17, 18], which were then subsequently expanded on in a survey documenting over 200 symptoms. Since then, there has been an explosion of research seeking to define and characterise Long Covid [19, 20, 21, 22, 23], identify potential causes of this condition [24], and explore potential treatments [25]. Despite community leadership in Long Covid research, persons with lived experiences are not always involved throughout the research process from inception to dissemination, nor meaningfully engaged in the process [26]. Tokenism, (unconscious) undervaluing of lived experiences, under‐investing in support for patient partners (e.g., lack of or insufficient compensation for patient partners), and under‐estimating the vulnerability of patient partners persist, highlighting gaps in community engagement in research [26].

The Long COVID and Episodic Disability Study aims to characterise and measure disability experienced among people living with Long Covid [27]. This international study was conceived by a member of the team (D.B.) who has lived experiences of Long Covid and was instrumental in the creation of the patient‐led organisation, Long COVID Physio [28]. As a physiotherapist with specialisation in HIV, his perspectives allowed him to understand that, like HIV, Long Covid can be understood as an episodic illness and that lessons learned from the conceptual Episodic Disability Framework could be usefully modified for Long Covid [29, 30].

Despite the existence of guidance documents and toolkits [13, 31], it is still possible for researchers to follow all the advice and ‘do’ apparent community engagement while being tokenistic [14, 32]. We do not wish to repeat the excellent practical advice which can already be found in numerous publications [33, 34, 35, 36, 37, 38]. In this paper, we aim to offer insights beyond the process elements of community engagement and instead elucidate the practices which allow research teams to move beyond tokenism. Specifically, we aim to: (1) describe our process of undertaking a community‐engaged Long Covid research study (methods); (2) evaluate our community‐engaged approach, highlighting strengths and limitations with our process (results); and (3) identify recommendations for engaging in community‐engaged patient‐oriented research in Long Covid (discussion).

The Long COVID and Episodic Disability Study is a community‐engaged research study involving 25 members, including 12 persons living with Long Covid, 13 researchers and 5 clinicians (categories are not mutually exclusive), referred to as the Full Team. Persons with lived experiences possessed a range of professional and personal experiences spanning research, clinical, policy and private sector/business contexts; team members wore multiple hats and perspectives which collectively strengthened the diversity of expertise, perspectives and insights to the team and process. Engagement of people with lived experiences with Long Covid ensured that the study was fully co‐created with people living with Long Covid. During the development of the study proposal, community partners from organisations in Canada, Ireland, the United Kingdom and the United States, who were linked to larger networks of people living with Long Covid, were purposefully invited to join the study team. Several Long Covid community networks and organisations, represented by persons living with Long Covid, were involved in all stages of the research, including: COVID Long‐Haulers Support Group Canada (S.G.) [43]; Long COVID Advocacy Ireland (I.O., S.O. and R.S.) [44]; Long COVID Ireland (N.R. and R.S.); Long COVID Physio (D.A.B. and C.T.) [45]; Long Covid Support UK (M.O.H.) [46]; and PLRC (L.M., N.M. and H.W.) [47]. These representatives, along with the Co‐PIs (K.K.O. and D.A.B.) and co‐ordinator (K.M.), comprised the Core Long COVID and Episodic Disability Community Collaborator Team (Core Team). Team members with lived experiences were provided yearly remuneration for their time and expertise dedicated to the study, either as an individual, or to the community organisation which they represented on the study according to their preference.

The Core Team guided all stages of the research study. The Core Team was comprised of Co‐PIs (K.K.O. and D.A.B.), persons with lived experiences representing the above community networks (S.G., I.O., S.O., R.S., N.R., R.S., C.T., M.O.H., L.M., N.M. and H.W.), and clinicians and researchers from St. James's Hospital (C.B., C.Ba. and L.T.). The Core Team was responsible for recruitment, refinement of data collection tools, interpretation of findings, and decisions surrounding ethical considerations as they related to safe engagement with persons with Long Covid and knowledge translation. The Core Team worked closely with the broader research team at all stages of the research. The Core Team met frequently (monthly to bimonthly) and also met with the broader research team (Full Team) at milestones throughout.

Research staff used several channels for communication among the Full Team and Core Team, including email, online platforms (Zoom), and circulating written materials via attachments in email. Staff used email to schedule and send reminders about team meetings. Staff arranged meeting times to accommodate team members across different time zones. Staff circulated documents approximately 1 week in advance of (e.g., agenda, summaries of results and meeting minutes) or after (e.g., minutes) meetings. The co‐ordinator (K.M.) circulated same‐day reminders for each meeting to aid those with cognitive constraints. Staff provided regular email updates on the progress of the study, including upcoming meetings, status of recruitment and upcoming knowledge translation opportunities. Out of respect for the limited time and energy of those contributing, staff encouraged open feedback about the need for flexibility in terms of means of study engagement and timelines for task completion. When team members were unable to attend meetings, the research staff offered phone or online calls at a convenient time to follow up one‐on‐one with the co‐ordinator or study Co‐PIs. Staff provided generous lead times for completion of tasks. However, timelines were also made flexible based on the capacity and schedule of team members.

Team members with lived experiences of Long Covid and representatives of community organisations led the recruitment of adults living with Long Covid to the study. Community partners promoted the study widely within their networks. Recruitment was an iterative and purposeful process whereby the Core Team met weekly during phase 1 (interviews) to discuss the characteristics of participants to date in relation to our aim to achieve diversity in age, gender identity, ethnicity, sexual orientation and time since initial Covid‐19 infection. The Core Team continued to meet bi‐weekly during phase 2 (web‐based survey) during the recruitment phase to discuss the process of recruitment, geographical (country) representation and any issues with recruitment or data collection.

Members of the Core Team were involved in refining the interview guide for phase 1 and developing and refining the PROMs for the web‐based questionnaire for phase 2. During phase 1 (interviews), the Core Team was involved in group‐based qualitative analysis of interview data using content analytical techniques [48]. The Core Team met twice to review coding and participant summaries for each interview and discussed interpretations of the analytical categories representing experiences of disability. After the interview analysis, the Core Team mapped categories of disability as experienced by adults living with Long Covid to the existing EDQ questions, originally derived from and with persons living with HIV [49, 50, 51, 52]. For phase 2, the team set out to identify a PROM to assess the presence, severity and episodic nature of disability among adults living with Long Covid. The Core Team met on two subsequent occasions to generate items for a supplemental questionnaire that represented categories of disability from the interviews that were not captured in the existing EDQ. The study team met twice (once as a Core Team and once as a Full Team) to review, prioritise and refine items, resulting in a draft Long COVID Supplement Questionnaire. The Full Team reviewed and refined the wording of the items and preamble to establish a Long COVID Episodic Disability Questionnaire (LC‐EDQ)—Supplement. The study team is currently assessing the measurement properties of the EDQ and LC‐EDQ Supplement.

The Core Team is involved in co‐authoring presentations, publications (including study protocol) and other knowledge translation outputs from the Long COVID and Episodic Disability Study [27, 39, 40]. Manuscript writing involves a co‐creation approach. Community partners provide feedback on all manuscript drafts. Comments and amendments are synthesised by the study team. Feedback on manuscripts is provided via email, track changes in a Word document, or Zoom. Research staff are flexible with timelines on feedback to accommodate capacity and competing responsibilities of team members. b.Evaluating Strengths and Limitations of Our Community‐Engaged Process

Authors (M.O.H., K.K.O. and K.M.) conducted a multi‐stage reflection with members of the Long COVID and Episodic Disability Study team (including community partners, researchers, clinicians and staff) regarding their experiences with community engagement in the study. We reached out to the REB, inquiring about our need for ethics review and approval. We received confirmation that, given this was an ongoing and iterative reflective process among members of the research team, we did not require REB approval for this evaluation.

At the end of each Core or Full Team meeting, team members were asked about their experiences and specifically asked to share their perspectives and feedback about the team process and what it was like to be involved in this study team.

Our evaluation was guided by a framework for describing patient engagement (4PI) [34] and Patient‐Led Research Scorecards [53] to identify strengths and challenges of community (or patient) engagement in the study. Evaluation of the effectiveness of patient engagement in research is not straightforward, and a number of impact evaluation frameworks exist [54, 55]. We chose to use the 4PI and Patient‐Led Research Scorecards as they are simple in their construction and easy to understand and were developed principally by people with lived experiences [34, 53]. The 4PI Framework is a commonly used framework for evaluation of PPI, and the lead author (M.O.H.) was familiar with its use [34, 53, 56]. Launched in 2013 and developed by the National Survivor User Network (NSUN), the 4PI Framework stands for ‘Principles; Purpose; Presence; Process; Impact’ [34]. This framework codifies ways of working between groups of people with a common purpose or project. The 4PI framework helps working groups think through their purpose, how they work together and how they will evaluate whether they have achieved their purpose. Examples of its use can be seen on the NSUN website [56].

The Patient‐Led Research Scorecards were published in 2023 as a patient‐centred, practical evaluation on the fairness and effectiveness of patient group and research partner collaboration [53]. Although the Scorecards are not specific to Long Covid, they were designed and written by the PLRC, including persons with lived experiences of Long Covid, and written with the needs of people with chronic health conditions in mind [53]. Some developers of the PLRC Scorecards are members of the Long COVID and Episodic Disability Study team and are co‐authors of this manuscript (H.W. and L.M.). We chose the Patient‐Led Scorecards because they were recently developed by persons with lived experiences with Long Covid. This was the first known study to use them in a formal evaluation of the community‐engaged research process. The 4PI framework is discursive, while the Scorecards provide numerical scores. Given the strengths of both, we used both to inform our approach.

In January 2024, the lead author and community partner (M.O.H.) met with other team members with lived experiences online (Zoom) to discuss the process and impact of patient engagement in the Long COVID and Episodic Disability Study. During this 90‐min discussion, community partners were asked to reflect on how they had worked together throughout the study according to the categories of the 4PI. MOH provided examples as a starter for discussion and then further populated the categories as the discussion ensued. M.O.H. asked team members to share their considerations for future patient engagement in Long Covid research. See Supplemental Filefor the guiding questions in the discussion. M.O.H. recorded the meeting and took notes to capture the discussion. No researchers were present at this meeting to enable community members to speak freely. 1

In March and April 2024, authors (M.O.H., K.K.O. and K.M.) transferred the contents of the PLRC Scorecards to an online questionnaire to make it easier to use and collect the data. This questionnaire was then sent to all members of the Long COVID and Episodic Disability Study team to obtain perspectives on patient engagement efforts during the study. Using the Scorecards as a foundation, authors asked all members of the team to reflect on and discuss their experiences with the study, either as an organisation or as an individual. The questionnaire included 14 items separated into the following four domains of the Scorecards: (1) Patient Burden; (2) Patient/Partner Governance; (3) Integration Into the Research Process; and (4) Research Organisation Readiness [53]. Team members were asked to rate their perception of patient engagement in the Long COVID and Episodic Disability Study team from non‐collaboration (−2) to ideal collaboration (+2). Each section included open‐ended text responses to allow for further expansion. The questionnaire also included two questions on the usefulness of the Scorecards to elicit perceptions on patient collaborative research efforts, with a range from −2 (not useful at all) to +2 (very useful) and asked about recommendations for how the Scorecards might be used to enhance collaborative engagement. M.O.H., K.K.O. and K.M. administered two versions of the questionnaire, one tailored to patient partners and the other tailored to researchers and clinicians on the team. See Supplemental File 2 for the questionnaire.

We consolidated comments derived from the guided discussion and Patient‐Led Research Scorecards to develop recommendations for engaging in community‐engaged patient‐oriented research in Long Covid. The lead author (M.O.H.) triangulated these two data sources with the aim of deriving practical recommendations that can be applied across different community‐engaged and participatory research contexts. All members of the co‐authorship team reviewed and refined the recommendations.

Of the 24 team members of the Long COVID and Episodic Disability Study, 10 (42%) completed the self‐reported Scorecard questionnaire. Of those 10 team members, five (50%) were researchers and five (50%) were community partners. Roles of the researchers (n = 5) on the team were self‐identified as ‘researcher’ (n = 3/5; 60%) and ‘clinician’ (n = 2/5; 20%). Among the five community partners, four specified their role on the team; one (1/4; 25%) identified as a ‘researcher’ and three (3/4; 75%) as a ‘person with lived experiences’. Four of the five community partners (80%) completed the questionnaire as individuals, and one (20%) completed the questionnaire in collaboration with other members of the community network they represented.

Team members ranked their agreement with the statements for each of the four domains of the Scorecards. The median scores (range −2 to +2) across questions in the Patient Burden and Research Organisation Readiness domains were +1 among researchers and +2 among community partners. The median scores for Patient/Partner Governance were +1 among both researchers and community partners. The median score for integration into the research process was +2 among both researchers and community partners. See Table 1 for details.

Comments from team members identified strengths and limitations in our approach, highlighting ways to consider improving our process moving forward. Strengths included establishing clear and simple processes for payment of community partners with honoraria; clear communications at a frequency to keep everyone updated but not be burdensome; arranging meetings at mutually convenient times; and providing plenty of notice when work was requested. Examples of areas for improvement in our approach included explicitly establishing terms of reference or culture at the foundation of our study; rules of engagement, and embedding trauma‐informed practices at the outset of a project with community partners; highlighting the need for more concise and clear email communication; and further establishing diversity of representation of the patient population on the team.

When asked about the usefulness of the Scorecards for eliciting perceptions on patient collaborative research efforts, the median score (range −2 to +2) was +1 among researchers and +2 among patient partners. The median for usefulness of the Scorecards for fostering collaborative efforts in research was +1 among researchers and +2 among patient partners. See Table 2 for details. Recommendations for how the Scorecards could be used to enhance collaborative engagement included: new researchers using the Scorecards to guide engagement planning; research team leads educating and preparing their teams for partnership with people with lived experiences; all members of a research team establishing a common language and understanding patient engagement; and funders evaluating patient‐engagement plans for proposed research projects.

Five team members with lived experiences (community partners), representing four community support groups and organisations, met with M.O.H. to discuss our experiences of engagement in the Long COVID and Episodic Disability Study and its impact.

While we used the 4PI framework to guide the discussion, the main focus was on the (1) principles and (2) impact sections. We describe the results of our discussion using the category headings below. We concentrate on these two domains in an attempt to uncover a deeper elucidation of the elements of partnership working which lead to meaningful engagement.

Our study featured many of the practical components shown to be important in community engagement, which can be found in most guidance documents. In Table 3, we propose a series of recommendations and considerations focused on practices that underpin meaningful community engagement and lead to impact and a positive experience for all concerned. These practices speak to the embodiment and manifestation of principles that pave the way for the sharing of power across a study team. Unique features of these recommendations are that they are directed to both persons with lived experiences as well as researchers, and they were derived from the perspectives of navigating the research landscape while living with a chronic and episodic condition such as Long Covid, which can be unique to other studies focused on different conditions. For these recommendations to be successfully implemented, community members must be sufficiently empowered to feel comfortable challenging the status quo of a research process. This makes it all the more important that research teams make efforts to ensure that conditions exist such that community partners feel enabled and supported to challenge the process where they feel it necessary.

This study was derived from an exemplar involving an international study involving a multidisciplinary team including persons with lived experiences of Long Covid, clinicians and researchers who were involved in the study since the conceptualisation of the study objectives and acquisition of funding. This study was derived from a longstanding international collaboration in HIV, disability and rehabilitation (D.B. and K.K.O.); the field of HIV and community‐based research was instrumental in informing the approaches among the research team. This study builds on the foundational literature of community‐based research and the GIPA/MEPA and the disability community principles of participation, of ‘nothing about us without us’ [7].

Nevertheless, our team and the Long COVID and Episodic Disability Study are situated in high‐income countries, with limited diversity, raising a query as to whether these lessons learned are transferable to other contexts.

This study was the first to use the Patient‐Led Scorecards to formally evaluate community engagement in the context of a Long Covid research study. Responses to the Patient‐Led Research Scorecard questionnaire are descriptive in nature. We developed the Patient‐Led Research questionnaire for the purposes of our consultation, whereby items in the questionnaire were directly derived from the Patient‐Led Research Scorecards and reviewed by members of PLRC on the team involved in the creation of the Scorecards (L.M. and H.W.). Nevertheless, the measurement properties of the tool are unknown.

While our consultation included a community‐led discussion with patient partners on the team, followed by an anonymous questionnaire, it is possible that team members may have felt reluctant to voice concerns or limitations in the team process, as the study is ongoing. The combination of lead author (M.O.H.) expertise in PPI with the co‐PIs' (K.K.O. and D.A.B.) experiences with community involvement in their research and clinical practices in the context of HIV, disability and rehabilitation meant we were able to bring a compilation of experiences and lessons learned to this study. Hence, we feel that the findings accurately reflect the strengths as well as considerations for improvement for engaging in this Long COVID and Episodic Disability Study. However, we acknowledge that the positive results of our process evaluation are a reflection of the community leaders living with Long Covid on this team who had strong coalitions and embedded academic expertise prior to the start of this study; otherwise, this dynamic process may not have been so fruitful or could have been limited. It is important to note that patients often lack institutional support that academics have access to; hence, ensuring individual comfort and empowerment, in conjunction with helping to support the communities behind the patient partners, is critical to this process. Finally, this manuscript is derived primarily from our process in phase 1 (interview) of the Long COVID and Episodic Disability Study; phase 2 (web‐based questionnaire with the EDQ and Long COVID Supplement) is still underway. We are 4 years into this study since our acquisition of funding in 2021. We continue to build this research programme and collaborate on other research related to Long Covid, disability and rehabilitation. Hence, our process and lessons learned will evolve as a team as the study continues over time.