Psychedelic-assisted therapy as a complex treatment: considerations for clinical trial design

The field of psychedelic medicine and psychedelic-assisted therapy (PAT) has seen rapid expansion in the last decade. With only a small number of clinical trials conducted before 2017, one analysis of clinicaltrials.gov showed that as of May 2024, 278 clinical trials had been registered for classic psychedelics, MDMA and related compounds.¹ Another analysis showed that as of April 2024, over 134 clinical trials for psilocybin alone had been registered studying 54 potential indications.² Despite this volume of clinical trials, some of the core theoretical principles for the development of PAT as an intervention (such as the role of any accompanying psychotherapy/psychological support) remain both understudied and unresolved. In this position piece, we argue that this lack of theoretical understanding of PAT derives in part from treating PAT as a simple pharmaceutical intervention rather than as a complex intervention. We introduce and apply the UK Medical Research Council's (MRC) well-established complex intervention framework³ to PAT and argue that this framework provides a more structured approach to answer not only key theoretical questions regarding PAT, but also to develop PAT in a way that will better align with existing healthcare systems and ultimately improve its utility to patients. Following this, we briefly recap some of the shortcomings of 'standard' explanatory clinical trials when applied to PAT and argue that PAT trials conducted in the context of the complex intervention framework could take more pragmatic design approaches. We then introduce the widely used PRECIS-2 tool⁴ for designing pragmatic trials and show how it can be applied to PAT trials, both past and present. In considering the positionality of various psychedelic research actors, we argue that academic/community-based investigators are well positioned to conduct the types of trials we envision. In particular, unlike typical drug development, the generic nature of key psychedelic drug compounds allows these types of trials to be conducted prior marketing approval being obtained from regulators. Indeed, data from the more diverse types of trials we propose could form part of a more pluralistic evidence assessment for the utility of PAT as a healthcare intervention.

Interventions are the basis of healthcare and can be defined as 'a treatment procedure or action taken to prevent disease or improve health in other ways'.⁵ Simple interventions tend to have direct linear pathways between an intervention and outcome, while complex interventions tend to have multiple interacting components and non-linear causal pathways.⁶ The definitions of simplicity/complexity are important as the tools used to research and evaluate complex interventions can be quite different than for simple interventions where randomised controlled trials are often the 'gold-standard'. The UK Medical Research Council's (MRC) latest guidance for developing and evaluating complex interventions³ considers properties that might define an intervention as complex including 'the number of components involved; the range of behaviours targeted; expertise and skills required by those delivering and receiving the intervention the number of groups, settings, or levels targeted; or the permitted level of flexibility of the intervention or its components (p.2)'. PAT typically involves preparation, dosing and integration sessions with one or more patients and therapists at a time. Delivering the intervention requires specialised clinical skills, specific therapeutic settings and flexibility in protocols – including the number and timing of sessions and the nature of therapist–patient interactions. PAT also interacts in complex ways with existing healthcare systems and is embedded within broader cultural histories of psychedelic drug use. Taken together, these features suggest that PAT fits well within the definition of a complex intervention (see also Noorani and Liebert).⁷

The framework devised by the MRC^3,8 challenges the view that measuring effectiveness is the primary goal of complex intervention development (See Figure 1 for definitions of effectiveness and efficacy and other key concepts discussed in the paper). Rather, the focus is broadened to topics such as improving theory and understanding how interventions contribute to change and interact with their context, which are considered equally important areas of inquiry as determining effectiveness. Complex intervention research consists of the following phases: intervention development, assessment of feasibility and design, evaluation of the intervention and its impact and implementation. These phases do not necessarily occur sequentially and progress through them may reverse as new knowledge is obtained. Six key questions can be asked at each phase: (1) How does the intervention interact with its context? (2) What is the underlying theory? (3) How can various stakeholder perspectives be included in the research? (4) What are the key uncertainties? (5) How can the intervention be refined? and (6) What are the resource consequences of the intervention? As these questions are answered, the phases of research can be progressed both forward and backward as required. Arguably, research on PAT has focused heavily on measuring its efficacy, often to the detriment of developing PAT as an intervention itself and exploring the myriad questions about how PAT might operate when embedded within existing healthcare systems and society at large. Figure 2 provides an overview of the phases of the complex intervention approach and the types of research questions that could be embedded within PAT clinical trials. We deliberately exclude effectiveness/efficacy assessment as this is the current default, and note that to answer some of these questions, outcome measures may involve surveys, questionnaires, interviews and focus groups (both quantitative and qualitative data). Many of the research questions displayed in Figure 2 could be answered by prospective cohort (open-label) studies. That said, in the following sections, we outline how it would be desirable for PAT researchers to embed some of these questions within pragmatic randomised controlled trials. We next provide further motivation for conducting such trials, given the limitations of interpreting data from psychedelic double-blinded randomised controlled trials (DB-RCTs).

DB-RCTs provide the best evidence for determining whether a non-complex treatment has a causal effect on outcomes. This is why they are considered the gold standard for many areas of research. We have previously written extensively on the theoretical basis for the randomised control trial based on the Rubin causal model and its underlying assumptions of positivity, consistency, exchangeability and non-interference, and do not recapitulate those details here^9,10). Even if all the assumptions of the RCT are met and the study has been perfectly controlled and executed, the knowledge gained from RCTs is easily misinterpreted/over-extrapolated as discussed by Deaton and Cartwright.¹¹ First, while randomisation is often thought to create balance between treatment and control groups, it actually only ensures balance in expectation over multiple trials, meaning individual trials may still suffer from imbalances, particularly in trials with smaller sample sizes or with numerous covariates. Second, although RCTs aim for unbiased estimates of the average treatment effect (ATE), unbiasedness does not guarantee precision; results can be highly variable between trials, leading to imprecise conclusions. Third, even when RCTs produce unbiased estimates of the ATE, this information alone may have limited practical value, especially when treatment effects vary widely across individuals or settings. The practical value of the ATE to predicting individual effects is limited because it represents the central tendency of a distribution of individual effects and it is not possible to know where any individual falls within that distribution. Similarly, even if the ATE = 0 in a sample, this does not mean that the treatment works for no one – there may simply be an unidentified sub-population that the treatment does work for – but the parameters that define this sub-population cannot be identified. Both these interpretative mistakes are created by misunderstanding the fundamental problem of causal inference (see refs. 9, 10, 12 for more detailed discussion). Furthermore, as pointed out by Kaptchuk,¹³ double-blinding in RCTs is not a neutral device as it can change the mindset of patients, potentially causing biases such as ambivalence, passivity, confusion, resentful demoralisation and voluntary submission. In fact, the 'uncertainty' created in patients by being blinded to the intervention may well modify treatment effects/therapeutic processes and this has received little explicit study in the psychiatric literature.¹⁴ Other biases present in RCTs include those caused by the informed consent process, investigator self-selection, Hawthorne effects and outcome measurements affecting the intervention.¹³ Taken together then, there are many reasons why the ATE measured in an RCT is a far from perfect representation of any underlying treatment effect.

Importantly, in the context of the current work, RCTs often struggle with external validity – that is, the validity of applying a study's conclusions to other settings, healthcare contexts, patient populations and so forth. The extrapolation fallacy of assuming that the effects obtained by RCTs simply extrapolate to other contexts,^11,15 may be a major issue for complex interventions including PAT. By contrast, internal validity refers to the validity of conclusions drawn within the context of a particular study. Typical explanatory RCTs with their tightly defined treatment protocols, eligibility criteria and other somewhat contrived features are said to prioritise internal validity at the expense of external validity (cf later with pragmatic trials).¹⁵ However, the internal validity of PAT trials is highly questionable given the notably high frequency of functional unblinding that occurs.^12,16,17 These functional unblinding effects likely cause over-estimates of the ATE found by PAT trials. This creates the uncomfortable situation in which existing PAT trials may lack both internal and external validity. The remedy for this situation may be to think more expansively about what PAT research could involve in terms of complex intervention design as previously outlined, while also incorporating principles of pragmatic RCT design.

PAT is a complex intervention and, as currently formulated, is heavily resource-dependent. Obtaining even a single data point in a PAT study is resource-intensive and trials have rarely exceeded 100 participants. For reference, the two phase III MDMA-AT studies conducted to date included 90¹⁸ and 104 patients¹⁹ – relatively small for phase III studies. As currently instantiated, we argue that PAT studies do not lend themselves easily to quantitative evaluation based on purely inferential statistical principles. Consider the generic power calculations for a study aiming to evaluate the differences between two groups (e.g. a parallel groups RCT). Entering standard parameters (with α = 05, (1–β) = 0.9, two-tailed) the required sample size as a function of effect size is plotted in Figure 3. It can be seen that for a small effect size of d = 0.3 a sample size of 465 is required and for d = 0.5 a sample size of 170 is required. Notably, standard antidepressant effect sizes have been found to be ~0.3, while an effect size of 0.5 is a moderate size. While some PAT trials have shown large effect sizes, these are likely over-estimates due to functional unblinding¹² and given the preceding logic, most PAT trials are probably only realistically powered to detect ATE estimates biased by functional unblinding. As such, almost all PAT trials, especially investigator-initiated trials, are likely to be underpowered in terms of having a singular focus on determining statistical significance for efficacy outcomes.

Should we abandon PAT clinical trials then? No. Increased emphasis on non-efficacy/effectiveness outcomes, descriptive statistics and qualitative outcomes would allow more numerous, smaller trials to more rapidly determine what works. Notably, qualitative research can often achieve data saturation (sufficient data to draw conclusions) at much smaller sample sizes (e.g. 9–17 participants²⁰). Such approaches would allow treatment protocols to be adjusted during a single study (e.g. in an umbrella protocol) or across multiple studies to allow faster optimisation of PAT factors. While overall evaluation of the efficacy of PAT may be a desirable long-term goal, it may be premature given the many unanswered questions regarding how complex PAT interventions might be implemented.

One suggested design to potentially decouple drug from therapy variables that we and others have proposed is the 2 × 2 design.^12,21,22 However, statistical considerations here are even more intimidating. Using power calculations like those presented above to detect a small interaction effect (d = 0.3) would require a total sample size of n = 922 and for a moderate effect size n = 338. From a pragmatic perspective, such samples are simply not realistic, and so this design should probably not be considered feasible.

In a landmark paper, Schwartz and Lellouch²³ distinguished between pragmatic and explanatory approaches to clinical trials, where the explanatory approach is aimed at understanding whether a difference exists between two treatments, often in idealised circumstances, while the pragmatic approach seeks to compare treatments under the conditions in which they will be applied in clinical practice. They give an example of testing the effect of an anti-cancer drug to be delivered for 30 days to sensitise the patient to subsequent radiotherapy. In an explanatory trial, the control group would wait (or be given a placebo) for 30 days and then be compared with the treatment group. However, in a pragmatic real-world trial, the control group would be given radiotherapy immediately, like they would in real-life practice. The explanatory trial seeks to understand whether the anti-cancer drug 'works' while the pragmatic approach determines how much difference it will make in real-life treatment. The pragmatic trial thus answers the criticism of clinicians that the research establishment often fails to provide answers to relevant clinical questions.²⁴ Based on these approaches, policymakers and trialists distinguish between the efficacy and effectiveness of an intervention. Explanatory trials aim to estimate efficacy – whether the intervention works under idealised circumstances while pragmatic trials aim to determine effectiveness – the degree of benefit in more real-world clinical settings.²⁵ That said, the definitions of efficacy versus effectiveness are not yet universally agreed upon. Most notably, the Food and Drug Administration (FDA) in their guidance documents for evaluating the evidence regarding drug products in general ²⁶ and for psychedelic drug development specifically²⁷ use the terms efficacy and effectiveness interchangeably (see also Schenberg¹⁵). Indeed, seen through the lens of pragmatic/explanatory trials, the FDA Psychopharmacologic Drugs Advisory Committee²⁸ when asked to evaluate the evidence of effectiveness (not efficacy) of MDMA-AT were in fact evaluating its efficacy as the MDMA-AT phase III trials^18,19 evaluated tested the efficacy of MDMA-AT (not its effectiveness).

It is important to bear in mind that pragmatic and explanatory trials are not a dichotomy but sit on a continuum.²⁹ The PRECIS-2 (PRagmatic Explanatory Continuum Indicator Summary) tool⁴ which supersedes the PRECIS tool³⁰ provides a formal framework for trial designers to assess how pragmatic or explanatory their trial design is across nine domains (see Figure 4), providing a method to quantify where trials exist on the explanatory/pragmatic continuum. These domains are essential for evaluating the applicability of trial results to real-world settings. The tool focuses on ensuring that the trial design aligns with the intended use of the intervention, making trials fit for application to routine clinical practice. Each PRECIS-2 domain can be scored using a Likert scale from one (very explanatory) to five (very pragmatic). For a given trial, the sum score of the domains gives an overall indication of how pragmatic the trial is (ranging from 9 to 45). While a previous perspective piece has called for increasing use of pragmatic trials for the development of psychedelic medicine³¹ detailed consideration for how this might be instantiated was not elucidated. Below, we consider each of the PRECIS-2 domains and propose variables that might influence the evaluation of each domain with respect to PAT. We then apply these criteria to existing PAT trials to demonstrate that the focus of previous trials has been largely explanatory, reflecting the current evidentiary requirements of drug regulators for medicines to be approvable.

In this section, three authors (SM, AF, TN) applied the PRECIS-2 criteria to a selection of existing PAT trials. This account is not intended to be exhaustive, and we focus on some of the major research trials and indications that have been published in PAT to date. In particular, we include one of the phase III Lykos trials of MDMA-AT for PTSD,¹⁸ phase II RCTs of psilocybin treatment for depression ^40–42 and a phase II trial of Lysergic acid diethylamide (LSD) for anxiety disorders.⁴³ The results of these are displayed in Figure 5. As can be seen, the PRECIS-2 scores for psychedelic RCTs are low, typically around 15–16, which is heavily skewed towards the explanatory side of the explanatory–pragmatic continuum. One counterargument is that, to some extent, existing PAT trials are in fact more pragmatic than how they are being presented. For example, trial publications often present schematics of the 'ideal' trial for participants – specifying the exact number of preparation/treatment/integration sessions. The degree to which trialists actually deviate from these schematics is not clear and almost never reported and indeed we (SM, AF) have previously called for greater transparency of reporting regarding what actually happens in PAT trials – this should include transparent reporting of the number and extent of protocol deviations.¹² A second possible counterargument is that it is simply not possible at this stage to develop a PAT clinical trial that is pragmatic. However, in our (SM, TN, AF) experience, this is not correct. In a phase II PAT protocol that we are developing, our PRECIS-2 self-evaluation for the trial achieved a score of 37, which is heavily skewed towards the pragmatic side of the continuum. We note that in most pharmaceutical development, new drugs are under the control of pharmaceutical companies and investigators cannot begin truly independent pragmatic trials until regulatory approval for the drugs is obtained and pragmatic Phase IV trials can then begin.⁴⁴ For PAT however, the major drugs of interest (e.g. psilocybin, MDMA, LSD) are available to Good Manufacturing Practise (GMP) standard from generic manufacturers, allowing pragmatic trials to be used earlier in the PAT development process. This is important as the purpose of pre-authorisation explanatory drug trials is to achieve product authorisation – not to provide practical measures of effectiveness, nor any understanding of potential therapeutic mechanism or underlying process of therapeutic change.⁴⁴

Pragmatic trials clearly have limitations on their interpretation. For example, a treatment assessed pragmatically in one region does not guarantee its transferability to another location/setting,⁴⁵ thus they are also vulnerable to the extrapolation fallacy. Moreover, it is not guaranteed that increasing study heterogeneity within a study necessarily increases the external validity of the study.⁴⁵ Pragmatic trials also have the limitation of generally not being able to identify placebo effects from treatment effects, and when results are non-significant, there is a much broader parameter space that could potentially explain null effects (for both efficacy and safety data). Because of this, pragmatic trials alone are not able to distinguish drugs with a causal role in a therapeutic effect from 'snake oil'. As such, it is critically important to view pragmatic and explanatory trials as complementary evidence-gathering tools. That said, the safety data from pragmatic trials is a much more realistic representation of real-world safety data and more likely to pick up adverse events, particularly given the more heterogeneous samples that are used. At the same time, from a health improvement perspective, a pragmatic trial demonstrating safety and effectiveness provides strong evidence for its own continuation, adjustment and even expansion as a treatment option that is already locally embedded.

One of the key elements of the complex intervention approach is to stress the importance of having a theory of change as to how an intervention works, what the key components of the intervention are and how these components might interact.³ In this regard, one could argue that PAT has neglected exploring the theoretical basis and necessary elements of PAT at the expense of efficacy testing. Most notably, a recent split emerged in the psychedelic literature between those therapeutic approaches that view the psychotherapeutic aspects of PAT as integral to the therapeutic process^46–48 as opposed to those that view only minimal 'psychological support' as required to ensure patient safety⁴⁹ (perhaps PAT vs psychedelic therapy (PT)). Gründer et al.⁵⁰ point out that in practice this distinction is fuzzy as even in PT frameworks, the essential psychological support provided is a form of psychotherapy. Remarkably, while the largest study to date testing psilocybin for depression referred to the treatment as 'psilocybin administered with psychological support'⁴¹ a preceding publication outlining the same industry-led study's therapist training programme describes 'psychological support' as a complex and structured set of psychological interventions, including highly process-directive psychotherapeutic techniques provided by experienced mental health professionals throughout preparatory, dosing and integration sessions.⁵¹ Although the authors avoid referring to their 'psychological support' model as psychotherapy, they explicitly acknowledge that it integrates psychotherapeutic models such as cognitive behavioural therapy, acceptance and commitment therapy and mindfulness-based therapy. One might then argue that the attempt to distinguish PT from PAT does not primarily reflect distinct theoretical assumptions about how psychedelics work. Rather, considering that regulatory bodies such as the FDA do not regulate psychotherapy, industry players' ongoing efforts to portray psychedelic therapy as anything but psychotherapy can be understood as a strategy to obtain regulatory approval for specific psychedelic drugs to be marketed.^52,53

While the distinction between PAT and PT may be fuzzy and is perhaps more often made for strategic rather than theoretical, practical or clinical reasons, it can, at least in principle, emerge from genuinely distinct philosophical positions on how psychedelics might work therapeutically and the underlying theories of change. The traditional DB-RCT framework is not at all concerned with the theory of how an intervention works, and all too often, trialists do not clearly specify what the core elements of their therapeutic approach are. However, we argue the philosophical position on PAT/PT greatly impacts how trials ought to be conceptualised and designed. To the theorist who hypothesises that most of the treatment's efficacy is attributable to the drug alone, while adjacent psychological support is a mere safety measure that does not contribute to efficacy (PT), traditional drug development designs seem most appropriate – for example, parallel groups RCTs with placebo, active placebo and dose–response control conditions. It behoves such theoretical approaches to grapple with issues such as expectancy and functional unblinding (see Aday et al.⁵⁴ and Muthukumaraswamy et al.,¹² for detailed consideration and trial design suggestions). However, even with psychological support models that are more clearly differentiated from psychotherapy than has been the case in studies to date – for example, by employing 'psychological supporters' who are not in fact trained psychotherapists – such designs cannot empirically test the fundamental theoretical assumption that psychological support (or its interaction with drug effects) is unimportant to treatment efficacy. To effectively examine this assumption, study designs more commonly used in psychotherapy research would be required. In this sense, critically examining the role of psychotherapy in psychedelic therapy, Goodwin⁵⁵ noted that 'it may be helpful if studies of hallucinogens are not thought of as drug studies at all, but as psychological treatment studies. (p.1201)'.

On the other hand, for the theorist who views psychedelic therapy as psychedelic-augmented psychotherapy, the central question addressed by DB-RCTs, that is, whether a drug is effective in treating a mental disorder, is inherently flawed. More aligned with the view that psychedelics are 'catalysts' of psychotherapy – a perspective widely upheld throughout the medical history of psychedelics – is the question of whether these drugs can induce or enhance psychotherapeutic processes. From this, a range of critical questions emerge concerning the optimisation of psychedelic therapies. For example, under what circumstances, and for whom, are psychedelics most likely to induce psychotherapeutically valuable experiences? What characteristics of an ongoing psychotherapy process indicate the optimal timing for scheduling a psychedelic session? Is it possible to purposefully influence the likelihood of specific sub-types of psychotherapeutically valuable psychedelic experiences by modulating drug-related factors (e.g. dosage, type of substance, or mode of administration) or contextual factors (e.g. preparatory interventions or different dosing settings)? What psychotherapeutic methods are most effective in harnessing therapeutic benefits from specific types of psychotherapeutically valuable psychedelic experiences? Although some hypothetical answers to such questions may be derived from psychotherapeutic theories and |conventional (non-psychedelic) psychotherapy research, such questions remain to be examined. It can therefore be surmised that RCTs to date have likely tested sub-optimal forms of PAT and reveal little information on how to optimise them.

Thus, for the theorist who believes that psychotherapy and its interaction with drug effects are critically important to the efficacy of psychedelic treatments, psychotherapy research designs are probably more appropriate than DB-RCTs. Schenberg⁵⁶ advanced the philosophical position that attempting to blind PAT studies is futile since these overtly consciousness-altering drugs profoundly affect the perceptions, thoughts and feelings of the patient. Furthermore, the distinction between these two positions imply two different causal claims, which further influence all stages from study planning, execution and data interpretations.¹⁵ Indeed, if the endpoint of the psychedelic administration is to create a malleable state that the psychotherapist can work with, then the closer an active placebo becomes to mimicking this state, the more it will itself promote efficacious psychotherapy. This critique of the DB-RCT methodology is similar to that raised by original psychedelic researchers Hoffer and Osmond.⁵⁷ This is a reasonable argument but, being axiomatic, difficult to falsify.

A key principle though is for investigators to centre their therapeutic theory in the clinical trial design process rather than forcing the method for evaluating it (e.g. the DB-RCT) onto how it is evaluated – see also Schenberg¹⁵ who arrives at a similar philosophical position but through different argumentation). The therapeutic/placebo theory of Grunbaum⁵⁸ modified by Howick⁵⁹ provides a useful backdrop (see Figure 6). The advance Grunbaum made was not to conceive of a treatment/placebo based on its physical characteristics, but to embed the treatment within a therapeutic theory. The therapy itself can then be divided into its characteristic features (causally relevant to the mechanism of action) and incidental features (not causally relevant to the mechanism of action, but nonetheless perhaps associated with outcome). This theoretical approach allows the placebo/treatment approach to be extended beyond the study of drugs to other interventions including psychotherapy.⁶⁰ Within the context of PAT/PT, theorists could define these features. For example, a PT theorist might define provisioning of psychological support as an incidental feature alongside music and eyeshades, whereas a PAT theorist might define these as characteristic features. Another distinction between PAT and PT that might be made is that PT can provide explanations with a simple linear causal framework (treatment to response), whereas the more complex PAT intervention can be envisioned as a dynamical (patient/drug/therapist/environment) system.

When applied to PAT, the issue arises of how to differentiate characteristic and incidental features for a particular psychotherapeutic modality – specifically where 'common factors' should be placed. Common factors are those factors shared by many types of psychotherapy such as goal consensus and collaboration, therapeutic alliance, empathy, positive affirmation, congruence/genuineness and expectation amongst others.⁶² More process-oriented conceptualisations of common factors such as Grawe's 'general change mechanisms'^63,64 also specify distinguishable types of therapeutic experiences that are shared by all effective psychotherapies. These factors are just as present in PAT as in other psychotherapies and presumably contribute to clinical response.^65,66 Regarding the placement of common factors in the Grunbaum model, some authors have categorized them as incidental.⁶⁷ Others have pointed out that 'common factors' should be considered characteristic features since they are commonly seen as integral treatment components in psychotherapeutic theories.⁶¹ Note that common factors can potentially be manipulated in a trial context and, importantly, PAT/PT theorists can define the placement of common factors according to the therapeutic theory they propose and test.

Following this argument, the design of PAT trials would then look more like trials conducted for psychotherapy than pharmaceutical trials – with the choice of control conditions being the major difference with such trials. Gold et al.⁶⁸ provide a pragmatic and useful decision framework for selecting control conditions for randomised trials of behavioural interventions, considering factors such as participant risk, resource requirements, trial size and anticipated effect size. Briefly, for smaller phase II trials treatment-as-usual, no treatment and waitlist controls are generally more appropriate, whereas non-specific factor controls/dismantling studies and active comparator studies require larger sample sizes, typically anticipate smaller effect sizes, and are more appropriate for later phases of research.⁶⁸ There are a number of other detailed sources specifying pros and cons for various control conditions in psychotherapy research^69–72 and their effect on outcomes⁷² which the reader is referred to rather than repeating the details here. That said, it is worth bearing in mind that there has been long disagreement in the psychotherapy literature about the 'dodo bird verdict' – the hypothesis that all validated forms of psychotherapies produce similar outcomes regardless of their specific components.^62,73 As pointed out Langlitz and Gearin²² even if forms of psychotherapy are therapeutically equivalent that does not mean they are ethically equivalent or indeed appropriate for particular clients when put in the context of PAT.

Based on this, some have argued that rather than 'tilting at windmills' in psychotherapy research evaluating the efficacy of psychotherapy there is no need to control the placebo effect.⁷⁴ Kirsch et al. argue that in psychotherapy research, no-treatment or usual care is the most appropriate control condition since dismantling studies, additive RCTs, and meta-analyses generally fail to show significant effects of specific hypothesised therapeutic components. Given that some recent critiques of PAT have come from researchers working in psychotherapy^75–77 it is worth reflecting on Klein's⁷⁸ contention 'The bottom line is that if the Food and Drug Administration (FDA) was responsible for the evaluation of psychotherapy, then no current psychotherapy would be approvable (p.84)' still holds true. Quite probably since such trials are generally unblinded,⁷⁹ infrequently carefully document adverse events,^79–81 and often lack rigorous intervention standardisation and fidelity reporting^82,83 compared to pharmaceuticals (the purpose of GMP), or trial oversight standards (external Good Clinical Practice monitoring and auditing).

Related to pragmatic trials, comparative effectiveness (CE) research – trials directly comparing active treatments as per usual practise – are important for helping clinicians and policymakers to make good decisions.⁸⁴ At the later stage of complex intervention development for PAT, large scale pragmatic CE trials would be particularly helpful and provide useful complementary evidence. To reduce blinding and allegiance biases such trials could involve cluster randomisation (by study site) or randomisation to treatment prior to informed consent being given. There is a good argument for pharmaceutical companies and regulatory authorities performing CE trials as part of their phase III research programmes. If one accepts the premise that it will be difficult to conduct phase III PT trials that are not affected by functional unblinding (or desirable if one takes the PAT approach), there may remain uncertainty as to whether treatment effects can be conclusively identified from placebo effects in these trials.¹⁰ Simply conducting more trials will not resolve the identification issue. For example, the second MAPS phase III for MDMA-AT trial for PTSD¹⁹ simply repeated the trial design of the first phase III trial¹⁸ and did nothing to resolve the treatment-effect identification issue.

Looking forward, publicly available information regarding the Compass Pathways⁸⁵ and Cybin⁸⁶ phase III psilocybin for depression programmes both include one placebo-controlled and one dose–response study. However, we predict with reasonable confidence that dose–response studies will not solve the identification issue since (1) the studies may not test for functional unblinding as it is not mandated by the FDA²⁷ and (2) unblinding will presumably covary with the dose, given confounding the interpretation of results. Pragmatic phase III CE trials would provide complementary evidence to explanatory trial results. If a CE trial were to show superiority to treatment-as-usual, then this would be strong evidence in favour of PAT. However, even demonstration of non-inferiority or equivalence provides some useful evidence, as at the level of an individual patient, approval of PAT would at least expand patient choice. Importantly, pharmacoeconomic evaluation of treatments provides more realistic estimates when based on effectiveness data and reduces the need for complex and potentially unrealistic economic modelling.⁸⁷ For example, recent economic modelling estimates of MDMA-AT for PTSD^37,88 are based on efficacy and not real-world effectiveness data, leading to great uncertainty in the validity of the estimates. Indeed, it has been proposed that as interventions get more complex, it is more likely that efficacy estimates will tend to be more favourable to the intervention than effectiveness estimates.⁸⁹

More generally, it is important to emphasise that, although strengths and weaknesses of explanatory and pragmatic trials have been known for a long time (e.g. Feinstein⁹⁰), industry regulators have become reliant almost exclusively on data from RCTs, largely dismissing other data sources and experimental designs. For pragmatic trial data to be really useful, policymakers and regulators should engage with recent advancements in the philosophy of evidence in medicine, including but not limited to proposals such as the updating of Evidence-Based Medicine (EBM) to EBM+,⁹¹ which arguably will be necessary to adequately assess evidence from a diversity of study designs with psychedelic compounds.^14,15

The choice of methodologies used and questions asked in biomedical research tend to be constrained by the requirements and norms of regulatory and ethical review bodies, investigator training, funding bodies and commercial incentive structures. We end with a structural analysis that suggests academic/community-based researchers are well placed to lead the way in conducting pragmatic trials towards PAT development.

Psychedelic research in this millennium initially re-emerged through several small investigator-initiated trials funded primarily through philanthropic sources. More recently – and particularly since the explosion in hype around psychedelic therapies – this has become a more diverse mixture ranging from industry-led trials hoping to register psychedelics as medicines to a sprawling portfolio of smaller investigator-initiated trials funded by both philanthropic and public sources. Given the requirements for large sample sizes to power explanatory trials outlined above, we consider it unlikely that industry actors will spend significant amounts of capital to also conduct the type of research we suggest here, as they seek to conduct trials that meet extant regulatory requirements and provide returns on investment for stakeholders. Working towards regulatory approval also suggests research strategies that avoid introducing a host of new variables in the research process, which we consider part of the epistemically generative power of pragmatic trials and complex intervention development approaches. One might view a future where the type of research suggested here could reshape regulatory requirements for medicine approval, but getting to this point itself would first require expansion in the use and normalisation of pragmatic drug trials.

By contrast with industry actors, we consider academic/community-based researchers to be particularly well placed to conduct pragmatic over explanatory trials. As noted, there are no IP obstacles to them independently accessing off-patent psychedelics. In addition, academics often bring existing relationships with local community groups and actors together with diverse disciplinary and methodological approaches to the study and improvement of community health. Sometimes these even include research and evaluation approaches specifically tailored to, and co-produced with, local community groups. University-based researchers also bring intellectual curiosity beyond what can be immediately instrumentalised, including reflecting upon their research findings and complicating their theoretical frameworks. There have been many attempts to articulate the potential of the changing role of universities, including through images of the 'engaged university'⁹² the 'transformative university'⁹³ and the 'ecological university'.⁹⁴ All offer versions of community–university partnership, with universities serving broader communities. Universities are being imagined as sites for the assembling of new 'publics' (e.g. Facer⁹⁵), each made up of diverse stakeholders bound together by some key matter of concern. These visions of the university align with the mission of conducting pragmatic trials embedded within existing community contexts and responding to a particular problem, whether that is problematic drug use, or high rates of depression or PTSD, in the ways that these manifest locally.

We do not mean to paint too rosy a picture of university research. Historically, their relationships with the communities they work amidst have been fraught, and today they are shaped by neoliberal pressures in ways that can compromise meaningful community engagement efforts.⁹⁵ Psychedelic research conducted by academics must also compete for funding – often with other medical research. Medical funding bodies and reviewers might not be familiar with the pragmatic psychedelic trial designs. In today's funding landscape, these kinds of trials may be more attractive to public health- and community-development-oriented funding bodies seeking immediate community-based health improvements. A final hope of this article is to provide a philosophical position and reference work for those pursuing such funding. Indeed, the subset of larger CER trials that we envisage would require the collaboration of large networks of academics/clinicians working under the same protocol. Although, we are not aware of such efforts yet occurring in psychedelic medicine, such large investigator-led multinational and institutional trials are common in other areas of medicine (e.g. surgery/anaesthesiology).

PAT was pioneered in the 1950s by researchers Humphrey Osmond, Abram Hoffer and Alfred Hubbard as LSD psychotherapy based on the principle that the LSD experience promoted changes in attitude, perspective and behaviour that could be harnessed to facilitate positive treatment outcomes.⁹⁶ In particular, these original developers considered that LSD psychotherapy was not a drug treatment per se but a form of psychotherapy. However, since LSD psychotherapy involved drug administration it came under the purview of the FDA and required evidence of efficacy gathered through controlled (double-blind) clinical trials to be approved as a treatment.^57,97 While the DB-RCT is well suited for testing simple pharmaceutical products, the application of the DB-RCT method to PAT leads to difficulties as many of the underlying assumptions required for causal inference can be violated.^10,57 Oram⁵⁷ argues that these conceptual and technical difficulties were a major cause of the decline in PAT research in the 1960's/1970's. Pioneering LSD researcher of the time, Sidney Cohen, reflected in his book⁹⁸ 'The difficulties of doing a clear-cut study would be far from solved even with these precautions. A control group of patients matched as well as possible with the LSD patients must be given the identical treatment except that LSD is not used. . .. It is quite impossible to keep the therapist in the dark about who is getting the LSD because of its pronounced action. Will he invest as much energy and dedication to his non-LSD patients? The patients themselves will quickly know whether they have received LSD or not. . .. These difficulties and others are the reasons why a decisive test of the efficacy of LSD has not yet been performed. The problems are great but surmountable (p.199)'. In the 2020s, PAT now finds itself facing similar conceptual difficulties as it did in the 1960s with the limitations of the DB-RCT approach. However, in 70 years since PAT was developed, other methodological approaches to evidence gathering that we have outlined here have been developed. These include complex intervention development, pragmatic trials, comparative effectiveness research, biomarkers, sound clinical pharmacological principles and pre-clinical models (which add mechanistic evidence and reasoning).¹⁵ By broadening the expanse of evidence gathering for PAT to include these approaches, and in inviting academic researchers to take the lead, we are hopeful that history need not repeat itself.