Non-criminal programs to reduce thinking and behavior linked to radicalization in children and teens: A review of how well they work and are applied

Interventions working with children and adolescents span all three levels of CVE prevention (Elshimi, 2020). They include primary interventions that work to build resilience to radicalisation amongst young people more generally (Wolfowicz et al., 2022), as well as more targeted secondary and tertiary interventions. Secondary interventions work to interrupt or counter the radicalisation of young people who are considered at risk of engaging in violent extremism (Elshimi, 2020). The identification of this risk may be informed by an assessment indicating that an individual is vulnerable or susceptible to radicalisation (e.g., based on the presence of relevant risk factors), and/or is considered already on a radicalisation pathway. Tertiary interventions work with those already engaged in violent extremism (Cherney, De Rooy, et al., 2022) and work to promote deradicalisation and/or disengagement (Elshimi, 2020). Typically, those engaged in tertiary interventions actively support or have engaged in violent extremist activities (Vergani et al., 2020), including committing a relevant criminal offence, such as joining a proscribed terrorist group. These more targeted secondary and tertiary interventions include those specifically designed for youth cohorts (e.g., Barracosa & March, 2022; Hirschfield et al., 2012), as well as those that work with younger clients as part of broader caseloads (e.g., Home Office, 2023).

This review focuses solely on secondary and tertiary interventions working with at risk and radicalised young people, and excludes primary interventions. We do so for two reasons. First, due to their broader focus on building resilience, primary interventions are largely concerned with delivering outcomes whose relationship to cognitive and behavioural radicalisation is less direct (Marsden et al., 2017). Second, a comprehensive review of these interventions has already been completed by Wolfowicz et al. (2022), and directly informs this review's analytical approach.

Targeted secondary and tertiary interventions face scrutiny due to concerns about their effectiveness in countering radicalisation (Lewis, Marsden, Cherney, et al., 2024), and their impact on young people in particular (Cherney, De Rooy, et al., 2022). Key questions relate to whether and how existing secondary and tertiary interventions are able to meet the needs of children and adolescents (Cherney, De Rooy, et al., 2022) owing to those important distinctions between processes of youth and adult radicalisation discussed in Section 1.1. above.

The literature therefore recognises the importance of tailoring CVE approaches to younger cohorts, whether through developing interventions specifically for children and adolescents or engaging with younger clients in age‐appropriate ways in the context of broader interventions (Cherney, De Rooy, et al., 2022). However, existing review articles have concluded that the effectiveness of current interventions for countering youth radicalisation remains poorly understood (Cherney, De Rooy, et al., 2022; Wallner, 2021; White, 2021). This is particularly the case in the context of countering radicalisation amongst children and adolescents due to the conceptual ambiguity over terms such as ‘youth’, ‘childhood’ and ‘adolescence’, and associated challenges in identifying relevant interventions, and evidence of effectiveness. For example, because ‘youth’ is often understood as a broader stage of life that spans 15–24 years of age (Sawyer et al., 2018), even those CVE interventions that are tailored to ‘youth’ might need to be adapted further if they are to effectively work with children and younger adolescents.

Such evidence gaps are not restricted to interventions working with younger cohorts, instead reflecting broader weaknesses in the evidence base underpinning the field of CVE (Lewis, Marsden, Cherney, et al., 2024). However, specific concerns have been raised about CVE work with children and adolescents. These include the risk that, owing to their younger age, children and adolescents are more vulnerable to the types of negative psychological impacts that can result from interacting with different parts of the counterterrorism system (Hewitt et al., 2022). These concerns are particularly pronounced in the context of interventions working in and through the criminal justice system, which can be experienced as particularly distressing and stigmatising by younger people (Kessels, 2017). The co‐occurrence of mental health conditions in a number of child and adolescent cases of radicalisation raises further ethical challenges (Duits et al., 2022; Schulten, 2022) because criminal justice responses have the potential to exacerbate existing mental health issues (Sugie & Turney, 2017).

Such issues point to the important role played by CVE interventions that work with children and adolescents outside of the criminal justice system, for example, in community (e.g., Puigvert et al., 2020) or clinical (e.g., Rousseau et al., 2023) settings. The importance of delivering secondary interventions in non‐criminal justice contexts is relatively well‐established in the field of CVE owing to their focus on young people who have yet to become engaged in violent extremism, or committed a crime (Davies, 2023; Puigvert et al., 2020). Notwithstanding the potential for these interventions to produce ‘well‐intentioned harm’ through this work (Vaughn, 2019), these interventions have the potential to help divert young people away from violent extremism and the need for harder forms of intervention delivered through the criminal justice system.

Some tertiary interventions also operate outside of the criminal justice system, particularly those delivered by non‐governmental organisations (NGOs) (e.g., Christensen, 2015). However, it is common for tertiary interventions to be delivered in and through the criminal justice system – for example, in correctional or probation contexts. This is because they work with individuals who, by engaging in some form of violent extremist activity, are likely to have violated counterterrorism legislation (Cherney, De Rooy, et al., 2022). Some of these interventions also span secondary prevention by engaging with individuals convicted for non‐extremist crimes who have been identified as being at risk of radicalisation (e.g., Cherney & Belton, 2021).

The use of the criminal justice system in youth contexts raises ethical and legal issues. According to the principles of the United Nations Convention on the Rights of the Child, the arrest, conviction, and imprisonment of children must be a last resort, including in the context of terrorism offending (UNODC, 2021). The United Nations Office on Drugs and Crime (UNODC) also states that ‘[a]lternatives to prosecution and detention should normally and generally be favoured’ (p. 50) when working with children (i.e., those aged under 18 years of age) in the context of countering terrorism.

The severity of some recent cases of adolescent radicalisation (Wallner, 2021) highlights that there may be no feasible alternative to detention in some instances; because of the threat posed by the individual, or when the failure to prosecute individuals for particular offences undermines the ethical and legal obligations governments have towards victims (UNODC, 2021). However, the intent, capability, and capacity for violence can vary markedly amongst young people convicted for different types of online and offline terrorism offences (Rose & Vale, 2023). This suggests that some of these individuals may not pose a significant threat of terrorist violence and could feasibly be managed outside of the criminal justice system. As well as being less distressing for the individual, and better able to motivate them to pursue positive change (Kessels, 2017), there is evidence from the broader field of violence prevention that diverting first‐time or lower‐order youth offenders outside of the criminal justice system can be effective in countering violence (Gaffney et al., 2021).

There appear to be practical and ethical benefits in seeking to prevent and counter childhood and adolescent radicalisation through secondary and tertiary interventions delivered outside of the criminal justice system. However, the reviews cited above highlight that the evidence base underpinning such interventions remains underdeveloped. No existing reviews have yet comprehensively synthesised evidence relating to how these interventions are being implemented in practice. The growing numbers of young people who are now being engaged through secondary and tertiary interventions means that it has become increasingly important to understand whether and how these interventions work to deliver positive outcomes relevant to countering radicalisation. This review aims to address this gap by examining the effectiveness and implementation of interventions working to counter the radicalisation of those aged 19 years or under outside the criminal justice system.

Determining whether interventions operate outside of the criminal justice system can be challenging because the boundaries of this system are hard to delineate (Levin, 2023). A key challenge relates to the increasing collaboration between criminal justice agencies such as the police, and professionals working outside of the criminal justice system (Levin, 2023). This challenge is particularly acute in the context of CVE due to the pervasiveness of multi‐agency interventions that include criminal justice agencies as a partner, or that are led by criminal justice agencies (Mazerolle et al., 2021).

Previous Campbell Collaboration reviews have defined criminal justice CVE interventions on the basis that at least one criminal justice agency, such as the police, courts, or corrections is involved (e.g., Sydes et al., 2022). Whilst this definition is useful, criminal justice agencies are often important partners in interventions that work with young people before they have committed any crime (e.g., Thompson & Leroux, 2022), as well as those offered as alternatives to formal processing through the criminal justice system (e.g., Wilson et al., 2018). Adopting a similar definition for this review would therefore mean excluding a range of interventions that work to prevent children and adolescents from ever requiring a formal intervention through the criminal justice system (i.e., at the arrest, prosecution, or imprisonment stage), or which aim to divert them away from the criminal justice system altogether. This emphasis on diversion and prevention is a defining feature of a non‐criminal justice intervention.

In order to capture the broadest possible range of relevant interventions, we therefore restrict our definition of criminal justice interventions to those that are delivered by criminal justice agencies (e.g., the police, correctional, or probation staff) through the criminal justice system to individuals prosecuted and convicted of relevant offences (e.g., in correctional, custodial, or post‐release contexts). In the broadest sense, the objectives of these interventions are comparable to interventions operating outside of the criminal justice system in that both aim to prevent and counter future or ongoing (re)engagement in violent extremism. As noted above, both criminal and non‐criminal justice interventions can also span secondary and tertiary prevention, and in turn seek to tackle cognitive and/or behavioural radicalisation, although the emphasis may vary. By definition, criminal justice interventions might be expected to more commonly work with clients who exhibit more severe forms of (behavioural) radicalisation, particularly those working in post‐conviction settings. In contrast, interventions outside of the criminal justice system might be expected to more commonly work with those at an earlier stage of (cognitive) radicalisation, and may even refer individuals at later stages of radicalisation to criminal justice agencies (HM Government, 2023). However, this distinction is not absolute: some criminal justice interventions work with individuals who are at an earlier stage of radicalisation (e.g., Cherney & Belton, 2021), whilst some non‐criminal justice interventions work with clients who have been behaviourally radicalised, including those who have joined violent groups (e.g., Christensen, 2015).

Instead, the fundamental distinction between the two types of intervention is that in contrast to interventions that operate through the criminal justice system as a consequence of arrest, prosecution, or imprisonment, interventions operating outside of the criminal justice system work to prevent or divert individuals away from arrest, prosecution, and imprisonment. This review therefore defines a non‐criminal justice intervention as any intervention that works to prevent young people from entering, or divert them away from, the criminal justice system, including those that involve some form of criminal justice agency participation. Examples include: Interventions working with at risk or radicalised young people that operate completely independently of the criminal justice system, such as community‐led programmes delivered by NGOs.Interventions engaging with youth at risk of radicalisation before they have committed or have been investigated for a violent extremist or terrorism offence, including those that partner with criminal justice agencies such as the police.Interventions designed to divert individuals who have been investigated or arrested for an offence away from the criminal justice system. This type of diversionary activity might occur at the point of arrest, before charges are made, or at the point of sentencing (Gaffney et al., 2021).

This review focuses specifically on childhood and adolescence as distinct stages of life, as opposed to the broader concept of youth which, as noted above, is often defined as extending up until 24 years of age.1 This narrower focus reflects the interest in children and adolescents specified in the original brief for the project.2 More substantively, this decision recognises that childhood and adolescence are distinctive stages of development during which the needs of individuals will differ to those of a young adult (Barracosa & March, 2022). Whilst there is no single definition of childhood and adolescence, common definitions of both stages of life often overlap. For example, the UN Convention on the Rights of the Child defines children as those under 18 (UNODC, 2021), whilst the WHO defines adolescence as the period from 10 to 19 years of age (WHO, undated). Following these definitions, we focus on interventions working with those aged 19 years or under, although we recognise some authors argue for extending the definition of adolescence further (Sawyer et al., 2018).

Studies reporting on secondary or tertiary interventions that work to prevent or counter the radicalisation of at risk (secondary) or radicalised (tertiary) individuals aged 19 years and under will be eligible. This includes interventions working directly with these cohorts, and/or with family members and/or broader social networks of at risk or radicalised children and adolescents seeking to prevent further radicalisation. To be eligible for inclusion, interventions must therefore work with at least one of the below: Clients explicitly described as being at risk of radicalisation or at an early stage of radicalisation and/or who have been assessed as such according to a specified set of criteria, for example, using a particular risk assessment tool, or based on their association with individuals who have become radicalised. Clients who have been cognitively or behaviourally radicalised, as evidenced by, for example, a conviction for a relevant extremist offence. Family members or other contacts of at risk or radicalised young people aged 19 years or under, when the express focus of this engagement is to prevent or counter the radicalisation of the young person in question.

There will be no additional socio‐demographic exclusion criteria.

Eligible interventions will include those working exclusively with clients aged 19 years or under, and those that work with this cohort as part of broader caseloads. Interventions working with broader caseloads may include those that work with children, adolescents, and adults; and those that are specifically designed for ‘youth’ cohorts (or related constructs), but which use a broader definition of youth as defined above or which do not make the age range of clients explicit (in which case, we will attempt to obtain this information from study authors). We will use objective‐specific criteria to determine whether studies reporting on interventions working with broader caseloads, or non‐defined youth caseloads are eligible for inclusion in the analysis: Effectiveness: Outcome data specific to youth (or related) constructs must be presented in the study or available from the study author(s). Wherever possible, we will isolate (or request) data specific to clients aged 19 years or under. When this data is not available, outcome data specific to ‘youth’ (or related constructs) presented in the study or available from the author will still be included in the meta‐analysis (e.g., when an intervention uses a broader definition of youth, or a study of a youth intervention does not specify the age range of clients). However, we will examine the effects of including these broader samples in the meta‐analysis through the use of sensitivity analysis which is discussed below. Implementation: Studies will be eligible for inclusion when they meet at least one of two conditions: (1) The caseload of the examined intervention must include 50% or more participants aged 19 years or under; (2) Studies must report on findings specific to working with youth clients (however defined).

This review focuses on secondary and tertiary interventions working to counter the radicalisation of young people aged 19 years or under. To be included, studies must therefore report on a secondary or tertiary intervention working to counter cognitive and/or behavioural radicalisation as defined above. This includes standalone interventions that focus solely on countering violent extremism, or interventions that aim to counter violent extremism and other harms (Thompson & Leroux, 2022). As noted above, this review excludes primary interventions due to their broader focus.

A diverse range of secondary and tertiary interventions work with children and adolescents outside of the criminal justice system. Examples of eligible interventions might include, but are not limited to, multi‐agency case management interventions that tailor support to the needs of each individual client (Lewis, Marsden, Cherney, et al., 2024); mentoring programmes (Christensen, 2015); and dialogical approaches that aim to challenge violent extremist beliefs by bringing at risk and radicalised young people into dialogue with others from outside their own extremist milieu (Hussain et al., 2019; Skiple, 2020). Some interventions might adopt less direct methods, for example, by supporting parents or peers to intervene in a young person's radicalisation (Haugstvedt, 2022). A variety of practitioners may be involved in delivering these interventions in different settings, including community‐based organisations, mental healthcare professionals, social workers, mentors, and dedicated CVE practitioners (Lewis, Marsden, Cherney, et al., 2024).

Interventions working to counter radicalisation relating to any form of violent extremism will be eligible. This includes those working to counter domestic forms of violent extremism as captured in categories used by DHS and the Federal Bureau of Investigation (FBI) (i.e., Racially or Ethnically Motivated Violent Extremism; Anti‐Government or Anti‐Authority Violent Extremism; Animal Rights/Environmental Violent Extremism: Abortion‐Related Violent Extremism; All Other Domestic Terrorism Threats); as well as international forms of violent extremism, such as those associated with Foreign Terrorist Organisations as designated by the United States. 3 4

When assessing if an intervention works to counter radicalisation as defined above, we will be guided by the definitions used by the US government, as well as common definitions used in the literature, including by Vergani et al. (2020). Interventions will be assessed by examining their objectives and aims as outlined in a study (if listed), and the extent to which it explicitly aims to counter cognitive or behavioural radicalisation as defined above. Where intervention aims and objectives are not explicit, eligibility will be determined based on assessing whether the intervention works with radicalised or at‐risk youth as defined above, and whether the outcomes used to evaluate the intervention relate to cognitive or behavioural radicalisation.

There is a lack of agreement in the literature as to how best to identify individuals at risk of radicalisation, and the risk factors that might be used for this purpose (Corner & Taylor, 2022). Rather than use specific indicators of risk, we will therefore rely on the descriptions of interventions, and intervention clients, when determining whether an intervention works with young people who are at risk of radicalisation and should therefore be classed as a secondary intervention. To be defined as such, interventions must be described as a secondary intervention and/or as working with those at risk of, or at an early stage of, radicalisation as defined in Section 3.1.2 above.

The line between secondary and tertiary prevention is somewhat blurred, as the extent to which clients are considered to be at risk of radicalisation, or already radicalised, will depend on how the term radicalisation is defined. However, when defining tertiary interventions, we will use a similar approach as for secondary interventions. To be defined as such, interventions must specifically be described as tertiary interventions or as working with individuals who have been cognitively or behaviourally radicalised, as defined in Section 3.1.2 above. One potential area of complexity here relates to competing conceptualisations of interventions that work with current or former members of violent extremist groups engaged in larger‐scale militancy in conflict zones. Interventions such as Operation Safe Corridor in Nigeria and the Serendi rehabilitation centre in Somalia work with former members of foreign terrorist organisations as designated by the United States (Boko Haram and Al‐Shabaab, respectively) but are often defined as Disarmament, Demobilisation and Reintegration (DDR) programmes, as opposed to CVE interventions (Khalil et al., 2019; Richards, 2017). Notwithstanding the important conceptual distinctions between CVE and DDR, such interventions would meet our definition of tertiary interventions owing to the fact that they have specifically been described as working with ‘violent extremists’ (e.g., Khalil et al., 2019; USAID, 2021), indicating cognitive and/or behavioural radicalisation. Interventions delivered in the context of larger‐scale militancy (i.e., DDR interventions) will therefore be included in the review (when meeting our other criteria) when it can be determined that they work with relevant clients as defined above.

Only interventions delivered outside of the criminal justice system will be eligible for inclusion. As noted above, criminal justice interventions are defined as those interventions that are delivered in correctional, custodial, or post‐release contexts. Any intervention delivered outside of these contexts – for example, in community, clinical, educational, or social work settings – will be included provided that they meet one of the below criteria: 1 Work with individuals before they formally come into contact with the criminal justice system (i.e., before any criminal investigation or arrest). 2 Provided as an alternative to arrest, charge, or imprisonment to individuals who are/have previously been subject to a criminal investigation or arrest. 3 Work with individuals who have previously been or are subject to a criminal investigation, arrest, prosecution, and/or sentence but which operate independently from criminal justice agencies, or the criminal justice system.

Interventions covered by points 1 and 2 may see some involvement by criminal justice agencies, for example, where the police, or multi‐agency panels involving criminal justice agencies, refer individuals into programmes (e.g., Thompson & Leroux, 2022). Because the role criminal justice agencies play within these processes can vary, we will only exclude interventions when a criminal justice agency plays a role in working directly with clients. No other restrictions will be placed on delivery agents, provided that interventions are delivered outside of, or aim to divert individuals from deeper engagement in, the criminal justice system (Wilson et al., 2018).

Eligible peer‐reviewed and non‐peer‐reviewed/grey literature studies will be included. Protocols and trial registries for eligible ongoing studies for which no published outputs can be identified will be included in a separate ‘References to ongoing studies’. However, should they link to a finalised study, such references will be classed as a dependent report of that study.

There are no geographical exclusion criteria.

Research published in English will be included.

Studies published after January 2000 will be included. Whilst some relevant interventions have a longer history, the policy framework of countering violent extremism only really emerged at the start of the 21st century (Neumann, 2013).

An information specialist (DS) developed a comprehensive core search strategy comprised of subject headings and keyword search terms in Ovid PsycINFO in consultation with the primary authors (JL/SM) via an iterative process through which the search terms from an earlier review (Lewis, Marsden, Cherney, et al., 2024) were adapted based on reviewing examples of studies eligible for inclusion in this review; reorienting the search terms to focus more explicitly on interventions for young people aged 19 years or under; and a series of stages of piloting. We present the full strategy for PsycINFO in Table 3. The information specialist will model strategies for the remaining databases (Table 4) on the strategy developed for PsycINFO, using appropriate syntax for each database and platform. Search fields for all databases will include, as a minimum, title, abstract, author supplied keywords, and controlled vocabulary indexing terms.

A number of supplementary sources will be searched by hand, including core journals on terrorism and violent extremism and topics relating to childhood and adolescence (Table 5); grey literature resources (Table 6); and clinical trial repositories (Table 7). When searching journals, all articles published since 2000 (including online first articles) will be reviewed. When searching other websites, the approach used will be tailored to the functionality of the platform, for example, whether it categorises outputs into topics easily identifiable as relevant/irrelevant or allows for detailed searches.

Extrapolating from previous reviews of CVE interventions (e.g., Lewis, Marsden, Cherney, et al., 2024), it is unlikely that this review will identify any eligible studies using experimental research designs given known challenges facing efforts to deploy these designs in the field of CVE. These include practical challenges relating to small sample sizes, and the potential ethical and legal implications that might result from withholding an intervention for the purposes of evaluating it (Lewis et al., 2020).

Previous Campbell Collaboration reviews of related interventions have highlighted that the use of quasi‐experimental designs remains limited (e.g., Lewis, Marsden, Cherney, et al., 2024; Mazerolle et al., 2021). A small number of authors have used quasi‐experimental research designs to evaluate CVE interventions (Mazerolle et al., 2021), which suggests that this review might identify quasi‐experimental evaluations of relevant interventions. However, a preliminary review of the literature suggests that only a very small number, if any, are likely to meet our definition of a ‘strong’ quasi‐experimental design outlined above. It is therefore likely that a significant proportion of the included studies will be other types of quasi‐experimental or non‐experimental quantitative studies (e.g., cross‐sectional surveys; observational research), and qualitative studies.

One potentially eligible quantitative study comes from Kolbe (2019). This study used secondary data (programme records) collected at baseline, intervention completion (3 months after baseline), and 9 months post‐intervention to examine whether the addition of a home‐based social work component to a group‐based ‘after school enrichment’ programme working with young people in an East African city (aged 13–18 years) explicitly identified as being ‘at risk’ of joining Al‐Shabaab improved levels of programme completion and decreased the likelihood of joining a violent extremist group. The evaluation was made possible by a naturally occurring experiment as the social work component was only introduced after the first cohort had completed the programme and was removed following funding cuts for a fourth and final cohort. Whilst this is one of the few studies identified to date using a control group, analysis of programme documentation is a feature of other quantitative and mixed methods studies of interventions working with at risk or radicalised young people (e.g., Cherney & Belton, 2021). Quantitative analysis of primary data collected from at risk and/or radicalised clients, as well as practitioners and other stakeholders (e.g., through surveys) may also be used to examine the implementation of relevant programmes, for example, when capturing feedback on an intervention or component of an intervention (see Lewis, Marsden, Cherney, et al., 2024). However, this type of data is more commonly captured using qualitative methods, such as focus groups and interviews with clients, practitioners and stakeholders, or observation of practice (see Lewis, Marsden, Cherney, et al., 2024).

Studies will be screened using a four‐step process: 1All studies identified through the search strategy above will be imported into EndNote. This initial corpus will be screened for ineligible (e.g., book reviews) and duplicate records before being uploaded into the online systematic review platform Covidence.6 Duplicate records will be identified and removed through a) the duplicate detection function included in EndNote and b) a review of all remaining records completed by one trained reviewer.2The Covidence platform will automatically screen the uploaded records for any remaining duplicates, and one trained reviewer will review any identified duplicates to ensure that only genuine duplicates are excluded. All remaining records will progress to title and abstract screening.3The lead author (JL) will train two reviewers to use the title/abstract screening tool shown in Appendix SI. The trained reviewers will use Covidence to review the first 50 records (sorted on relevance), and their inclusion decisions will be compared to the decisions made by JL to ensure that both reviewers are applying the inclusion criteria correctly and consistently. An online meeting will be held to discuss disagreements, and, if necessary, the exercise repeated.4Both reviewers will then screen the remaining abstracts (i.e., abstracts will be screened in duplicate) using this same tool. JL will review 5% of studies excluded by each reviewer to identify any false negatives upon completion of this process, following previous Campbell Collaboration reviews (e.g., Mazerolle, Cherney, et al., 2020). In the event of a high number of false negatives, JL will manually review all records excluded by that reviewer. Once title/abstract screening is completed, all reviewers will meet to discuss any conflicts. In the event that these conflicts cannot be resolved, SM will make a final decision.5The same two reviewers will conduct a full text screening of remaining studies in Covidence, guided by a full text screening tool shown in Appendix SI. The same approach outlined above for piloting the screening tool, reviewing exclusion decisions, and resolving any conflicts between reviewers at the title and abstract screening stage will be used.

Data extraction will be guided by the data extraction tool presented in Appendix. This tool contains a series of domains to capture bibliographic information about the study (e.g., author(s); title; year and place of publication); details about the intervention being examined (e.g., secondary or tertiary intervention; nature of client base; delivery context; types of practitioner, etc.); methodological information (e.g., type and source of data; research design; sample details, including e.g., whether it includes clients or practitioners; approach to analysis; outcome measures used); data relating to effectiveness and/or implementation (including authors' own conclusions); funding sources and declarations of interest; and the risk of bias as assessed by the reviewers. SIII

All data will be extracted and managed in Microsoft Excel following the data extraction template included as an Appendix. When extracting effect size data relating to effectiveness (Objective 1), effect size data will be independently double‐coded to ensure the accuracy of data inputted into any meta‐analysis.

The conventions for analysing this data are outlined in the sections below.

The risk of bias of studies reporting evidence of effectiveness (Objective 1) will be assessed using either Version 2 of the Cochrane risk‐of‐bias tool for randomised trials (RoB 2) or the Cochrane Risk of Bias in Non‐Randomised Studies – of Interventions (ROBINS‐I) tool (Sterne et al., 2016, 2019). Both tools are organised around a series of domains containing ‘signalling’ questions that will be used to assess the risk of bias present at each domain, and in turn the overall risk of bias (Table 9).

Criteria for determining a study's overall risk of bias are shown in Tables 10 and 11.

Each study will be assessed against the individual domains of the relevant tool shown in Table 9, with these ratings informing an overall risk of bias assessment for each study following the definitions shown in Tables 10 and 11. Assessments for each domain and for a study's overall risk of bias will be presented in summary tables, and sensitivity analysis will be conducted (if possible) to understand the effects of any identified bias.

The approach for considering risk of bias within the meta‐analysis will be determined by the number of eligible studies included, and the identified variation in their risk of bias. Following Mazerolle, Cherney, et al. (2020), we may therefore ‘stratify’ the analysis according to different levels of risk or present a singular analysis alongside a ‘narrative discussion of the risk of bias’ (p. 8). The approach taken will be determined by the statistical lead (AS), in consultation with the rest of the research team.

The risk of bias of other types of quantitative (e.g., non‐experimental) studies reporting on implementation (Objectives 2 and 3) will be assessed using the Effective Public Health Practice Project (EPHPP) Quality Assessment Tool for Quantitative Studies, which contains a number of questions relating to selection bias; study design; confounders; blinding; data collection methods; and withdrawals and dropouts.7 Qualitative studies, including qualitative components of mixed‐methods studies, will be assessed against a widely used checklist developed by the Critical Appraisal Skills Programme (CASP), which is organised around ten ‘Yes’, ‘No’, or ‘Can't tell’ questions relating to study quality.8 Following the conventions set out in Lewis et al. (2023) and Mazerolle, Cherney, et al. (2020), studies for which the answer to the following questions are ‘No’ or ‘Can't Tell’ will be excluded from the review: –Was the research design appropriate to address the aims of the research?–Was the recruitment strategy appropriate to the aims of the research?

The results of assessments completed using the CASP and EPHPP tools will be presented in summary tables, and an overall summary of the quality of evidence will be included within the analysis of Objectives 2 and 3.

Given the focus of the current review on providing evidence for the effectiveness of interventions, only designs that allow for computation of different outcomes between treatments and between treatment and control conditions will be included in the analyses (i.e., RCTs; Cross‐over design; Matched control group design; Unmatched control group designs where the control group has face validity; Unmatched control group designs allowing difference‐in‐difference analysis; Time‐series designs).

For ease of interpretation all effects will be coded such that higher scores would indicate a more positive result (i.e., decrease in support for violent extremism). Some evaluations will likely use continuous measures to report on relevant primary and secondary outcomes (e.g., responses to questions using Likert scales). We will calculate Hedges' g (standardised mean differences) for each continuous measure. In the event that evaluations also report on binary outcomes (e.g., whether or not an individual has re‐offended post‐intervention), effect sizes will be computed as odds ratios, and transformed into Hedges' g for the meta‐analysis (Borenstein et al., 2021).

Hedge's g will be calculated using the formulas described in Borenstein et al. (2021): g=J×d,Vg=J2×Vd;SE=Vg,where: J=1−34df−1.

Similarly, odds ratios will be transformed into standardised mean difference scores (d) and then into Hedge's g by using the formulas described in Borenstein et al. (2021): d=Logoddsratio×3π.

If other inferential tests are reported that allow for transformations to find our desired effects (e.g., F, t), these transformations will be applied in line with Borenstein et al. (2021).

It is expected that some unit‐of‐analysis issues will be detected in the data. These will be evaluated on a case‐by‐case basis and suitable methods will be used to ensure the effect sizes that are used in the meta‐analysis are properly calculated. The approaches for addressing issues considered most likely to appear in relevant studies – clustering; multiple intervention groups; and multiple follow‐up periods – are discussed below.

If studies report results for designs with clustering among participants (e.g., assignment of school classes, participants from the same location, or otherwise grouped individuals to treatments), we will adjust for this by inflating standard errors (as described in Higgins et al., 2023, section 23.1.5). Specifically, we will calculate ‘design effect’ using the methods described in Higgins et al. (2023) and use it to adjust the standard error of the effect size estimates. All adjustments will be clearly described in the report for full transparency.

Where a study reports more than one treatment group (i.e., where multiple groups are compared with a single control group), our primary approach would be to avoid double‐counting by splitting the sample size of the shared group (Harrer et al., 2021; Higgins et al., 2019). However, to provide a sensitivity test we will also combine the treatment groups to create a single comparison (i.e., combined treatments vs. control).

If studies produce results on one or more follow‐up periods, our main comparison will be between the baseline and the most immediate follow‐up. However, for exploratory purposes, we will also calculate the effects for the difference between the baseline and the last follow‐up for each study that reports more than one follow‐up to gauge whether the effects hold over time (Higgins et al., 2023, section 6.2.4).

The review is likely to identify multiple records reporting on the same empirical findings (i.e., the same research study). Any overlap between records will be determined based on an assessment of whether there is any identifiable overlap in authors; research dates and research design; intervention; or references to other included records. In such cases, we will extract and code data from all records meeting our inclusion criteria, but only count individual studies once.

Where two or more overlapping records using an eligible research design present relevant outcome data drawn from the same study, only the record presenting the most complete data will be included in any quantitative analysis. This includes any meta‐analysis or alternative synthesis (see Section). Qualitative data synthesis will include data drawn from all overlapping records, but individual studies will only be included once in presented counts, to avoid overstating the strength of the evidence. 3.3.11

The potential issue of dependency in the data may arise in the likely event that studies report on multiple relevant outcome measures (e.g., studies may present data for two or more outcomes capturing a reduction in support for violent extremism). To address this, we will employ robust variance estimation procedures (Hedges et al., 2010; Pustejovsky & Tipton, 2022; Tipton & Pustejovsky, 2015) that take into account the dependency stemming from collecting multiple measures in the same study (i.e., in an RVE analysis we will include individual measures, not combined effects).

If identified and included studies have missing data, we will take the following actions to address this issue: 1 Search any supplemental materials that may be provided with the study. 2 Reach out to the author(s) to request the missing information (either in the form of providing the missing statistics or the data set from the reported study). 3 Search for other outputs by the author(s) that use the same data.

Studies will be excluded from the meta‐analysis if we are unable to obtain the required information from the authors. However, these studies will still be included in study summaries and in the qualitative assessment, where they present relevant data. When analysing data drawn from studies where a correlation matrix is not available or where not enough information is provided to calculate Hedges g, we will calculate the effect sizes for outcomes for which descriptive statistics are sufficiently detailed following Lipsey and Wilson (2001), using the ‘Practical Meta‐Analysis Effect Size Calculator’.

We will assess heterogeneity using both Cochran's Q and the I² statistic (Higgins, 2003) as well as the Tau (Higgins et al., 2023).

We will assess publication bias using the following methods: 1We will use funnel plots with the Egger's Regression Test (Egger et al., 1997) to identify asymmetry between published and non‐published studies and, if possible, examine identified asymmetry through sub‐group analysis.2The trim and fill method (Duval & Tweedie, 2000) which makes it possible to check the robustness of the initial findings against publication bias, unless there is high heterogeneity among studies as this can produce unreliable results.3If appropriate – that is, if the number of identified studies is greater than 20 and heterogeneity is less than 80% – we will also use the PET‐PEESE method to calculate the precision‐adjusted effect estimate (Stanley, 2017).4In the case that heterogeneity of the identified studies is small‐to‐moderate (I² < 50%), we will use the p‐curve analysis. This analysis makes it possible to estimate the true effect size and verify whether there is evidence that the intervention effect differs from zero, given the p‐values that were obtained in the studies (for heterogeneity >50% such analyses may be performed in subgroups).

Previous reviews of youth CVE interventions highlight the diversity of interventions, and associated implementation factors and moderators, and outcome measures (e.g., White, 2021). Reflecting this diversity, we intend to compare the effects of different interventions and the impact of moderator variables in the analyses by utilising meta‐regression (Borenstein et al., 2021; Harrer et al., 2021). We will use the subgroup correlated effects model (described by Pustejovsky & Tipton, 2022) to conduct moderator analysis for each conceptually similar outcome when an appropriate (ideally at least 10) number of studies for each outcome is identified. For smaller numbers any effects identified could be spurious. Should there be enough studies (i.e., at least 10 that provide us with variables of interest) to warrant such analyses we will assess the impact of the following characteristics of the studies on the relevant effect sizes:

Methodological factors 1 Intervention type (decided based on interventions identified through the review). 2 Type of comparison group (e.g., empty control vs. active control). 3 Methodology (comparing effects found in RCTs vs. quasi‐experimental designs).

Contextual factors 1 Intervention type (decided based on interventions identified through the review). 2 Form of prevention (i.e., secondary vs. tertiary interventions). 3 Intervention length. 4 Location. 5 Delivery context (e.g., clinical, community, etc.). 6 Delivery agents (e.g., clinicians, NGOs, social workers, etc.). 7 Client demographics (e.g., ages, type of extremist ideology, etc.).

The narrative synthesis will also consider differences between findings drawn from studies working with different types of samples as defined in the participant characteristics section, that is, whether the findings specific to working with clients aged 19 years of age or under differ from those drawn from studies examining broader or non‐defined samples of youth.

In the event that the meta‐analysis contains outcome data drawn from studies reporting on different types of sample (e.g., youth aged 19 years of age or under; youth aged over 19 years of age; youth samples that are not clearly defined) we will conduct sensitivity analysis (i.e., a leave‐one‐out analysis) to examine the impact of including studies reporting on older or non‐defined youth samples on the effect estimates.

Sensitivity analysis will also be performed to examine other differences in study design. For example, we will compare effects found in experimental and quasi‐experimental designs by performing a leave‐one‐out meta‐analysis to assess whether any studies have an undue effect on the findings.

Qualitative research will be included when assessing implementation and associated implementation factors and moderators (Objectives 2 and 3) and will be included in the framework synthesis as outlined above. Qualitative data will not be examined in the analysis of effectiveness and will therefore not be included in any quantitative synthesis (i.e., data will not be converted into quantitative metrics).

We do not plan to include Summary of findings and assessment of the certainty of the evidence.