School‐based interventions for reducing disciplinary school exclusion: a systematic review

Discipline problems are frequent in schools and they may have a harmful effect on pupils’ learning outcomes. A lack of discipline and the subsequent potential increase in school disorder (e.g. bullying, substance misuse) can seriously threaten the quality of instruction that teachers provide, hamper pupils acquisition of academic skills and subsequently reduce their attachment to the education system (Gottfredson, Cook, & Na, 2012).

As such, discipline represents a serious concern for parents and teachers, demanding significant efforts and resources from schools (Kaplan, Gheen, & Midgley, 2002). The PISA 2009 report (OECD, 2010) stated that schools registering higher levels of disciplinary problems result in teachers spending less time on learning in order to deal with such issues. In its 2012 version, the PISA report asked students about school discipline. Results found that “28% of students reported that teachers had to wait a long time to quiet down every class, or almost all classes”(OECD, 2013). Being more precise, the Teaching and Learning International Survey (TALIS) revealed that teachers spend at least 20% of lesson time dealing with disruption and administrative tasks. In the United Kingdom, the Behaviour Survey 2010 states that 80% of school teachers felt their ability to teach effectively was impaired by students’ poor behaviour (Massey, 2011). On a global level, evidence suggests that 13% of teachers’ time is, on average, spent maintaining order (OECD, 2009).

Schools use different procedures to manage discipline, including a range of punitive responses (e.g., loss of privileges, additional homework or detention). Among these, exclusion is normally seen as one of the most serious punishments. Although types and lengths vary from country to country, school exclusion (also known as school suspension in the United States)1 can be broadly defined as a disciplinary sanction imposed in reaction to students’ behaviour (i.e. violations of school policies) by the responsible authority. In concrete terms, exclusion entails a removal from regular teaching for a period of time during which students are also not allowed to be present on school premises. Specifically, fixed‐term exclusions consist of a limited number of hours or days (Cornell, Gregory, & Fan, 2011), whereas permanent exclusion (i.e., expulsion) involves the pupil being transferred to a different school, or educated outside of the regular education system (Spink, 2011; Webb & Vulliamy, 2004).

Even if school policies suggest that exclusion should be used as a last resort, reserved for only the most serious and persistent offences (Gregory & Weinstein, 2008a; Skiba & Peterson, 1999; Skiba, Trachok, Chung, Baker, & Hughes, 2012), research evidence suggests that minor offences can also provoke this type of punishment (Munn, Cullen, Johnstone, & Lloyd, 2001; Skiba, 2014). Fenning et al., (2012) provide a case in point: their research concluded that suspension and expulsion were the most common types of punishment for minor problems such astardiness and school truancy. These findings were also confirmed by Liu (2013) who found that 48% of suspensions lasting a maximum of five days targeted minor disorder or disruptive behaviours.

In terms of prevalence, data provided by the UK Department for Education (academic year 2011/12) shows that in England fixed‐term exclusion affects 3.5% of the school population whereas permanent exclusion applies to only 0.06%. The national figures suggest that students in secondary‐level education (6.8% of the school population) as well as those in special education (14.7%) are the most likely to experience fixed‐term exclusion (DfE, 2013). In the United States, data provided by the Department of Education (academic year 2011/12) concluded that 7.4% (3.5 million) of students were suspended in school, 7% (3.45 million) were suspended out‐of‐school, and less than one per cent were subject to expulsion (around 130,000 students). Black students and those presenting disabilities are, respectively, three and two times more likely to be excluded compared to White and non‐disabled pupils (U.S. Department of Education, 2012).

International comparisons of exclusion prevalence rates are not available in the literature examined. Indeed, differences in use, extent and recording (i.e., unreported exclusions) make an international estimation challenging. In Table 1, the reader will find information regarding the use of exclusion in a sample of high‐ and middle‐income countries. The information is limited to a convenience sample involving twelve different cases to allow an overview of i) the types of exclusion used in these countries, ii) the length of the sanction, iii) the authority responsible for determining this sanction, iv) the behaviours for which school exclusion is permissible, and, in cases where information was available, the table also includes v) the local prevalence of exclusion. This does not claim to be a representative sample of all countries, but as an initial approach will help provide a more complex picture of the phenomenon. In addition, this comparison was intended to help with searches for studies that could be potentially included in the systematic review. For instance, by comparing exclusion in different countries, it was found that the same school sanction had different names in different countries (e.g., “stand‐down” in New Zealand, “exclusion” in the UK and “suspension” in the US).

The comparative data incorporated in the table above suggests heterogeneity in the application of exclusion. For instance, in the US, Norway and England, educational systems distinguish between fixed and permanent exclusion. However, in some educational systems, such as Finland's, the law only permits fixed‐term exclusion. Concerning length, England limits fixed‐term exclusions to a maximum of 45 days per school year while New Zealand's legislation allows exclusions for a maximum of 10 days per year. On the other hand, it is important to note that in some countries – such as France – specific laws define and regulate exclusion, whereas in others – like Chile and Colombia – the ability to set the length of the sanction is granted to each school.

Although the data on prevalence is limited to a few countries, the percentage of in‐school exclusion seems to be larger than out‐of‐school exclusion and expulsion. In New Zealand, the data suggests that the use of exclusion is marginal particularly when compared with some areas in the US and the UK.

From a normative point of view, school exclusion is a punitive response for misbehaviour. In that sense, behavioural problems seem to be the most obvious empirical predictor for exclusion. Reinke, Herman, Petras, & Ialongo, (2008) illustrate the role of problem behaviour in exclusion by conducting a latent class analysis. Participants in the subclass of boys exhibiting behavioural problems only (i.e., isolating other academic/learning difficulties) were almost 4 times more likely to be suspended (OR = 3.42; 95%CI 1.36 to 8.58; p < .05) than their non‐problematic peers. Similarly, Pas, Bradshaw, Hershfeldt, & Leaf, (2010) found that after controlling for student, teacher, classroom, and school level covariates, the strongest predictor for out‐of‐school suspension was disruptive behaviour (OR = 4.83; 95%CI 4.10 to 5.68; p < .05).

Despite the role of behaviour in school exclusion, research suggests that it is not the sole or even the most prominent predictor. In fact, previous findings show a more complex scenario where exclusion is also strongly predicted by gender, ethnicity, age, economic background, and special educational needs (Costenbader & Markson, 1998a; Mcloughlin & Noltemeyer, 2010; Monroe, 2005; Nickerson & Spears, 2007; Noltemeyer & Mcloughlin, 2010; Skiba et al., 2011; Yudof, 1975). In the following paragraphs, we offer an overview of the role of these variables in predicting school exclusion.

Supporters of zero tolerance policies have pointed out that the use of exclusion can persuade students to account for their behaviour and lead to a decrease in rule‐breaking (Bear, 2012). However, most of the research has consistently documented the negative impact of these types of sanctions (APA Zero Tolerance Task Force, 2008; Chin, Dowdy, Jimerson, & Rime, 2012; Hemphill, Toumbourou, Herrenkohl, McMorris, & Catalano, 2006; Sharkey &Fenning, 2012). In particular, previous research suggests that school exclusion is related to serious negative outcomes in at least three dimensions of young people's development: behavioural, academic, and future social inclusion.

The prevalence of exclusion and its adverse correlated consequences have caught the attention of policy makers and programme developers. As a result, a range of interventions have been designed and implemented to improve school discipline. In the present review, we include different types of school‐based intervention aimed at reducing school exclusion as a punishment for inappropriate behaviour. These interventions include those targeting individual risk factors or school‐related factors, as well as those using a more comprehensive strategy that includes parents, teachers, school administrators, and the community.

Interventions targeting individual risk factors include, for instance, cognitive‐behavioural approaches such as anger management programmes or skills training for children (e.g., Humphrey & Brooks, 2006). Another type of intervention focusing on student behaviour – or, more precisely, students’ skills for conflict resolution – are restorative justice programmes (e.g., Schellenberg & Parks‐Savage, 2007; Shapiro, Burgoon, Welker, & Clough, 2002)In general, these interventions target motivated children and train them in practical skills to deal with anger, solve conflicts or become more assertive in social relationships. Such interventions are normally organised within a curriculum and implemented during school hours. The curriculum involves a package of group or one‐to‐one sessions using a wide range of techniques such as instruction, modelling, role‐play, feedback, and reinforcement, among others (Gottfredson, Cook, & Na, 2012; Schindler & Yoshikawa, 2012).

At the classroom level, interventions may target teachers’ abilities in classroom management (Pane, Rocco, Miller, & Salmon, 2013). The training for teachers encompasses instructional skills, such as guidelines for teaching rules and maintaining attendance, and non‐instructional skills, such as group management techniques, reinforcing positive conduct, and techniques to explain expected behaviour. Both skill sets are aimed at improving the learning process, preventing misbehaviour and encouraging positive participation by pupils (Averdijk, Eisner, Luciano, Valdebenito, & Obsuth, 2014).

Some schools offer mental health services independently or via community agencies. Experienced clinicians are located in schools in order to deliver individual, group, and/or family therapy. Clinicians may also be available for teacher consultation on matters related to students’ behavioural and emotional issues. All these interventions may target a reduction in out‐of‐school exclusion (Bruns, Moore, Stephan, Pruitt, & Weist, 2005).

Alternatively, comprehensive prevention strategies target students, families, teachers and school managers as well as the community as a whole (Bradshaw, Waasdorp, & Leaf, 2012; Flay & Allred, 2003; Colin Pritchard & Williams, 2001; Snyder et al., 2010). A well‐known comprehensive programme is the School‐Wide Positive Behavioural Interventions and Supports (SWPBIS). The programme aims to provide support for positive conduct by building proactive school‐wide disciplinary procedures (i.e., improving school climate and reducing problem behaviours). SWPBIS incorporates a multi‐level approach: from whole school prevention, to group‐based intervention for problematic pupils, and personalised, tailored interventions for high‐risk students. The basic elements of the programme are: i) building a school culture for both social and academic attainment, ii) early prevention of problem behaviours, iii) teaching social skills to all students, iv) using behaviour support practices, and v) actively using data for decision‐making. Research reports promising results, although further and stronger evaluation designs need to be undertaken (Gottfredson et al., 2012; Maag, 2012).

Our original proposal was to include both randomised controlled trials and high‐quality quasi‐experimental studies (defined as studies using a comparison group, pre‐post testing and a statistical matching approach). To be eligible for inclusion, we stated that manuscripts must clearly report the method used to ensure equivalence between treatment and control groups, taking into account major risk factors (e.g. behavioural measures) and demographic characteristics. 2014001034

In this review, we only present results from randomised controlled trials (RCTs). There were three reasons for our decision: 1First, even though a number of quasi‐experimental studies initially fulfilling our inclusion criteria were found by our searches, many of them fail to report baseline measures (e.g., Guardino, 2013; Hawkins, Catalano, Kosterman, Abbott, & Hill, 1999; Munoz, Fischetti, & Prather, 2014), the matching procedures were not described (e.g., Hasson, 2011; Risner, Vanderhaar, Muñoz, & Abati, 2005; St James‐Roberts & Singh, 2001) or the balance procedures did not produce statistical equivalence (e.g., Gao, Hallar, & Hartman, 2014).2A number of the school‐based intervention programmes included in this review presented several studies, involving quasi‐experiments as well as RCTs. Some examples involve interventions such as the Positive Action Program or the School‐Wide Positive Behavioural Interventions and Supports (e.g., Lewis et al., 2013; Snyder et al., 2010). In both cases, RCTs (e.g., Lewis et al., 2013; Snyder et al., 2010) were preceded by quasi‐experimental studies (Barrett, Bradshaw, & Lewis‐Palmer, 2008; Flay & Allred, 2003) . In this context, we decided to keep the strongest study design.3RCTs are regarded to be the most compelling methodological design to test the impact of a particular treatment. This type of study has the strengths of isolating confounding factors, reducing the likelihood of alternative explanations for observed effects (Shadish, Cook, & Campbell, 2002; Sherman, Farrington, Welsh, & Mackenzie, 2002). We believe that by selecting only these studies we will achieve a more precise final estimation of the effect of school‐based interventions.

To offer a broad overview of the research testing the impact of school‐based intervention at reducing school exclusion, a list of the quasi‐experimental studies can be provided on request.

Qualitative studies were excluded from the present review as stated in the published protocol.

The present review is focused on the general population of students in primary and secondary schools irrespective of nationality, ethnicity, language, and cultural or socio‐economical background. By targeting primary and secondary schools, participants could theoretically be aged from 4 to 18 years of age.

Reports involving students who presented special education needs, disabilities or learning problems but were educated in mainstream schools were included in this review. However, reports involving students with serious mental disabilities or those in need of special schools were excluded. The rationale for this is that the results of this review are intended to be generalisable to mainstream populations of students in non‐specialised schools from all the included countries.

Students in college or higher levels of education have been excluded. Their exclusion from the review is based on previous evidence suggesting the largest number of exclusions affect pupils aged about 10 to 15 (e.g., Liu, 2013; Raush & Skiba, 2004; DfE, 2012).

We include interventions defined as school‐based: that is, delivered on school premises, or supported by schools with at least one component implemented in the school setting. In the present review, we include interventions explicitly aimed at preventing/reducing school exclusion or those measuring exclusion as an outcome.

Interventions in the present review cover a wide range of psychosocial strategies for targeting students (e.g., Cook et al., 2014), teachers (e.g., Ialongo, Poduska, Werthamer, & Kellam, 2001), or the whole school (e.g., Bradshaw, Waasdorp, & Leaf, 2012). Types of intervention include, for example, those focused on: instructing students to identify risky behaviours and expanding their alternatives for responding appropriately to risks or harms (e.g., social skills training)developing teachers’ skills to improve the quality of their classroom management (e.g., reward schemes)cognitive‐behavioural treatment, such as anger management, counselling, social work, and mentoring programmes;school‐wide interventions.

Since there was no previous review analysing school‐based prevention programmes for reducing exclusionary discipline, we wanted to include a wide range of school‐based interventions that could be effective for reducing exclusionary practices.

We excluded studies where the intervention was not school‐based or school supported. Even though some of these interventions targeted school students, they were community programmes or purely focused on mental health issues without any connection to schools (e.g., Henderson & Green, 2014; Schwartz, Rhodes, Spencer, & Grossman, 2013; Wiggins et al., 2009).

We also excluded interventions designed for children or adolescents who have committed a crime, that is, specialised interventions aimed at reducing reoffending or reconviction. Although suspended students may commit offences, such specialised interventions were excluded from the present review because they exceed the strategies used by schools to prevent misbehaviour and their levels of complexity make them too specific for a general population of students. School‐based prevention programmes targeting outcomes related only to students’ physical health (e.g., AIDS/ HIV prevention programmes, programmes to develop healthy eating programmes) were also excluded.

Databases and journals were searched from 1980 onwards with the aim of comprising more contemporary interventions or prevention programmes. Eligible studies included both published and unpublished book chapters, journal articles, government reports, and Doctoral theses. When the same data was published in more than one source (e.g., a book chapter and a journal article) we used all the linked manuscripts but the most complete report or the report measuring suspension was defined as the main source of data (see Section 3.4.5). That way we kept as much information as possible from a specific study but avoided over‐estimation of effect sizes. In cases where it was not clear if the manuscripts referred to the same study, we contacted the main author for further information (e.g., email communication with Barnes, Bauza, & Treiber, 2003; Bradshaw et al., 2012; Lewis, Romi, & Roache, 2012; Snyder et al., 2010; Sprague, Biglan, Rusby, Gau, & Vincent, 2016).

Below we list details of the 27 electronic databases searched. As noted above, these databases included both published (e.g., ISI web of knowledge, PsycINFO) and unpublished reports (e.g., Dissertation Abstracts, EThOS) as well as reports from Latin‐American countries (e.g., Scientific Electronic Library Online – SciELO).

For each database, we ran pilot searches including the key terms described in Table 3. Four categories of key words were used, including: i) type of study; ii) type of intervention; iii) population; and iv) outcomes. The pilot searches were useful to adjust the terms, synonyms and wildcards as appropriate. They were also helpful in creating combinations of terms that capture relevant sets of studies in each database.

We kept a record with the date of searches, number of reports found, number of reports retrieved, key terms included, synonyms, and wildcards used when appropriate. Further details of electronic searches are presented in Section 13.

As planned, we contacted key authors requesting information on primary studies that could potentially be integrated in this systematic review and meta‐analysis. We also reviewed reference lists of previous primary studies or reviews related to the intervention/outcomes (e.g., Burrell, Zirbel, & Allen, 2003; Gottfredson, Cook, & Na, 2012; Johnson, 2012; Mytton, DiGuiseppi, Gough, Taylor, & Logan, 2006; Wilson, Tanner‐Smith, Lipsey, Steinka‐Fry, & Morrison, 2011).

Eligible studies met the following criteria: Reported results of interventions from 1980 onwards Tested the impact of a school‐based intervention on different types of exclusion (e.g., in‐school, out‐of‐school, expulsion) Included students from primary and secondary school levels settled in mainstream schools Based on an experimental design, where participants are randomly allocated treatment or control conditions Reported statistical results for computed an effect size

Data extraction was the responsibility of two researchers (AC & SV). Descriptive data of all studies potentially includable in the meta‐analysis was extracted using the data collection instrument presented in Section 12.2. The instrument facilitated the extraction of the following information: Bibliographical data (e.g., type of publication, year of publication, name of the publication, main author discipline) Ethics (e.g. declaration of conflicts of interest, use of informed consent) Research methods (e.g., type of design, units of randomisation, unit of analysis, variables used for matching) Sample selection (e.g., methods to select sample, attrition) Primary outcome coding (e.g., type of exclusion, duration of exclusion) Secondary outcomes coding (e.g., internalising and externalising behaviours, name of the instrument used to measure the outcome data) Base‐line measurements (e.g., source of data, quantitative measure of the primary outcome) Programme delivery (e.g., programme deliverer, training, type of intervention, frequency of the intervention) Post‐intervention and follow‐up measurement (e.g. official records, surveys) Data for calculation on effect sizes

The same two researchers extracted data for effect‐size calculations. The process was carried out independently. In general, discrepancies were solved by agreement but when the information reported was contentious, we asked for input from the more senior members of the team (ME & DF). Details on the data extracted from each included report can be found in section 9.1, 9.2 and 9.3 of the present review.

When data for calculation of effect sizes was incomplete we used two different strategies. First, we tried to find more details in other sources (e.g., published protocols or reports). Secondly, the lead researcher or members of the research team were contacted regarding the additional data needed.

Endnote X7 software was used to manage references, citations and documents. Data extracted to characterise studies was inputted in STATA v.13 in order to produce inferential/descriptive statistics. Effect sizes were inputted in Version 3.0 of the Comprehensive Meta‐Analysis software.

To check code consistency across studies, or inter‐rate reliability, we use Cohen's Kappa coefficient (Cohen, 1960). In the event of two coders making inclusion/exclusion decisions, Cohen's Kappa coefficient computes a standardised index across studies based on cross‐tabulated ratings. The index is given by the difference between the observed percentage of agreement in ratings across studies (P_o), and the probability of expected agreement due to chance (P_e), divided by 1‐P_e.

Cohen's Kappa was calculated based on the following formula: (1) k = P o P e − 1 − P e

To a good approximation, we calculated the standard deviation of Cohen's Kappa, following the expression (Cohen, 1960): (2)SDk=Po(1−Po)(1−Pe)2

We planned to assess the risk of bias in included studies by using two different instruments. In the case of RCTs, we intended to use the Cochrane Collaboration risk of bias tool (Higgins & Green, 2011). To analyse risk of bias of studies involving quasi‐experimental designs we planned to use the ACROBAT‐NRSI, another Cochrane Risk of Bias Assessment Tool for Non‐Randomised Studies (Stern, Higgins, & Reeves, 2014). Both instruments were supposed to assist the research team in evaluating the external validity of the included reports.

At the end of January 2016, we began the assessment with the originally proposed instruments. We soon realised that the instruments and their categories seemed more suited to medical trials than school‐based experiments. Therefore, possible alternatives were explored. In consultation with the coordinating group editor, we selected the EPOC risk of bias tool suggested in the methods section of the Campbell Collaboration website (see the tool in section 12.3). The instrument proposes the following eight criteria for the assessment of quality bias: Sequence generation Allocation concealment Baseline outcome equivalence Baseline characteristics equivalence Incomplete outcome data Blinding of outcome assessment Protection against contamination Selective outcome reporting

Each of these domains was judged on a 3‐point scale (i.e., low risk, high risk, unclear risk). EPOC tool provides guidance and examples for each domain that facilitate the decision of assigning low, high or unclear risk. Two members of the team performed the assessment of risk of bias (AS & SV). Assessment of bias was performed independently and the final results represent the agreement of both evaluators.

Since violations of the assumptions of independence in meta‐analysis would lead to incorrect estimates of the variance for pooled effect sizes (Higgins & Green, 2011 b; Romano & Kromrey, 2009),we used some strategies to deal with dependency in the data extracted from primary studies.

First, since we included book chapters, journal articles, government reports, and academic PhD theses, we anticipated the case where the same results would be published in more than one source (e.g., a book chapter and a journal article). The protocol stated that in those cases we would code only one outcome (e.g., the most complete, or the most outdated). In practice, we excluded 11 reports whose results were reported in more than one publication. They are grouped in four cases: The study did not report enough statistical data for effect size calculation. For instance, Vincent, Sprague, Pavel, Tobin, & Gau, (2015) did not report enough data for meta‐calculation. Although the main author was contacted, we were not able to access more details. For that reason we decided to include Sprague, Biglan, Rusby, Gau, & Vincent, (2016). This latter case used the same data but reported enough information for meta‐analysis. There was no overlap.RCT data was merged with quasi‐experimental data. For instance, Allen & Philliber, (2001) merged the RCT sample with another sample of students. As a result, the study was not an RCT anymore (For further details see Allen & Philliber 2001, p. 641).The same results were reported in more than one manuscript. That was the case of Panayiotopoulos & Kerfoot. The results of the study were reported in two papers (2004 and 2007). Results were identical in both publications (same dataset, same analysis and same outcomes). There was no overlap.The study did not report the outcome measured (i.e., suspension). A case in point is Arter, (2007). The author reported the results of her thesis in a journal. The journal article did not describe the outcome suspension, probably because no effect was found. We included the thesis since it reported all the outcomes measured.

Based on that, those 11 manuscripts were not included in the meta‐analysis. There was no overlap among them, consequently, no dependence in the outcomes was observed.

Secondly, included studies reported multiple time points, for instance, multiple follow‐up measures. The inclusion of multiple follow‐ups would create statistical dependence because the different measures are based on the same subjects (i.e., correlated with each other). We calculated effect sizes separately for those studies reporting short‐term and long‐term follow‐up measures. We also corrected variances estimation (see 3.4.8).

We use Standardised Means Differences (SMD or Cohen's d) to measure the treatment effects of the school‐based interventions included in the review. The decision to use this specific effect size is based on the fact that most of the included manuscripts report results where measurements are expressed in continuous scales (see section 9.3). The standardised mean effect size for a non‐clustered study is given by (3)d=X¯e−X¯cSp, where X¯e and X¯c represent the experimental and control group means, respectively, S_p is the pooled sample standard deviation given by (4)Sp=(ne−1)se2+(nc−1)sc2(ne−1)+(nc−1) where n_e and n_c are the sample size in each group, and se2 and sc2 are the experimental and control group standard deviation, respectively.

In the present review, we anticipated the inclusion of primary studies involving individually randomised as well as cluster‐randomised unit (e.g., schools or classrooms). One key issue emerges when meta‐analyses include cluster‐randomised studies: participants nested in the same cluster tend to be more similar to one another (as measured by the intra class correlation – ICC). Furthermore, when units of randomisation are clusters instead of individuals, we need to deal with the fact that the data presents different levels of variation (i.e., within clusters variation, between clusters variation and the total variance). This issue needs to be taken into account when computing effect size estimates. When this correlation is not accounted for, standard errors, confidence intervals and p‐values will tend to be too small. These conditions affect the meta‐analysis in two different ways. Firstly, the primary trial gets a mistakenly large weight. Secondly, the pooled result produces estimated effect sizes with an overly small standard error (Borenstein, Hedges, Higgins, & Rothstein, 2009; Higgins et al., 2011).

For the case of clustered data with dichotomous outcome measures (e.g., odds ratios), we followed the strategy proposed by the Cochrane Handbook for Systematic Reviews of Interventions (Higgins & Green, 2011), which corrects standard errors of effect sizes. The handbook suggests that the effective sample size in a cluster‐randomised trial can be obtained by dividing the original sample size by the design effect, which is calculated via (5)1+(M−1)×ICC.

In this equation, M is the average cluster size (units per cluster) and ICC is the intra‐cluster correlation coefficient. Once we were able to identify the design effect, the squared root of the design effect could be multiplied by the original standard error of the log Odds Ratio. Since ICC is rarely reported in primary studies, we have assumed a value of .05, based on the review of multiple meta‐analyses testing similar populations, produced by Ahn, Myers, & Jin, (2012).

In the case of clustered studies with continuous outcomes (e.g., school level means and standard deviations), we followed the strategy suggested by Hedges (2007) and Spier et al. (2013). Effect sizes were computed using d_T2 assuming equal cluster sample size: (6)dT2=[X¯¯■E−X¯¯■CST]1−2(n−1)ρN−2

In this equation X¯¯■E and X¯¯■C represent the overall means of the experimental and control group and S_T is the total sample standard deviation estimated from the pooled sample standard deviation across the experimental and the control group. Rho (is the notation used to represent the intra class correlation. N is the total sample size and the sample size of the clusters is represented by n. Based on the characteristics of our data and following Spier et al., (2013) we assume equal cluster size in our calculations.5 When the clusters have different sizes, we will take a conservative approach, including the smallest cluster size in our calculation.

The variance of the effect size will be calculated by (7) V n ρ { } = ( + ( − ) ) + d T 2 ( ) N E N C + N E N C 1 1 d T 2 2 ( ) ( ) N − 2 ( − ) + ( − ) + ( − ) ( − ) 1 2 2 2 1 ρ n N n N n ρ ρ ( − ) 1 ρ 2 ρ 2 2 2 2 2 1 ( − ) [ ( − ) − ( − ) ] N N n ρ

In this equation, N^E and N^C represent the experimental and control group sample across clusters. As suggested by Higgins & Green, (2011), in the event that the value of ρ is not reported, analysts are advised to assume a reasonable value based in previous studies with similar population (Ahn et al., 2012). As detailed previously, we have assumed a value of (ρ)= .05.

In those reports where key statistical information was missing, we attempted to obtain data from principal investigators. When that was not possible, the study was excluded from calculations of effect sizes. All the studies excluded for this reason have been identified and systematically reported in. section 4.1.3

In the present review, many of our selected studies involved repeated measures of the outcome exclusion. In fact, manuscripts reported measures of exclusion at baseline and post treatment (e.g., Hawkins, Doueck, & Lishner, 1988) or posttreatment and follow‐up (e.g., Farrell, Meyer, & White, 2001). In these cases, we have calculated a synthesis index or effect size of the difference, representing the change between those different measures (i.e., time points). Consequently, the change in exclusion from the baseline is computed by subtracting the means (X) as follows: (8)Ydiff=X2−X1

One issue arises when pursuing this strategy. Because measures at baseline and posttreatment are positively correlated, the calculation of the variance must be corrected. If we avoid the correction, assuming the two measures to be independent (correlation equals zero), we could be overestimating the variance and underestimating the precision of the difference (Borenstein et al., 2009). For a fair approximation of the value of the variance, we would need to know the correlation between the pre‐ and post‐measures (covariation). However, the covariance is not usually reported in primary research (and this was commonly the case in our set of included studies). Consequently, we proceed to assume a value for that correlation. After checking previous meta‐analysis of similar populations, testing school‐based interventions with estimates for the stability of serious problem behaviours (e.g., Farrington & Ttofi, 2009; Wilson et al., 2011), we concluded that the value of the pre‐post correlation should be assumed to be equal to .75. We then calculated the variance of the difference by using equation 9 below (Borenstein et al., 2009), where V₁ and V₂ represent the variances of the original point estimates and r represents the pre‐post correlation value: (9)VYdiff=V1+V2−2rV1V2.

As suggested by Higgins & Green, (2011), we undertook sensitivity analyses to determine whether the overall result of the analysis is robust in the use of imputed correlation coefficient.

We report weighted mean effect sizes, under a random model using 95% confidence intervals and accompanied with graphical representation (i.e., forest plots). For investigating heterogeneity, we use the estimates suggested by Borenstein et al. (2009), specifically; Tau‐squared, Q‐statistic and I².

Tau‐squared, or the difference between the total variance or variance observed and the within‐studies variance, will be estimated and reported.

The final calculation of the Q‐statistic includes reporting its value, degrees of freedom and p‐values. Significant p‐values provide evidence of heterogeneity in intervention effects.

Bearing in mind that Q can appear distorted when the number of studies meta‐analysed is small (Higgins, Thompson, Deeks, & Altman, 2003), we also report I2. I2 “is the proportion of observed dispersion that is real rather than spurious” (Borenstein et al., 2009). High percentages will be interpreted as an indication of high heterogeneity, meaning that the study‐to‐study dispersion is due to real differences in true effect size and not attributable to random error.

Since our review has a wide scope, we use the random effect inverse variance weighted models for meta‐analytical calculations. The random effect model is the most appropriate when effect sizes are not homogeneous or consistently coming from a single population (Borenstein et al., 2009). Under a random effects model the variance includes the original (within‐studies) variance plus the between‐studies variance, Tau².

Following Chandler, Churchill, Higgins, Lasserson, & Tovey, (2013), effect sizes will be coded such that a positive effect will reflect the outcomes favouring the treatment group. To illustrate our analysis, we provide summary forest plots displaying the estimated effect sizes along with their 95% confidence intervals.

In the present review, we use moderator analysis involving categorical variables estimating models analogous to ANOVA. Analyses are run under a random‐effect model assuming separate variance components for each group. Meta regression has been run in order to explore heterogeneity.

The distribution of SMD effect sizes was examined to determine the presence of outliers. Following Lipsey & Wilson, (2001), outliers were defined as those values which are more than two standard deviations from the overall mean of effect sizes. One outlier was detected (i.e., Collier, 2002) and it was windsorised to the next closest value (Lipsey & Wilson, 2001).

Since the present meta‐analysis involved a wide range of decisions, we conducted sensitivity analysis to test the robustness of these decisions (Higgins & Green, 2011). Specifically, we ran sensitivity analysis for the pre‐post correlations (i.e., covariance) assumed to be .75. We re‐ran the analysis using a correlation equal .50. As expected there was no change in the effect sizes and no relevant difference in standard errors. We also ran sensitivity analysis testing the impact of the outlier and the impact of the windsorisation (see section 4.7).

As stated in our protocol, we ran sensitivity analysis testing differences between published and unpublished reports.

Included studies reported multiple time points data, for instance, multiple follow‐up measures. Since the inclusion of multiple follow‐ups would create statistical dependence due to the different measures based on the same subjects (i.e., correlated with each other), we calculate effect sizes separately for those studies reporting short‐term (i.e., post‐treatment) and long‐term (i.e., follow‐up) measures.

We attempted to identify and retrieve the body of published and unpublished studies that met our inclusion criteria. Figure 1 shows a PRISMA flow diagram describing the results of our searches.

At the beginning, different combination of terms produced a total of 42,749 references from different electronic databases, of which we kept 1,474 relevant hits.

The 1,474 hits were selected based on screening the title, abstract and key words. We targeted studies broadly defined as evaluations, testing the impact of interventions on school exclusion.

As originally planned, we complemented electronic searches for papers with two extra strategies: i) reviewing lists of references from retrieved manuscripts, and ii) communication with key authors. Based on these steps, an additional 22 manuscripts were added to our sample. The full list of studies and their references were imported into Endnote X7. After removing duplicates, a total of 516 unique manuscripts were saved for further assessment.

Efforts were made to retrieve the full text copies of all 516 selected manuscripts. Notably, a high percentage of them corresponded to unpublished reports (48.6%), mainly PhD theses from American universities and Technical or Governmental reports. In the end, we were able to retrieve almost all full text manuscripts. Only eight studies out of 516 were categorised as “studies awaiting classification” and they are reported in Table 6.

The next round of evaluation was based on reading the information available in abstracts, methods and results sections using the eligibility checklist (see Section 12.1). The checklist included the basic criteria for inclusion in this systematic review. We concluded the second round of evaluation with 471 manuscripts excluded for reasons laid out in. Thirty‐seven papers presented enough statistical data for inclusion in our meta‐analysis. section 4.1.3

Two trained researchers (AC & SV) independently assessed the 516 pre‐selected manuscripts for inclusion or exclusion. We calculated Cohen's Kappa for testing inter‐rater reliability (Cohen, 1960; Sim & Wright, 2005). The value of Kappa ranges between 0 and 1, where a value of 1 represents perfect agreement between the two raters and a value of 0 indicates no more agreement than that expected by chance. We obtained a Cohen's Kappa= .76; SD=.81; SE=.036, reflecting a high level of agreement between coders. After calculating the agreement between coders, they went through the papers where they found disagreement. Differences were solved by further analysis and discussion. When needed, a more senior member of the team was consulted.

Following our published protocol, we excluded a total of 471 manuscripts. Table 4 summarises the reasons for the exclusion of each report. References of the excluded papers are availablein section 7.2.

Outcome. We excluded 52 reports (11%) because they did not present a suitable measure of school exclusion. In some specific cases the primary outcome was not reported (e.g., Gage, Sugai, Lewis, & Brzozowy, 2015; Webster‐Stratton, Rinaldi, & Reid, 2011) or it was reported in a composite measure along with other disciplinary measures which did not involve any type of exclusion from school (e.g., De Blank, 2009; Wright, Offord, John, Duku, & DeWit, 2005). Since it was not possible to isolate our primary outcome we excluded those reports. In other exceptional cases exclusion was measured as a predictor instead of an outcome (e.g., Rosenbaum, 2012).

Type of intervention. Following our protocol, we excluded 53 (11.2%) studies because the tested intervention was not delivered in schools, supported by schools or with at least one component implemented in school settings. Consequently, we excluded community programmes when they had no alliance with a school (.e.g., Henderson & Green, 2014; Wiggins et al., 2009). Alternative schools for high‐risk students (e.g., Rhea, 2010)and intervention in special schools (e.g., Kutash, Duchnowski, Green, & Ferron, 2013) were also excluded. Since our protocol stated that targeted intervention must be an “alternative to school exclusion”, we dismissed studies testing for instance the impact of restorative justice in the context of an in‐school exclusion programme (e.g., Brown‐Kersey, 2011), or conflict resolution in the context of an in‐school exclusion programme (e.g., Devlin, 2006). In both cases, the tested intervention was delivered in addition to exclusion more than being an alternative to it.

Methodological design. 339 (72%) studies were excluded because they did not satisfy the methodological characteristics defined in the protocol. We excluded studies lacking a control group (e.g., McDaniel‐Campbell, 2011; Nocera, Whitbread, & Nocera, 2014) and those studies where the control group was not equivalent in demographics and risk factor variables (e.g., Kilian, Fish, & Maniago, 2006; May, Stokes, Oliver, & McClure, 2015).

During the searches, we kept 28 manuscripts which corresponded to literature reviews (e.g., Fronius, Persson, Guckenburg, Hurley, & Petrosino, 2016; Gonzalez, 2012), systematic reviews (e.g., Wolgemuth, Cobb, & Dugan, 2007) or meta‐analysis (e.g., Noltemeyer & Ward, 2015) related to school exclusion or behavioural problems in schools. These types of manuscripts were initially retained on the understanding that they could be a source for identifying extra primary research reports. All 28 of these studies were excluded in the second round of assessment once we had checked their citation lists.

We identified nine manuscripts evaluating the impact of obligatory use of school uniform on levels of school exclusion (i.e., Draa, 2005; Gentile & Imberman, 2012; Gouge, 2011; Johnson, 2010; Samuels & Bishop, 2003; Shimizu & Peterson, 2000; Stevenson III & Brooks II, 1999; Vaughan, 2001; Washington‐Labat & Ginn, 2003). Even if they represented a particular type of intervention that could be interesting to meta‐analyse and compare, none of these eight interventions presented the research design targeted by our review. For that reason, they were excluded.

Finally, under the method design criteria we excluded a number of qualitative studies (Huston, 1999; Maguire, Macrae, & Milbourne, 2003; Rose, 2008) and case studies (e.g., Navarro, Aguilar, Aguilar, Alcalde, & Marchena, 2007) since none of them contributed with statistical data for meta‐analysis.

Participants. Five reports (1.1%) presented data focused on students with special needs (e.g., Cramer, 1990), or young offenders (e.g., Parkes, 2008). Since these participants were not targeted in our protocol, all these reports were excluded from our analysis.

Time span. Five studies (1.1%) were excluded because they were published before 1980 (i.e., Rogers, 1972)or because they reported the evaluation of an intervention implemented before 1980 (i.e., Feldi, 1980; Fisher, 1980; Herzog, 1980; Safer, Heaton, & Parker, 1981).

Reports based on the same dataset. Finally, we excluded 11 reports (2.3%) because they presented additional results based on the same data reported elsewhere (e.g., Allen & Philliber, 2001; Arter, 2007; Panayiotopoulos & Kerfoot, 2007; Vincent, Sprague, Pavel, Tobin, & Gau, 2015). In these cases, we kept the most complete report and the additional manuscripts assisted in a better understanding of the included research; they were however defined as excluded.

Pilot study. We excluded a single report presenting data from a pilot study (Bonell et al., 2015). In page 12 of the cited report, the author states that the aim of the pilot was “to evaluate feasibility and acceptability of the intervention and trial methods, and not to estimate intervention effects.” For those reasons, even though it was reporting results, the study was excluded. We kept references to the report in the category of ongoing research (see section 4.1.4).

Not enough data for meta‐calculations. Five reports (1.1%) were excluded because they did not present enough data for calculation of effect sizes (e.g., Grinage, 2005).

We identified the protocol of four ongoing studies whose outcomes had not been published at the end of the searches in December 2015. As observed in Table 5, all of them are cluster randomised control trials testing the impact of school‐based interventions and measuring school exclusion as an outcome.

We were unable to classify eight studies. We selected them based on title, abstract and key words but we have not been able to locate the full text copies. A list of studies that could be potentially included in a future updated version of this review is given in Table 6.

Thirty‐seven studies reporting 38 interventions’ effect sizes were included in this meta‐analysis. As we mention before, all of them were randomised controlled trials. In 23 studies the control group received no‐treatment (62.2%); six studies reported controls receiving intervention or business as usual (16.2%), four experiments offered a placebo to the control group (10.8%) and four studies allocated controls in a waiting list (10.8%).

Following sections provide a general overview of 37 included studies. Data is organised by characteristics of the included studies, characteristics of participants and characteristics of the delivered interventions.

As observed in Table 8, in the present meta‐analysis, sampled students most frequently come from the higher school grades (i.e., middle and high schools), with a M_age=12.9; SD= 2.8.

In addition, data shows that students were nested in schools with a high percentage of Black and Latino students and, subsequently, with a low percentage of White‐Caucasian peers (24.6%). In fact, five of the included studies are based in schools where a 100% of the pupils were Black (i.e., Barnes, Bauza, & Treiber, 2003; Brett, 1993; Collier, 2002; Mack, 2001; Reese, Murphy, & Filipczak, 1981).

As stated by Hobbs & Vignoles, (2010) and Sutherland, Ilie, & Vignoles, (2015), in this report the percentage of students with access to free school meals was understood as an index of the vulnerability of the population. Across our selected studies the pupils receiving meals funded by the school was predominant (M_fsm=66.2; SD=23.9%).

The present review includes a wide range of school‐based interventions covering, for instance, strategies such as counselling, mentoring, community services, classroom management, after‐school academic support, and school‐wide strategies, among others. Taking a broad view, 27 % of the interventions were focused on a change at the level of the school or teacher, and a 73% anticipating that a change in pupils’ skills/behaviours could affect suspension/exclusion rates.

As shown in Table 9, school staff (32.4%) or school staff assisted by external facilitators (24.3%) delivered most of the interventions. This means that schools interested in reducing exclusion typically put resources into the intervention process. Most of those delivering the interventions were school psychologists/counsellors (32.2%) and, in two particular cases (5.4%), the intervention was delivered by a police or probation officers, which seems to be an increasing trend in the United States (e.g., Gottfredson et al., 2012; Hirschfield, 2008).

We coded data on the role of the evaluator. We identified four potential approaches taken by evaluators; i) delivering the programme, meaning that the researcher implemented the intervention as well as acting as evaluator (e.g., Harding, 2011); ii) designing the programme, meaning that the evaluator defined or took part in the defining the theoretical base, aims and activities of the intervention (e.g., Cornell et al., 2012), iii) both, designing and delivering the programme (Mack, 2001) and iv) independent evaluation, referring to those researchers or research teams where any member was involved in any stage of the designed or delivery (e.g., Bradshaw et al., 2012; Dynarski et al., 2003). A high percentage of interventions were designed and/or delivered by the same researcher who evaluated the impact of the intervention. Independent evaluators conducted 48.6% of the RCTs included in this review.

A high percentage of the interventions were curricular (67.6%). For instance, social skills trainings or anger management training were based on a pre‐designed curriculum with detailed aims and activities for each session. In the present review, non‐curricular interventions are those targeting school‐wide change or those focused on counselling and individual therapies.

On average, included interventions lasted M=20; SD=11.5 weeks. 37.8% of the interventions lasted less than 12 weeks and an equal percentage were delivered over more than 24 weeks. In general, school‐wide interventions lasted longer in our sample (i.e., 35 weeks or more).

Table 10 provides a description of the included school‐based programmes targeting school exclusion as a primary or secondary outcome. In some specific cases this grouping could be restrictive, because some of the interventions involve multiple components, however we have attempted to create an exhaustive list of categories.

Enhancement of academic skills. We found two effect sizes targeting the enhancement of academic skills as a strategy in order to improve academic performance, increase motivation and promote more adaptive behaviour. Edmunds et al., (2012) tested an intervention to boost the academic progress of students in order to facilitate their future access to college while in the case of Cook et al., (2014), the intervention involved academic remediation plus social skills training.

After‐school programmes. Two effect sizes come from interventions that offered students after‐school activities. While the intervention tested by Dynarski et al., (2003) was more focused on academic support and recreational activities, Hirsch, Hedges, Stawicki, & Mekinda, (2011) tested an after‐school programme offering students paid apprenticeships.

Mentoring/monitoring programmes. Five effect sizes reported the impact of interventions focused on mentoring/monitoring. These programmes involved structured and supportive relationships between a young person who presents academic, emotional or behavioural difficulties and a non‐parental adult, their mentor. Mentoring entails a volunteer member of the community serving as a role model and providing support to a younger person over an extended period of time (e.g., Brett, 1993; Wyman et al., 2010). In other cases, adults served as tutors for the students, supervising their performance, providing advice or counselling, and assisting the students with academic tasks (Johnson, 1983; Peck, 2006; Reese et al., 1981). Such tutors were normally schoolteachers or school counsellors.

Social skills training for students. We found nine effect sizes representing the impact of social skills training. These programmes were based on social learning and cognitive behavioural theories (e.g., Burcham, 2002; Collier, 2002; Harding, 2011; Hostetler & Vondracek, 1995; Shetgiri, Kataoka, Lin, & Flores, 2011; Smith, 2004) and their goal is to enhance individuals’ socio‐cognitive, socio‐emotional, and behavioural skills in order to regulate maladaptive conducts. Social skills training programmes typically consist of a curriculum with focused training modules. Some more specific programmes target communication skills (e.g., Obsuth et al., 2016) or approaches to reducing stress (e.g., Barnes, Bauza, & Treiber, 2003). Group‐based sessions and occasionally one‐to‐one sessions (e.g., Russell, 2007) offer the opportunity to implement specific techniques (e.g., instruction, modelling, role‐playing, feedback and reinforcement) in a “real‐world environment”.

Skills training for teachers. We found four independent interventions targeting teachers’ skills. The training provides preventive strategies and techniques that help to maintain classroom discipline, create a supportive educational environment, and enhance students’ positive behaviour. These involve training in facilitating mutual respect between teacher and student (e.g., Okonofua, Pauneskua, & Walton, 2016) as well as training to establish clear classroom rules (e.g., Hawkins, Doueck, & Lishner, 1988). Skills for teachers also involve strategies for working in an alliance with parents to promote students’ engagement to the school activities (e.g., Ialongo et al., 2001).

School‐wide interventions. Six effect sizes represent comprehensive interventions targeting systemic changes across the whole school. They involve pupils, teachers, parents, and sometimes also the community where the school is based. These programmes aim to create positive environments, with clear rules that promote good behaviour, learning and safety. School‐wide interventions are capable of addressing the needs of schools as institutions as well as the particular needs of individual school children. We found six studies testing school‐wide strategies: Bradshaw et al., (2012); Cornell et al., (2012); Lewis et al., (2013); Snyder et al., (2010); Sprague et al., (2016); Ward & Gersten, (2013).

Violence reduction. We included three effect sizes measuring the impact of violence reduction programmes. Although these interventions could be classified as skills training, we have isolated them, because they are specifically targeted at increasing self‐control and reducing violence (e.g., Feindler, Marriott, & Iwata, 1984; Mack, 2001). We also included anger management programmes encouraging peaceful responses to conflict (e.g., Farrell et al., 2010).

Counselling and mental health interventions. We included three effect sizes primarily focused on the provision of counselling in schools (e.g., Berlanga, 2004; Tilghman, 1988)and on a more specialised provision from community mental health services (i.e., Panayiotopoulos & Kerfoot, 2004).

Other interventions. Four effect sizes were classified in this general category. They encompass a community services programme (Allen, Philliber, & Herrling, 1997), a multicomponent programme (Arter, 2005), a career awareness intervention (Bragdon, 2010), and a programme focused on character‐building education, promoting civic behaviour and national values (Crowder, 2001).

Adequate sequence generation is intended to produce comparable groups in an experimental evaluation. Accurate methods used to generate the allocation are, for instance, the use of random number tables or a computer random number generator (Higgins & Green, 2011). Regardless of the selected methods, it should be clearly described in enough detail to allow an assessment of whether the sequence generation was adequate.

As shown in Figure 2, only three studies (8%) presented a high risk of bias with respect to allocation sequence. A case in point:Allen, Philliber, Herrling, & Kuperminc, (1997) used three different methods of randomisation (i.e., picking names from a hat, coin toss and using an alphabetical list of names) as well as running randomisation at different levels in the same study (i.e., individuals and classrooms). In our opinion this mixture of methods could lead to a high risk of bias in the final results: in particular, the use of a list of names is known not to be appropriate for allocating cases. In the case of Barnes, Bauza, & Treiber, (2003:2), the author mentions that “each school was alternately assigned to either TM or health education control.” Since the study involves only two schools, allocation would be predictable and at a high risk of bias.

Although we identify a low percentage of high‐risk cases, it is important to note that a large proportion of studies (59%) were categorised as “unclear risk”. This is because many reports presented succinct descriptions of the randomisation process without detailing the methods of sequence generation (e.g., Arter, 2005; Brett, 1993; Feindler, Marriott, & Iwata, 1984; Reese, Murphy, & Filipczak, 1981). 32% of manuscripts were defined as presenting a low risk of bias (e.g., Bradshaw et al., 2012; Hirsch et al., 2011; Lewis et al., 2013).

Correct allocation concealment safeguards a rigorous implementation of the randomisation process by not allowing researchers or participants prior knowledge of the results of assignment (i.e., by using sealed envelopes or other procedure that prevent knowledge about the condition that the participant is going to be allocated). To achieve this aim it has been suggested that allocation should be centralised and executed at the beginning of the study (Schulz & Grimes, 2002).

In the present meta‐analysis, only 5% of the studies were evaluated as having a high risk of bias for allocation concealment. Some examples of high‐risk studies are those where reports suggested that schoolteachers instead of researchers performed the random allocation (e.g., Crowder, 2001) or that the randomisation was performed after the participants’ screening (i.e., Mack, 2001). Even if we cannot be sure that the results of the screening biased the randomisation, this conduct could potentially interfere with the chances of participants being placed in the control or treatment group.

Once again, we found a high percentage of studies (54%) reporting minimal details of allocation concealment. In concrete terms, 20 out of 37 included reports were classified as “unclear risk” of bias. The remaining 41% (i.e. 15 studies) were classified as having a low risk of bias.

A key element of a randomised controlled trial is that it ensures, in theory, that participants (and their associated outcomes and characteristics) are distributed by chance in the control and treatment groups. Pre‐existing baseline differences between groups – particularly of outcomes – could suggest problems in randomisation, hence it is a key focus when assessing risk of bias (Shadish et al., 2002). It is important to mention that we did not focus on whether there were statistically significant differences between groups, as that is a function of sample size; instead we reviewed means and distributions.

We found seven studies (19%) whose description of the baseline equivalence suggested a high‐risk. This was found, for instance, in cases where the control group presented higher levels of exclusion than the treatment group and there was not clear mention of adjustment (e.g., Panayiotopoulos & Kerfoot, 2004). Another example concerns the imbalance of one specific type of exclusion: specifically, expulsion and out‐of‐school exclusion were equivalent in the treatment and the control groups, but display substantial imbalance in the case of in‐school exclusion (e.g., Berlanga, 2004).

Again, it is important to mention that nearly 30% of assessed studies reported limited information, and for that reason they were assigned an unclear risk of bias. The remaining 49% of our included randomised controlled trials were evaluated as low risk of bias.

Regarding this criterion, the instrument considers the balance in demographics or any other behavioural outcomes, which again should be equivalent in the treatment and the control group if randomisation has been successful.8 We found 11 studies (30%) displaying non‐equivalent results between treatment and control groups. Studies reporting imbalance in gender (e.g., Berlanga, 2004), ethnicity (e.g., Hostetler & Fisher, 1997), initial levels of problem behaviour, self‐control or size of the school (e.g., Harding, 2011)were all classified as a potentially having a high risk of bias.

A further 22% (eight studies) did not report enough data for judgement (i.e., unclear risk) and around half (49%) of the studies were assessed as presenting a low risk of bias.

The fact that our included studies pursued more than one measure across time makes it likely that attrition or other forms of missingnes affected sample sizes.

We assess a high risk of bias when i) substantial attrition was present in the study and the researchers did not mention a strategy to deal with that issue (e.g., Peck, 2006); ii) when they used list‐wise deletion (e.g., Russell, 2007), and iii) in those cases where the attrition affected the treatment or control group in an unequal proportion of missing cases across arms (e.g. Hostetler & Fisher, 1997). Using these criteria, we judged that eight studies (22%) presented a high risk of bias.

As in previous cases, a high number of studies (30%) did not report enough data to be judged. Seventeen studies (46%) were evaluated as presenting a low risk of bias. Low risk cases involved i) those studies reporting zero attrition; ii) studies where attrition was represented by a low percentage of cases; iii) when missingnes was equivalent in the treatment and the control group, and iv) when the researcher reported attrition, analysed it and used methods to deal with attrition (e.g., multiple imputation, full information maximum likelihood or intention to treat rather than assessing the effect of treatment on the treated).

This criterion covers bias arising from the fact that those collecting outcome data are aware of the condition assigned to each participant (e.g. individuals, classrooms, schools). In our evaluation, 35% or 13 of the coded studies were assessed as having a high risk of bias. We assigned a high risk of bias when those in charge of delivering the intervention were also collecting data (e.g., Harding, 2011) or when teachers rated the students’ behaviour while being aware of the allocated condition (e.g., Tilghman, 1988). Although the outcome was in the majority of studies, based on official records, we did not assume this meant blind assessment. In most cases, teachers or school staff who imposed exclusion were not necessarily blind to the allocated condition of participants. In any case, it must be said that in school‐based experiments blinding is likely infeasible. Most of the studies require at least a minimal participation of school staff and allocated condition is mostly evident for participants (Hutchison & Styles, 2010).

As in previous cases, the number of reports lacking the data to evaluate blindness of the outcome assessment (i.e., unclear risk) was high (54%) and only four studies (11%) were assessed as low risk bias.

Protection against contamination refers to the measures taken to avoid a spill‐over of the treatment into the control group. Specifically, contamination is the risk that the control group might accidentally receive the intervention, but also the risk that the two groups could influence each other's outcomes. If the groups cannot remain isolated during the experimental study, contamination would threaten the validity of the results. In our meta‐analysis 76% of the studies presented a high risk of contamination. The main explanation for high risk of bias was the fact that schools in the trials host both the treatment and the control participants (e.g., Cook et al., 2014; Smith, 2004).

One single study (3%) did not report enough data for evaluation and it was categorised as unclear risk of bias. Eight studies (22%) were classified as low risk of bias. Normally low risk studies were cluster‐randomised experiments where control and treatment participants were in different schools (e.g., Bradshaw et al., 2012; Snyder et al., 2010).

Selective outcome reporting occurs when there is a difference between the proposed outcomes for evaluation and those finally reported. In our evaluation, four studies (11%) presented a high risk of selective outcome reporting (e.g., Panayiotopoulos & Kerfoot, 2004; Russell, 2007). In the case of thesis or trials without published protocols, the assessment was only based on discrepancies between outcomes proposed and reported in those documents9. Thirty‐two studies (86%) displayed a low risk of bias and only 3% did not report enough data for judgement.

Overall, our assessment demonstrates that a high number of the included studies lacked enough information to use the EPOC tool to judge risk of bias in all areas. Specifically, 22 studies were not clear on how they allocated units to treatment and control groups. Similarly, blinding to allocation was another area infrequently reported with enough detail to allow assessment (i.e., in the case of 21 studies). These points aside, contamination or spill‐over is probably the main threat to the validity of results among the RCTs included in our review.

Arguably the seven methodological strongest studies were Bradshaw et al., (2012), Cook et al., (2014), Hirsch et al., (2011), Johnson, (1983), Lewis et al., (2013), Obsuth et al., (2016) and Wyman et al., (2010). All of them presented low risk in the randomisation process and most of them are clustered studies. In particular, these studies achieved low risk of bias in six or more of the eight EPOC criteria.

The present analysis incorporates 38 effect sizes across 37 studies producing enough statistical information for meta‐analysis. These studies represent a total sample of 31,273 students (M_age=12.5; SD=2.85) partaking in completed trials as treatment, control or placebo groups.

On average, school exclusion was significantly reduced in the experimental group compared with the control group, post‐treatment (i.e.,sixmonths on average). Under a random effects model, the standardised mean reduction was SMD=.30; (95% CI .20 to .41; p<.001). Figure 3 shows that results are positive and statistically significant, meaning that those participating in school‐based interventions were less likely to be excluded than those in the control group. Results exhibit significant heterogeneity (Q=301.3; df= 37; p<.001; I²= 87.7; τ²=.078) which was expected in this meta‐analysis bearing in mind that we include different school‐based programmes, administering different “doses of treatment” with participants from different locations and in different school grades.

As suggested by Lipsey and Wilson (2001, p. 153) the U3 statistic for a SMD= .30 is 62%. This would indicate that 62% of the treatment group is above the median of the control group. Stated differently, if we assume a 50/50 success rate for both groups, the treatment group sees 62% success, versus 38% in the control group.

As proposed, we ran an analysis for the different outcome measures: in‐school exclusion, out‐of‐school exclusion, expulsion, and general exclusion. Results of the four independent meta‐analysesare summarised below in Table 12.

In‐school exclusion. Only six trials were concerned with the effect of school‐based intervention on reducing in‐school exclusion. Under a random model, the effect of the intervention remained positive and statistically significant. The final effect described in Figure 7 was SMD=.35 (95% CI .11 to .58; p<.005). Once again heterogeneity was tested, finding significant but smaller variability across studies (Q=11.62; df=5; p<.05; I²=57; τ²=.045).

Out‐of‐school exclusion. Correspondingly, nine studies reported data for the impact of interventions on out‐of‐school exclusion. Figure 8 shows an effect close to zero, non‐statistically significant of SMD=.02 (95% CI ‐.16 to .19; p>.05).

Expulsion. The impact of school‐based interventions on expulsion was significantly higher than any other impact described so far. Figure 9 shows that expulsion was reduced by SMD=.53 (95% CI .07 to .98; p<.05) with significant heterogeneity (Q=13.66; df=3; p<.05; I²=78; τ²=.14) but based on only four reports presenting data for analysis. Therefore, these results must be evaluated cautiously.

General Suspension. Finally, a number of studies presented data on suspension as a broad and general measure. These studies did not describe operational definitions about the type of disciplinary suspensions involved in the outcome. We have defined this category as General Suspension. In the aim of transparency, we report these results separately, although this measure could involve any of the previous outcomes reported above and, therefore, be a subset of the overall effect size reported at the beginning of this section.

Figure 10 shows 27 studies reporting 28 independent effect sizes concerned with the impact of targeted interventions on general suspension. The effect of school‐based interventions was positive SMD= .32 (95% CI .21 to .43; p<.001), it was statistically significant and similar to the overall effect size reported in Figure 3 above. In addition, heterogeneity remained substantial (Q=171.45; df=27 p<.001; I²=84; τ²=.056).

As stated in our protocol, for any identified study reporting data on school exclusion, we also coded a secondary outcome referring to internalising and externalising behaviours. Data on internalising behaviour was not very prolific in our set of included studies. Only five trials presented statistical results but sometimes the data was insufficient for effect size calculations. For that reason, we are not running meta‐analysis on internalising behaviours, but we summarise the results in Table 13.

We found a diverse range of measures referring to externalising behaviours, such as substance misuse, violence, aggression, and problematic behaviour in school. To pool together comparable measures, we ran a meta‐analysis only on behaviours that could be categorised as antisocial such as aggression, physical fights, delinquency, bullying and conduct disorder. 2014001034

Fourteen studies reported complete data for a composite measure of antisocial behaviour (See Figure 11). The fourteen studies provided 15 independent effect sizes. Unusually, Wyman et al. (2010) reported a measure of behaviour control (e.g., children accepting imposed limits). We reversed the results as a proxy of antisocial behaviour and included this information in our calculations. The same procedure was followed with Feindler et al. (1984) who reported a measure of increase in self‐control. Once again, we reversed the effects size as a proxy of antisocial behaviour.

As reported in Figure 11, the overall impact of school‐based interventions on antisocial behaviour, under a random effects model, was not statistically different from zero SMD= ‐.005 (95% CI ‐.097 to .09; p >.05) indicating an overall null effect of these programmes at reducing antisocial behaviour.

It must be highlighted that some of the included studies in this overall measure reported negative effect sizes, meaning that in some specific cases the intervention had iatrogenic impact (Hawkins et al., 1988; Hostetler & Fisher, 1997; Obsuth et al., 2016). For instance, Hawkins et al., (1988) reported the impact of an intervention focused on interactive teaching and co‐operative learning targeting low achievers in mainstream schools. The treatment group showed a reduction in the number of exclusions but an increase in the mean value for serious crime. The study reported by Hostetler & Fisher, (1997) described a similar case. For its part, Obsuth et al., (2016) reported the results of a clustered randomized controlled trial testing the impact of “Engage in Education‐London”. The intervention targeted high risk students and it was aimed at improving communication and broader social skills. Results suggest that the iatrogenic effects were observable not only for exclusion but also for the case of antisocial behavior.

It is important to mention that Feindler et al., 1984; as well as Wyman et al., (2010) were reverted and used as proxies of antisocial behaviour. For this reason, they should not be understood as interventions which report iatrogenic results.

Of the 38 effects reported in this meta‐analysis, 11 were tested in schools whose population was predominantly male (i.e., more than 60% of students were male), and 19 effects were tested in schools presenting a mixed population (i.e., neither gender exceeded 60 per cent).

Post‐intervention effects were different for both groups. For studies targeting predominantly male schools, the standardised mean difference was SMD=.41; (95% CI .10 to .72; p<.05). In studies targeting mixed‐gender schools, the impact was lower (SMD=.17; 95% CI .02 to .32: p<.05). Differences between groups were not statistically significant (Q=1.84; df=1; p>.05).

In terms of age, the best proxy variable was school grade. This information was reported in 34 studies and 35 independent effects. To test the hypothesis that effect sizes vary by age, we ran sub‐group analysis for 12 studies involving elementary school students (SMD=.27; 95%CI .09 to .45; p<.05), 16 targeting middle schools (SMD=.23; 95% CI .04 to .41; p<.05), and eight targeting high schools (SMD=.45; 95%CI .18 to .72; p=.001). The effect was statistically significant in each sub‐group and larger in high school populations. However, the between effect difference was not statistically significant (Q=1.81; df=2; p=.41) meaning that there is no evidence that the effect of school‐based interventions differs by age (i.e., grade at school) in the present meta‐analysis.

Ethnicity was reported as a continuous variable (i.e., percentage of each ethnic group). For that reason, the role of ethnicity in explaining the effect of school‐based programmes will be explored in a meta‐regression analysis.

In our review, only a limited number of studies presented data on behaviour (i.e., internalising or externalising behaviours). In addition, the overall impact of school‐based interventions on antisocial behaviour was not statistically different from zero.

Pupils involved in included studies shared a similar high‐risk condition. They were registered in schools with a high percentage of ethnic minorities and more than 60% of the students in those schools received free school meals, which is an indicator of disadvantaged socio‐economic backgrounds. In that sense, participants do not display high variability.

One of the aims of the present meta‐analysis was to compare the effect of different interventions on the reduction of school exclusion. Table 14 presents the standardised mean differences, confidence intervals and p‐values as well as measures of heterogeneity for each of the ninetypes of programmes included in the review. The typology is the same as described in section 4.1.7.

First of all, as observed in Table 14,most of the interventions are represented by a restricted number of effect sizes, and for this reason, these results should be interpreted cautiously.

Secondly, the standardised mean differences of only five types of programmes present positive (small to moderate effect sizes) and statistically significant results in favour of the reduction of school exclusion. Those programmes are: i) Enhancement of academic skills, ii) Mentoring/monitoring, iii) Skills training for teachers, iv) Counselling/mental health services, and v) Other programmes. Since “other programmes” involve a mixture of different interventions we believe they cannot be interpreted in the same way as the remaining four types. Similarly, when it comes to the number of studies included in each sub‐group, it seems that the most stable results are Mentoring/monitoring and Skills training for teachers since they are based on a larger number of studies.

Thirdly, to test the hypothesis that differences were significant among the compared sub‐types, we ran further analysis. The comparison demonstrates that differences are statistically significant (Q=18.4; df= 8; p<. 05), meaning that variation in effect sizes can be explained by the type of intervention implemented.

Reported information on the theoretical bases of the interventions was not very comprehensive. Selected studies described components of interventions more frequently than reporting the theory or set of theories framing the praxis. Based on the provided details, 20 interventions were clearly based on a cognitive behavioural frame, while another five were concerned with ecological ideas, targeting a change of the school system as a whole. The remaining effects refer to different theories such as emotional intelligence (i.e., Smith, 2004), empathy (i.e., Okonofua et al., 2016), civic values (i.e., Crowder, 2001) or developmental theories (i.e., Allen et al., 1997). A number of studies did not provide enough information to make a judgement. We tried to run sub‐group analysis on the theoretical bases of the interventions but it demonstrated low power, given that the number of effects was, in some cases, as low as one per sub‐group. In that scenario, we decided that a measure of the level of targeted change could inform more consistent data on the theory framing the interventions. We then divided the studies into those targeting a systemic change versus those targeting a change in pupils.

The standardised mean difference for the 10 evaluations targeting a change at school level provideda significant reduction with a value of SMD=.25 (95%CI .04 to .45), whereas the 28 evaluations targeting a change at the pupil's level reported a significant reduction with a value of SMD=.33; (95%CI .19 to .48). Both independent effects are statistically significant; however, the between‐group comparison reported non‐significant differences (Q=.48; df=1; p>.05) meaning that there is no evidence that the effect differs by level of targeted change.

Previous research demonstrates that well‐implemented programmes – those including training, monitoring and supervision – display larger and more consistent effect sizes (e.g., Durlak et al., 2011; Gottfredson & Wilson, 2003; Lösel & Beelmann, 2006). We tested this hypothesis based on two variables, namely, “training before implementation” and “monitoring during intervention”.

Twenty‐five studies (reporting 26 independent effect sizes) clearly stated the presence of training hours before the intervention was delivered (i.e., training hours for those delivering the intervention). In the remaining 12 studies, authors did not mention any kind of training. We ran sub‐group analysis to test the hypothesis that those that reported training could produce a significantly different effect. In the end, those reporting prior training yielded a result equal to SMD=.29 (95% CI .16 to .43; p<.001) whereas those that did not report training produced an effect equal to SMD=.34 (95% CI .15 to .53; p<.001). Both effects were positive and significant, however the test of the difference between the two sub‐groups of studies yielded a result of Q=.16 with df=1 and p>.05, meaning that there is not enough evidence that effects differ by presence/absence of prior training.

Fifteen studies that reported monitoring the implementation of the programme during the trial yielded a result equal to SMD=.20 (95%CI .05 to .35; p<.05). In parallel, the 23 studies that did not report monitoring produced an SMD=.37 (95%CI .25 to .50; p<.001). Both results were positive and statistically significant. The test for differences between sub‐groups showed non‐significant differences (Q=2.89; df=1; p>.05).

Based on descriptive analysis of data, and also based on previous research findings (Beelmann & Lösel, 2006; Farrington, Ttofi, & Lösel, 2016), we selected three post‐hoc moderators, namely: i) reasons for conducting the research; ii) evaluator role, and iii) risk of quality bias. Their results are described below:

Reasons for conducting the research. We coded data related to the reasons for conducting research. Two categories are explored: i) demonstration study, referring to those studies testing the impact of an intervention under highly controlled optimal conditions and ii) routine evaluation, focusing on testing established programmes under circumstances that approach real‐life conditions (Singal, Higgins, & Waljee, 2014).

Sub‐group analysis produced different effects for the 16 studies conducted for demonstration (SMD=.43; 95% CI .26 t0.59; p<.001), and the 18 studies carried out for routine evaluation (SMD= .13; (95% CI .00 to .25; p<.05). Demonstration studies reported the highest effect. The between‐studies comparison results were significant (Q=8.15; df=1; p<.05), meaning that the effect varies depending on the reasons for conducting the research.

Evaluator role. We coded data identifying the role of the evaluator. Independent evaluators were those not taking part in the design or implementation of the evaluated programme. Dependent evaluators were those also contributing to the design and/or the implementation of the programme.

The 18 trials carried out by independent evaluators produced an SMD= .13 (95% CI .00 to .25; p<.05). Unsurprisingly, based on what is known about developer‐led trials, the 16 RCTs conducted by dependent evaluators (i.e., those who also developed and/or designed the intervention) yielded results where SMD=.47 (95% CI .32 to .62; p<.001). The between‐groups comparison demonstrated that effect sizes are lower for studies conducted by independent evaluators. These differences were statistically significant (Q=12.36; df=1; p<.001).

Risk of quality bias. As reported in section 4.2, we ran EPOC tool for assessing risk of quality bias. Results show that studies vary in the amount of information available for judging their level of risk as well as their risk of bias. We speculated that the quality of studies could explain some of the heterogeneity present in our results. In order to use the results reported by EPOC, we transformed the “low risk”, “unclear risk” and “high risk” into continuous 0, 2 and 3 values respectively. The sum of the results of the eight EPOC criteria resulted in an index that, in our case, was used as a representation of the level of risk of each single study. We tested the role of RoB in meta‐regression.

Instruction: use one data‐coding instrument for each manuscript. When more than one manuscript reports the same research project, select one of them as the principal (e.g., the older) and give it an ID number. The following manuscripts should use the same ID but it must be registered in the Crossref field.

[STUDYID] Study ID number:

[CROSSREF1] Cross reference document identifier:

[CROSSREF2] Cross reference document identifier:

[CROSSREF3] Cross reference document identifier:

[DATESCR] Date of screening:

[CODER] Coder Initials:

Before completing this section, please be sure that the manuscript is correctly uploaded in the reference manager programme.

[AUTHOR] Name of the main author(s):

[AFFIL] Main author affiliation:

[DATEPUB] Year of publication:

[DATEFIEDW] Year of fieldwork (usually reported in a range):

[COISTATEMENT] Has the paper included a conflict of interest statement? ⑥ 1. Yes⑥ 0. No

[LANGPUB] Language of the publication: ⑥ 1. English⑥ 2. German⑥ 3. Italian⑥ 4. Spanish⑥ 5. Portuguese⑥ 999. Other:________

[COUNTPUB] Country of publication: ⑥ 1. UK⑥ 2. USA⑥ 3. Canada⑥ 4. Australia⑥ 999. Other:_________⑥ 99. Unknown

[TYPUB] Type of publication: ⑥ 1. Journal⑥ 2. Book/book chapter⑥ 3. Masters thesis⑥ 4. PhD/doctoral thesis⑥ 5. Technical/governmental report⑥ 6. Conference proceedings⑥ 999. Other:________

[AUTDIS] Main author discipline: ⑥ 1. Education⑥ 2. Social Work⑥ 3. Psychology⑥ 4. Criminal Justice⑥ 5. Sociology⑥ 6. Psychiatry/Medicine⑥ 999. Other:________⑥ 99. Unknown

[LOCAT] How was the study/report located? ⑥ 1. Electronic database⑥ 2. Web search⑥ 3. Reference in a book/paper. Please specify:⑥ 4. Hand search in specialised journal⑥ 5. Peer/expert suggestion⑥ 999. Other. Specify:____________

Section C. Ethics

[CONSENT] Did the study declare the use of “consent agreement forms”? ⑥ 1. Yes⑥ 0. No⑥ 999. Other:________⑥ 99. Unknown

[SIGNCONS] Who signed the consent? ⑥ 1. Students⑥ 2. Parents⑥ 3. Teachers⑥ 4. Schools⑥ 5. Parents and student⑥ 999. Other. Specify:______________⑥ 99. Unknown

The present systematic review includes randomised control trials as well as quasi‐experimental reports (before/after measure plus a control or comparison group). If the control/comparison group is randomly allocated, non‐randomly allocated or matched and no intervention expected to produce impact is provided to it, you will be able to code that group as CONTROL. Subsequently, the TREATMENT group could be understood as the group that receives the intervention, no matter if that condition has been randomly allocated or not.

Please select always the data that is related with the sample effectively analysed.

[DESTYPE] What kind of design is this paper based on? ⑥ 1. Randomised controlled trial (true experiment)⑥ 2. Before‐and‐after with control/comparison group/s⑥ 3. Instrumental variable⑥ 4. Propensity score matching⑥ 5. Interrupted time series⑥ 6. Pre/post measures with unmatched control/comparison group⑥ 7. Inverse probability weighting⑥ 999. Other. Specify:____________

[RANDUNIT] Units of randomization ⑥ 1. Individuals⑥ 2. Clusters/groups (classroom, schools)⑥ 999. Other. Specify:____________⑥ 99. Unknown

[ANALUNIT] Unit of analysis ⑥ 1. Students⑥ 2. Clusters/groups (classroom, schools)⑥ 999. Other. Specify:____________⑥ 99. Unknown

[COMPVAR] Variables measured to create comparability? (e.g., variables used to match the control and treatment groups)

__________________________________________________________________________________________

[MAINSTAT] What is the main statistical analysis used to produce the final results? ⑥ 1. Multilevel modelling⑥ 2. Differences of means⑥ 3. MANOVA⑥ 4. Chi‐squared⑥ 5. Propensity Score Matching⑥ 999. Other. Specify:____________

[SAMPSELECT] How was the sample selected? ⑥ 1. Randomly⑥ 2. Assessment⑥ 3. Self‐selection⑥ 999. Other. Specify:___________

[INSAMP] Initial sample size (i.e., individuals/schools):

[NUMBFOLL] Nº of follow‐up:

[FOLLSAMP1] Follow‐up 1 sample size:

[FOLLSAMP2] Follow‐up 2 sample size:

[FOLLSAMP3] Follow‐up 3 sample size:

[NSCHOOL] Initial number of schools:

[NSFOLL1] Follow‐up 1 sample size:

[NSFOLL2] Follow‐up 2 sample size:

[NSFOLL3] Follow‐up 3 sample size:

[NCLASS] Initial number of classes:

[NCFOLL1] Follow‐up 1 sample size:

[NCFOLL2] Follow‐up 2 sample size:

[NCFOLL3] Follow‐up 3 sample size:

[MEANAGE] Mean age and standard deviation of overall sample at beginning of intervention:

[GENDER] Gender ⑥ % of males⑥ % of females⑥ 99. Unknown

[LOCAT] Location of programme ⑥ Urban area⑥ Suburban area⑥ Rural area⑥ Mixture of areas⑥ 99. Not enough information to determine

[GRADEX] Grade level of students ⑥ % of students in Elementary school or equivalent⑥ % of students in Secondary school or equivalent⑥ % of students in High school or equivalent⑥ 4. Other:⑥ 99. Unknown

[ETHNI] Predominant ethnicity13⑥ 1. % of Caucasian:⑥ 2. % of Black:⑥ 3. % of Hispanic:⑥ 4. % of Asian:⑥ 5. % of other mixed background:⑥ 99. Unknown

[COUNTRY] Please state the name of the country where schools and sample of students were located when tested.

______________________ (99 if unknown)

[LUNEX] Socio‐economic status

% of students receiving free/reduced school lunch:

99. Unknown

[SENEX] Special Educational Needs

% of students declaring SEN:

99. Unknown

[EXCLUSION] Is the manuscript reporting outcomes for school exclusion?⑥ 1. Yes⑥ 0. No

[TYPEXC] Type of exclusion measured ⑥ 1. In‐school exclusion⑥ 2. Out‐of‐school exclusion⑥ 99. Unknown

[CHEKTIP] Duration of school exclusion measured ⑥ 1. Days of Fixed‐term exclusion(Expressed in number or days, frequencies, percentages)⑥ 2. Days of Permanent exclusion(Expressed in number or days, frequencies, percentages)⑥ 99. Unknown

[ICCEXCLU] If the statistical analysis include cluster in MLM, please register the ICC for Exclusion:

[BEHAVMES] Did the study include measures on behaviour domains? ⑥ 1. Yes⑥ 0. No⑥ 99. Unknown

What types of the following behaviours are measured? [PROSO] Pro‐social behaviour (e.g., helping, empathy). Specify:____________ ⑥ 1. Yes⑥ 0. No

[MEPROSO] Measure(s) used to test the behaviour (name):

[PAGEPROSO] Number of the page from where you extract statistical data:

[ICCPROSO] If the statistical analysis include cluster in MLM, please register the ICC for behavioural outcomes: [INTERNAL] Internalising problem behaviour(e.g., anxiety, depression, attention‐deficit and hyperactivity disorder (ADHD), attention deficit, hyperactivity). Specify:____________ ⑥ 1. Yes⑥ 0. No

[MINTERNAL] Measure (s) used to test the behaviour (name):

[PAGEINTERNAL] Number of the page from where you extract statistical data:

[ICCINTERNAL] If the statistical analysis include cluster in MLM, please register the ICC for behavioural outcomes: [NAEXTERNAL] Non‐aggressive externalising problem behaviour(e.g., stealing, lying, graffiti, illegal drugs). Specify: _____________ ⑥ 1. Yes⑥ 0. No

[MNAEXTERNAL] Measure used to test the behaviour (name):

[ALPHANAEXTER]⑥ Reliability test. Specify alpha value:_____________⑥ Non reported

[PAGENAXTERN] Number of the page from where you extract statistical data:

[ICCNAEXT] If the statistical analysis include cluster in MLM, please register the ICC for behavioural outcomes: [AAGRESEXT] Aggressive externalising problem behaviour(e.g., Opposition/defiance, physical aggression, indirect aggression, instrumental aggressions/dominance, reactive aggression, school bullying). Specify:_____________ ⑥ 1. Yes⑥ 0. No

[MAGRESSEXT] Measure used to test the behaviour (name):

[ALPHAAEXT]⑥ Reliability test. Specify alpha value:_____________⑥ Non reported

[AGRESPAGE] Number of the page from where you extract statistical data:

[ICCAGREEX] If the statistical analysis include cluster in MLM, please register the ICC for Behavioural outcomes:

[DATABAS] Date of baseline assessment:

What measures were used?

[SRMES] Self‐report ⑥ 1. Yes⑥ 0. No⑥ 99. Unknown

[TRMES] Teachers’ report ⑥ 1. Yes⑥ 0. No⑥ 99. Unknown

[SCHRMES] School records ⑥ 1. Yes⑥ 0. No⑥ 99. Unknown

[PAREP] Parents ⑥ 1. Yes⑥ 0. No⑥ 99. Unknown

[OMES] Other:_________________

[EXCBL] Frequency of exclusion at baseline (register any measure given by the study)

This section aims to codify data on the delivery process. Be aware that sometimes final reports do not describe all the data related to delivery. In those cases it would be helpful to search for registered protocols or earlier publications reporting more data on this.

[PRONAME] Name of the programme:

[PROCURRI] Was the programme curricular? ⑥ 1. Yes⑥ 0. No⑥ 99. Unknown⑥ 999. Other. Specify:____________

[PROEND] The programme was conducted for: ⑥ 1. Research ends⑥ 2. Demonstration ends⑥ 3. Routine⑥ 99. Unknown⑥ 999. Other. Specify:_____________

[PROSIT] Primary programme site: ⑥ 1. Public school⑥ 2. Private school⑥ 3. Other, (specify):_________⑥ 99. Unknown

[PROSCH] Was at least one of the components of the intervention was settled at school? ⑥ 1. Yes⑥ 0. No⑥ 99. Unknown

[PRODEL] Who delivered the programme? ⑥ 1. External facilitators⑥ 2. School facilitators⑥ 3. Both⑥ 99. Unknown

[PDBACK] Deliverer's background 1 ⑥ 1. Social worker⑥ 2. Psychologist⑥ 3. Teacher⑥ 4. Police officers⑥ 5. Peers⑥ 999. Other. Specify:_____________⑥ 99. Unknown

[PDBACK] Deliverer's background 2 ⑥ 1. Social worker⑥ 2. Psychologist⑥ 3. Teacher⑥ 4. Police officers⑥ 5. Peers⑥ 999. Other. Specify:_____________⑥ 99. Unknown

[TRAINBEF] Did the deliverer receive training BEFORE implementing the programme? ⑥ 1. Yes.⑥ 0. No.⑥ 99. Unknown

[THOURS] How long was the training in hours?:____________

[TRAINDUR] Did the deliverer receive training DURING the implementation? ⑥ 1. Yes.⑥ 0. No.⑥ 99. Unknown

[THOURS2] How long was the training in hours?:

[PROCONT] What happened to the control group? ⑥ 1. No intervention⑥ 2. Wait‐list control⑥ 3. Minimal contact⑥ 4. Treatment as usual⑥ 5. Alternative treatment⑥ 5. Placebo⑥ 999. Other. Specify:____________

[PROFORM] Delivery format: ⑥ 1. Manualised programme⑥ 2. Unstructured programme⑥ 3. Mixed⑥ 99. Unknown⑥ 999. Other. Specify:____________

What was the programme dosage?

[PRODOSW] AVERAGE Duration in weeks:

[PRODOSH] AVERAGE Hours per week:

[PROFREQ] What was the frequency of the programme counted? ⑥ 1. Less than a week⑥ 2. Once a week⑥ 3. Twice a week⑥ 4. 3‐4 times a week⑥ 5. Daily⑥ 99. Unknown

[EVROLE] What was the “evaluator” role? ⑥ 1. Deliver the programme⑥ 2. Designed the programme⑥ 3. Both design and delivery⑥ 4. Independent evaluator⑥ 99. Unknown

[MONITOR] Was the programme implementation monitored? ⑥ 1. Yes⑥ 0. No⑥ 99. Unknown. Not enough information

[IMPROB] Does the report provide information about implementation problems? ⑥ 1. Yes, there were clear problems which are reported⑥ 0. No, non‐reported problems, reasonably well implemented⑥ 2. Possible problems based on the description of the intervention⑥ 99. Unknown. Not enough information

[PROCOST] Is the cost of the intervention mentioned? ⑥ 1. Yes⑥ 0. No

[AMOUNT] Cost:

[UNITCURR] Currency:

[DATEFALL] Date of follow up:

Multiple follow‐ups

[MONTHFO1] Nº of months from baseline to 1^st follow‐up:

[MONTHFO2] Nº of months from baseline to 2^nd follow‐up:

[MONTHFO3] Nº of months from baseline to 3^rd follow‐up:

[MONTHFO4] Nº of months from baseline to 4^th follow‐up:

What measures were used?

[POSTSR] Children/adolescent self‐report ⑥ 1. Yes⑥ 0. No

[POSTTR] Teachers’ report ⑥ 1. Yes⑥ 0. No

[POSTSR] School records ⑥ 1. Yes⑥ 0. No

[POSTPR] Parents report ⑥ 1. Yes⑥ 0. No

[POSTO] Other:__________

[FREQEXFOLL] Frequency of exclusion at follow‐up (register any measure given by the study)

Effect size: outcomes expressed in continuous data.

[CSSEX] Sample size for the ES (Treatment group)

[CSSCON] Sample size for the ES (Control group)

[MEANEX] Mean (Treatment group)

[MEANCON] Mean (Control group)

[MEANADJ] Are the Means adjusted? ⑥ 1. Yes.⑥ 0. No

[ADJBY] Adjusted by (describe):______________

[SDEX] Standard deviation (Treatment group)

[SDCON] Standard deviation (Control group)

[SEEX] Standard error (Treatment group)

[SECON] Standard error (Control group)

[CORREX] Correlation coefficient + p value (Treatment group)

[CORRCON] Correlation coefficient + p value (Control group)

[SMDTREAT] Standardised mean difference + confidence intervals Effect size: outcomes expressed in dichotomous data.

[DSSTRE] Sample size for the ES (Treatment group)

[DSSCONT] Sample size for the ES (Control group)

[NUMTRE] Treatment group; number of successful cases:

[NUMCON] Control group; number of successful cases:

[PROPTRE] Treatment group; proportion of successful cases:

[PROPCON] Control group; proportion of successful cases:

[ORTRE] Treatment group; odds ratios: Confidence Intervals:p‐value:

[ORCON] Control group; odds ratios: Confidence Intervals:p‐value:

[ORADJ] Are the odds ratios adjusted? ⑥ 1. Yes.⑥ 0. No

Adjusted by (explain):_________

[CHISC] X² value with df:

[PAGEEFFECT] Number of the page from where you extract statistical data: Effect sizes at follow‐up

[ESFOLLOW1] Calculated effect at follows up 1:______

[ESFOLL1] Number of months after intervention for follow‐up 1:______

[ESFOLLOW2] Calculated effect at follows up 2:______

[ESFOLL2] Number of months after intervention for follow‐up 2:______

[ESFOLLOW3] Calculated effect at follows up 3:______

[ESFOLL3] Number of months after intervention for follow‐up 3:______

[ESFOLLOW4] Calculated effect at follows up 4:______

[ESFOLL4] Number of months after intervention for follow‐up 4:______