Study methods

This chapter describes the methods for the feasibility trial. The study objectives are described in Chapter 1, Aims and objectives. The development of the STASH study intervention (including methods of the development stage) can be found in Chapter 2. A full description of the post-pilot intervention can be found in Chapter 2, The STASH study intervention design for the feasibility trial.

In this chapter we describe the overall study design, and present the progression criteria regarding the feasibility and acceptability of the intervention and evaluation methods (the primary outcome for the trial). We then describe the process evaluation [including social network analysis (SNA)], piloting of outcome measures and economic evaluation. Within each of these sections, we describe the aims, methods, measures and analysis.

Study design and setting

The STASH study was a non-randomised feasibility study conducted with students in six mixed-sex, state-funded secondary schools in Lothian, Scotland, during the 2017/18 academic year (August 2017 to June 2018).

All six study schools received the intervention and provided both intervention and control data. The latter were provided by students in the year above the intervention group and were collected prior to the delivery of the intervention. These were then compared with data from the intervention year group collected 1 year later. Control and intervention participants were therefore the same age when they completed the control and follow-up questionnaire, respectively, but the intervention year group had received the intervention and the control group had not. A flow chart for the study is shown in Figure 5 and a flow chart for participant involvement in the intervention and evaluation is shown in Figure 6.

FIGURE 5 The STASH study flow chart. Reproduced from the STASH protocol, Forsyth et al.62 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/. The figure includes minor additions and formatting changes to the original figure. (Figure not reproduced here; see the original article.)

FIGURE 6 Movement of participants through intervention and evaluation. (Figure not reproduced here; see the original article.)

Participants

Participants were year 4 students (aged 14–16 years) in secondary schools meeting the study inclusion criteria. (In Scotland, secondary school education begins at age 11 years and comprises 6 years, with compulsory education ending after 4 years.)

Intervention participants were students in S4 during the academic year (2017/18) when the STASH study was delivered. Control participants were in S4 in the previous academic year (2016/17).

Inclusion criteria

Secondary 4 students (aged 14–16 years) in state-funded schools who had received, or were receiving, teacher-led sex education, regardless of their sexual experience or individual level of risk, were eligible to participate.

Exclusion criteria

Students attending private or independent schools or state schools not currently delivering comprehensive sex education were ineligible.

The STASH study was designed to augment rather than replace usual sex education. Therefore, to participate, schools had to be already providing SRE. At the time of study, most Lothian schools were delivering a version of the Sexual Health and Relationships Education (SHARE) package85 as part of their personal and social education curriculum. Faith schools were included in the invitation to participate, but none took up the offer.

Sample size

As a feasibility study, the STASH study was not designed to identify an estimate of effect and thus a standard power calculation was not appropriate. We anticipated that 1400 participants would be sufficient to allow qualitative and quantitative progression criteria to be assessed and provide information on key parameters for the design of a future trial, and that a cluster of six schools would enable us to generate an estimate – albeit imprecise – of ICC, when combined with a range of ICC estimates from published evaluations of similar interventions, using the model outlined by Turner et al.86 The average year size for West Lothian is 160 students and, allowing for a non-response rate of 15% due to student absence, we estimated a sample size across six schools of approximately 700 intervention participants and 700 control participants. We thus aimed to recruit six schools.

Selection and recruitment of schools

Schools in the study area were introduced to the STASH study by management staff at West Lothian and South West Edinburgh education authorities. The research team followed up these introductions by e-mail and visited those expressing interest. All interested schools received a STASH study brochure outlining project steps, roles and responsibilities. HTs and DHTs agreed to participate on behalf of the school and usually handed on responsibility for the STASH study to a contact teacher (typically someone with responsibility for personal and social education, physical education or pastoral care). A senior staff member was asked to sign a research agreement, after which point the school was considered formally recruited into the study.

At the end of the evaluation activity, schools received £500 compensation for any disruption caused by the intervention or evaluation activities and an extra £500 to cover backfill costs for teachers to attend sessions such as the STASH study training.

Criteria for progression to a full trial

The main aim of the feasibility trial was to assess the feasibility and acceptability of the intervention and evaluation methods to PSs, the target group and stakeholders. The primary study outcome was whether or not the study met progression criteria regarding the feasibility and acceptability of the intervention and evaluation methods.

The criteria, outlined in Table 1 were discussed within the TMG and agreed via discussion with the TSC after the pilot and before the main feasibility stage. The results reported in Chapters 5–7 are focused on ascertaining whether or not these criteria were met, as well as contextualising the broader experience of the trial in relation to the criteria and study objectives. There was some debate within the TMG around criterion 1. The figure of 60% (for role uptake among nominated students) was set based on learning from ASSIST67 and in view of the need to ensure that PSs had reach or influence across their social networks. However, it was acknowledged that with an older year group in a public examination year and a more sensitive intervention topic, it would be harder to recruit PSs in the STASH trial than in ASSIST. The question of whether or not 60% was still required in an intervention engaging social media as well as face-to-face interaction was also raised. These issues are discussed in Chapters 5 and 6 and revisited in Chapter 9.

A traffic light system was adopted, in which meeting the green target was a very strong indication to proceed, amber was a medium indication to proceed and red an indication of doubt (i.e. whether or not to proceed). Amber and red targets required discussion with TSC and an identified mitigating strategy. In the case of a red, other indicators should be amber or green to proceed. Table 1 describes the criteria, target, measure, method and time of measurement.

Process evaluation

Purpose of process evaluation

The process evaluation was undertaken primarily to assess whether or not progression criteria were met (see Table 1), but more broadly to evaluate the following:87

• Implementation: what was put in place and exactly how was it implemented?
• Mechanisms of impact: how did the intervention produce change?
• Context: how did contextual factors shape implementation and outcomes?

In line with the study objectives, we sought to assess (1) the extent to which it was possible to recruit and retain PSs, (2) the acceptability of the role to PSs and key stakeholders and (3) the extent of intervention fidelity (i.e. whether or not PSs, trainers and schools were able to carry out the intervention activities as intended). The process evaluation also sought to identify barriers to and facilitators of successful implementation, to contribute to refining and testing the programme theory and to investigate the potential and limits of using social media to diffuse messages. We also evaluated the extent of friendships outside the school and therefore estimated potential contamination in a full-scale cluster randomised trial.

We undertook a SNA as part of the process evaluation. This sought to assess the potential reach of PSs across the S4 year group, by examining their position within the year group network. We wanted to understand the extent to which the nomination and recruitment process had resulted in PSs who could serve as ‘information hubs’ (network centrality) and who, as a group, could span an effective number of friendship subgroups within the year.

We also used SNA to investigate how friendship interactions were shaped and sustained by network properties, as well as individual characteristics, before and after the intervention. The analysis highlighted how perceived friendships interacted with the PS role. We also looked at homophily, that is whether subgroups within each school network were formed around gender (e.g. did young women tend to be friends with young women?) or sexual behaviour (did sexually active students mainly befriend other sexually active individuals?), and whether or not students influenced each other’s behaviours.

Process methods

We undertook basic process evaluation activities in all six schools and enhanced data collection in four ‘case study’ schools. One school was excluded as a case study school, as it had recently been part of a highly intensive research study, whereas another was specifically included because we were interested in exploring intervention feasibility in a very small school. In selecting two case studies from the remaining four schools, we sought balance in terms of urban/periurban location, school size and proportion of free school meals (FSM) (see Chapter 4).

Evaluation activities for all schools comprised:

• A monitoring log (spreadsheet) of all intervention activities (e.g. attendance at training and follow-up meetings by gender, field notes from school visits and informal conversations between research team/teachers).
• An anonymous evaluation questionnaire on PS training, completed by PSs (n = 104) and school staff (n = 12). It was completed immediately following training and capturing views on training activities, students’ perceptions of what they gained and staff views of content and organisation.
• An anonymous online questionnaire for all PSs (n = 88, 85% response rate). It was completed in a computer room during the final follow-up session. PSs received a £10 voucher on completion. Data were captured on the specifics of PS activity and engagement (see Measures used in process evaluation).
• Interviews with the STASH study trainer (n = 3). Conducted separately with each trainer, these addressed perceptions of intervention components (including what worked and what did not work, and why), their role and potential future improvements.
• Reflective record of all activity sessions (recruitment, training and follow-up) completed by trainers (n = 48). These captured student attendance and engagement, and any modifications made to the session plan.
• PS activity log (spreadsheet) recording Facebook group membership (by gender) and posting activity. Anonymous data were extracted via the STASH study trainer account (with PS consent obtained at the intervention’s outset). The number of face-to-face conversations self-reported at follow-up sessions was also logged (extracted from trainer reflective records).
• Items on a follow-up questionnaire assessing engagement in the STASH study intervention activities, views on acceptability of format and content.
• Items on friendship ties from baseline and follow-up questionnaires for SNA (see Measures used in process evaluation).

Enhanced process evaluation activities comprised:

• Structured observation conducted by the research team at recruitment (n = 4), PS training (n = 8) and follow-up (n = 8). A non-participant observer sat in a discreet place and, using an observation guide, recorded data on fidelity (activities carried out as intended), acceptability (apparent responses of/interaction with students, challenges arising), exposure (degree of student engagement) and any relevant contextual information.
• Semistructured interviews with PS and non-PS students (11 groups/pairs, n = 42 participants). PS interviewees were selected to give a mix of gender and engagement (whether using Facebook or face-to-face interaction only). Non-PS students were recruited via PSs (with support from the contact teacher) as we were interested in understanding views of students who had some exposure to the intervention activities. Interviews addressed awareness of the STASH study, acceptability and engagement with messages and perceptions of impact.
• Semistructured individual or paired interviews with school staff and the STASH study contact teacher plus a member of senior management team with pastoral responsibilities and some awareness of the STASH study intervention (seven interviews, n = 8 participants). Interviews focused on perceived value and impact of intervention, and potential barriers and facilitators from the schools’ perspectives.

Measures used in process evaluation

The PS questionnaire (see Report Supplementary Material 2) focused on PS experiences regarding the feasibility and acceptability of the role. It comprised a wide range of items, measuring PS activity, time spent on the role, how easy or difficult they found the role, extent to which they were able to meet challenges and difficulties (including technical issues), views on the role and the STASH study website and training content, perceptions of parent and teacher views on their involvement, how their peer group responded and interest in ongoing involvement.

The process evaluation also used measures from the baseline and follow-up questionnaires (see Piloting of outcome measures). These focused on measuring exposure to the STASH study activities among all students; the acceptability of intervention content and activities among those exposed; the use of Facebook; whether the STASH study was more or less relevant, helpful or interesting compared with usual classroom SRE; whether or not it was age appropriate; and the likelihood of using the STASH study website or talking to a PS in future.

For the SNA, students were asked to name up to six friends and for each to give information on gender, if in year group, if connected on Facebook, if they would share private information with the person, and if time spent together was in school, outside school and/or online (yes/no options).

Analysis of process data

The process evaluation framework details data sources for all process research questions, whereas the qualitative analysis plan gives more detail on the analysis (see Report Supplementary Material 3). Quantitative data from the follow-up questionnaire (see Report Supplementary Material 4) and PS questionnaire (see Report Supplementary Material 2) were summarised descriptively and stratified by gender, as relevant. A descriptive summary was produced from the PS activity log (Microsoft Excel^® file; Microsoft Corporation, Redmond, WA, USA), collating group membership numbers, total posts, total reactions to posts and reported face-to-face activity. Information on Facebook group membership was documented by school for the SNA.

Project monitoring data were entered into a different Excel document and subsequently summarised descriptively. Data included specifics on PS nomination, recruitment, training, attendance at follow-ups and completion, broken down by gender and school; contemporaneous events, such as relevant news stories; significant communication with schools; and key school information.

Transcribed qualitative data from observations (recruitment, training and follow-up) and interviews (teachers, trainers, PSs and non-PSs) were entered into NVivo 11 qualitative analysis software (QSR International, Warrington, UK) to manage data retrieval. We undertook a thematic analytic approach informed by the framework method.88 A coding framework was developed and applied to transcripts, based on the key aims of the process evaluation [i.e. fidelity, acceptability, exposure, reach, contextual factors (including barriers to and facilitators of implementation), plus researcher reflections on the process of implementation]. From this descriptive stage, data were interpreted further to establish links between themes, develop potential explanations around such links and to better understand the functioning of the intervention. Quotations were selected both to exemplify and illuminate key findings, and to illustrate variation in participant experiences and perspectives. The primary aim was to assess the overall feasibility of the intervention, based on analysis of these constituent themes.

Social network analysis

We conducted a SNA study to evaluate the potential reach of PSs across their social networks. From a SNA perspective, a network is composed of a set of nodes and a set of ties connecting some of the nodes. In this case, nodes corresponded to students in school and a tie appears each time there is a reported friendship relationship among them.

In principle, it can be assumed that each node is capable of connecting with anyone else in its network. However, individuals tend to have preferred patterns of connections because they tend to enjoy some ties more than others. These preferences, usually reciprocated, generate a specific pattern among all members of the network.

In general, SNA measures and routines help in the rigorous and systematic visualisation of the distribution of preferences represented by a tie among pairs of individuals and in testing and exploring the properties of the network (e.g. closeness, cohesion and clustering).

In the present study, students were asked to name up to six friends and for each to give information on gender, if in year group, if connected on Facebook, if they would share private information with the person, and if time spent together was in school, outside school and/or online (yes/no options).

First, we used a direct reach measure, that is the number of students directly connected to a PS (% of the total) to evaluate the potential reach of PS through their direct friendship ties. Second, we measured target reach, corresponding to the number of friendship clusters containing a PS, expressed as a percentage, and revealing the distribution of PS within friendship subgroups. In this case, we applied the algorithm developed by Girvan and Newman89 based on the identification of ties whose removal contribute to the breakdown of the network in small subgroups. Third, we applied standard SNA centrality measures to compare (1) students who were nominated but declined the PS role, (2) students who were nominated and took up the PS role and (3) other students. In particular, we used indegree, two-step reach90 centrality and eigenvector centrality. Indegree corresponds to the number of incoming ties, that is the number of times an individual was named as a friend. This can be interpreted as a measure of popularity, as high indegree scores indicate students who have been named as a friend by many others. The two-step reach centrality measure indicates the potential of PSs to spread the STASH study information with other students within two steps (e.g. via word of mouth). This measure examines how close each student is to all the others. Eigenvector centrality indicates how well connected an individual is by considering how well connected their friends are. This measure can also be considered as an indication of popularity: students with high eigenvector centrality are connected to other students who are themselves well connected. Table 2 summarises the measures and their visual representations.

To analyse changes in ties from baseline to follow-up, we applied a stochastic actor-oriented model for network dynamics.91–93 These classes of statistical simulations are commonly used to test hypotheses regarding the co-evolution of behaviours and networks related to a set of individuals. In our case, we used stochastic actor-oriented models to analyse the formation of ties among students considering individual characteristics (gender, being a PS or not being a PS), behavioural characteristics (sexual experience, self-assessed confidence to refuse unprotected sex, self-assessed knowledge of STI risk) and the influence of PSs in promoting preventative healthy sexual behaviours.

Each network was analysed separately and then a Fisher’s test (which combines independent p-values of parameter estimates and standard errors across the networks) was used to determine the presence of significant differences among schools. We used meta-analysis to aggregate the resulting values of the parameter estimates for each school and compare them on a common scale (see Appendix 3). Finally, we used the goodness of fit test to examine whether or not our data samples fit the simulated networks.94 These analyses were conducted in the simulation investigation for empirical network analysis package (version 1.2–14, released 5 December 2018) in the R statistical software (The R Foundation for Statistical Computing, Vienna, Austria).

Piloting of outcome measures

Purpose

A key study objective was to identify key trial design parameters for a future evaluation, including candidate primary and secondary outcomes, ICC and sample size. Since setting this objective, the DELTA2 guidance has been published, which advises against using estimates of effect from pilot studies to calculate the sample size for a full trial because of their imprecision.95 We therefore did not seek to fully meet this objective. The purpose of collecting these outcomes was to pilot our outcome measures and analysis methods to assess the utility of these for a future definitive trial. The quantitative analysis was not intended to offer any guide to the intervention’s effectiveness due to lack of statistical power.

Outcome methods

The control questionnaire was administered first (March–June 2017) to the S4 year group above the intervention cohort. We administered the baseline questionnaire in August/September 2017 to the intervention cohort, who had just moved up to become the new S4 year group. We administered the follow-up questionnaire to this same S4 group post intervention (March–June 2018). Compared with the control questionnaire, the baseline questionnaire had a reduced set of outcome measures and the follow-up questionnaire had additional exposure measures.

Outcome measures

Baseline characteristics

Student characteristics describing the study sample were collected at baseline. These were:

• gender
• age
• SIMD
• residence
• ethnicity
• number of examinations being taken
• plans for leaving school
• views on religion.

School characteristics were collected at baseline for each school:

• total number of students (student roll)
• urban/rural
• FSM (per cent)
• National 5 examination results.

Potential primary and secondary outcome measures

The end outcomes for the trial we sought to test were:

• delayed initiation or abstinence from sexual activity
• consistent condom use among those who are sexually active.

We considered a range of potential primary outcome indicators to set an indicative primary outcome for use in any subsequent trial. Given that sexual behaviour would be expected to increase from baseline to follow-up due to maturation effects, the primary comparisons were between intervention and controls at follow-up.

Potential primary outcomes collected were:

• condom use at last vaginal intercourse
• condom or dental dam use at last oral sex
• number of sexual partners in last 3 months and whether or not a condom was used
• frequency of condom use in the last 6 months
• proportion of students who have not had sex in the past 6 months or have not had sex at all.

Potential secondary outcomes collected were:

• STI prevention and sexual health-related knowledge (drawn from, and adaptations of, existing survey items, e.g. Sexunzipped96), including knowing where to seek help locally (adapted from the Healthy Respect study97)
• ease of talking about sex with parents/carers and friends [adapted Natsal measure;98 URL: www.natsal.ac.uk/natsal-3/questionnaire.aspx (accessed 5 May 2020)]
• ease of talking about sex with a boyfriend or girlfriend [adapted Natsal measure;98 URL: www.natsal.ac.uk/natsal-3/questionnaire.aspx (accessed 5 May 2020)]
• confidence in STI prevention skills (adapted from range of existing survey items, e.g. Sexunzipped96)
• ‘competence’ at first intercourse or competence at last intercourse [adapted from Natsal measure;99 URL: www.natsal.ac.uk/natsal-3/questionnaire.aspx (accessed 5 May 2020)]
• sexual attitudes and adherence to sexual health norms (12 new items, adapted from a range of existing survey items)
• perception of whether or not others are sexually active (newly designed item)
• intimate relationship well-being (seven newly designed items)
• distress about sex life (adapted Natsal measure100)
• use of internet and social media for finding sexual health information, sexting and viewing sexual images (six newly designed items)
• Short Warwick–Edinburgh Mental Well-Being Scale (SWEMWBS)101
• conversations about STASH study-related topics (10 newly designed items)
• self-esteem – two items from the Randomised Intervention trial of PuPil-Led sex Education (RIPPLE) study,33 plus a single-item global measure102
• self-reported sexual activity.

Effect modifiers

A number of effect modifiers were measured. These included the student characteristics (see Baseline characteristics), along with:

• self-reported risk behaviours in peer group
• school climate and engagement103 (items drawn from the Beyond Blue School Climate Questionnaire)
• parental monitoring (three newly designed items, drawing from existing measures)
• sexual attraction (modified version of the Kinsey scale) and sexual identity (adapted item from Natsal)
• self-regulation104 (three items drawn from the 36-item Adolescent Self-Regulatory Inventory104)
• importance of social media to social life (two newly designed items).

Student roles and exposure

Derived variables were created to categorise students by their role in the intervention and by their level of exposure. These measures were used both in the process and outcome analysis. They are summarised below.

Intervention identification (intervention group only)

• PS: those who attended training, regardless of whether or not they completed the role.
• Nominee: nominated and invited to recruitment meeting, but did not attend training.
• Other student: neither of the above.

Level of exposure (intervention group only)

• Trained PS.
• Exposed non-PS student reporting one or more of the following:
  • been shown the STASH study website by a PS or accessed it themselves
  • joined, viewed posts or followed links in a STASH study Facebook group
  • talked to friends about STASH study Facebook posts
  • talked to or asked a PS about a STASH study topic.
• Unexposed: non-PS student who did not meet any above condition.
• Unknown: non-PS student who did not complete the follow-up questionnaire (meaning exposed/unexposed status could not be assigned).

Development of measures over subsequent survey administrations

As a key purpose of the outcome analysis was to pilot measures, a small number of changes were made across subsequent survey administrations. Some items were improved, for instance the item on gender became two stage to allow students to report sex at birth and current gender identity. Minor modifications were made to response options (e.g. separated out ‘never or almost never’ options for items on parental monitoring) and small tweaks were made to instructions or explanations (e.g. ‘a penis in a vagina’ added as explanation to item on sexual intercourse in the follow-up questionnaire). Other changes were substantive. For instance, the question bank on ‘competence’ (extent to which sex is wanted, protected, not under influence of drugs/alcohol and not regretted) was switched from being asked about first sex to most recent sex, because the number eligible to report on their most recent sexual encounter was much larger.

Outcome data analysis

The full details of the outcome measures are given in the statistical analysis plan (see Report Supplementary Material 5).

Continuous variables were summarised by the number of observations, number of missing values, the mean and standard deviation (SD), the median and interquartile range, and the minimum and maximum values. Categorical variables were summarised by the number of observations, number of missing values and the number and percentage of subjects who are in each of the categories. No missing data were imputed.

Baseline student characteristics were summarised overall and by study group. Characteristics were also summarised separately for those who did not complete the follow-up survey, to investigate any systematic dropout. School characteristics were summarised overall and listed for each school. Feasibility and acceptability outcomes were summarised overall and by school. Outcomes collected for both the control and intervention groups were summarised by study group.

Questionnaire validation was carried out for the potential primary and secondary outcomes. Principal component analysis (PCA) was applied to the entire questionnaire (at baseline) to assess whether or not there was a small number of individual domains that explained the majority of the variability in the data set. PCA was also applied to items with certain question banks (groups of questions assessing the same topic or underlying construct). Principal components representing clear question domains (i.e. combinations of questionnaire items) were taken forward into the primary and secondary outcomes analysis. Validated measures were not included in the PCA process.

To assess the effect of the intervention on the potential primary outcome, mixed-effect logistic regression models were fitted to study group, with a fixed effect for study group and a random effect for school. Similar models, adjusted for appropriate effect modifiers, were run to identify the factors that had an effect on the outcome. This helped indicate whether or not future studies should adjust for such factors.

The model-estimated odds ratios (ORs) and corresponding 95% confidence intervals (CIs) were reported for the intervention effect and any effect modifiers, for each of the models. The ICC is a key parameter that the study sought to estimate to inform decisions about the sample size for a full evaluation of the STASH intervention. This is based on generalised linear mixed models fitted to the primary outcome, with a random effect for school and no fixed effects. Models were fitted to the data from all follow-up and control participants and separately by study group. Sample size estimates were provided for all ICCs estimated, as estimates of the ICC can vary between study groups. The ICC was estimated using methods detailed in Wu et al.105 and implemented using R code [URL: https://github.com/timothyslau/ICC.merMod/blob/master/functions.R (accessed 5 May 2020)].

These analyses were exploratory and focused on identifying appropriate outcome measures. As the study lacks statistical power, little value can be given to the estimates of difference between the control and intervention groups. Furthermore, there were notable differences between the control group and intervention group, particularly in relation to gender, which would be much less likely to occur in an appropriately powered study. Therefore, estimates of effect are liable to be affected not only by random error, but also by confounders (including unmeasured factors), which cannot be fully controlled for.

Economic evaluation

Aim

The aim of the economic evaluation was to determine the key components of a future cost-effectiveness analysis and test data collection methods.

Measures

To calculate STI-related health-care costs, we assessed the feasibility of collecting self-reported health-related health-care resource use (STI testing and treatment, contraception, including out-of-pocket costs) at follow-up. This included contacts with school specialists, health-care professionals in STI services and primary care, STI treatment and prevention, and pregnancy-related costs. Control and intervention students were asked at the end of S4 to report the number of times they had used each service in the past 6 months. We report descriptive statistics, including missing data, to evaluate the feasibility and acceptability of collecting this information.

The National Institute for Health and Care Excellence recommends that evaluations of new interventions include a cost-effectiveness analysis: the incremental cost per unit gain in an effectiveness measure. Quality-adjusted life-years (QALYs) calculated using the EuroQol-5 Dimensions (EQ-5D) are the preferred effectiveness measure in cost-effectiveness analyses.106 There is no preference-based tariff for the EQ-5D that can be used to calculate QALYs in children. Instead, we explored the Child Health Utility 9D (CHU-9D) to calculate utility scores,107 completed at baseline and at the end of S4 in control schools, and at the end of S4 in intervention schools.

Quality-adjusted life-years are not always the most suitable effectiveness measure in economic evaluations of public health interventions.108 We evaluated if the CHU-9D was sensitive to changes in the potential primary outcome, to determine its suitability in an economic evaluation of the STASH study. We also report descriptive statistics for the CHU-9D and its relationship with (1) SWEMWBS and (2) self-esteem, to provide a picture of how it functions within this population group.

Calculating the cost of the STASH study

The key cost of the STASH study is the cost associated with the identification and training of the PSs, as well as trainer-led activities (e.g. follow-up sessions). Based on data collected, we calculated the average number of hours spent by trainers in each activity. This was multiplied by the hourly cost of a trainer to give the average cost per trainer. Any additional costs of the training, in particular the cost of the 2-day training, including room hire and refreshments, was also collected and included as part of the total cost per trainer. We also collected information on the amount of time spent by teachers on STASH study-related activities, including booking rooms, co-ordinating with other teachers and attending sessions. Teacher time spent on the STASH study was multiplied by average hourly gross earnings of a UK secondary school teacher in 2017 of £23.24,109 to calculate the total cost of teacher time spent on the STASH study. This is considered the ‘opportunity cost of STASH’. Although it is not a direct cost to the school, it is a cost as teachers could have spent this time doing other education-related activities, which may have had a greater or lesser benefit to students than time spent on the STASH study.

The total cost of PS training plus the total cost of teacher time was divided by the number of students in the intervention arm to obtain the cost per student of the STASH study. Ninety-five per cent CIs were calculated based on 3000 bootstrap replications.

Calculating the cost of sexually transmitted infection-related health-care use

Resource use for each student was multiplied by unit costs from the most recent Unit Costs of Health and Social Care published by the Personal Social Services Research Unit,110 reference costs111 and other published sources, when required (Table 3). To calculate the mean difference in costs between control and intervention students, we used a general linear model with random effects for school and controlling for imbalances between control and intervention groups. Ninety-five per cent CIs were calculated based on 3000 bootstrap replications.

Quality of Life

Descriptive statistics are reported for CHU-9D utility scores by gender, sexual orientation and FSM (a proxy for socioeconomic status).

Descriptive statistics for the mean utility values by intervention and control students, and at each time point, are also reported. The difference between control and intervention students was calculated using a multilevel model, with time and the intervention as a fixed effect, including schools as a random effect and adjusting for imbalances between control and intervention groups.

Multilevel model linear regression models accounting for individual and time were used to describe the linear relationship between the CHU-9D and (1) well-being (SWEMWBS) and (2) self-esteem. These were also used to test the sensitivity of the CHU-9D in changes in the proposed primary outcome and, hence, its suitability as an effectiveness measure in a cost-effectiveness analysis of the STASH study.

Analysis

Cost-effectiveness analysis

Although the study was not designed as a confirmatory trial, an analysis was conducted to provide an initial estimate of the probability that the STASH study intervention is cost-effective compared with current practice from a health-care and wider cost perspective.

We estimated the mean incremental cost per 1-point gain in utility of the intervention compared with control from a health-care cost perspective, in line with the NICE guidance106 on economic evaluations. Costs and utilities are from the bootstrap adjusted analyses. A secondary analysis included the cost of the STASH study and out-of-pocket costs.

We used the bootstrap results to calculate the cost-effectiveness acceptability curve: the probability that the intervention is cost-effective compared with control for a range of values of willingness to pay for a gain in utility. We also reported a cost-effectiveness plane of the bootstrap results.

Missing data

In the follow-up questionnaire, free-text responses were allowed for the number of times a sexual health resource was used. As a result, students erroneously entered responses, such as the clinic they used, or entered the response ‘a few’, rather than the number of times that they used that service or resource. For each variable when this occurred, the results for that resource were interrogated and the most likely number was selected; for example, if the student entered a clinic name for resource use and the mode for that response was ‘1’ then the value ‘1’ was entered, and if they entered ‘a few’ and the mode was ‘2’ then the value ‘2’ was entered.

All analyses are complete case.

Subgroup and sensitivity analyses

We conducted a subgroup analysis for gender (boys and girls). Different assumptions regarding the cost of the STASH study are included as sensitivity analyses.

Ethics and governance

Ethics approval

Ethics approval was granted by the University of Glasgow Medical, Veterinary and Life Sciences College Ethics Committee in two stages:

1. intervention development/pilot, project number 200150088
2. feasibility study, project number 20160002.

Written consent was sought from all participants taking part in research activities, using information sheets and oral explanations by researchers. The exception was observations made at PS activities; here, participants were given a verbal explanation of the researcher’s presence and advised that no identifying information would be recorded. All students and their parents in the S4 intervention group were sent information on the STASH study. Parents and carers were given the opportunity to opt out their child from the survey and qualitative interviews. PSs and their parents gave written (opt-in) consent to take on the PS role. PSs were advised that the research team would have access to activity in secret Facebook groups to log activity (such as the number of posts and ‘likes’), but that no personal data would be recorded. At all study stages, participants were informed that they could withdraw from any research component at any time without prejudicing their experience at school. Research participants were reassured that their answers would be treated in confidence. In qualitative interviews, participants were advised that the researcher would only break confidentiality if they made a disclosure suggesting that they might be at risk of serious harm or of harming others. The intervention website included links to local and national referral services for sexual health and well-being, as well as child protection services.

All researchers and fieldworkers working in schools had Protecting Vulnerable Groups clearance.

Assessment of harms

The intervention was low risk to participants. The team anticipated that the main risk would be psychological harm resulting from online bullying and breaches of private information by peers. The issue of ‘harm’ was investigated in the process evaluation. Procedures were developed for reporting trial protection concerns in schools (see Report Supplementary Material 6 and 7). The former was designed in consultation with the child protection lead for West Lothian Health Education Authority and the latter in consultation with the TSC.

A number of steps were taken to mitigate the possibility of harm: PSs were required to sign up to a code of conduct (the STASH study charter); social media use was confined to private (‘secret’, i.e. non-visible, invite-only) Facebook groups, of which a STASH study trainer was a member; trainers undertook monitoring ‘spot checks’ and PSs were encouraged to report inappropriate posting to the trainer and/or STASH study contact teacher; PSs had the option of privately messaging the trainer with any concerns; and trainers were advised to pass on any concerns to the contact teacher without delay. Procedures for dealing with inappropriate online behaviour followed the information and communication technologies code of conduct and discipline code of participating schools.

Peer supporters were trained on how to respond to any sensitive disclosures from their peers (such as respecting privacy) and on when to share a disclosure with the staff contact teacher (or school teacher designated to handle child protection issues). They were advised that any disclosures that made them feel worried or uncomfortable should be reported without delay to the contact teacher/designated member of staff. After this point, the school procedure for handling disclosures relating to child protection would be followed.

In terms of reporting ‘harms’ to the TSC and funders, the procedure set out that if alerted to a harm, the principal investigators would triage according to severity and report to the TSC and funders as appropriate. If a harm was reported for which causality by intervention was deemed possible, probable or definite, the principal investigators would communicate to the TSC within 15 days.

Governance

Trial management, data monitoring and quality assurance

The study was overseen by a TSC, consisting of an independent chairperson and three other independent members, including a lay representative (a parent and sex education specialist) and the two principal investigators (LM and KRM). The TSC met three times in total to examine the proposed methods, assess progress following the pilot and assess whether or not emerging trial results suggested that progression to full-scale evaluation was warranted. With the agreement of the TSC, no separate Data Monitoring and Ethics Committee was established because the study was deemed low risk, non-randomised and did not include interim analysis. The TSC were asked to cover the functions of the Data Monitoring and Ethics Committee, in particular in relation to ethical issues, and monitor any unintended outcomes and the continuation of the trial. The TMG met 6-weekly for the duration of the study and comprised the principal investigators, the co-investigators, study manager (RF) and process evaluation lead (CP).

Patient and public involvement

The study was co-produced with young people and professional stakeholders via an extensive series of consultations and development workshops (see Chapter 2). Working with a local youth organisation, we established a Young People’s Advisory Group to advise on study design. The group met once in the initial phases of the project. A subsequent meeting was planned, but proved too difficult to arrange because many of the original members were no longer in contact with the youth group. Instead, the study sought input from teachers and students involved in the pilot to optimise the feasibility study stage. In terms of study oversight, our TSC included a lay member who had a strong professional interest in young people’s sexual health but no training in the evaluation of complex interventions.