Health economic analysis

This chapter addresses the key components of a future economic evaluation and reports on the piloting of data collection methods.

Cost of the STASH study

Table 14 provides a summary of the total cost of the STASH study intervention. On average, teachers spent 23.5 hours per school on planning and organisation associated with the STASH study intervention, including attending the training, at a cost of £545 per school or £3270 in total over six schools. The bottom-up costing of £545 for organisation and delivery was very similar to a payment of £500 paid per school by the trial as compensation for teacher time to attend the training. The compensation payment was omitted from the calculation, as this would be double counting.

Time spent on the STASH study reported by teachers varied by school from 8 hours to 39 hours, or from a cost of £185.92 for teacher time per school to £906.36. Once the costs of teacher time were included, the total average cost per student ranged from £38.28 to £45.36 (assuming 8 hours or 39 hours per school, respectively). School sizes also differed, with the smallest school having 46 students per year and the largest having 183 students per year. Assuming similar fixed costs regardless of school size, the average cost per student for a smaller school is £92.56 and for a larger school is £23.27.

The costs shown in Table 14 are exclusive of intervention design and preparation carried out by research team (design of website, design of training manual and training of trainers).

Cost of sexually transmitted infection-related resource use by STASH study participants

Descriptive statistics for sexual health-related resource use and rates of missing data are reported in Table 15. For the intervention year, means (number of visits, purchases, etc.) are reported both for numeric data (mean) and as imputed means, which included numeric responses plus imputed numbers where free text had been given instead of a number (e.g. ‘several times’ or ‘a few’). Overall, the number of missing data was relatively small (4.9–10%), with higher rates seen in the intervention group. Condoms from the NHS were the most common resource used, with 12% (n = 70) using condoms in the control group and 8.8% (n = 45) using condoms in the intervention group.

The total adjusted cost per student of sexual health services was £30.87 in the control group (95% CI £18.68 to £43.24) and £15.64 in the intervention group (95% CI £2.14 to £29.04), with a total cost saving of £15.23 (95% CI –£30.38 to –£0.08; bootstrapped estimate –£12.77, 95% CI –£31.59 to £6.05). When out-of-pocket costs were included, the total mean cost per student in the control group was £32.88 (95% CI £20.61 to £45.31) and £16.06 (95% CI £2.43 to £29.58) in the intervention group, with a total mean cost saving per student of £16.82 (95% CI –£32.40 to –£1.24; bootstrapped estimate –£16.95, 95% CI –£35.75 to £1.85).

When the cost per student of the STASH study intervention (£41.81) is added to the average cost savings per student for sexual health services, the total cost of the intervention is £24.99 per student.

Combining costs for all students with the ‘always safe sex’ outcome, for each additional student classified as ‘always safe sex’ this corresponds to an average cost saving of –£85.72 (95% CI –£107.56 to –£63.88).

Measurement of quality of life and cost-effectiveness

Descriptive statistics for the CHU-9D are reported in Table 16. Young women and students identifying as ‘other’ had significantly lower utility scores than boys, with an average utility decrement of –0.068 (95% CI –0.081 to –0.056) for young women compared with young men, and –0.114 (95% CI –0.162 to –0.066) for students who identify as ‘other’ compared with young men. Students who identified as gay, lesbian, bisexual or other also had significantly lower utility scores than those who identified as heterosexual. There was no difference in utility scores between students eligible for FSM and those ineligible (our marker for potential socioeconomic deprivation).

Mean unadjusted utilities derived from the CHU-9D at each time point are reported in Table 17. There was no significant difference between control and intervention year group utility scores at follow-up in the adjusted multilevel model, including imbalances in gender (–0.010, 95% CI –0.035 to 0.015; bootstrapped estimates –0.010, 95% CI –0.022 to 0.002).

To investigate the reliability of the CHU-9D as a measure of student health-related quality of life, we investigated whether or not it was positively associated with other measures of well-being and self-esteem, as hypothesised. The CHU-9D had a significant positive correlation with the SWEMWBS measure of well-being (0.46, p < 0.0001). For every 1-point increment on the SWEMWBS, there was a corresponding 0.014 (95% CI 0.013 to 0.015) gain in utility on the CHU-9D.

There was also a significant decrease in utility for each step away from ‘strongly agree’ response options for the three measures of self-esteem. For the item ‘I like myself’, there was a utility decrement of –0.047 (95% CI –0.052 to –0.042) for each step from ‘strongly agree’ to ‘strongly disagree’. For the item ‘Most of the time, I like the way I look’, there was a utility decrement of –0.042 (95% CI –0.047 to –0.037) for each step from ‘strongly agree’ to ‘strongly disagree’. For the item ‘I have high self-esteem’, there was a utility decrement of 0.042 (95% CI –0.046 to –0.037) for each step from ‘strongly agree’ to ‘strongly disagree’.

There was no significant relationship between the CHU-9D utility scores and the proposed primary outcome of ‘always safe sex’ (increase of 0.003 in CHU-9D utility scores for each additional case of ‘always safe sex’, 95% CI –0.017 to 0.022; p = 0.775).

Incremental cost-effectiveness ratio, cost-effectiveness acceptability curve and cost-effectiveness plane

The incremental cost-effectiveness ratio for the intervention compared with current practice from the health-care cost perspective is £1303 (–12.77/–0.010) per 1-point gain in utility. Including out-of-pocket costs, it is £1715 (–16.95/–0.010). This is dominated by the control group if the cost of the STASH study intervention is introduced (additional cost per student of £24.99 and a utility loss of –0.010). The cost-effectiveness plane is reported in Figure 15. As most of the values sit in the south-west quadrant, where the intervention costs less but also results in a lower utility, the resulting cost-effectiveness acceptability curve (Figure 16) is difficult to interpret. There is a 93% probability that the intervention is cost-effective for very low values (close to £0) of willingness to pay for a utility, but a 2% probability for a willingness to pay > £6500. The probability of cost-effectiveness decreases as willingness to pay increases because, as willingness to pay for utility increases, the reduction in utility for the intervention year plays a greater role in the probability of cost-effectiveness than in the cost savings.

FIGURE 15 Cost-effectiveness plane for all costs, including the cost of STASH study, out-of-pocket costs and NHS costs for intervention year compared with control year. (Figure not reproduced here; see the original article.)

FIGURE 16 Cost-effectiveness acceptability curve of NHS costs, out-of-pocket costs and intervention costs for intervention year compared with control year. Int, intervention; OOP, out of pocket. (Figure not reproduced here; see the original article.)

If the proposed primary outcome of ‘always safe sex’ was used instead of the CHU-9D, the cost per additional person having ‘always safe sex’ would be £1315, assuming that 85% of students had ‘always safe sex’ in the control group and 87% of students had ‘always safe sex’ in the intervention group, and a cost per student of the intervention of £24.99 compared with the cost savings.

Subgroup analysis

For young men, there was a difference in utility of 0.008 (95% CI –0.025 to 0.042) for the intervention year compared with the control year, with an average cost saving per student of –£10.78 (95% CI –£38.63 to £17.07) for NHS only costs, an average cost saving per student of –£17.78 (95% CI –£46.12 to £10.57) when out-of-pocket costs are included and a mean total of £24.03 per student when the cost of the intervention is added. This corresponds to an incremental cost-effectiveness ratio of £3004 (24.03/0.008) per 1-point change in utility.

For young women, there was a utility decrement of –0.036 (95% CI –0.077 to 0.005) for the intervention year compared with the control year, with an additional cost of £15.61 (95% CI –£8.50 to £39.71) for NHS costs and £15.65 when out-of-pocket costs are included (95% CI –£8.59 to £39.89). The intervention is dominated by the control.

Interpretation of findings

The total cost of the STASH study intervention was approximately £42 per student or £4258 per school for a year group of approximately 100 students. This varied from £23 per student to £93 per student, dependent on the number of students in a year group. In comparison with school size, assumptions about the duration of time teachers spent on the preparation and delivery of the STASH study intervention had less of an impact on potential costs (£38–45 per student). It is possible that the cost of the intervention could be reduced if taken to scale, for example by reducing costs of travel or meals, to result in a lower cost per student.

It was feasible to collect STI health-care resource use data from students. Costs of sexual health services were significantly lower for students in the STASH study intervention year than for control students, with a cost saving of £15.23 per student to the NHS and £16.82 if out-of-pocket costs are included. These data should be interpreted with caution, given that this is a feasibility trial and also because some of these costs relate to preventative behaviours that might represent a saving to the NHS in the longer term. The net mean total cost per student was therefore £25 when NHS cost saving and out-of-pocket costs were incorporated. This was mostly due to a reduction in condom use and pregnancy tests. The reduction in condom use might be because the proportion of students having sex in the past 6 months was lower in the intervention year than in the control year, although the difference is small (19% in control and 17% in intervention) (see Chapter 7). The reduced use of pregnancy tests in the intervention year may be because of an increase in safe sex in the intervention year, although again the difference is small (85% ‘always have safe sex’ in the control group and 87% in the intervention group) (see Chapter 7).

The analysis included costs that occurred over the duration of the trial only. Looking at within-trial costs for all students in the study, each additional case of ‘always safer sex’ corresponded to a reduced cost to sexual health services of £86. Over a longer time horizon, additional cost savings could accrue via reductions in STIs and unintended pregnancies that were not captured over the duration of this intervention.

It was feasible to collect CHU-9D data and we found hypothesised significant relationships with the variables expected (well-being and self-esteem). However, there was no relationship with the proposed primary outcome of ‘always safe sex’.

The small, non-significant, reduction in CHU-9D in the intervention year makes the cost-effectiveness of the intervention difficult to interpret in the standard cost per QALY gained framework recommended by NICE. The suitability of using the CHU-9D to calculate QALYs in a future evaluation of the STASH study is questionable. A key issue for a cost-effectiveness analysis as cost per outcome gained, is that it is hard to envisage what an appropriate effectiveness outcome would be, as costs per STI or averted unplanned pregnancy are difficult to interpret. Averted unplanned pregnancies can have significant long-term savings, with an estimated 10-year cost saving of £23,909 per pregnancy averted (including the cost of health care, social security benefits and child education costs).125 Calculating the cost of STI prevention is more complex because different STIs have different costs and consequences. Furthermore, STIs can be transmitted to other individuals, resulting in additional costs to the system. A systematic review and 1-year statistical model of the costs and QALYs of behavioural interventions to prevent STIs in young people aged 13–19 years found that for a cohort of 1000 boys and 1000 girls a behavioural intervention would avert three STI cases, with an additional 0.5 of a QALY.25 Owing to data limitations and uncertainty around intervention effect size, the authors considered the findings purely illustrative and they were unable to project effectiveness or cost beyond the 1-year time horizon.

The results of public health interventions for which the costs occur in different parts of the system can make policy decisions challenging. For instance, in Scotland, third-sector partners and NHS boards are responsible for ensuring that schools are adequately equipped to teach relationship, sexual health and parenthood education.126 As a result, all parties in the system have a vested interest in ensuring suitable sexual health education. In England, schools are responsible for delivering SRE,127 with discretion as to how they achieve this. Meanwhile, provision of sexual health services falls within the budget of local authorities, contraception costs can fall within both local authority budgets and the NHS, and the costs of unintended pregnancies falls across NHS, local authorities and the Department of Work and Pensions (if social security benefits are included). To understand the relative impact of sexual health education in schools on each of these government departments, it is recommended that costs and benefits to different sectors are reported separately,108 and that a cost–consequences analysis may be more appropriate, although this is not the option preferred by NICE.106 Return on investment, reporting all costs to publicly funded organisations and budget impact analysis, when the costs to specific payers and service providers are reported separately, are other potential options. A budget impact analysis may be attractive given the potential cost savings to the NHS. Different models could be explored, in which the intervention is delivered at a lower cost to schools and no cost to local government. There is no perfect option for analysis of the cost perspective, and the choice will involve a trade-off between capturing all relevant costs but providing no specific answer to any one decision-maker (societal analysis), or specificity to one decision-maker but not capturing costs to other affected stakeholders (health-care cost perspective).

An alternative approach is to model long-term outcomes of STIs and unintended pregnancies related to increases in safe sex. This may be more appropriate to capturing the impact of the STASH study intervention on both costs and consequences. Such a model would allow for cost-effectiveness as cost per QALY to be calculated by including the disutility of STIs in the model. A decision model was beyond the scope of this project, but would be a useful component in a future evaluation of the STASH study.

Any future evaluation will need to collect sufficient, relevant data to facilitate a range of analysis options, so that the results can be reported in ways that are useful to different audiences, including schools, local authorities, the NHS and national policy-makers, such as the Scottish Government and NICE.