Economic evaluation

Parts of this chapter have been reproduced from Jago et al.54 © 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

Aims

This chapter describes the methodology and results of an economic evaluation of the Action 3:30 intervention. Drawing on components of the RE-AIM framework,12 the aims of the economic evaluation of the project were to:

• assess the potential for examining cost variation in Action 3:30 delivery across school settings (implementation)
• assess the potential for change in HRQoL (effectiveness)
• assess the potential for sustaining the outcome of the intervention from a cost perspective (maintenance).

This short-term analysis estimates the cost-effectiveness and cost-utility of the Action 3:30 intervention, compared with no active intervention, over the 1-year period of the feasibility study. For these analyses, it was assumed that there were no differences in costs or intervention effects between the intervention arm and the control arm beyond the 1-year follow-up period. The intervention occurred over the course of one school year, and, as a result, costs were not discounted. Analysis was conducted in SPSS (Statistical Product and Service Solutions) version 22 (IBM Corporation, Armonk, NY, USA).

Methods

Costing methods

To assess the feasibility of collecting cost data in a definitive trial, the project team used a resource use checklist, adapted from previous studies,18,57 to collate data on the resource use and actual costs incurred by TAs over the 1-year period of the feasibility study (September 2017–August 2018). In the UK, many primary schools hire external agencies to provide extracurricular physical activity19 and therefore identifying the potential cost of an in-house extracurricular club delivered by paid TAs is important. Prices were drawn from time sheet data collected by the project team (e.g. printing costs, sports equipment costs and venue hire costs) and from published and established sources. Costs were categorised according to stage of programme delivery and were stratified by school as follows: one-off training resources, recurrent programme preparation resources and recurrent programme delivery resources. Consistent with methods used in the Bristol Girls Dance Project,57 recruitment and marketing costs were identified separately because they depend on the implementation context for participation in each school setting.

Cost-effectiveness

To estimate the cost-effectiveness of Action 3:30 compared with no active intervention, objectively measured follow-up MVPA for the intervention arm and the control arm and data collected on Action 3:30-related resources and costs were examined. The cost of Action 3:30 producing an additional minute of weekday MVPA compared with no active intervention was estimated by dividing the cost per pupil by the difference between the intervention arm and the control arm weekday minutes of MVPA at follow-up (incremental cost-effectiveness ratio). Average weekday minutes of MVPA at follow-up were calculated using multivariable mixed-effects linear regression, adjusting for intervention arm, local authority, recruitment strategy and baseline mean minutes of weekday MVPA.

Health-related quality of life

To assess the potential for change in HRQoL as a result of Action 3:30, pupils were asked to complete the KIDSCREEN-1040,41 and the CHU9D42,43 measures at baseline and follow-up.

KIDSCREEN-10

KIDSCREEN-10 is an instrument for measuring the well-being and HRQoL among children and adolescents aged 8–18 years. It has 12 items, 10 of which start with the stem ‘Thinking about the last week . . .’ (e.g. ‘. . . have you felt lonely?’). Responses are given on a 5-point Likert scale from 0 (not at all) to 5 (extremely). Item 11 assesses whether pupils have a long-term disability, illness or medical condition (response options of yes or no; if yes, which one), and item 12 assesses general health, with five response options ranging from 1 (poor) to 5 (excellent).

KIDSCREEN-10 data were handled in accordance with the KIDSCREEN handbook.60 Negatively formulated items (‘Have you felt sad?’ and ‘Have you felt lonely?’) were recoded from 1 to 5. All items were then summed to generate T-scores with a scale mean of around 50 and a SD of around 10. A higher T-score was indicative of better HRQoL.

Child Health Utility 9D

The CHU9D is a preference-based instrument for measuring HRQoL in young people. It consists of nine dimensions: worried, sad, pain, tired, annoyed, schoolwork/homework, sleep, daily routine, and ability to join in activities. There are five levels representing increasing levels of severity within each dimension (e.g. I don’t feel worried today, I feel a little worried today, I feel a bit worried today, I feel quite worried today, I feel very worried today).

Responses to the CHU9D were converted into utility scores42 ranging from 0.00 (equivalent to being dead) to 1.00 (perfect health). A higher utility score was indicative of better HRQoL.

After-school physical activity provision

To compare the costs and resources associated with Action 3:30 with those of existing extracurricular club provision at participating schools, at baseline and follow-up key contacts at each participating school were asked to provide a description of each existing after-school club, including club duration and cost to the school and parents/guardians.

Results

Costs associated with the Action 3:30 intervention

The Action 3:30 intervention was delivered in four schools. The total cost of the programme, including training, preparation and delivery costs, was £7422.15, with an average cost per school of £1855.55 (based on 2017–18 prices) (Table 28). To estimate the costs associated with mainstream delivery of the intervention after 1 year, training costs were excluded, reducing the total indicative cost to £5929.73 and the average cost per school to £1482.44.

One-off training costs were £1494.42, with the cost of the lead trainer the biggest contributor (£750.00), followed by teaching cover costs to release TAs for training (£442.42) and costs associated with venue hire (£300.00). Recurrent programme preparation costs comprised printing costs and the costs of sports equipment, totalling £1113.83 across the four participating schools. Recurrent programme delivery resources were estimated at £4815.90, with programme delivery by TAs the main contributor to the indicative total cost (£3828.30). This involved 240 hours of intervention delivery time and equated to just over 50% of the total indicative cost of Action 3:30. Recruitment and marketing costs were identified separately and totalled £3560.67. These included the costs of the printed information sheets and consent forms issued to all eligible students, costs of the thank-you gifts for participating (frisbees and bouncy balls) and the cost of hiring trained coaches to deliver the taster sessions for the enhanced recruitment method.

The total cost of Action 3:30R per pupil, including training, preparation and delivery costs, and based on a class of 30 pupils attending 30 1-hour sessions, was £61.85 (95% CI £55.44 to £68.26). The estimated total cost per pupil per individual session was £2.06 (95% CI £1.85 to £2.28). After 1 year, when training costs were excluded, the total cost per pupil was reduced to £49.41 (95% CI £43.00 to £55.83) and the estimated total cost per pupil per individual session was £1.64 (95% CI £1.43 to £1.86).

The indicative total cost of Action 3:30R per pupil (£62) was comparable with that reported in the earlier Action 3:30 feasibility trial (£81), and the mainstream cost per pupil after 1 year is almost identical to that reported in the earlier Action 3:30 feasibility trial (£49 per pupil).18 Few other studies report the costs of children’s physical activity, but these estimates are comparable with those that have been reported in recent years.57,61,62

Cost-effectiveness

As this was a feasibility trial and not powered to detect significant differences in MVPA, there was no firm basis for creating a cost-effectiveness ratio to compare the change in costs with the change in outcomes (MVPA) through a full economic evaluation. However, a partial economic evaluation was viable and this was conducted on cost changes alone (see Table 28).

Health-related quality of life

KIDSCREEN-10

Intervention and control condition KIDSCREEN-10 responses were comparable at baseline and follow-up (Table 29). Mean T-scores were found to fall within the normative range of European and UK T-scores for children aged 8–11 years.60 As responses were found to be negatively skewed, median-derived T-scores were used to generate CIs (95%). The lack of difference between the intervention arm and the control arm responses at baseline and follow-up is reflected in the 95% CIs presented in Table 29.

Child Health Utility 9D

The CHU9D-derived utility values were found to be similar at baseline and follow-up (Table 30). Similar to the KIDSCREEN-10 responses, CHU9D responses were negatively skewed and, thus, median-derived utility values were used in analysis. The findings reported in Table 30 revealed no difference in intervention and control arm utility values at baseline or follow-up (95% CIs).

These findings are consistent with previous studies in which HRQoL measures were found to be unresponsive among school children with ‘healthy’ profiles,57 and supports the case for alternative HRQoL measures that are appropriate for use with this population.

After-school physical activity provision

Data on extracurricular physical activity provision and costs were collected from all intervention and control schools at baseline (see Appendix 8) and follow-up (see Appendix 9). Study schools provided a range of extracurricular physical activity opportunities at a cost ranging from £0 to £45 per session for schools and from £0 to £8.50 per session for parents. As shown in Table 31, on average intervention schools provided more extracurricular physical activity clubs per week at baseline (intervention, mean 3.25 ± 0.96; control, mean 2.83 ± 1.47) and at follow-up (intervention, mean 3.75 ± 2.06; control, mean 3.00 ± 1.67). The average club duration across time and condition was approximately 60 minutes. The delivery of extracurricular physical activity clubs cost intervention schools more, on average, at baseline (intervention, mean £4.62 per session ± £11.27; control, mean £0.41 per session ± £1.16) and at follow-up (intervention, mean £9.00 per session ± £15.83; control, mean £3.33 per session ± 9.85) compared with control school costs. Costs to parents were broadly similar across time and condition, with the mean cost per session ranging from £1.32 ± £1.62 to £2.61 ± £2.81.

A comparison of existing extracurricular physical activity provision with estimated Action 3:30 costs suggests that Action 3:30 may offer a more economically viable option for schools than delivery of alternative active after-school clubs. The average cost to schools per pupil attending one 1-hour session of existing extracurricular physical activity at follow-up was found to cost £5.91 (see Table 31), compared with an estimated mainstream cost of £1.64 (95% CI £1.43 to £1.86) per pupil per Action 3:30 session (see Table 31). However, this cannot be confirmed without a full trial in which the variation in costs between schools is considered.

Summary

A partial economic evaluation was conducted on cost changes alone. Action 3:30 is a low-cost intervention, comparing favourably with other after-school activities in that respect, and helps children to engage in non-sedentary activities. There was no evidence of a potential effect of the intervention on MVPA and, therefore, the potential cost-effectiveness could not be determined.

Self-reported responses to HRQoL measures by participants in the intervention arm and the control arm were similar at baseline and follow-up. This finding is consistent with previous studies57,62 in which HRQoL measures were found to be unresponsive among school children with ‘healthy’ profiles and supports the case for considering alternative HRQoL measures that are appropriate for use with this population.

The findings of this economic evaluation demonstrate that there is potential to examine cost variation in Action 3:30 intervention delivery across school settings and to estimate mainstream costs. The estimated cost of the Action 3:30 intervention is comparable with the cost of after-school physical activity interventions that have been reported in recent years. Findings indicate that Action 3:30 is inexpensive, with an estimated mainstream cost after 1 year of £1.64 per pupil per session, compared with the average school-level costs associated with existing extracurricular physical activity (£5.91 per pupil per session). As such, Action 3:30 may provide a more economically viable option for schools than existing school provision.