Economic evaluation

Methodology overview

The economic evaluation considers the resource use, costs and cost-effectiveness of the SHIFT intervention compared with usual practice, using evidence from the SHIFT RCT and other sources. Costs were measured in GBP (2019–20) from a public sector perspective (i.e. NHS and Personal Social Services). A private sector perspective (haulage firm) was also considered for secondary analyses.106,107 Health outcomes were primarily measured in QALYs and based on the EQ-5D-5L questionnaire.81,108 Other measures, including productivity, employee well-being and absenteeism, were also considered. Missing data were populated using multiple imputation by chained equations.109 Within-trial costs and QALYs were estimated using multilevel econometric modelling to control for participant co-variables and cross-cluster variation.110 Decision-analytic models were used to extrapolate the results over a longer time horizon, based on any observed differences in physical activity between trial arms.111 In line with UK guidelines,106 costs and QALYs were discounted at 3.5% per annum. Cost-effectiveness was measured using ICERs and incremental net health benefits (INHBs) at cost-effectiveness thresholds of £15,000, £20,000 and £30,000 per QALY.106,112

Probabilistic sensitivity analysis was used to characterise the uncertainty, and decision uncertainty was assessed across alternate cost-effectiveness thresholds. Scenario, sensitivity and threshold analyses were also conducted to consider the implications of alternate methods and modelling assumptions on study findings.106 Further details are available in the health economic analysis plan (see Report Supplementary Material 4).

Resource use and costs

Health-related resource use was collected from participants using a service use questionnaire at baseline and at 6 months and 16–18 months. Responses at each time point related to participants’ resource use in the 6 months prior. Health-care costs were calculated for each trial participant by resource use category and at each follow-up period by applying unit costs to any resources used. It was assumed that the 16- to 18-month follow-up occurred at 18 months, and resource use between 6 and 12 months was equal to resource use between 12 and 18 months. Health-related resources and costs were categorised into primary care, secondary care, mental health care and occupational services. Unit costs were measured in GBP (2019–20) and were sourced from published UK sources (see Appendix 3, Table 43).91,113 Unit costs were inflated to 2019–20 prices using inflation indices where necessary.91 Costs of absenteeism were calculated based on firm-reported full-day driver-replacement costs.

The resource use and associated intervention costs for the SHIFT intervention comprised (1) exercise-related devices, equipment and materials [e.g. wearable device (i.e. a Fitbit Charge 2), THERABAND^® bands (Akron, OH, USA), intervention booklet] and (2) a 6-hour education session that required driver time, course materials and staff training. It was assumed that the education session would require a full worker day for each attendee and that the session was delivered by existing facilitators and, therefore, incurred no additional costs. The average SHIFT intervention cost per driver was treated as an up-front cost (i.e. with no follow-up costs) and calculated on an ITT basis. Usual practice was assumed to incur no intervention-related costs. Intervention unit costs were based directly on those incurred during the trial.

Outcomes

The primary outcome used in the cost-effectiveness analysis was QALYs, which is a generic measure of health that combines both length of life and HRQoL (1 QALY is equal to 1 year in perfect health).108 Participants’ HRQoL was assessed using the EQ-5D-5L questionnaire, collected at baseline and at 6 months and 16–18 months. The EQ-5D-5L is a descriptive HRQoL instrument that comprises five levels of severity across the following five health dimensions: (1) mobility, (2) self-care, (3) usual activity, (4) pain/discomfort and (5) anxiety/depression.81 In accordance with National Institute for Health and Care Excellence (NICE) recommendations, HRQoL weights were calculated from a published mapping of EQ-5D-5L responses onto HRQoL values calculated for the EuroQol-5 Dimensions, three-level version (EQ-5D-3L) instrument from a survey of the UK general public.106,114 Trial QALYs were estimated through an area under the curve approach, with linear interpolation between observations.108 It was assumed at the 16- to 18-month follow-up occurred at 18 months to allow for a common time horizon for estimation of QALYs. Longer-term cost-effectiveness analyses calculated QALYs using decision-analytic models (see Decision-analytic model and longer-term cost-effectiveness), which combined estimated within-trial QALY differences (i.e. within-trial analysis) with QALYs estimated over a longer time horizon, based on any observed impacts on levels of physical activity. It was assumed that the HRQoL of drivers beyond the trial were equal between arms (i.e. any within-trial differences did not persist beyond the trial period) and equivalent to age-specific values observed in the trial. The removal of estimated within-trial differences in outcomes and the use of HRQoL weights from the EQ-5D-5L value set were explored in scenario analyses.

Secondary outcomes included productivity, employee well-being and absenteeism. Productivity considered employee-assessed job performance and work ability on Likert scales that ranged from 0 to 7 and 0 to 10, respectively (with higher scores denoting more favourable outcomes). Participants’ work-related well-being considered employee-assessed job satisfaction on a Likert scale that ranged from 0 to 7 (with higher scores denoting greater satisfaction) and presenteeism according to the number of days drivers have worked despite feeling unwell.

Methods for analysis

Analysis

A health economics analysis plan (see Report Supplementary Material 4) was created before the health economics analysts had access to the data. From a public sector perspective, the cost-effectiveness of the SHIFT intervention (compared with usual practice) was assessed according to (1) the estimated differences in the QALYs gained by drivers and (2) the incremental costs to public services incurring over a 16- to 18-month time horizon (i.e. within trial) and over the longer term. Secondary analyses considered cost-effectiveness from a private sector perspective by assessing the differences in changes to measures of productivity and employee well-being at 6 months and 16–18 months from baseline, as well as the incremental costs. From a public perspective, intervention costs and all costs relating to health resource usage were considered. Only intervention and absenteeism costs were considered from the private sector perspective. Both public and private sector perspectives included SHIFT intervention costs, as these could feasibly be financed by either source.

For the public sector analysis, estimated costs and outcomes in each arm, and their differentials, are presented alongside ICERs and INHBs. ICERs represented the cost per additional unit of outcome for the SHIFT intervention compared with usual practice. The INHBs for the SHIFT intervention measured the intervention’s health gain less the health that would have otherwise been generated elsewhere had the additional resources (compared with usual practice) been allocated for alternative purposes (i.e. the opportunity cost estimated using a given cost-effectiveness threshold). Three cost-effectiveness thresholds (i.e. measures of health opportunity cost) were considered in the analysis: (1) £15,000 per QALY [i.e. the Department of Health and Social Care’s usual threshold (based on recent empirical estimates)],112,115 (2) £20,000 per QALY and (3) £30,000 per QALY. The thresholds (i.e. £15,000, £20,000 and £30,000) are used by NICE to assess cost-effectiveness of health-care interventions.106 At a given threshold, the SHIFT intervention is considered cost-effective, compared with usual practice, when its ICER is below the chosen cost-effectiveness threshold and it has positive INHB. For the private sector perspective, ICERs are presented, showing the cost per unit change in the measure of productivity or well-being.

Within-trial analysis

Estimated within-trial costs and QALYs in each treatment arm were obtained using multilevel linear regression models on multiply imputed data that controlled for a set of relevant participant co-variables and accounted for cross-cluster variation (i.e. variation between sites). Regression analyses controlled for age (i.e. 40–49 years, 50–59 years and ≥ 60 years vs. < 40 years), sex (i.e. female vs. male), ethnicity (i.e. white vs. other), BMI category (i.e. overweight, morbidly obese and obese vs. healthy), presence of diabetes, smoking status (i.e. ex-smoker and current smoker vs. never smoked) and cluster size. QALY regression analyses also controlled for baseline EQ-5D scores to account for differences in baseline HRQoL.116 Scenarios considered the following alternative approaches: (1) a generalised linear model using a log-link transformation and gamma family form to account for the positive and right-skewed nature of the cost data; (2) excluding the costs from inpatient-related services, given the potential for random imbalances in hospital procedures to affect cost differentials between arms; and (3) an analysis that considers only participants with complete data (i.e. complete-case analysis).

Missing data

Missing cost and outcome data were populated via multiple imputation using chained equations, with predictive mean matching used to match predicted missing values with the closest observed value.116 The imputation model controlled for all the covariates considered in the within-trial regression models, including clusters. Using Rubin’s rules, overall imputed mean estimates and standard errors were calculated from 20 imputed data sets to reflect the variability within and across imputations.116,117

Decision-analytic model and longer-term cost-effectiveness

The longer-term cost-effectiveness of the SHIFT intervention was assessed using decision-analytic models that sought to capture the longer-term benefits of physical activity for HGV drivers. Given the uncertainties associated with modelling the impact of physical activity on public health,118 alternative measures of physical activity and modelling approaches were considered to estimate outcomes.

The first decision-analytic model was a two-state Markov model, where the cohort starts in an alive state and will either remain in that state or transition into an absorbing dead state. Beyond the first year, the model captured QALYs using the age-specific HRQoL observed in the trial and did not consider additional costs. For usual practice, transitions to the death state were based on age- and sex-adjusted English general population mortality rates.119 Mortality rates for the SHIFT arm were then adjusted according to estimated changes in one of two alternative measures of physical activity: (1) time spent in MVPA (i.e. the MVPA-based model) and (2) time spent sedentary (i.e. the sedentary-based model) (see Chapter 2). The dose–response relationship between changes in accelerometer-measured physical activity (specifically MVPA and sedentary time) and all-cause mortality was based on hazard estimates reported in Ekelund et al.’s meta-analysis.120 Polynomial functions were used to interpolate between all-cause mortality hazard ratios, relative risks and 95% CI point estimates (see Appendix 3, Figures 4 and 5). Physical activity in the usual-practice arm was assumed to follow baseline values (across arms) and to remain constant over time (see Table 7). In the SHIFT arm, the average change in physical activity relative to usual practice in the first year was assumed to be equal to estimated differences at the yearly mid-point (i.e. 6 months) (see Table 9). After the first year, SHIFT-associated differentials in physical activity were reduced exponentially at a 50% decay rate per annum from estimated differences at final follow-up, although the decay rate was varied in sensitivity analyses. The model was run using average participant characteristics over a lifetime horizon.

An alternative decision-analytic model considered was the Model for estimating the Outcomes and Values in the Economics of Sport (MOVES) tool, version 2.0, which was developed for Sport England.92,121 The MOVES tool estimates risk reductions across seven completing diseases (i.e. dementia, depression, colon cancer, type 2 diabetes, breast cancer, ischaemic heart disease and stroke) from changes in physical activity measured by metabolic-equivalent hours per week. The model was modified to have the same treatment effect schedule as the first decision-analytic model, with first year exercise differences equal to those estimated at 6 months, second year differences equal to those estimated at 16–18 months, and differences thereafter reduced exponentially from 16–18 months at a 50% decay rate per annum. The underlying HRQoL of the cohort over time was aligned with the age-specific HRQoL observed in the trial. Minutes in MVPA were translated into metabolic-equivalent hours per week by assuming the intensity of activity within MVPA follows a uniform distribution between 3 (i.e. light moderate exercise) and 6 (i.e. vigorous exercise). Metabolic-equivalent hours per week in each arm were defined according to baseline MVPA minutes (pooled across arms), estimated treatment differentials in MVPA minutes and presumed metabolic intensity (drawn from identical uniform distributions) (see Table 9). Exercise differentials were bounded such that no individual could undertake a negative number of minutes exercise. The model was run using average participant characteristics over a 25-year time horizon.

In all models, treatment-associated changes in physical activity were applied as common effects (i.e. irrespective of participant characteristics). Estimated within-trial cost and QALY treatment differences were incorporated in the first year, but were not extrapolated thereafter. In accordance with NICE guidance,106 costs and QALYs were discounted at a 3.5% discount rate. Scenario analyses considered removing all estimated trial-specific treatment differentials in QALYs and non-intervention costs, as well as an annual discount rate of 1.5%.122

Uncertainty

Monte Carlo simulation was used to propagate the uncertainty in the analyses to estimate overall decision uncertainty surrounding the adoption of the SHIFT intervention. It was assumed that (1) event probabilities followed beta distributions (MOVES92,121), (2) baseline physical activity (i.e. usual practice), treatment effects and all-cause mortality hazard ratios (Ekelund et al.120) were normally distributed and (3) regression coefficients (i.e. within-trial non-intervention costs and outcomes) followed multivariate normality.111 Regression parameter correlations were accounted for using Cholesky transformations of the variance–covariance matrix.111

Uncertainty was reported at 95% credible intervals around mean cost, QALY and INHB values, alongside the probabilities of the SHIFT intervention/usual practice being the most costly, clinically effective and cost-effective alternative. INHB and the probability of being cost-effective are presented for cost-effectiveness thresholds of £15,000, £20,000, and £30,000 (i.e. the health that would have been generated elsewhere using the same resources). Cost-effectiveness acceptability curves illustrated the probability of the SHIFT intervention or usual practice being cost-effective up to a threshold of £50,000 per QALY.

Key uncertainties were explored in a range of scenario analyses involving alternative methodological approaches (see above) and the removal of all estimated within-trial differences in non-intervention costs and QALYs between trial arms. Sensitivity and threshold analyses explored the impacts of two alternative degrees of treatment maintenance on study findings: (1) the annual rate of decay in the treatment effect on physical activity beyond the first year (i.e. a 50% decay rate used in the base case) and (2) the continuation of the treatment effect on physical activity observed at 6 months.

Results

The SHIFT trial involved 382 participants (SHIFT intervention, n = 183; usual practice, n = 199). Health-care resource use forms were fully completed by approximately 93.5%, 63.9% and 46.9% of participants at baseline, 6 months and 16–18 months, respectively. Complete-case resource use was achieved by 40.8% of participants. EQ-5D-5L questionnaires were complete for 98.2% of participants at baseline, and for 66.8% and 49.2% of participants at 6 months and 16–18 months, respectively. Complete-case EQ-5D-5L responses amounted to 44.5% of participants. Secondary outcome data (i.e. productivity, employee work-related well-being and absenteeism) had a comparable degree of missingness, ranging between 98.7% and 99.5% at baseline, between 67.0% and 67.3% at 6 months and between 48.4% and 50.5% at 16–18 months. Participant characteristics are reported in Table 5.

Treatment effect

Baseline physical activity (i.e. usual practice) was modelled at 15.36 minutes per day in MVPA and 670.31 minutes of sedentary time per day. In the first year, the SHIFT intervention was associated with an additional 5.84 minutes per day in MVPA and a reduction in sedentary time of 24.37 minutes per day (see Table 9). In the second year, the SHIFT intervention was associated with an additional 1.6 minutes per day in MVPA and a reduction in sedentary time of 12.16 minutes per day. In the subsequent years, the treatment effect was extrapolated from second year differentials (i.e. a 1.6-minute/day increase in MVPA and a 12.16-minute/day reduction in sedentary time).

Resource use and costs

Intervention-level costs

Table 30 provides the average per driver costs of delivering each element of the SHIFT intervention. The average total cost of delivering the SHIFT intervention was £369.57 per driver. The equipment (£182.49) and education (£187.08) elements of the SHIFT intervention had comparable costs. Education costs mostly comprised driver replacement costs.

Health-care resource use and non-intervention costs

Resource use and associated costs were broadly balanced between the SHIFT intervention and usual practice over the course of the trial (Table 31). Differences within and across resource categories were small in magnitude and inconsistent in direction of effect (see Appendix 3, Tables 44–53). The total imputed costs were lower for usual practice (£637.66) than for the SHIFT intervention (£1,162.50). When controlling for participant covariates, the SHIFT intervention was associated with an additional £181.50 in non-intervention costs compared with usual care (see Appendix 3, Table 57). Appendix 3, Tables 44–47, present available- and complete-case breakdowns of resource use for each comparator by resource category and follow-up period. Appendix 3, Tables 48–51, describe available-case, complete-case and imputed costs by resource category and follow-up period for each trial arm.

Outcomes

Quality-adjusted life-years and HRQoL scores were similar between the arms, albeit with modest differences in baseline values and changes at 6 months’ follow-up (Table 32). In the SHIFT arm, EQ-5D-5L and mapped EQ-5D-3L scores fell over the trial period. In the usual-practice arm, scores rose between baseline and 6 months, before declining to levels below baseline (see Appendix 3, Table 54). Adjusted QALY estimates (i.e. QALY estimates estimated using imputed data while controlling for participant covariates) found –0.028 and –0.015 QALY decrements associated with the SHIFT intervention compared with usual practice over the trial horizon when using mapped EQ-5D-3L and EQ-5D-5L preference weights, respectively (see Appendix 3, Tables 61 and 62). Secondary outcomes are reported in Table 33. There were small and largely inconsistent changes within and between arms for each outcome considered.

Cost-effectiveness analysis

The within-trial and longer-term base-case mean cost, QALY and cost-effectiveness estimates for each arm and modelling approach are reported in Table 34.

The within-trial analysis found the SHIFT intervention to be more costly and less effective than usual practice, resulting in it being dominated. The probability of the SHIFT intervention being cost-effective in the within-trial period was low, with a probability of between 0.009 and 0.011 for the range of cost-effectiveness thresholds considered.

For the MVPA-based model, when using Ekelund et al.120 all-cause mortality estimates with changes to MVPA minutes per day, the SHIFT intervention was found to be more costly and less effective than usual practice and, thereby, dominated. Incremental costs (£555) were the same as the within-trial analysis (the model did not extrapolate costs); however, QALY decrements were reduced to –0.022 per driver because of the increased physical activity in the SHIFT group reducing mortality. Similar results were found when using the sedentary-based model, with costs aligned to within-trial results and QALY decrements of –0.021.

The SHIFT intervention was also found to be more costly and less effective when using the MOVES tool. The inclusion of lifetime costs in the MOVES model increased overall cost estimates, but resulted in a small reduction in the incremental costs for the SHIFT group (£507). QALY decrements were reduced to –0.016 in the SHIFT group, relative to usual practice.

In all base-case analyses, the SHIFT intervention was found to be dominated by usual practice. The 95% credible intervals around incremental QALYs overlapped zero in the MOVES model, suggesting a significant level of uncertainty in the QALY differentials of the SHIFT intervention compared with usual practice. Appendix 3, Tables 57–63, present each regression analysis used to inform the cost-effectiveness analysis. Appendix 3, Figure 8, shows the cost-effectiveness acceptability curves for each modelling approach.

Scenario analyses

Table 35 presents the incremental cost, incremental QALY and associated ICER estimates for the scenario analyses considered for the long-term analyses, based on the three decision-analytic models. Base-case estimates are also provided for reference. A more detailed breakdown of each scenario can be found in Appendix 3, Tables 64–69.

In each modelling approach, removing estimated trial-specific differentials in non-intervention costs and QALYs between arms resulted in smaller incremental costs for the SHIFT intervention and positive incremental QALYs, compared with usual practice. For the MVPA- and sedentary-based models, the resulting ICERs were £65,072 and £51,174 per QALY, respectively, which are above the cost-effectiveness thresholds considered. For the MOVES model, SHIFT was associated with an ICER of £29,287 per QALY, thereby falling below the highest cost-effectiveness threshold considered. Under this scenario, for the MOVES model, the SHIFT intervention had a 12.4%, 26.1% and 46.6% probability of being cost-effective at the cost-effectiveness thresholds £15,000, £20,000 and £30,000 per QALY, respectively (see Appendix 3, Table 64).

Scenarios concerning the application of EQ-5D-5L preference weights, a 1.5% discount rate, generalised linear models to estimate costs, participant costs omitting inpatient-related resource utilisation and a complete-case analysis framework did not have marked impacts on results and, therefore, did not change base-case findings (i.e. the SHIFT intervention dominated by usual services).

Sensitivity and threshold analyses

Study findings were largely insensitive to changes in the decay rate of the treatment effect on physical activity. At base-case settings, the INHB of the SHIFT intervention remained negative for each threshold considered across the range of decay rates examined (i.e. 10–100%) in all models considered (Figure 3). When removing estimated trial-specific differentials in non-intervention costs and QALYs between arms, the INHB for the SHIFT intervention was positive at a decay rate of approximately 20% at a £30,000 per QALY threshold for the sedentary-based model. Likewise, for MVPA-based model, the SHIFT intervention was positive at a decay rate of approximately 15% at a £30,000 per QALY threshold (see Appendix 3, Figure 6). For the MOVES model, when within-trial differences were removed, the SHIFT intervention was cost-effective at all cost-effectiveness threshold values considered at a decay rate at, or below, approximately 20%.

FIGURE 3 Incremental net health benefit of the SHIFT intervention relative to usual practice for alternative rates of treatment decay: MOVES model. (Figure not reproduced here; see the original article.)

The base-case cost-effectiveness results were also largely insensitive to extensions in the duration of treatment effect on physical activity observed at 6 months. A comparison of SHIFT intervention ICERs (relative to usual practice) for alternative additional intervention costs and extension periods in treatment benefit are displayed in Table 36. With all other things remaining equal, the SHIFT intervention remained dominated for extensions up to and including 7 years for the MVPA- and sedentary-based models. For the MOVES model, ICERs only fell below £30,000 at a 6-year extension (see Table 36). Cost-effectiveness results were more sensitive to extensions in treatment effect when removing estimated trial-specific differentials in non-intervention costs and QALYs between arms. For the MVPA-based model, ICERs fell below £30,000 per QALY at 3 years, below £20,000 per QALY at 5 years and below £15,000 per QALY at 7 years. For the sedentary-based model, ICERs fell below £30,000 per QALY at 3 years, below £20,000 per QALY at 5 years and below £15,000 per QALY at 6 years. For the MOVES model, ICERs fell below £20,000 per QALY at a 1-year extension and below £15,000 per QALY for a 2-year extension (see Appendix 3, Table 72).

Secondary cost-effectiveness analysis from a private sector perspective

Absenteeism-related cost differentials between the SHIFT intervention and usual practice at baseline equated to a cost saving of £57.98 between the two arms (SHIFT intervention, £628.50; usual practice, £686.46). At the 6-month follow-up, larger reductions in absenteeism in the SHIFT arm (see Table 33) resulted in a cost saving equivalent to £169.43, compared with baseline differences. At the 16- to 18-month follow-up, an increase in absenteeism amounted to an additional £552.05, compared with baseline differences. Over the course of the trial, changes in absenteeism costs were £324.64 higher in the SHIFT group than in the usual-practice group, relative to baseline values. Incremental private costs per driver equated to £200.16 at 6 months and amounted to £694.23 at final follow-up (see Appendix 3, Table 72).

Employee-assessed job performance and presenteeism at 6 months were less favourable in the SHIFT arm, relative to the differences in changes from baseline. At 6 months, differences in employee-assessed work ability and satisfaction rose relative to baseline differences by approximately 0.148 and 0.036, respectively, in favour of the SHIFT arm, equating to a £1353 and £5560 cost per unit increase on each respective Likert scale. At the final follow-up, relative to baseline differences, differences in employee-assessed job performance, work ability, presenteeism and satisfaction changed in favour of the SHIFT intervention by approximately 0.081, 0.154, 1.19 and 0.151 less days worked while sick, respectively, equating to a £6816, £3585, £465 and £3656 cost per unit increase on each respective Likert scale. Average difference in results over the trial equated to the SHIFT intervention being dominated in presenteeism (compared with usual service), while having ICERs of £17,142, £4598 and £7425 with respect to one unit increases in employee-assessed job performance, work ability and job satisfaction, respectively (see Appendix 3, Table 72). Given the modest changes over time and largely inconsistent differences between arms, caution must be taken when interpreting the differences in costs, outcomes and associated ICERs.

Given the additional productivity costs (i.e. lost driver days), QALY decrements and higher public costs associated with the SHIFT intervention, relative to usual practice, a broader perspective that considers public costs and productivity would fail to alter base-case study findings.