Work package 3.5: health economic evaluation

Introduction

This section presents the methods and results of economic analyses undertaken to assess the cost-effectiveness of the CFHealthHub intervention evaluated within the RCT (Dr Martin Wildman, Sheffield Teaching Hospitals, 2019, personal communication).

We undertook two related economic analyses to assess whether or not the intervention represents good value for money for the NHS: (1) a short-term ‘within-trial’ economic evaluation alongside a clinical trial (EEACT) that compares health gains and costs for the intervention and usual-care arms using individual patient-level data (IPD) from the trial (Dr Martin Wildman, personal communication) only, and (2) a model-based analysis that compares the intervention with usual care over a lifetime horizon. The scope of the analyses is summarised in Table 8. Both analyses evaluate the cost-effectiveness of the intervention in a population of PWCF aged ≥ 16 years who are taking inhaled mucolytics or antibiotics from the perspective of the NHS and PSS. The within-trial analysis is restricted to the health outcomes and costs accrued in the trial follow-up period, whereas the model-based analysis includes the extrapolation of health outcomes and costs over a lifetime horizon and includes the use of additional evidence beyond that collected in the RCT.

We used Consolidated Health Economic Evaluation Reporting Standards (CHEERS) reporting guidance (see Report Supplementary Material 6).56

Within-trial economic evaluation alongside a clinical trial

Within-trial economic evaluation alongside a clinical trial methods

The EEACT was undertaken using IPD from the RCT (Dr Martin Wildman, personal communication) Quality-adjusted life-year (QALY) gains for each patient were estimated as the ‘background QALYs’ accrued based on EQ-5D-5L assessments measured during routine clinic visits, less health losses associated with exacerbations (days spent receiving i.v. antibiotics). EQ-5D-5L measurements were mapped to the EuroQol-5 Dimensions, three-level version (EQ-5D-3L), using van Hout et al.57

Background QALYs for each patient were calculated under four assumptions:

1. Mean utility was calculated using an area under the curve approach via the trapezium rule, assuming a constant rate of improvement/worsening in HRQoL between successive EQ-5D-5L measurements.
2. For patients with a final EQ-5D-5L visit beyond the primary outcome cut-off, the patient’s EQ-5D-5L score was estimated at day 365 based on assumption 1.
3. For patients with a final EQ-5D-5L visit before the primary outcome cut-off, the patient’s mean utility during the observed period was assumed to apply during the unobserved period (i.e. mean utility is ‘stretched’ over the whole primary outcome window).
4. Patients with missing baseline EQ-5D-5L measurements and those with fewer than two EQ-5D-5L assessments over the primary outcome window were treated as missing.

All QALY losses associated with i.v. days were estimated as the mean loss of utility between the first and last exacerbation-related EQ-5D-5L measurements (‘visit E3’, which took place approximately 4 weeks after the start of the exacerbation, minus ‘visit E1’, which took place at the start of the exacerbation) for the whole patient population multiplied by the number of i.v. therapy-days incurred by each individual patient. Total QALYs gained per patient were calculated by subtracting i.v.-related QALY losses from the background QALYs.

The EEACT includes the following resource costs: the CFHealthHub adherence intervention (applied to the intervention arm only); i.v. therapy-days in hospital; i.v. therapy-days at home; i.v drugs and consumables; visits from health-care practitioners; and nebuliser devices. Resource use estimates were derived from data collected using a standardised resource use form in the trial as well as expert opinion. The costs of nebulised drugs were not included in this analysis. Unit costs were taken from NHS Reference Costs 2017/18,58 the Personal Social Services Research Unit (PSSRU)59 and personal communication [Misbah Tahir, Clinical Pharmacist, Sheffield Teaching Hospitals NHS Foundation Trust, 2019; Carola Fuchs, PARI GmbH, 2019; Pauline Whelan, Farr Institute, and ACtiF Project Management Group (PMG), 2019]. The resource and cost inputs applied in the EEACT are presented in Appendix 13.

Missing data were handled using multiple imputation by chained equations over 10 iterations.60 A complete-case analysis was undertaken as a secondary analysis. Total QALY gains and costs (with or without imputation) were included in a seemingly unrelated regression (SUR) model,61 which adjusts for baseline imbalances between the two groups in terms of baseline EQ-5D-5L and i.v. therapy-days in the previous 12 months (a stratification factor in the ACtiF RCT).

Within-trial economic evaluation alongside a clinical trial results

The results of the EEACT are summarised in Table 9. The base-case results for the EEACT suggest that the intervention generated 0.01 additional QALYs at an additional cost of £865.91 per patient; the corresponding incremental cost-effectiveness ratio (ICER) is £71,136 per QALY gained. The complete-case analysis suggests a higher ICER of £109,754 per QALY gained.

The results of this analysis should be interpreted with caution for two reasons. First, the short time horizon for the EEACT incorporates all of the upfront costs of the intervention but does not include the potential longer-term benefits that accrue as a result of these. Second, the allocation of the intervention costs across the trial group is unlikely to reflect the actual per-patient costs of delivering the intervention in practice, further inflating costs and associated ICERs.

Model-based analysis

Model structure and assumptions

The model-based analysis uses evidence from the RCT (Dr Martin Wildman, personal communication) and other external sources to estimate the cost-effectiveness of the intervention versus usual care over a lifetime horizon. The model structure was based on the model used to inform the National Institute for Health and Care Excellence (NICE) technology appraisal of dry powder antibiotics.62 The model adopts a state transition approach, defined according to five health states: (1) FEV₁ ≥ 70% predicted, (2) FEV₁ 40–69% predicted, (3) FEV₁< 40% predicted, (4) post transplant and (5) death (Figure 7). During each annual model cycle, patients may remain in their current FEV₁% predicted state, transition to an improved or worsened FEV₁% predicted state or die. A small proportion of patients in the worst lung function state (FEV₁ < 40%) may undergo lung (or heart and lung) transplantation and do not subsequently receive further nebulised treatment. HRQoL is modelled according to FEV₁% predicted stratum and transplant history, with QALY losses applied according to the proportion of time spent receiving i.v. therapy. The model uses a 12-month cycle length over 70 cycles; a half-cycle correction was applied to account for the timing of events. Total QALYs are calculated as the sojourn time in each health state weighted by state-specific utility scores, less any i.v.-related QALY losses. The model includes the costs associated with (1) the adherence intervention (intervention arm only), (2) health state resource use conditional on FEV₁% predicted stratum (based on the trial resource use form used in the trial), (3) i.v. therapy-days at home or in hospital (including hospital inpatient stays, i.v. drugs and consumables), (4) nebuliser devices, (5) nebulised drugs and (6) lung or heart and lung transplant.

FIGURE 7 State transition model structure. (Figure not reproduced here; see the original article.)

The model employs the following key structural assumptions:

• At model entry, patients are assumed to be aged 30 years, based on the mean age of patients at entry into the RCT (Dr Martin Wildman, personal communication).
• Patients in any FEV₁% predicted stratum can progress/regress to any other FEV₁% predicted stratum during any model cycle.
• Patients who undergo transplantation cannot revert back to other alive health states; hence, the only remaining event for these patients is death.
• Mortality risk is conditional on model health state. General population and CF-specific mortality constraints63,64 are applied to ensure that the survival projection is plausible over the entire time horizon.
• Exacerbation risk is dependent on FEV₁% predicted stratum.
• HRQoL is dependent on lung function and whether or not the patient is receiving i.v. therapy for exacerbations.
• The impact of the adherence intervention is assumed to manifest through two mechanisms: (1) risk ratios (RRs) of switching FEV₁% predicted strata in a given year and (2) a relative rate ratio (RRR) for the number of i.v. therapy-days incurred (which is, in turn, assumed to be constant across all three FEV₁% predicted states):
• The model assumes that the treatment effects and costs related to the intervention accrue only in the years in which the intervention is given.
• In the base-case analysis, the intervention is assumed to be given for 10 years; this assumption is tested in sensitivity analyses.
• The intervention is not assumed to directly affect the probability of undergoing transplantation or death.
• The costs of antibiotics and ‘high-cost therapies’ are independent of adherence to those therapies. This assumption is tested extensively in sensitivity analyses.

Evidence used to inform the model’s parameters

The evidence sources used to inform the model’s parameters are summarised in Table 10; these are briefly described below. All model parameters are presented in detail in Appendix 15.

Baseline characteristics

Initial patient characteristics (age, sex and the distribution of patients across FEV₁% predicted strata at baseline) were based on pooled data for both arms of the RCT (Dr Martin Wildman, personal communication).

Transition probabilities: usual care

The probabilities of transitioning between the model health states under usual care were informed by analyses of a bespoke registry data set provided by the Cystic Fibrosis Trust.65 The data set contained patient-level records for all patients included in the CF registry aged ≥ 16 years who had been prescribed nebulised mucolytics or antibiotics at any time point (data years 2006–15). The overall data set included 44,464 records across 7144 patients. We fitted a series of homogenous continuous time multistate models to the available data using the msm package in R. This method assumes that event hazards are constant with respect to time; hence, fitting a single model to the available data would have failed to reflect the increasing risk of death for older patients. To address this structural limitation in the approach, we fitted five separate multistate models to subsets of the data relating to patients in the following age categories: 30–34 years, 35–39 years, 40–44 years, 45–49 years and ≥ 50 years. Annual transition probabilities for each age category were estimated using the msm pmatrix function. In the economic model, a constraint was applied to ensure that the risk of death for any CF patient in the model is at least as high as that for the general population.63 A further constraint was applied to all cycles from the age of 60 years to ensure that the risk of death from any state is at least as high as that estimated by the flexible parametric survival models reported by Keogh et al.;64 this constraint was applied to prevent underestimation of the mortality rate caused by limited numbers of older patients in the registry data set.

Frequency of intravenous therapy-days: usual care

The expected number of i.v. therapy-days per year conditional on FEV₁% predicted stratum under usual care was estimated directly from the overall CF registry data set.65

Treatment effect models, risk ratios of transitioning between states

The FEV₁% predicted states were modelled using a cumulative link logit model with a nominal treatment group effect, baseline FEV₁% predicted state and adjustment for the previous year’s i.v. therapy-days, fitted to data from the RCT. The model was used to generate predicted probabilities for each follow-up FEV₁% predicted state, given baseline state and treatment allocation, from which RRs for between-state transitions by treatment group were derived. A bootstrap approach was used to estimate the standard errors (SEs) of the ratios on a log normal scale.

Treatment effect models, relative rate ratio for intravenous therapy-days

The i.v. therapy day counts in the 12-month period following consent into the RCT were modelled using zero-inflated negative binomial regression with a treatment group factor, adjustment for baseline FEV₁ status and the previous year’s i.v. therapy-days, and an offset for follow-up time. The treatment coefficient was exponentiated to give the IRR and the SE estimated using the delta method.

Health-related quality of life

Health utility values for the FEV₁% predicted states were estimated using a de novo function developed to map from absolute FEV₁% predicted scores to the EQ-5D-3L using data collected in the RCT (Dr Martin Wildman, personal communication). The data used to estimate the mapping model included repeated observations on 608 patients. Observations were not included in the analysis if EQ-5D-3L and FEV₁% predicted were missing or not measured in the same day/visit (2308 excluded observations). The final number of observations used was 2386 (out of 4694), corresponding to 576 patients with a mean of 3.4 clinic visits (SD 2.07). The mapping function developed by van Hout et al.57 was employed to transform the EQ-5D-5L descriptive system into EQ-5D-3L utility values. The mean EQ-5D-3L and FEV₁% predicted values were 0.78 (SD 0.2) and 58.7 (SD 0.23). An adjusted limited dependent variable mixture model67,68 (ALDVMM) was used to estimate the relationship between FEV₁% predicted and the mapped EQ-5D-3L values. The ALDVMM was developed specifically for mapping preference-based measures and has been shown to outperform other models.69 Robust clustered SEs were used to take into account the presence of repeated observations per patient. Models using two to four components were estimated. On the basis the Akaike information criterion (AIC), the Bayesian Information Criterion (BIC) and the mean absolute error (MAE), a model with three components was selected for inclusion in the final economic model.70 An assessment of the goodness of fit of the alternative models is presented in Appendix 15.

Utility values for patients undergoing transplantation and for patients receiving i.v. treatments were excluded from the mapping analysis. The utility value for the transplant state was taken from Anyanwu et al.66 QALY losses associated with receiving i.v. treatments were estimated non-parametrically using the RCT data (using the same disutility applied in the EEACT; see Within-trial economic evaluation alongside a clinical trial methods). Health utility values were age-adjusted using Ara and Brazier.71

Resource costs

The cost of the adherence intervention includes an initial cost associated with training and set-up (year 1 only), ongoing annual costs (including data transfer, monitoring, maintenance and hosting of the data platform), fidelity support and delivery of the intervention by interventionists. The model-based analysis assumes that 30 interventionists can support 5900 adult CF patients receiving nebulised therapies.

Resource use estimates and unit costs were taken from NHS Reference Costs 2017/18,58 the PSSRU,58 the British National Formulary (BNF),72 the ACtiF RCT (Dr Martin Wildman, personal communication) and personal communication (Misbah Tahir, Clinical Pharmacist, Sheffield Teaching Hospitals NHS Foundation Trust; Carola Fuchs, PARI GmbH; Pauline Whelan, Farr Institute, and the ACtiF PMG). Further details of the cost and resource use inputs are provided in Appendix 13.

Model evaluation methods

The model estimates the incremental cost-effectiveness of the intervention compared with usual care in terms of the incremental cost per QALY gained. Central estimates of cost-effectiveness were based on the expectation of the mean derived from the probabilistic version of the model, based on 2000 Monte Carlo samples. Parameter uncertainty was evaluated using both deterministic sensitivity analysis (DSA) and probabilistic sensitivity analysis (PSA). DSAs were used to identify key drivers of the cost-effectiveness. PSA was used to estimate the probability that the intervention is cost-effective across a range of willingness-to-pay (WTP) thresholds. Uncertainty was represented using cost-effectiveness planes and cost-effectiveness acceptability curves.

Validation

A number of activities were undertaken to ensure internal validity and credibility of the model; these are summarised in Appendix 16.

Model-based analysis: results

Table 11 presents the results of the base case analysis; the results of additional sensitivity analyses can be found in Appendix 17. Based on the probabilistic version of the model, the intervention is expected to generate 0.17 additional QALYs and cost savings of £1790 compared with usual care; therefore, the adherence intervention is expected to dominate usual care (i.e. it is expected to be more effective and less expensive). The results of the deterministic model are very similar. Assuming a WTP threshold of £20,000 per QALY gained, the probability that the intervention generates more net benefit than usual care is approximately 0.89.

The DSAs indicate that the conclusions of the economic analysis are sensitive to assumptions regarding:

• the duration over which health effects and costs of the adherence intervention apply
• the impact of the adherence intervention on the patient’s FEV₁% predicted stratum
• increases in adherence to nebulised treatments and associated effects on drug-prescribing levels.