Economic evaluation: within-trial analysis

Introduction

This chapter reports on the within-trial economic evaluation of laparoscopic cholecystectomy compared with observation/conservative management for the treatment of adults presenting uncomplicated symptomatic gallstones in a secondary care setting conducted during the C-GALL trial.1 The analysis considers a 24-month time horizon. A longer time horizon is explored with a Markov model extrapolation allowing for further relevant costs and consequences associated with gallstones, laparoscopic cholecystectomy and observation/conservative management in Chapter 5. Health service resources are scarce and healthcare technologies should be adopted or maintained as part of the NHS healthcare package only if they can be demonstrated to provide good value for money. Economic evaluation can aid decision-making on the adoption, or not, of new healthcare technologies by providing information on the relative efficiency of the technologies under consideration.

Objectives of the economic evaluation

The primary economic objective of the within-trial cost-effectiveness analysis was to estimate the difference in cost and QALYs between observation/conservative management and laparoscopic cholecystectomy at out to 24 months post randomisation. The secondary economic objective addressed in Chapter 5 is to model the longer-term cost-effectiveness of observation/conservative management versus laparoscopic cholecystectomy.

Methods

This economic evaluation followed established methods42,43 and reporting standards44 with a prespecified protocol and health economics analysis plan. The UK NHS healthcare system perspective was adopted for all the economic analyses (see Chapters 4 and 5).

Study design and participants

The C-GALL trial protocol1 and Chapter 2 provide details of the trial design. The within-trial economic analysis follows the ITT principle and is based on the same participants randomised and analysed for the main trial analysis.

Cost and outcome assessment

Case report forms and PQs were used to assess costs and outcomes. A RN completed a CRF at the time of surgery providing details of the operative procedures, perioperative complications and resource use in hospital [e.g. transfer to high dependency unit (HDU) or intensive care unit (ICU)]. Costs of the initial intervention procedures were estimated from resource use data recorded on the CRFs coupled with routine unit cost data.31 Costs associated with subsequent contacts with primary and secondary care (due to symptomatic gallstones or post surgery) were estimated from PQs at 3, 9, 12, 18 and 24 months post randomisation. Health-related QoL weights were estimated from participants’ responses to the SF-36 questionnaires at 3, 9, 12, 18 and 24 months post randomisation, and were used to estimate QALYs.45,46

Assessment of health service costs

The aim of the economic evaluation is to inform the efficient allocation of the NHS budget. Therefore, an NHS perspective was adopted for the analysis and NHS costs were estimated for health service use by the trial participants in both secondary and primary care. Costs are expressed in Great British pounds and reported using the 2019–20 price year.

Cost of the primary interventions

Cholecystectomy

The cost of the initial surgical intervention was estimated from resource use data recorded in the C-GALL Surgery CRF for each participant. The CRF recorded the type of procedure carried out (i.e. laparoscopic cholecystectomy, open cholecystectomy, laparoscopic converted to open cholecystectomy), operation time (i.e. incision to skin closure), grade of surgeon (and if the main surgeon was supervised by a consultant), intraoperative complications and anaesthetic complications. Further information such as dates for admission and discharge, and destination post operation (i.e. type of ward, HDU or ICU) was provided by the C-GALL postsurgery CRF.

In order to capture patient variation, the primary costing approach assigned costs to individual components of resource use (micro-costing). Laparoscopic cholecystectomy is one of the most common NHS operational procedures with more than 66,000 procedures completed in England in 2020–1.47 The C-GALL Surgery CRF asked only for time from incision to skin closure. Upon discussion within the PMG and clinical opinion, time in the anaesthetic room and recovery room were estimated to be similar to those observed in the Hysterectomy or Endometrial AbLation Trial for Heavy menstrual bleeding (HEALTH)48 which compared laparoscopic supracervical hysterectomy and second-generation endometrial ablation for the treatment of heavy menstrual bleeding. Time in the anaesthetic room was costed using the cost per hour (incorporating overheads) for a consultant anaesthetist and an anaesthetic nurse (Table 11). The unit cost of the recorded grade of surgeon and a consultant anaesthetist was applied to the time in theatre (incision to skin closure time). Nursing staff were costed as the requirement for general day surgery: one anaesthetic nurse, a scrub nurse and two further theatre nurses.

In addition, a published unit cost was applied for time in theatre to reflect the average cost of other staff, supplies and consumables, and allocated capital charges and overheads.49 While this detailed unit cost of theatre time is only available for Scottish hospitals, the average cost per theatre hour in general hospitals in Scotland (£1072 including medical and nursing staff) is comparable with a published estimate for England (£1200 per hour).50 Therefore, the Scottish estimate for the average cost for time in theatre for gastroenterology (£550 per hour, excluding medical and nursing staff) was applied to the time in theatre for the base-case analysis (see Table 11).

The unit cost of a band 6 nurse (inclusive of overheads) assuming one-to-one care was used to cost the time in recovery after surgery. Time on the ward following recovery was costed using an estimate of the cost per excess bed-day following cholecystectomy. Cost for excess bed-days were last published by NHS England in 2018.51 Therefore, these unit costs were applied after being adjusted for inflation using the NHS cost inflation index.52

The National Schedule of NHS costs provides an alternative source for costing the initial cholecystectomy procedure.31 For this, each patient record was mapped to the appropriate Healthcare Resource Group (HRG). The core HRG code for cholecystectomy is GA10K (laparoscopic cholecystectomy, 19 years and over, with complexity and comorbidity score 0). The C-GALL Surgery CRF contained information about the cholecystectomy being conducted as an elective or emergency procedure. Thus, the NHS cost for cholecystectomy was applied as either a day-case (patient discharged same day) or an elective inpatient admission (stay ≥ 1 day), non-elective short stay (up to 1 overnight stay), or non-elective long stay (more than 1 overnight stay).

Observation/conservative management

Observation/conservative management involves the prescription of analgesics to relieve the biliary pain (paracetamol, NSAIDs, e.g. ibuprofen etc.; narcotic analgesics, e.g. opiates; antispasmodics, e.g. Buscopan), together with generic lifestyle advice.2 Furthermore, active monitoring is recommended for individuals without symptoms. This means that participants could let their GP know if any changes in symptoms occurred. Data on medications and GP contacts were collected through PQs. The British National Formulary (BNF)53 was used to value medications. GP contacts were costed utilising the cost per contact (i.e. visit, telephone consultation or online consultation) reported by Curtis and Burns.52 The cost for a GP-led telephone triage was used to cost telephone consultations with the GP.

Costs of hospital admissions, perioperative complications and hospital re-admissions

The costs of clinical management of any procedure due to perioperative complications as well as any hospital admissions were based on NHS reference costs.31 The information on the type of complication experienced and details of the procedures undertaken were obtained from the C-GALL CRFs and associated SAE forms. These included events such as biliary colic, cholecystitis, gallstone pancreatitis and choledocholithiasis (see Table 8, Chapter 3). Participants who reported no hospital admissions or had no relevant CRF completed were assumed to have zero cost due to hospital admissions.

Costs of hospital outpatient and primary care healthcare utilisation

Primary and secondary outpatient contacts related to gallstone disease incurred over the 24-month follow-up period were obtained from the PQs. Medications prescribed for any ongoing problems related to the patient’s gallstone condition were also recorded in these questionnaires. All the primary care contacts were costed using the unit cost of Health and Social Care,52 and outpatient visits were costed using the NHS reference cost for a general surgery outpatient visit.31 The number of visits for each patient was multiplied by the appropriate unit cost. The list of medications and quantities primarily prescribed in primary care were costed using prices recorded in the BNF.53

Outcome measures

Quality-adjusted life-years were defined as the measure of effectiveness for the cost–utility analysis. QALYs were estimated using participants’ responses to the SF-36 questionnaire completed at baseline, 3, 9, 12, 18 and 24 months post randomisation. The SF-36 questionnaire was selected as the recall time for this instrument is the last 4 weeks. We believe this recall time has a better chance to reflect participants’ QoL variations due to acute events associated with gallstones compared to other instruments such as the EuroQol-5 Dimensions (EQ-5D) with a focus on ‘your health today’.55 Participant responses were assigned a utility score based on the Short Form-6 Dimensions (SF-6D) UK tariff.45,46 The appropriate algorithm for the SF-36 version 2 was recreated in Stata56 to obtain the health state utility weights. QALYs were estimated using the AUC approach, assuming linear change in utility between the observed follow-up time points.

Statistical analysis of trial economic data

Aggregating costs and effects

Costs and QoL data were summed up for each participant for the 24-month follow-up period and total cost and QALYs per participant were obtained. Mirroring the statistical analysis, the principles of ITT analysis were followed to compare cost and QALYs between groups.

Missing data

Reliance on complete case data for cost-effectiveness analysis can introduce bias unless the data are missing completely at random. The total estimated cost is the sum of numerous components over the observed 24-month follow-up period of the trial. Besides, QALYs can only be computed where participants have responded to the relevant QoL questionnaires at every follow-up point. Therefore, economic evaluations based on participant-level trial data are likely to face problems with missing data. MI43,56 was implemented as part of the primary analysis, using chained equations with predicted mean matching (kth-nearest-neighbour = 5). Implemented in Stata,56 the method starts with the variable with fewer missing values. Predictive mean matching imputes an observed value from another individual whose predicted value is close to the predicted value of the individual with the missing observation. Twenty imputed data sets were generated with plausible fitted values assigned for missing cost and utility elements. The imputation was conducted at the cost variable level (e.g. inpatient care, outpatient visits and primary care contacts) and SF-6D utility score level. The imputation model included all the variables in the analysis model (age, gender and treatment group allocation) and auxiliary variables that may help to explain missingness (trial centre, indicator for having surgery and type of procedure). Rubin’s rules were used to pool estimates across the MI data sets.57

Incremental cost-effectiveness analysis

The economic analysis estimated the incremental cost and QALYs between observation/conservative management group and laparoscopic cholecystectomy group up to 24 months post randomisation. General linear regression models adjusted for minimisation factors (centre, age and gender) and baseline SF-6D score were used. Family function was based on a modified Park test, while a modified Hosmer and Lemeshow, Pearson correlation and Pregibon link tests were used to select the link function43 (see Appendix 4, Tables 49 and 50, for details). The test results suggested a Gaussian family and identity link as the best model for estimating cost and QALY differences. Adjusted mean values by treatment allocation and the incremental difference between the groups were obtained using the methods of recycled predictions.43 The incremental cost-effectiveness ratio (ICER) for observation/conservative management group versus laparoscopic cholecystectomy group was calculated as the difference in mean costs divided by the difference in mean QALYs. Moreover, the uncertainty surrounding the joint incremental costs and effects was characterised using non-parametric bootstrapping with 1000 iterations with the MI process (k = 5 and 20 simulated data sets) nested within the bootstrapping process.58 Results from the bootstrap iterations were used to obtain 95% credible intervals (95% CrI) for the incremental cost and incremental QALYs. While the micro-costing approach makes better use of the trial data, the HRG-based costing might better reflect the opportunity costs for the UK NHS. Therefore, the HRG-based costing was defined as the base-case analysis and all sensitivity analyses were conducted with the HRG-based costing.

Sensitivity analyses

Sensitivity analyses were conducted by predefined subgroups and explored the cost-effectiveness of observation/conservative management versus laparoscopic cholecystectomy according to gender (male/female), age (< 35; 35–64; ≥ 65 years) and ethnicity (white; non-white). Treatment allocation by subgroup interaction terms was defined in the regression models for this analysis. A further analysis was conducted using micro costing for the index surgical intervention.

Results

Resource use and costs

Resource use and cost from the NHS perspective by treatment allocation are presented in Table 12. Cholecystectomies in the observation/conservative management group required slightly longer time in theatre (mean 83 minutes) than those in the laparoscopic cholecystectomy group (mean 72 minutes) and longer hospital stays (1.4 days vs. 0.6 days). HRG-based costing was used as the primary costing method to cost cholecystectomy episodes with the cost of the episode also reported in Table 12. The costs of time in the anaesthetic room (16 minutes) and recovery room (71 minutes) were added to the costs of the time in theatre and hospital stay for the micro-costing of the cholecystectomy episode. One hundred and ninety-five participants provided complete data on time in theatre and length of hospital stay. The average cost per cholecystectomy episode was £839 and £1912 for the observation/conservative management and laparoscopic cholecystectomy groups, respectively. NHS reference costs were used as the alternative costing method. The mean cost of cholecystectomies is substantially lower in the observation/conservative management group (£1014; N = 217) compared to the laparoscopic cholecystectomy group (£2190; N = 217), due to the lower percentage undergoing treatment.

Table 12 also reports follow-up resource use up to 24 months post randomisation. Participants in the observation/conservative management group spent more time in hospital in the follow-up period. Reasons for hospital admissions included biliary colic, cholecystitis, gallstone pancreatitis and choledocholithiasis. The average costs of these events were higher in the observation/conservative management group (£242 vs. £121). There were no differences in the mean number of outpatient visits between groups, but there was a higher mean number of visits to A&E department in the observation/conservative management group. While GP face-to-face and practice nurse contacts were higher in the surgical group, GP phone consultations were lower. However, none of these differences appear statistically significant by visual inspection of the SD for each group.

The overall costs for the initial and follow-up healthcare result in a total average NHS cost of £885 for observation/conservative management and £2184 for cholecystectomy, giving an unadjusted difference of £1299.

Utility scores and quality-adjusted life-years

Mean health state utility scores based on the SF-6D algorithm45,46 by treatment allocation are reported in Table 13. QoL data were collected at baseline, and at 3, 9, 12, 18 and 24 months post randomisation. At baseline, there was a small non-significant difference in the mean utility score favouring observation/conservative management. Mean utility score is also slightly higher for this group at 3 months. However, the mean score for laparoscopic cholecystectomy group increases from month 3 to 9 surpassing the corresponding score for the observation/conservative management group. This probably reflects participants going through, and recovering from, surgery in the laparoscopic cholecystectomy group as the median time to surgery was 4.7 months (compared with 9 months for observation/conservative management group). Both study groups show an increase in the mean utility score from 9 months to 18 months and a fall mean utility score value at 24 months. A small non-significant mean QALY difference favouring laparoscopic cholecystectomy emerges from the calculation of QALYs based on 30% of participants reporting complete data.

Cost–utility analysis results

The base-case cost–utility analysis results based on the MI data set and adjusted by minimisation factors and baseline SF-6D utility score are reported in Table 14.

The adjusted mean costs per participant were £2510 and £1477 for the laparoscopic cholecystectomy and observation/conservative management groups, respectively, resulting in an adjusted cost difference of –£1033 (95% CrI: –1413 to –632). The mean adjusted QALYs per participant were 1.413 and 1.395 for the laparoscopic cholecystectomy and observation/conservative management groups, respectively, producing an adjusted QALY difference of –0.019 (95% CrI: –0.06 to 0.02) for the 24-month follow-up period. Therefore, ITT with laparoscopic cholecystectomy as the first option results in significantly higher mean costs and non-significantly higher-mean QALYs compared with observation/conservative management. The ICER between observation/conservative management and laparoscopic cholecystectomy was £55,235. Therefore, moving from the standard practice of laparoscopic cholecystectomy to observation/conservative management would result, on average, in lower costs and QALYs with a saving of £55,235 per QALY forgone.

The results from the 1000 bootstrapped iterations of the regression analysis nested MI are presented in the cost-effectiveness plane (Figure 7). Incremental costs and QALYs for observation/conservative management versus laparoscopic cholecystectomy are reported in this scatterplot. All the iterations show negative mean cost differences, meaning that observation/conservative management is less costly than laparoscopic cholecystectomy. Although 16% of the iterations resulted in higher QALYs for observation/conservative management, the MD in QALYs (–0.019) favoured laparoscopic cholecystectomy. The red dashed line in Figure 7 represents the £20,000 cost-effectiveness threshold. This threshold separates iterations that result in either trial group being cost-effective. Iterations sitting to the right (left) and below (above) the red dashed line indicate that observation/conservative management (laparoscopic cholecystectomy) is cost-effective. Just 7% of the iterations sit above and to the left of the £20,000 threshold59 dashed line; that is just 7% of the iterations show QALY differences large enough for laparoscopic cholecystectomy to be cost-effective (see Figure 7).

FIGURE 7 Trial-based incremental cost-effectiveness scatterplot for observation/conservative management vs. laparoscopic cholecystectomy (base case; imputed data – 1000 bootstrap iterations). (Figure not reproduced here; see the original article.)

The cost-effectiveness acceptability curves (CEACs) for laparoscopic cholecystectomy and observation/conservative management are reported in Figure 8. The CEACs show the probability of observation/conservative management or laparoscopic cholecystectomy being cost-effective at alternative cost-effectiveness threshold values. The CEAC for observation/conservative management shows a 94% probability of being cost-effective at £20,000 threshold value. This probability decreases at higher threshold levels reflecting the higher mean cost and QALYs resulting from laparoscopic cholecystectomy, but conservative management has the higher probability of cost-effectiveness up to a cost-effectiveness threshold of £55,000.

FIGURE 8 Cost-effectiveness acceptability curves for observation/conservative management and laparoscopic cholecystectomy groups (trial-based analysis; base case; imputed data – 1000 bootstrap iterations). (Figure not reproduced here; see the original article.)

Alternative costing method and subgroup analyses

Results for the alternative costing methodology for the index laparoscopic cholecystectomy and the subgroup analyses are summarised in Table 15. The analysis using micro costing to value cholecystectomies generates a slightly smaller cost difference between the study groups (–£930 compared to –£1033) resulting in a lower ICER (£49,747). As for the base-case analysis, this ICER is above the usual cost-effectiveness threshold used in the UK.59

The subgroup analyses for gender and age give similar results to those observed for the base case. Although observation/conservative management shows higher mean QALYs for males and for participants over 65 years old, and the subgroup analysis for non-white favours laparoscopic cholecystectomy with an ICER of £15,627, the results for these analyses should be interpreted with caution as they are due to small numbers. It should be noted that all p-values for the interaction terms between the subgroup indicator and treatment effect variables were > 0.05 (see Appendix 4, Table 51); that is, there were no statistically significant differences in the incremental cost or QALYs between the subgroup.

Summary and discussion

The results for the within-trial cost–utility analysis reported in this chapter indicate that ITT with observation/conservative management was, over a 24-month follow-up period, less costly than cholecystectomy (MD –£1033). This cost difference was driven by the higher number of cholecystectomy procedures undertaken in the laparoscopic cholecystectomy group as there was no difference in the cost of other events triggering hospital admissions between the groups. A non-significant QALY difference of –0.019 was observed. This is consistent with the C-GALL trial main statistical analysis that showed no difference in primary outcome of SF-36 bodily pain AUC up to 24 months. The significant cost difference, together with a small non-significant QALY difference favouring cholecystectomy (cholecystectomy is more effective by 0.019 QALYs), resulted in an ICER of £55,235. That is, moving from the standard practice of ITT with laparoscopic cholecystectomy to observation/conservative management would result, on average, in lower costs and QALYs with a saving of £55,235 per QALY lost. The probabilistic analysis showed observation/conservative management having a high probability of being cost-effective (97%) and the sensitivity analyses showed these results to be robust to the alternative costing approach using micro-costing for cholecystectomies. Prespecified subgroup analyses (by gender, age or ethnicity) showed that these findings were generally consistent across subgroups.

The main strength of this within-trial analysis relates to the randomised controlled trial (RCT) study design. The resource use and health-related QoL data used were collected prospectively for individual participants as part of the C-GALL trial. Unbiased and accurate estimation of costs and QALY differences are possible due to the randomised treatment allocation. The pragmatic nature of the C-GALL trial together with the ITT principles facilitates the generalisability of the findings to the patient population treated routinely in the UK NHS. A further strength is that the economic analysis was conducted according to a prespecified and agreed health economics analysis plan.

The pragmatic RCT design is a strength of the analysis. The imposed restrictions on elective procedures such as cholecystectomy might have de facto implemented observation/conservative management on both trial groups for some participants for a limited period of time. A second important limitation relates to the short follow-up and the natural history of gallstone disease. Cholecystectomy might be indicated in the future for those participants in the observation/conservative management group as well as those still waiting for surgery in the cholecystectomy groups. Schmidt et al.60 reported on 14-year follow-up for a RCT that randomised 137 participants to observation or cholecystectomy. The authors reported that median time to cholecystectomy for those participants in the observation group was 28 months. The cost of cholecystectomy was the driver of the cost difference. Participants crossing over from observation/conservative management to the laparoscopic cholecystectomy group may result in improved cost-effectiveness for laparoscopic cholecystectomy over a longer time horizon. Therefore, there is a clear case for a longer-term follow-up of the C-GALL trial. Until then, an extrapolation exercise beyond the trial follow-up is conducted in Chapter 5 using a Markov model over a 10-year time horizon.