Health economic evaluation

This chapter presents details of the economic evaluation. It includes information on the use and cost of services and the cost-effectiveness of CBT + SMC compared with SMC alone.

Introduction

People with DSs tend to have high use of emergency medical services and other hospital-based services,138,139 particularly associated with delayed diagnosis and access to appropriate treatment.66 There has been an increase in the diagnosis of DSs,140 which suggests a substantial burden of illness to patients, health systems, carers and society at large.65 DS-specific CBT has been evaluated in this study as a potentially effective intervention to determine whether or not it could reduce seizure frequency and improve quality of life, as well as psychosocial functioning. However, any change to the way that care is provided is likely to have resource consequences, and this is certainly the case with psychological interventions given the limited supply of trained therapists. Therefore, it is essential to assess the cost-effectiveness of the CBT + SMC intervention compared with SMC alone. As healthcare decision-makers have to make comparisons across diverse clinical areas, it is also necessary to use a generic measure of health outcome, specifically quality-adjusted life-years (QALYs).

The aims of the health economic component are to (1) compare the service use and costs of the CBT + SMC arm with those of the SMC-alone arm at baseline and over the follow-up period, and (2) assess the cost-effectiveness of CBT + SMC relative to SMC alone at the 12-month follow-up in terms of cost per QALY gained.

Economic evaluation methods

Perspective and resource use

The primary economic evaluation adopted a health and social care perspective. This approach is in line with recommendations from the National Institute for Health and Care Excellence (NICE) and is a convention followed in most health technology assessments conducted in the UK. However, it is often the case that interventions have effects outside the health and social care system, so we approach a societal perspective in the secondary analyses. To do this, we include costs associated with lost employment and costs of care provided by family and friends alongside the health and social care costs. We do not include patient time costs, costs associated with lost leisure/recreational activities or costs of any health impacts on carers; therefore, this is not a complete societal perspective.

The number of therapist contacts was recorded as part of the study. Other health and social care service use, receipt of care from family members and time off work were measured at baseline and at 6 and 12 months’ follow-up using a bespoke version of the Client Service Receipt Inventory (CSRI).141 The CSRI is a widely used questionnaire that is usually adapted for each specific study and allows for the collection of retrospective data on health and social care use. The CSRI listed key services and to measure contacts with individual professionals the participant was asked whether or not this had occurred, how many times in the previous 6 months and what the typical duration of the contact was. For inpatient care, the CSRI asked for information on the specialty and number of days in hospital. For outpatient and accident and emergency (A&E) care it collected information on the number of visits and those visits made by an ambulance. Medication received (for all reasons) was also recorded. If an individual had a missing number of service contacts, then the median of others with these data was used.

The questions on care from family or friends specified specific areas: help in the home, help outside the home, personal care and help with medical problems. The participant was asked to state how many hours of care per week were provided in these areas because of their health problems. Participants were asked to state how many days they had lost from work during the previous 6 months because of health problems. Some services were measured with the CSRI but not costed owing to lack of appropriate values. This included self-help groups and online resources.

In most studies, including CODES, the CSRI relies on participant self-reporting and so may be prone to recall accuracy problems. To address this, we also used data from the HES as an alternative for assessing use and estimating the hospital inpatient, outpatient and emergency care costs. Data were requested from NHS Digital, Electronic Data Research and Innovation Service (NHS Scotland) and NHS Wales Informatics Service, and permission was granted for the use of these data. Ethics approval was granted for us to apply for HES data for participants who had not formally withdrawn from the study; we supplied these organisations with the participants’ personal identifier numbers and NHS (or in Scotland Community Health Index) numbers, and we were sent the data by personal identifier number but without other identifiers. Information was recorded differently for England, Scotland and Wales, although it was still possible to combine the data.

Cost calculations

Intervention costs were derived from information on salaries, overheads, training and supervision. The salaries were based on self-reported bands of the therapists taking part in the study. Costs of other health and social care contacts were calculated by combining the service use data with recognised national unit costs.142,143 Average wage rates144 were used to value lost work and care provided by family/friends, assuming that time spent providing this could be spent on work or leisure. The use of wage rates in principle reflects the opportunity cost of this time. Medication costs were calculated using information in the British National Formulary.145 Nationally applicable unit costs were also applied to the HES data that consisted of inpatient days, A&E visits and outpatient contacts. Costs are reported in 2017/18 Great British pounds. The list of unit costs used here is given in Appendix 8.

Quality-adjusted life-years

The main outcome measure in the economic evaluation was QALYs derived from EQ-5D-5L.116 QALYs are a composite measure combining HRQoL and time, and allow comparisons to be made across different conditions. The EQ-5D-5L is a short scale that is used to measure HRQoL considering five dimensions: mobility, self-care, usual activities, pain/discomfort and anxiety/depression. For each domain, participants rate their current health using a five-point scale to describe the extent to which they can perform activities (ranging from ‘No problems/symptoms’ to ‘Unable to/extreme problems’). This gives rise to 3125 unique health states ranging from 11111 to 55555. The scale also includes a VAS that has been described in Chapter 2. Participants completed the EQ-5D-5L at all study points. Weights that are assumed to be proxies for ‘utilities’ were applied to the health states measured by the EQ-5D-5L. These weights are anchored by 1 (reflecting full health) and 0 (reflecting death), and were obtained from work conducted by Devlin et al.146 QALYs were generated from the utility scores using area under the curve methods with an assumption of linear change between adjacent time points.

Analysis

Analyses were conducted using Stata software version 15. The use and cost of individual services were described but statistical tests were not conducted to analyse the group differences. Total health and social care costs and societal costs were compared between the two arms over the follow-up period. This was carried out using an ordinary least-squares linear-regression model with follow-up cost as the dependent variable and the baseline costs and group identifier included as independent variables. Cost data usually follow a skewed distribution and, therefore, bootstrapping methods were used to produce 95% CIs around the cost differences. Comparisons of QALYs over the follow-up period were also made using a regression model, controlling for baseline utility scores.147

To assess cost-effectiveness, costs were combined with QALYs and the primary clinical outcome measure. Missing costs and QALYs were imputed using a best-subset regression procedure in Stata. Incremental costs and outcomes were estimated using regression models and 1000 differences obtained from bootstrapped resamples were plotted on a cost-effectiveness plane to investigate uncertainty around the incremental cost-effectiveness ratio obtained from point estimates. The intervention is considered ‘dominant’ if it is less expensive and more effective than SMC. If it is more expensive and more effective, incremental cost-effectiveness ratios were constructed to indicate the extra cost incurred to achieve a one-unit reduction in DS frequency or one extra QALY. To explore uncertainty around cost-effectiveness estimates (based on QALYs), we plotted cost-effectiveness planes (based on incremental cost–outcome pairs from 1000 bootstrapped resamples) and plotted cost-effectiveness acceptability curves (CEACs) (that were derived using the net benefit approach). We did not produce cost-effectiveness planes or CEACs for the analyses based on the difference in seizure frequency. CEACs reveal to decision-makers the probability of the intervention being cost-effective compared with the alternative, given different (implicit monetary) values placed on incremental improvements in the outcome measurement.

The HES data were analysed by examining the numbers of participants receiving inpatient care, receiving outpatient care or visiting A&E departments at baseline and in the 6 months prior to the 12-month follow-up, and making comparisons between the two arms. Numbers of contacts/days were also compared, as were mean costs. Owing to restrictions agreed when applying for the data, we did not link HES data to any variables other than participant ID and group idenifier. For this reason, these could not be directly used in the cost-effectiveness analysis.

Sensitivity analyses

In the sensitivity analyses, the Short Form questionnaire-6 Dimensions (i.e. utility score) (SF-6D), generated from the SF-12v2, was used to derive QALYs.148 The SF-12v2 contains a wider range of items than the EQ-5D-5L and, thus, could reflect the impact of DSs more appropriately. A comparison of HRQoL has demonstrated that QALYs derived from the SF-6D performed well for people with epilepsy in discriminating between those with and those without seizures over a 2-year follow-up period,149 although there was no relevant follow-up data for DS patients. Further sensitivity analyses were conducted by increasing and decreasing the intervention costs by 25% and 50%. We examined the impact on societal costs of using alternative unit costs for informal care and lost employment. Specifically, we used the healthcare worker unit cost (£26 per hour) to derive informal care costs and a minimum wage of £7.50 for informal care and lost employment.

Results

Service use

Service use during the 6-month period prior to baseline and at the 6- and 12-month follow-up period is described in Table 18. At baseline, > 80% of the participants in each arm had contact with GPs. Other community services were used by far fewer participants, with the next most commonly used being practice nurses. The mean number of contacts with community services was largest for GPs, followed by home helps. There were no obvious differences in the use of community services between the arms. During the 12-month follow-up period, around three-quarters of each trial arm had contacts with GPs. Other community services were again used by relatively few participants, although there were some differences between arms with the CBT + SMC arm participants more likely to have contact with practice nurses and the SMC-alone arm more likely to have contact with physiotherapists. It is also evident that a small number of participants were reporting receipt of CBT at baseline in each of the arms. At follow-up, this occurred for the CBT + SMC arm only. It may be that the seven participants who reported such contacts were referring to the actual trial intervention, or it may be that they were in fact receiving non-intervention CBT.

Turning to hospital use, Table 18 shows that around one-third of each arm had inpatient stays during the baseline period, with a similar average number of inpatient days. Outpatient appointments also occurred for about one-third of each arm. Use of an ambulance was slightly more likely in the SMC-alone arm, as were visits to A&E departments. Contacts with psychiatrists occurred for about three-quarters of each arm and contacts with neurologists for > 80%. During the entire 12-month follow-up period, there was less use of inpatient care, especially in the CBT + SMC arm. Outpatient use remained similar to baseline and there was reduction for both trial arms in visits to A&E departments. There were also reductions for both trial arms in contacts with psychiatrists and neurologists.

Receipt of informal care was very high in both trial arms, with nearly half of the participants at baseline reporting having received help with personal care and over half reporting having received help inside and outside the home. Over one-third reported that informal carers spent some time ‘on-call’ (i.e. had to remain with the participant in the home even if no care was being directly provided). When this occurred, it accounted for the most hours of care received per week. During the follow-up period, levels of informal care remained high and with no clear differences between the arms.

Service costs

The mean number of CBT sessions attended by participants in the CBT + SMC arm was 10. Combined with the unit cost of a clinical psychologist (used because this is the largest professional grouping of the current therapists) produced associated mean costs of £1064. One participant in the SMC-alone arm had access to the intervention and the mean cost for the arm was estimated at £7.

Mean costs of community services during the baseline period were highest for GP contacts and home help providing both personal care and support for household tasks (Table 19). The latter was more expensive for the SMC-alone arm. At follow-up, intervention costs were the highest healthcare cost for the CBT + SMC arm. At both the 6-month and the 12-month follow-up, GPs and home help service items again accounted for relatively high costs. Medication costs at baseline were similar between the arms, whereas at both follow-up points they were somewhat higher for the SMC-alone arm. Inpatient costs were similar at baseline but higher at follow-up for the CBT + SMC arm. However, the difference at 12 months of £332 is equivalent to < 1 extra inpatient day. Other hospital costs were relatively similar between trial arms at both baseline and follow-up, as were the total hospital costs.

Informal care costs were substantial at all time periods and for both trial arms. No major differences between the arms were apparent. Lost employment costs were higher for the SMC-alone arm at both baseline and follow-up.

The total mean health and social care costs at baseline were £4092 for the SMC-alone arm and £3798 for the CBT + SMC arm. From baseline to the 6-month follow-up, the costs were £2560 and £2922, respectively, whereas at the 12-month follow-up they were £2683 and £2915, respectively. The difference in health and social care costs over the entire follow-up period, adjusted for baseline, was £1834 (bootstrapped 95% CI £478 to £3475), which is around £840 more than the extra cost accounted for by the intervention itself.

The mean societal costs at baseline were £33,261 for the SMC-alone arm and £29,066 for the CBT + SMC arm. At 6 months, the societal costs were £22,828 for the SMC-alone arm and £26,183 for the CBT + SMC arm. During the combined 12-month follow-up period, the societal costs were £55,503 and £52,933, respectively. The imputed difference, adjusted for baseline costs, was £6566 (bootstrapped 95% CI –£5909 to £18,919).

Quality-adjusted life-years

Table 20 reports the utility scores and QALYs derived from both the EQ-5D-5L and the SF-12v2 measures. At baseline and at the 6- and 12-month follow-ups, the CBT + SMC arm had a slightly higher utility score. The CC analysis shows a QALY difference of 0.0416 in favour of the CBT + SMC arm. After imputation, the difference is similar at 0.0457. However, the QALY difference after controlling for baseline is less: 0.0152 for imputed cases (bootstrapped 95% CI –0.0106 to 0.0392).

When the SF-12v2 was used to generate the SF-6D, it was shown from the CC analysis that CBT + SMC accrued 0.0293 more QALYs than SMC alone. After imputation, the difference was 0.0233 QALYs. When baseline utility was controlled for, the differences became 0.0231 and 0.0157 QALYs, respectively. We also used the crosswalk method150 to convert the EQ-5D-5L scores to tariffs derived for the EuroQol-5 Dimensions, three-level version (EQ-5D-3L). This resulted in lower utility scores for both trial arms and an incremental QALY gain for CBT + SMC of 0.0089 QALYs.

Cost-effectiveness results

After imputation and controlling for baseline costs and utility, the CBT + SMC arm was shown to have incremental costs of £1834 and incremental QALYs based on the EQ-5D-5L of 0.0152 (Table 21). This gives an incremental cost per QALY of £120,658 compared with SMC alone. The incremental QALYs based on the SF-6D were 0.0157 QALYs, resulting in an incremental cost per QALY of £116,815.

Uncertainty around the cost-effectiveness results based on the EQ-5D-5L is depicted in Figure 9. Most of the bootstrapped replications fall in the top-right quadrant in which costs and QALYs are higher for the CBT + SMC arm. The percentage of replications falling below the red line indicating that the intervention is cost-effective based on a threshold of £20,000 per QALY is 27%. The corresponding CEAC (Figure 10) indicates that the probability of cost-effectiveness increases as the threshold value on a QALY rises, but is relatively low. Similar results apply to the cost-effectiveness results derived from the SF-6D (Figures 11 and 12).

FIGURE 9 Cost-effectiveness plane using the EQ-5D-5L and costs measured from the health and social care perspective. (Figure not reproduced here; see the original article.)

FIGURE 10 Cost-effectiveness acceptability curve using the EQ-5D-5L and costs measured from the health and social care perspective. (Figure not reproduced here; see the original article.)

FIGURE 11 Cost-effectiveness plane using the SF-6D measure derived from the SF-12v2 and costs measured from the health and social care perspective. (Figure not reproduced here; see the original article.)

FIGURE 12 Cost-effectiveness acceptability curve using the SF-6D derived from the SF-12v2 and costs measured from the health and social care perspective. (Figure not reproduced here; see the original article.)

With the crosswalk method used for the utility scores, the incremental cost-effectiveness ratio (ICER) for CBT + SMC increased to £206,067 per QALY. When the intervention costs were reduced by 25%, the ICER fell to £103,224 per QALY (Table 22) and further to £85,724 when the costs were reduced by 50%. By definition, the ICERs rise with higher intervention costs to £138,158 and £155,592 for 25% and 50% increases in intervention costs, respectively.

We have shown that societal costs are higher by £6566 for the CBT + SMC arm. With our main analysis showing a QALY difference of 0.0152, this results in an ICER of £431,974 per QALY.

Applying the unit cost of a healthcare worker resulted in increased informal care costs for both trial arms (£66,239 for the SMC-alone arm and £65,486 for the CBT + SMC arm). The difference between the arms (£2359) was not statistically significant (95% CI –£19,583 to £24,280). Applying the minimum wage of £7.50 to informal care would reduce the informal care cost to £19,209 and £18,990, respectively. Lost employment costs would be reduced from £2303 to £1174 for the SMC-alone arm and from £1721 to £903 for the CBT + SMC arm using the minimum wage.

Cost-effectiveness based on the number of seizures

The difference in median seizure rate between the two arms at 12 months is three seizures (see Table 14). The difference in costs is £1251 for the CC analysis and £1834 after imputation, producing ICERs of £41,740 and £611, respectively. The results imply that the cost for an avoided seizure over the 4 weeks prior to the 12-month follow-up is £611. There is, however, no accepted (cost-effectiveness) value of a seizure avoided and, therefore, it is simply a value judgement as to whether or not this represents value for money.

Hospital Episode Statistics analysis

The findings from the analysis of the HES are shown in Table 23. Between baseline and 6 months, > 80% of each arm had outpatient contacts, which fell substantially for the follow-up period. The number of outpatient contacts was very similar between arms and over time. Direct comparison with the CSRI data is difficult when we look at all outpatient contacts because we separated out psychiatrist and neurologist contacts with the CSRI data. At baseline, the CSRI data report 76% of patients having psychiatrist contacts in the CBT + SMC arm and 78% in the SMC-alone arm. The figures for neurologist contacts are 83% and 84%, respectively. The figure for other hospital care is 13% for both trial arms. Although all participants should have had neurologist and psychiatrist contacts, both the CSRI and the HES data indicate similar levels of use, and this is < 100%. Between one-fifth and one-quarter of patients in both trial arms at both time points have missed appointments. This information is not provided in the CSRI.

If we combine the cost of outpatient care from Table 23 with the psychiatrist and neurologist costs (these categories should be mutually exclusive), we find similarities with the costs in Table 19. (Other hospital costs are not included here.) At baseline, this cost is £497 for SMC alone and £491 for CBT + SMC (whereas the HES data show costs of £498 and £487, respectively). In the 6 months prior to the 12-month follow-up, the costs are £384 for SMC alone and £359 for CBT + SMC. The HES-based costs are £350 and £436, respectively. Therefore, there is reasonably good agreement between the CSRI and the HES data for outpatient care, except for CBT + SMC for which HES gives higher costs. At follow-up, the CBT + SMC arm does make more use of this than the SMC-alone arm, but the difference is not large.

The HES data also show very similar rates of A&E department contacts between trial arms, with participants of both arms less likely to attend in the follow-up period. The rate of use at baseline for the CBT + SMC arm is very similar when comparing HES (46%) with CSRI (47%) data; there is more of a difference for the SMC-alone arm (45% and 57%, respectively). The HES-based costs of A&E care are rather different at baseline from the CSRI-based costs (with higher costs for the CSRI), particularly for the CBT + SMC arm. However, at the 12-month follow-up there are very similar costs (£119 and £138 for the SMC-alone arm and £116 and £131 for the CBT + SMC arm, respectively).

At baseline, HES report that 31% of the SMC-alone arm and 33% of the CBT + SMC arm use inpatient care. The CSRI data are 36% for each arm. However, the costs of care are noticeably different when using the CSRI or the HES data. Costs based on the former are higher at baseline for both trial arms and for the CBT + SMC arm at follow-up. The implication is that although participants report a similar rate of inpatient use to HES, they also report more days in hospital. The difference at baseline between methods is comparable for the two arms, whereas at follow-up it seems to result in costs that are potentially overestimated by £341 for the CBT + SMC arm. If we were to reduce the overall incremental costs of £1834 for the CBT + SMC arm compared with the SMC-alone arm by this amount, the ICER is reduced to £95,096 per QALY (based on the EQ-5D-5L).

Discussion

Main findings

To our knowledge, this is the first economic evaluation of CBT for people with DSs participating in an RCT evaluating a psychotherapeutic intervention. The main clinical results of the study have shown that although CBT + SMC was superior to SMC alone on most of the secondary outcomes, the primary outcome was not significantly different between the trial arms (see Chapter 3). In this chapter, we have found that CBT + SMC produces slightly more QALYs than SMC alone, but that the difference is not substantial. This is the case whether the EQ-5D-5L or the SF-6D was used to generate QALYs. The costs for the CBT + SMC arm were higher than those for the SMC-alone arm, with most of the extra cost accounted for by the intervention itself. It was apparent that the intervention did not seem to offset costs in other sectors. It is not unusual for interventions such as this to result in higher health costs than usual care, and this is not in itself evidence against cost-effectiveness. It is recognised that to achieve improved outcomes it is often necessary to have higher costs of care and this is where the incremental cost per QALY is relevant. Interventions that are assessed by NICE are usually considered to be cost-effective if they achieve a cost per QALY below £20,000–30,000. In this study, the cost per QALY for CBT + SMC was £120,658 compared with SMC alone. The figure was similar when QALYs were generated from the SF-6D. For this reason, the addition of CBT to SMC would be unlikely to be considered cost-effective. Costs were not markedly different between the trial arms when informal care and lost employment costs were included.

It was of interest that costs in both trial arms fell for some services. This is not unusual in trials, given that recruitment often requires various investigations, and health improvements (often experienced by receiving care in the control groups) should lead to reductions in use. The study was not, however, designed to explore changes in costs. There were not obvious cost differences between the two arms (other than for the therapy itself). However, medication costs were higher for SMC alone. This may be because of compensation for the lack of CBT in that arm.

Limitations

The analyses were conducted in a way that is consistent with similar NIHR-funded economic evaluations. However, a number of limitations need to be considered when interpreting these results. First, we relied on self-report data for the evaluation. The CSRI asked participants to recall what services they had received in the previous 6 months, which may have led to inaccuracies. There is, however, no reason to suppose that potential inaccuracy would differ between arms, and for most services that we wanted to collect data on it was the only option. The possibility of errors did, however, lead us to examine HES data in addition to those collected with the CSRI. For outpatient use, the findings from the CSRI and from HES were remarkably similar. For A&E visits, the costs were somewhat different at baseline but at follow-up were very similar using the different methods. The methods produced the largest discrepancies for inpatient care, with higher costs indicated by the CSRI. When patients are admitted they are likely to be particularly unwell and recollection of exact numbers of days may be exaggerated. When we produced an ICER taking this into account, it fell to £95,096.

What is clear from the analyses is that both the CSRI and the HES suggest that fewer than 100% of participants had psychiatrist and neurologist contacts during the baseline period. The trial procedures were that such contacts should take place for recruitment to occur. The CSRI data could quite easily be underestimates of real use, but HES data seem to validate the CSRI. Although HES should capture all contacts, it is still reliant on contacts being recorded and reported, which may not always have occurred. Some of the missed appointments reported in HES will also have been for neurologists and psychiatrists. One clinician in one site provided both neurological and psychiatric input (see Appendix 6), but it is unlikely that would be the reason for the figures reported here.

Second, although CSRI data covered baseline and the whole follow-up period, the HES data covered the baseline period and the 6 months prior to the 12-month follow-up. This means that we are not able to make full comparisons between the two methods. However, the comparisons we have made are still informative and largely validate the CSRI method.

Third, the outcome measure used in the economic evaluation was QALYs generated with the EQ-5D-5L. This is the approach recommended by NICE and used by most UK health economists. However, it is unclear whether or not the EQ-5D-5L is a sensitive measure in this patient group, and given that most secondary outcomes favoured CBT + SMC we might have expected larger differences. Use of the SF-6D-based QALYs revealed smaller differences between groups.

Fourth, the tariffs that were attached to the health states were taken from work conducted by Devlin et al.146 Since we embarked on this study, concerns have been raised about this value set,151 with some arguing for conversion to tariffs derived for the EQ-5D-3L.151

Fifth, the time horizon may be too limited to fully assess cost-effectiveness; it may be that the effects of CBT continue long term. Although a 1-year follow-up is reasonable for a trial, we might wish to conduct modelling work to determine the effects beyond this period. For CBT + SMC to be cost-effective, there would need to be ongoing differences in HRQoL compared with SMC alone. If ‘top-ups’ were required, then this would have cost implications. Continued health improvements would hopefully lead to future reductions in health service use that would also improve cost-effectiveness. We addressed this in sensitivity analyses and the ICERs rose substantially when using the crosswalk method.

Finally, there was one person in the SMC-alone arm who received the intervention. There were also participants in both trial arms who reported other CBT and talk therapy. However, the mean costs were low (see Table 19) and this would not substantially affect the results. In addition, we were not able to verify the nature of this additional therapy owing to the risk of unblinding the research workers. We cannot therefore be sure as to its nature or extent and had no analogous HES data against which to compare it.

Conclusions

The study found that CBT when added to SMC resulted in increased healthcare costs. QALYs were also increased but not sufficiently for the intervention to be considered cost-effective at the 12-month follow-up.