Pilot randomised controlled trial: health economic analysis

The aim of the economic analysis was to evaluate the level and changes in health service resource use and quality of life to understand the potential economic impact of the CHEMIST ESI in a pilot RCT. For this reason, the economic analysis compared the ESI with the usual-care arm in terms of overall response rates, item completion rates and any trends in the level and changes in health service resource use and health-related quality of life. The pilot RCT analysis was not intended to provide a definite estimate of relative cost-effectiveness, but to inform the design of a larger RCT.

Health services resource use

Data completion

At baseline and 4-month follow-up, all of the participants responded to at least one question in the resource use questionnaire (i.e. there were no participants who completely failed to answer any resource use items). Three participants in the ESI arm did not complete 4-month follow-up and, therefore, had no data available (note that this is also reflected in the overall questionnaire completion rate in the study).

However, a small number of data items were missing. Table 24 shows each resource use item with the corresponding number of missing data at each time point. The data completion rate was high in both treatment arms and at both time points: only one participant had missing data on certain items in each treatment arm. As a result, the response rate was comparable in the two treatment arms. When missing response items were compared, this indicated that the question about consultations with a GP at home and on the telephone may be more likely to have missing data (although this is observed in the case of only two participants in the study). Overall, the results indicate that participants were able to respond to the resource use items.

Frequency of health service use

Table 25 shows the frequency of use of each type of health service in the last 4 months, measured at baseline and the 4-month follow-up. Given the small number of participants, the aim of this section was to summarise the results for each treatment arm, but not to draw statistical inference regarding differences between arms.

At baseline, GP consultations and pharmacy visits were the most common categories of resource use. The mean number of GP consultations at the clinic was 4.21 and 1.65 in the intervention and usual-care arms, respectively, whereas the mean number of pharmacy visits was 5.5 and 4.75, respectively. At baseline, only a small proportion of visits in both treatment arms were because of low mood. The numerical range of the number of visits per participant was plausible for all resource use categories for both treatment arms. For almost all categories of primary/community care, the number of visits reported at baseline (except NHS direct visits) was higher in the intervention arm than in the usual-care arm. However, given the small sample size, the difference between treatment arms can be driven by a small number of observations; for instance, in the present case, the difference in GP visits was primarily driven by four participants in the intervention arm who had 10–15 consultations in the last 4 months. The observed pattern of resource use indicates that the cost-effectiveness analysis should control for any differences in baseline level of resource use.

At the 4-month follow-up, participants in the ESI arm reported a reduction in the number of all primary/community care consultations except for pharmacy and occupational therapist visits, the number of which had increased slightly. The usual-care arm showed a mixed pattern, with a slight reduction in GP consultations, NHS direct visits, and walk-in visits and a small increase in all other categories of resource use.

Table 26 shows the frequency of use of mental health and hospital-based services in both treatment arms at the two time points. The use of mental health services was very low in both treatment arms at baseline, with further reduction at the 4-month follow-up. None of the study participants had consulted a clinical psychologist or psychiatrist, which is not surprising given that the study participants were recruited into CHEMIST with subthreshold depression.

Table 26 also shows the frequency of use of hospital-based services in the two treatment arms at both time points. Only a small proportion of participants used hospital services. In the intervention arm, the number of outpatient appointments at baseline was 2.63 appointments over a 4-month period, which reduced to 1.1 appointments at follow-up. In the usual-care arm, the number of outpatient appointments was 1.95 appointments at baseline, which increased slightly to 2.1 appointments at the 4-month follow-up. Resource use in other categories was low in both treatment arms and time points, and the range of values were within plausible range. It should be noted that given that hospital inpatient visits (particularly those with overnight stays) are costly, a small number of outliers in this category can be highly influential in determining the difference in overall costs between the treatment arms. In the current study, one participant in the intervention arm had inpatient stays lasting 23 nights over three admissions. This was clearly an outlier in the data set and, therefore, was not included in the final cost analysis.

Unit costs of health service use

Unit costs were obtained from national databases, when available, and otherwise from other published sources. Sources and assumptions for unit costs are presented in Table 27.

Intervention cost

The intervention consisted of up to six sessions that were delivered by a trained member of pharmacy support staff (ESI facilitator). Table 28 summarises the number and duration of ESI sessions as well as the cost of the ESI. The mean number of sessions was four, with most participants having at least three sessions, and the range was one to six sessions (one session, n = 2 participants; two sessions, n = 1 participant; three sessions, n = 3 participants; four sessions, n = 3 participants; six sessions, n = 9 participants). Each session lasted an average of 31 minutes, with the range being 5–90 minutes. The total session delivery time per participant (across all sessions) was 136 minutes (mean), with the range being 20–305 minutes. In addition, data were also collected on the time spent on administrative tasks related to the sessions. The average administrative time (across all sessions) was 49 minutes per participant, with the range being 5–170 minutes.

The total ESI time (i.e. session plus administration time) was 185 minutes (mean), with the range being 25–360 minutes. In addition, each session required clinical supervision, lasting an average of 15 minutes, equal to 60 minutes over four sessions per study participant. The total time of the ESI facilitator was multiplied by their hourly cost (assumed to be a member of pharmacy support staff, employed at £9.88 per hour, i.e. scale 4 for local government employees).83 Clinical supervision was provided by a grade 6 researcher (employed at £20.90 per hour).84 The mean total cost of the ESI was £51.40 per study participant, with the range being £9–91 per study participant.

In addition to the above, the cost of training ESI facilitators on the CHEMIST intervention was estimated using the following information: each ESI facilitator attended a 2-day training workshop conducted by a mental health researcher (academic grade 6; hourly rate of £20.90) and a nurse consultant (NHS band 8b; hourly rate of £80.30), and the average cost of venue hire, catering and travel claims per workshop was £500. The average number of ESI facilitators trained in each workshop was 11.7. Based on this information, the cost of training an ESI facilitator was £181.70. This cost is in addition to the intervention delivery cost presented in Table 26.

Total cost of health service resource use

Figure 8 presents the total health service resource use cost in the last 4 months, by treatment arm and time points. Four cost components are presented separately: primary/community care resource use cost, mental health service use, hospital-based service use and intervention cost. These are in line with the data presented above.

FIGURE 8 Pilot RCT: health service resource use cost, by time point and treatment arm. (Figure not reproduced here; see the original article.)

At baseline, primary/community care resource use cost was higher in the intervention arm than in the usual-care arm. This is mainly driven by the larger number of GP consultations in the intervention arm. At the 4-month follow-up, these costs reduced significantly for the intervention arm, but remained stable for the usual-care arm.

The cost of mental health services was very small at baseline and follow-up for both treatment arms. The cost of hospital care was significantly higher in the usual-care arm at baseline and remained higher at the 4-month follow-up, although the difference became smaller. These differences were primarily driven by outpatient appointments and inpatient admissions without an overnight stay.

The total cost per study participant at baseline was £1145 in the usual-care arm compared with £933 in the intervention arm (difference: £212). The total costs were lower for both treatment arms at the 4-month follow-up (i.e. £728 for the usual-care arm and £490.70 for the intervention arm, including the cost of the intervention). The difference in cost was slightly higher at the 4-month follow-up (i.e. £237.30). However, these figures should be interpreted with caution because of the small number of participants.

Health-related quality of life

Health-related quality-of-life data were collected using the EQ-5D and the SF-12v267,73 questionnaires at baseline and at the 4-month follow-up. EQ-5D data were subsequently used to derive utility values using the UK population tariff.

Data completion

Data completion was 100% for all domains of EQ-5D and SF-12v2 at baseline and the 4-month follow-up in both the intervention and the usual-care arms. This indicates that participants found it feasible to complete these questions. This finding was consistent with results of the feasibility study.

Frequency and pattern of EuroQol-5 Dimensions item responses

Figure 9 presents the distribution of completed responses for each domain or item of the EQ-5D questionnaire at baseline and the 4-month follow-up in the intervention arm. At baseline, most participants had some level of problem in at least one of the five domains; study participants reported lowest level of problems in the self-care domain (i.e. 71% participants reported no problems). For the anxiety/depression domain, 6 out of 24, 15 out of 24 and 3 out of 24 participants reported no problems, some problems and severe problems, respectively (i.e. 75% of participants reported experiencing symptoms of anxiety/depression). This is not surprising given that participants with subthreshold depression were recruited in the study. At the 4-month follow-up, there was little change in the mobility, self-care and usual activities domains, a slight increase in the number of study participants reporting severe pain/discomfort and a slight decrease in the number of participants experiencing moderate anxiety/depression. However, the numbers are too small to draw any conclusions about the trend.

FIGURE 9 Pilot RCT: EQ-5D responses at baseline and at the 4-month follow-up for the intervention arm. (Figure not reproduced here; see the original article.)

Figure 10 presents the distribution of completed responses for each domain or item of the EQ-5D questionnaire at baseline and at the 4-month follow-up in the usual-care arm. At baseline, most participants had some level of problem in at least one of the five domains; as with the intervention arm, study participants reported the lowest level of problems in the self-care domain (i.e. 80% participants reported no problems). For the anxiety/depression domain, 75% of participants reported experiencing symptoms of anxiety/depression (i.e. the same as the intervention arm at baseline). However, in general, compared with the intervention arm, fewer study participants in the usual-care arm reported problems at baseline: this was true for all EQ-5D domains. This is important to note for a full-scale RCT, which would need to take account of differences in baseline quality of life between treatment arms: adjusting for baseline utility is recommended in the literature to avoid biasing results of the cost-effectiveness analysis.85 At the 4-month follow-up, there was little change in the mobility, self-care and usual activities domains (same as the intervention arm), a slight increase in the number of study participants reporting severe pain/discomfort and a decrease in the number of participants experiencing severe anxiety/depression. However, the numbers are too small to draw any conclusions about the trend.

FIGURE 10 Pilot RCT: EQ-5D responses at baseline and at the 4-month follow-up for the usual-care arm. (Figure not reproduced here; see the original article.)

Frequency and pattern of Short Form-12 item responses

Figure 11 presents the distribution of responses for each item and domain of the SF-12v2 questionnaire at baseline and at the 4-month follow-up in the intervention arm. All items have five levels, except the two physical health questions (SF-2 and SF-3) that have three levels (see Chapter 2 for scoring of levels). At baseline, most participants had some level of problem in at least one of the domains; participants reported the lowest level of problems in the social functioning domain (which asks about physical health or emotional problems interfering with social activities): in this domain, at least 20% of participants reported no problems at baseline and at the 4-month follow-up. The domains with the highest level of problems were ‘general health’, ‘physical health’, ‘role physical’ and ‘vitality’. On the mental health domain, most participants (around 90%) reported some level of problems at baseline, with most of them reporting problems ‘some of the time’. This is consistent with subthreshold depression in this population. At the 4-month follow-up, there was a trend towards slight improvement in general health (SF-1). In addition, physical health problems (SF-2 and SF-3) appeared to get marginally worse over time. At the same time, the proportion of participants reporting extreme problems decreased in one domain (i.e. SF-11: feeling downhearted/low). Overall, although there was a slight trend towards health improvement in some domains, the numbers are too small to draw any conclusions about the trend.

FIGURE 11 Pilot RCT: SF-12v2 responses at baseline and at the 4-month follow-up for the intervention arm. 4M, 4-month follow-up; BL, baseline. a, For SF-1, SF-8, SF-9 and SF-10, lower level (i.e. 1) indicates better health. For all other questions, lower level indicates worse health. (Figure not reproduced here; see the original article.)

Figure 12 presents the distribution of SF-12v2 responses for the usual-care arm. At baseline, the pattern was similar to that in the intervention arm, that is most participants had some level of problem in at least one of the domains. Study participants reported the lowest level of problems in the social functioning and bodily pain domains: for these domains, at least 15% of participants reported no problems at baseline and at the 4-month follow-up. The domains with the highest level of problems were ‘general health’, ‘physical health’ and ‘vitality’. On the mental health domain, almost all study participants reported some level of problems at baseline. At the 4-month follow-up, there was no clear trend in quality of life, with some domains showing some improvement and others showing slight worsening. The proportion of participants reporting extreme problems remained stable at the 4-month follow-up. Overall, the trend was stable with marginal changes in some domains; however, the numbers are too small to draw any conclusions about the trend.

FIGURE 12 Pilot RCT: SF-12v2 responses at baseline and at the 4-month follow-up for the usual-care arm. a, For SF-1, SF-8, SF-9 and SF-10, lower level (i.e. 1) indicates better health. For all other questions, lower level indicates worse health. (Figure not reproduced here; see the original article.)

Health-related quality of life (utility values)

Figure 13 presents health-related quality of life (or utility) at baseline and at the 4-month follow-up (based on EQ-5D 3L responses, using UK tariff). The figure shows that participants in both treatment arms had low quality of life at baseline, which reduced further during the follow-up period. This is not surprising given the eligibility criteria of the study (i.e. patients with LTCs and subthreshold depression). On average, baseline utility level was lower among participants in the intervention arm than among those in the usual-care arm (difference: 0.096). During the 4-month post-randomisation period, utility values fell slightly in both treatment arms, but the difference remained almost the same (i.e. 0.101). However, as with the cost data, these results should be interpreted with caution as they are based on a small sample size.

FIGURE 13 Pilot RCT: utility values based on EQ-5D 3L responses at baseline and at the 4-month follow-up by treatment arm. Numbers are rounded to second decimal place. (Figure not reproduced here; see the original article.)

Chapter summary

The economic analysis evaluated the overall response rates, item completion rates and trends in the level and changes in health service resource use and health-related quality of life. The ESI cost per study participant was £51.40 (range £9.30–91), including the cost of ESI sessions, administrative work and clinical supervision. Questionnaire response rate, item completion rate and changes in resource use and quality of life over the trial follow-up period were also evaluated. Both health service resource use and quality-of-life questionnaires had very high completion rates among study participants. The overall rate of missing items was also very low. Visits to the GP, nurse and pharmacist were the most common types of resource use items. It was noted that a very small number of study participants had inpatient admissions with high costs. These may not be related to the intervention; therefore, analysis of the full trial should be conducted with and without the cost of inpatient admissions. Regarding the EQ-5D and SF-12v2 questionnaires, there were no missing data among participants; however, an imbalance in baseline utility was observed. Given the observed difference in cost and utility at baseline, the analysis of any future full-scale RCT should adjust for these in a regression analysis.

During the study follow-up, the level of primary and community care use reduced in the intervention arm compared with the usual-care arm, which could be partly attributable to the intervention; however, the sample size was too small to draw an inference. In addition, during the follow-up period there were small changes across quality-of-life domains in both treatment arms, but the overall utility values reduced by the same quantity in both treatment arms. Overall, the economic analysis suggested a high level of questionnaire completion, a low level of item missingness and no out-of-range responses. Moreover, there were differences between treatment arms at baseline that should be adjusted for. Finally, there was an indication that the ESI might reduce primary care costs, although the study sample size was insufficient to draw any statistical conclusions.