Introduction
The economic analysis aimed to assess the feasibility of a within-trial economic analysis in the context of a definitive trial. The secondary aim was to conduct a value of information analysis to determine the optimal sample size for a definitive trial based on the marginal gain from enrolling an additional individual in a trial compared with the additional cost.
Methods
Data
The target population and characteristics of the sample are described in Chapter 5. All cost data for the delivery of the intervention were obtained from the practitioner logs. An example of a log sheet is presented in Appendix 1. All cost data presented are in 2017 Great British pounds.
The participant questionnaire collected data on service usage, and health was measured by the EQ-5D-5L.115 The EQ-5D-5L was chosen for this study as the sample is aged between 12 and 20 years. Thus, given this age range and recommendations from the EuroQol-5 Dimensions (EQ-5D) user guide,116 the EQ-5D-5L adult version is more appropriate for this sample. For ages 12–15 years, both the youth and adult versions can be used, and for age ≥ 16 years the adult version is recommended. The EQ-5D-5L questionnaire was administered online and respondents had to provide an answer to a question before being allowed to move on to the next question. Thus, there are no missing responses for individuals who agreed to fill in the questionnaire.
Practitioner logs
Practitioners were asked to record every session for every client to enable us to calculate how cost-effective the interventions are to deliver. We asked practitioners to record (for each session) the date the session took place, the session number (1–6), if any travelling was required to deliver the intervention (practitioners were requested to record mileage and travel time in minutes) and the number of people present at the session (not including themselves).
Practitioners were asked to record the type of casework being undertaken. Activities would include any form of patient contact (e.g. telephone calls) and non-patient contact (including preparation work and completion of forms for referrals, etc.). We also requested that, for each session, practitioners recorded all activities associated with the preparation and delivery of the session, and the duration of each activity.
Once sessions had been completed, the practitioner log captured how many sessions had been offered, how many had been attended, the reason provided for non-completion of treatment, if known, and whether or not a referral into mainstream services was made.
There were 15 practitioners, and 76 young people were randomised to the intervention. Sixty of the young people contacted by practitioners declined to engage or participate with the intervention (the number of contact attempts and the reason for the young person deciding not to participate was recorded and is reported in Chapter 5). With the exception of one participant, practitioners did not fill out the second page of the log sheet, which captured the time spent contacting the young person and establishing that they did not want to attend the additional service offer of the SBNT/MET interventions.
Service usage
A total of 60 young people out of 112 who were eligible to participate in the follow-up agreed to take part in the health service usage survey. To better understand service usage of young people, all young people who agreed to participate in the follow-up were asked to complete a questionnaire on service usage on the tablet computer during the 12-month follow-up period. The method by which the electronic questionnaire was delivered meant that, in order to go on to the next question, each young person needed to provide a response to the previous question, thus ensuring completeness of data. The response rate was 54% (n = 60 young people). Five of the young people who participated in the intervention completed the follow-up questionnaire.
Primary outcome measure
The primary outcome measure for the health economic analysis was health-related quality of life, measured using EQ-5D-5L.115 The EQ-5D-5L questionnaire was administered at baseline and at the 12-month follow-up to the control and intervention groups. The EQ-5D-5L contains five descriptive questions, tapping five dimensions of health-related quality of life: (1) mobility, (2) self-care, (3) usual activities, (4) pain/discomfort and (5) anxiety/depression. Each question can be answered on a five-point response scale: (1) no problems, (2) slight problems, (3) moderate problems, (4) severe problems and (5) unable. The ranking of states is ordinal and this means that, for example, a move from 1 to 2 and from 2 to 3 does not represent an equal change in health.
Analytic framework
A health and social care perspective was chosen to focus on the cost and benefits of the intervention to the health sector.
Feasibility of a definitive within trial economic evaluation
The within-trial feasibility study looked at the response rates and data provided in the practitioner logs and participant questionnaires to inform the development of survey material for a definitive trial.
Costs
We present the time spent on each activity, the hourly wage of the practitioner undertaking the activity and the cost per element of each activity. Mean, SD and range of travel time of practitioners to deliver the intervention, intervention delivery time and number of participants are presented. Costs for the first two sessions are presented, as only one young person participated in more than two sessions.
Benefits
Owing to a lack of engagement with the intervention, to avoid identification of the small number of participants who engaged with the intervention, the control and intervention groups are combined. Means and medians are presented for health service usage at 12 months’ follow-up (health service usage information was not collected at baseline). Proportions are presented for the five dimensions of the EQ-5D-5L across baseline and follow-up, with means, SDs and ranges presented for mean EQ-5D score at baseline and at follow-up.
Value of information analysis
It was originally proposed that the economic evaluation component for the feasibility study should involve a value of information analysis.117 This analysis would have incorporated the data from the literature and data from the feasibility study. The data needed for the value of information analysis include cost data for the intervention and outcome data measured by quality-adjusted life-years. These data could come from the study and be supplemented with data from the literature. The purpose of this analysis would have been to help clarify the economic case for a definitive study and, by using a variant of value of information analysis, utilise expected value of sampling of information to inform decisions on the optimal sample size for a definitive trial.67 Unfortunately, no useable data were available from the feasibility study and insufficient data were available from the literature to conduct the economic evaluation modelling exercise that underpins a value of information analysis.
Exploratory return on investment analysis was conducted to identify the range of values for benefits and total costs consistent with a return of investment of a minimum level; this is available on request to the authors.
Results
Costs
Summary statistics from the log files of the practitioners for the 16 young people who did engage with the intervention are reported in Table 24. Practitioners successfully returned a complete practitioner log for children in care who attended any sessions.
| Staff costs | Practitioner | Cost (£) per element | |
|---|---|---|---|
| Time spent (minutes), (SD) [range] | Cost (£) of practitioner time (per hour) | ||
| Monthly clinical supervisiona | 60 | 12.85 | 12.85 |
| 2-day training to deliver the intervention | 960 | 12.85 | 205.60 |
| Mean travel time to deliver first session of interventionb | 28 (23) [0–80] | 12.85 | 6.45 |
| Mean session time for first sessionb | 47 (29) [0–90] | 12.85 | 9.64 |
| Mean number of participants in first session | 1.4 (1.1) [0–3] | 12.85 | |
| Mean travel time to deliver second session of interventionb | 23 (18) [0–52] | 12.85 | 4.24 |
| Mean session time for second sessionb | 35 (30) [0–90] | 12.85 | 7.07 |
| Mean number of participants in second session | 1.4 (1.5) [0–4] | 12.85 | |
| Total cost of intervention delivery | 4256 | ||
Health service usage
The mean and median for health service usage over a 12-month period are presented in Table 25.
| Type of service | Number of visits (n = 60) | |
|---|---|---|
| Mean | Median | |
| A&E admissions | 2.38 | 1 |
| Nights in hospital | 1.27 | 0 |
| Number of hospital admissions | 0.83 | 0 |
| Number of outpatient appointments | 1.77 | 0 |
| GP appointments | 5.33 | 3 |
| GP home appointments | 0.03 | 0 |
| Nurse appointments | 2.57 | 2 |
| Nurse home appointments | 0.48 | 0 |
| Number of prescriptions | 5.18 | 2 |
| Social worker home visits | 7.58 | 6 |
| Social worker office appointments | 2.88 | 0 |
| Case worker home visits | 4.68 | 0 |
| Case worker office visits | 1.63 | 0 |
In free-text boxes, other health service usage reported by respondents was with midwives (n = 2) and CAMHS (n = 6). In this free-text box we do not know how many appointments they had with these health professionals.
EuroQol-5 Dimensions, five-level version (health-related quality of life)
Table 26 reports the proportions reporting each category in the baseline and follow-up questionnaire. These results are purely descriptive.
| EQ-5D category | Response | ||||
|---|---|---|---|---|---|
| No problems | Slight problems | Moderate problems | Severe problems | Unable | |
| Baseline (N = 111), n/N (%) | |||||
| Mobility | 99/111 (89) | 8/111 (7) | 3/111 (2.7) | 1/111 (0.01) | 0/111 (0) |
| Self-care | 108/111 (97) | 3/111 (2.7) | 0/111 (0) | 0/111 (0) | 0/111 (0) |
| Usual activities | 91/111 (82) | 11/111 (10) | 6/111 (5.4) | 3/111 (2.7) | 0/111 (0) |
| Pain/discomfort | 80/111 (72) | 19/111 (17.1) | 7/111 (6.3) | 4/111 (3.6) | 1/111 (0.01) |
| Anxiety/depression | 48/111 (43.2) | 27/111 (24.3) | 17/111 (15.3) | 12/111 (10.8) | 7/111 (6.3) |
| Follow-up (N = 60), n/N (%) | |||||
| Mobility | 52/60 (86.7) | 3/60 (5) | 3/60 (5) | 2/60 (3.3) | 0/60 (0) |
| Self-care | 57/60 (95) | 1/60 (1.6) | 2/60 (3.3) | 0/60 (0) | 0/60 (0) |
| Usual activities | 49/60 (81.6) | 9/60 (15) | 2/60 (3.3) | 0/60 (0) | 0/60 (0) |
| Pain/discomfort | 47/60 (78.3) | 8/60 (13.3) | 4/60 (6.6) | 0/60 (0) | 1/60 (1.6) |
| Anxiety/depression | 20/60 (33.3) | 16/60 (26.6) | 15/60 (25) | 6/60 (10) | 3/60 (5) |
At the 12-month follow-up, 60 (54%) of the young people who completed the baseline questionnaire completed the follow-up questionnaire. Five of the young people who participated in the intervention completed the questionnaire. As with the service usage questionnaire, the mode of data collection ensured that the young person could not move on to the next question without completing the previous question. Thus, there are no missing responses in the EQ-5D-5L questionnaires competed.
In Table 27, we use the EQ-5D-5L values to calculate the mean EQ-5D utility scores at baseline and follow-up.
| Time point | Mean utility score (SD), minimum/maximum |
|---|---|
| Baseline | 0.92 (0.14), 0.39/1 |
| Follow-up | 0.93 (0.13), 0.31/1 |
Discussion
Within-trial economic evaluation
This pilot study showed that overall the tools used to collect the cost information and participant data on service usage and health-related quality of life seemed feasible. Practitioners and participants engaged with these materials. However, we have some suggestions to make to further refine these tools to inform future research in the area.
Cost data
If the intervention was to be used in a definitive trial, an additional log sheet should be provided to practitioners to get a better understanding of non-engagement with the intervention. This log sheet should contain information outlining the means of contacting the young person, how many times they were contacted and the length of time they spoke with the young person, as this would be useful to get a better idea of the true cost of the intervention. Data also need to be collected on administrative support, room costs and building maintenance for when the interventions were delivered, if this is relevant.
Health service use
The information from the free-text boxes in the questionnaires suggests that, in a definitive trial, CAMHS should potentially be included in a health service usage questionnaire. As a very small percentage of respondents had GP home appointments, this could potentially be removed from a questionnaire that would be part of a definitive trial.
EuroQol-5 Dimensions
From looking at the proportion of responses to each of the dimensions of the EQ-5D, a large proportion of respondents reported suffering from anxiety and depression. However, it is possible that the health attributes in the EQ-5D may be too narrow for this client group. In a definitive trial, it may be worthwhile including a condition-specific tool that measures quality of life related to mental health.118 In addition, the EQ-5D has ceiling effects that another utility tool, such as the Short Form questionnaire-6 Dimensions, does not (although that tool has floor effects), which may make it a more appropriate outcome measure for a definitive trial.67
Summary
The limited engagement of young people with the intervention means that we cannot be certain if the practitioner logs would be feasible in a definitive trial. The health service usage questionnaire with some minor modifications related to including CAMHS and removing GP home appointments would be appropriate for a definitive trial. In a definitive trial it may also be appropriate to use a condition-specific tool that measures quality of life related to mental health.