Workstream 4 addresses the third objective of the programme: to determine the clinical and cost-effectiveness of online behavioural interventions compared to standard clinical care.
This work has been published:
https://doi.org/10.1136/bmjopen-2020-045583.2
https://doi.org/10.1136/bmj-2022-072007.52
https://doi.org/10.1007/s10198-023-01649-9.53
The trial was registered with the International Standard Randomised Controlled Trial Number registry (ISRCTN79282252) on 28 November 2019, prior to enrolling the first participant. The trials were approved by South Central–Oxford A Research Ethics Committee (19/SC/0351).
Multicentre randomised controlled trials
Two separate, pragmatic, multicentre, unmasked individually randomised superiority trials were conducted to evaluate the clinical and cost-effectiveness of the two ECO interventions developed in workstream 3: ECO for parents/carers of children with eczema, and ECO for young people with eczema.
The trials were conducted within GP in the UK NHS. Most trial processes were online from the start, with the exception of consent forms from parents/carers of 13- to 15-year-olds which were originally paper-based. In response to COVID-19, these consent forms were digitised, making all trial processes fully online. Participants were invited via search of electronic health records and postal invitation from participating practices around four regional centres: Wessex, West of England, East Midlands, and Thames Valley and South Midlands. Potential participants were sent an invitation pack containing a participant information sheet and the study uniform resource locator (URL) to register if they wished to take part. After registering, participants were asked to provide informed consent and complete screening and baseline measures online. For children under 16, the invitation was sent to their parents/carers. In the trial for parents/carers, informed consent and questionnaires were completed by the parent/carer. In the trial for young people, parental consent and young person’s assent were sought for participants aged under 16 and young person’s consent was sought for participants aged 16 and over. Young people aged 13–25 years were asked to complete their own questionnaires.
Participants were eligible if they were a young person aged 13–25 with eczema or the parent/carer of a child aged 0–12 years with eczema, had a GP electronic record code for eczema and had a prescribed eczema treatment in the past 12 months. On baseline screening, participants were excluded if they had a baseline POEM54 score of < 5 to exclude very mild or inactive eczema. Participants were also excluded if they were unable to give informed consent; were unable to read and write English (as the intervention content and outcome measures were in English); had taken part in another eczema study in the past 3 months; or had no internet access. Only one person per household could take part in either trial, as the intervention content was similar. This has been reported in Santer et al.52
Outcome measures
All participant-reported outcome measures were collected online via LifeGuide software.51 Automated reminder texts and e-mails were sent by the software, and non-responders were followed up by the trial team via text message, e-mail or telephone call.
The primary outcome for both trials was the difference in patient-reported eczema severity between the usual care and intervention group, measured by POEM every 4 weeks over 24 weeks.54,55 POEM includes seven questions about the frequency of eczema symptoms over the previous week that give a total score from 0 (no eczema) to 28 (worst possible eczema).
Secondary outcomes included POEM scores 4-weekly over 52 weeks, eczema control at 24 and 52 weeks, measured by RECAP,56 itch intensity57 at 24 and 52 weeks (young people only), patient enablement at 24 and 52 weeks,58 quality of life at 24 and 52 weeks: measured by Child Health Utility Nine Dimensions (CHU-9D)59 for children aged 2–12 years and EuroQol-5 Dimensions, five-level version (EQ-5D-5L)60 among young people aged 13–25. Health service use and medication use were measured by medical notes review for the 3-month period prior to baseline and the whole 52-week trial period.
We also measured prior beliefs about the effectiveness of the intervention and other online resource use (websites or apps for eczema). Process measures included self-reported barriers to adherence to eczema treatments measured at 24 and 52 weeks using the Problematic Experiences of Therapy Scale (PETS) and frequency of eczema treatment use (treatment adherence) was measured by self-report at 24 and 52 weeks. Intervention usage data (e.g. time spent on the intervention, number of logins, pages viewed) for each participant were recorded by LifeGuide software for the duration of the 52-week trial period. See Report Supplementary Material 4 for the questionnaire booklet of outcome and process measures.
Sample size and randomisation
The sample size calculation was based on 4-weekly POEM scores using repeated measures from baseline to 24 weeks, seeking to detect a MCID of 2.5 points between groups [standard deviation (SD) 6.5]. Assuming a correlation between repeated measures of 0.70, with 90% power and 5% significance, this gave a target sample size of 121 per group in each of the two trials. Allowing for 20% loss to follow-up resulted in a target sample size of 303 in each of the two trials. Participants were randomised online via LifeGuide software to either (1) usual eczema care or (2) online intervention plus usual care. Randomisation was carried out in random permuted blocks (size 4 and 6) and stratified by age (children 0–5 vs. 6–12 years; young people 13–17 vs. 18–25 years), baseline eczema severity [POEM categories55 6–7 (mild); 8–16 (moderate); 17–28 (severe)] and recruitment region (4 regions, as above).
Key findings
Analysis was conducted following a statistical analysis plan (SAP) agreed in advance with the independent Trial Steering/Data Monitoring Committee and reported according to Consolidated Standards of Reporting Trials (CONSORT) guidelines,61,62 and the two trials were analysed separately. Full results have been reported elsewhere.52
Three hundred and forty parents/carers of children (169 usual care; 171 intervention) and 337 young people (169 usual care; 168 intervention) were randomised into the trials. The mean baseline POEM was 12.8 (SD5.3) for children and 15.2 (SDd5.4) for young people. Three young people withdrew from follow-up but did not withdraw their data. All randomised participants with available outcome data were included in their randomised arm, in the intention-to-treat analyses. Follow-up rates were excellent: 92.4% (314/340) parents/carers and 90.2% (304/337) young people at 24 weeks. CONSORT diagrams for the trials can be found in Appendix 5, Figures 9 and 10 (see Appendix 6, Tables 7 and 8 for full baseline characteristics).
We found that our two brief online behavioural interventions to enable self-management for eczema for parents/carers of children and for young people provided a useful benefit in eczema severity at 24 weeks. After controlling for baseline severity and prespecified covariates (age, recruiting centre, sex, ethnicity, prior belief in the intervention, previous use of a website for eczema, and parental education in the parent and carer trial), compared with usual care over 24 weeks, eczema severity (POEM) improved in the intervention groups: −1.5 [95% confidence interval (CI) −2.5 to −0.6] for parents/carers, and −1.9 (95% CI −3.0 to −0.8) for young people. Effects were sustained for 52 weeks in both trials. The magnitude of the treatment effects ranges from a small improvement to larger than the MCID of 2.5 points. No harm or unintended effects were identified in either group.
We did not detect a difference in the use of eczema treatments between groups, yet did find significant differences between groups in patient enablement instrument (PEI) scores. Enablement showed an important difference favouring the intervention group in both trials [adjusted mean difference at 24 weeks −0.7 (95% CI −1.0 to −0.4) for parents/carers and −0.9 (95% CI −1.3 to −0.6) for young people].
Economic evaluation
Aim
To undertake a within-trial economic evaluation of the two online interventions from an NHS perspective.
Methods
We conducted two primary cost–utility analysis (CUA), using individual-level data collected within the trials, to estimate the cost-effectiveness for each of the interventions plus usual care compared to usual care alone in terms of incremental cost per QALYs at 52 weeks. A secondary cost-effectiveness analysis (CEA) was conducted for each of the two trials, and the two trials combined, using the trial primary outcome measure POEM.
The interventions were developed using research funding and therefore the ‘development costs’ were not included in the economic evaluations but are reported separately.63 However, ‘maintenance costs’ likely to be incurred to keep the intervention running were included in the analysis. This included e-mail support, server-to-host interventions, server updates and domain names. The maintenance costs were split equally among participants, although when implemented on a larger scale, the per participant cost would likely be smaller.
Wider NHS resource use for eczema (primary and secondary care medication and service use) was captured via medical notes from GP practices. This was for a 52-week period plus a 3-month pre-baseline period to allow for adjustments for baseline costs in the adjusted analyses. All resource use was valued using UK published unit costs (in Great British pounds 2020–1).64,65
In terms of outcomes, quality-adjusted life-years (QALYs) were estimated via utility scores elicited using the proxy CHU-9D for parents/carers and the EQ-5D-5L instrument for young people. Utility measurements were collected at baseline, 24 and 52 weeks via online questionnaire. The responses received were converted to utility scores using the Stevens59 value set for the CHU-9D and using UK preference weights in line with recommendations at the time of analysis for the EQ-5D-5L.66,67 These utility values were used to estimate the number of QALYs generated over the 52-week trial period, using both linear interpolation and area under the curve analysis with and without baseline adjustment.68 Separate CUAs were conducted to estimate the incremental cost per QALY for each trial.
The CEA was undertaken using the primary outcome for the trial, the POEM, where a lower score represents less severe eczema. Since a two-point difference in POEM score is deemed to be a difference that would be noticeable and important to people with eczema,69 we use this to estimate the incremental cost per success.
The mean (SD) cost per participant was estimated for each randomised group in order to estimate the mean difference (95% CI) in cost per participant between groups. Alongside this, the mean (SD) utility and mean (SD) QALYS per participant per randomised group are presented along with the mean difference (95% CI) in utility and QALYs between groups.
A regression-based approach70 using complete-case data was used in the base-case CUAs. Both unadjusted and adjusted results are presented, but the adjusted analyses constitute the main base-case analysis adjusting for baseline POEM/utility/cost (as appropriate), recruitment region and the covariates which were pre-specified in the SAP as possible confounders: age, gender, ethnicity, prior belief in the intervention, carer education if appropriate, and prior use of a website or app for eczema information or advice. Secondary CEA is presented for each trial and both trials combined as POEM was used in both trials (unlike utility which was captured using a different instrument in each trial to reflect the age of participants). Incremental cost per success is presented. Generalised linear models were used to estimate adjusted incremental costs and effects in the CEAs given the binary outcome; this assumes that costs and benefits are not correlated.
Since costs and outcomes were likely to be skewed, non-parametric bootstrapping was used to determine the level of sampling uncertainty surrounding the mean incremental cost-effectiveness ratios (ICERs) by generating 10,000 estimates of incremental costs and benefits. Cost-effectiveness acceptability curves (CEACs) were produced, which show the probability that the interventions are cost-effective for different values of willingness to pay (WTP). The National Institute for Health and Care Excellence threshold of £20,000–30,000 per QALY is used.66
Sensitivity analyses (SAs) to explore the impact of missing data on results were undertaken using multiple imputation.71
Key findings
The trial interventions were low cost in terms of maintenance and ongoing delivery costs, with a mean cost per participant of £1.32 in the parent/carer trial and £1.36 in the young person’s trial.
Incremental results are presented in Appendix 7, Table 9. In the base-case complete-case CUAs for the parents/carers trial, the adjusted analysis had an incremental cost of −£34.15 (95% CI−104.54 to 36.24) and was associated with incremental QALYs of −0.003 (95% CI −0.021 to 0.015). The ICER was £12,466 per QALY, which, since it falls in the Southwest quadrant of the cost-effectiveness plane (i.e. the intervention is cost-saving but has a slightly worse outcome), is unlikely to be considered cost-effective because the ICER is not greater than the WTP threshold. Another way of presenting this is as a net monetary benefit (NMB; estimated as incremental benefit × WTP threshold − incremental cost) which is positive (suggesting interventions are cost-effective) in all analyses except for the adjusted complete-case analysis where the NMB is negative (see Appendix 7, Table 9) at both a £20,000 and £30,000 WTP threshold indicating that the intervention would not appear cost-effective. For the interested reader who wishes to understand the vagaries of estimating and interpreting ICERs and NMBs, please see Paulden.72 For the trial including young people, the intervention was dominant in both the adjusted and unadjusted analyses. The adjusted analysis had an incremental cost of −£20.82 (95% CI −71.77 to 30.13) and was associated with incremental QALYs of 0.012 (95% CI −0.017 to 0.041). In the SA to estimate the impact of missing data, the intervention was dominant for both trials, with at least a 65% probability of being cost-effective at the £20,000 WTP threshold. Given the level of missing data, particularly in trial 1, the SA using multiple imputation is likely to give a more valid estimate of cost-effectiveness.
In the complete-case CEAs, the parent/carers trial’s adjusted analysis estimated an incremental cost of −£27.66 (95% CI −79.63 to 24.31) and an incremental success of 8.6% (95% CI –3.0% to 20.2%). In the young person’s trial, the adjusted incremental cost was −£23.57 (95% CI –74.22 to 23.07) with an incremental success of 10.4% (95% CI −2.4% to 23.2%). The CEA analysis combining both trials showed an adjusted incremental cost of −£20.35 (95% CI –55.41 to 14.70) with an incremental success of 10.3% (95% CI 2.3% to 18.1%). In all CEA analyses, the intervention was dominant (cheaper and more effective than usual care).
Conclusions
Eczema Care Online interventions for parents/carers of children with eczema and for young people with eczema are robustly developed evidence-based resources which were found to be low cost and cost-effective.
A small amount of benefit at low cost, with no identifiable harms, for a condition that impacts many people can lead to substantial health benefits for the public in absolute terms. The findings reinforce the importance of health professionals signposting people with eczema to self-management support.