Workstream 5 addresses the third objective of the programme: to formulate and initiate an implementation plan for integrating interventions into clinical practice and facilitating their uptake (if clinically and cost-effective), informed by process evaluation. It also includes the redevelopment of the online behavioural interventions into a marketable product ready for dissemination. We examined two aspects of implementation: (1) how people and families with eczema might use ECO outside a trial, and (2) how health professionals and organisations may signpost to ECO.
Workstream 5.1: process evaluation
This work has been published or is currently under review:
https://doi.org/10.1093/bjd/ljac115.73
https://doi.org/10.1002/ski2.59.74
Aim
- To understand likely causal mechanisms for the interventions, how effects might vary between user groups and settings, and inform the implementation of the interventions.
Methods
Design
Qualitative and quantitative process evaluation embedded in the two RCTs (workstream 4) using three data sources:
- Qualitative interviews with young people and parents/carers participating in the RCTs.73
- Objective intervention usage data collected across a 52-week study period.
- Quantitative process questionnaires.
Recruitment
The methods of recruitment for the RCT are described in Workstream 4: determine the clinical and cost-effectiveness of the Eczema Care Online interventions. For the qualitative interview study, all trial participants were asked if they consented to be contacted by the research team on study sign-up. Those who consented were recruited via purposive sampling to recruit a range of ages, gender, ethnicity, eczema severity, socioeconomic status, recruitment site, trial groups and intervention usage. Participants were contacted by the research team and invited to take part. Consenting participants were sent an information sheet and completed an online consent form. For young people aged 13–15 years, their parents or legal representatives provided online consent for them to take part. These young people were sent an information sheet and provided verbal assent at the outset of the interview. Participants were given a £10 voucher for taking part.
Data collection and analysis
Qualitative: Semistructured telephone interviews were carried out at least 3 months after randomisation by four researchers. A topic guide was developed with feedback from a PPI representative and included questions exploring people’s views of the website content and delivery features, any cognitive or behavioural changes that resulted from using the website, impact of the COVID-19 pandemic on eczema management or intervention usage, and reasons for any low intervention usage. Interviews from both groups were analysed together using inductive thematic analysis.75 Intervention modifications were identified using the person-based approach table of changes method.50
Quantitative: There were two hypothesised behavioural mechanisms: emollient use and TCS/TCI use, for which no differences were observed in the trials. There were also two psychological mechanisms: patient enablement (the self-perceived ability to understand and cope with health issues) and perceived barriers to treatment. Intervention usage data were collected to describe patterns of intervention usage for all intervention participants in the intervention arm (young people n = 168; parents/carers n = 171) for both trials across the 52-week study period. Data were analysed using descriptive statistics. Per-protocol and complier-average causal effect analyses were carried out to examine the effect of high intervention use on eczema severity (POEM questionnaire). High intervention use included users who had finished at least one of the treatment modules (moisturising creams or flare control creams) or an optional module. Mediation analysis was used to determine whether patient enablement, treatment use or barriers to adherence mediate the intervention effect on eczema severity. Subgroup analysis was carried out to explore whether the intervention effect was different among pre-specified categories of baseline variables. Logistic regression explored associations between higher intervention use and various demographic and baseline factors.
Key findings
Participants
Seventeen parents/carers and 17 young people from the two intervention groups took part in the qualitative interviews. Thereafter, 314/340 (92.4%) parents/carers completed the 24-week POEM and 304/337 (90.2%) young people completed the 24-week POEM (primary timepoint). See Appendix 8, Table 10 for participant characteristics.
Qualitative interviews
Generally, both parents/carers and young people found the interventions easy to use and relatable, were able to engage with them competently, believed they were trustworthy and provided value to them and believed they helped them manage eczema. Our analysis suggested that the interventions may reduce eczema severity by facilitating empowerment among its users. Parents/carers and young people reported that ECO helped them to understand and feel confident in managing eczema; improve their use of topical treatments; avoid irritants and triggers that make eczema worse; engage in productive treatment conversations with their health professional; and involve their child in eczema management (parents/carers). Many parents and young people valued how the interventions helped them, or their child, normalise or accept eczema. Although many reported how the intervention relieved concerns they had about the safety of eczema treatments, two young people still held negative beliefs about TCSs, views that were not influenced by the intervention. Some participants believed they were already knowledgeable about eczema, had a good treatment regimen, or had their eczema under control and, therefore, felt that the interventions were not valuable to them. Several minor modifications were made to the intervention including improving the website design to make it more visually appealing, improving navigation to locate specific informational content more easily within a module, and adding information on TCS withdrawal. See Appendix 9, Table 11 for an excerpt from the Table of Changes.
Intervention usage
Most intervention participants in the parent/carer trial (88%; n = 151) and the young people trial (93%; n = 156) reached the minimal engagement threshold within the trial period, which was defined as having viewed the core introductory content containing all the key content that we felt was necessary for behaviour change. Users spent a relatively short amount of time on the interventions (median = 21–27 minutes). See Appendix 10, Table 12 for details.
Quantitative process analysis
For parents/carers, about 30% of the intervention effect on the POEM score at 24 weeks was mediated by increasing patient enablement. For young people, about half of the intervention effect was mediated by increasing enablement. However, as enablement and POEM were measured at the same time point, as is common in trials, these results need to be interpreted with caution. There was no evidence of a mediating effect of treatment use or perceived barriers to treatment in either trial. See Appendix 11, Table 13 for details.
Most of the associations between user characteristics and outcomes were not statistically significant, but these are exploratory analyses which are likely to be underpowered for the tests undertaken. However, those in the parent/carer trial with severe eczema at baseline had a significantly larger treatment effect than those with mild eczema (−4.0 vs. 0.8, adjusted interaction term −4.0, 95% CI −7.7 to −0.2). This difference is larger than the MCID of 2.5 points for the POEM outcome. Higher baseline eczema severity (young people; combined parents/carer and young people data), greater baseline emollient use (combined parents/carer and young people data), having a degree (parent/carer), having uncertainty about how to carry out treatment and having doubts about treatment efficacy (parent/carer) were significantly associated with higher use of the online intervention, although higher intervention usage was not linked to improved outcomes. See Appendices 12–14 for details.
Conclusions
In summary, the process evaluation suggests that these interventions were acceptable to our parents/carers and young people and provided benefits with little time commitment. It is likely that all people with eczema will benefit from using the intervention. Together with the RCT findings, these studies provide additional support for the wider implementation of ECO.
Redevelopment of interventions
We worked with a commercial software provider to redevelop the trial versions of the online interventions into marketable products ready for dissemination and implementation. The intervention content that was trialled remained the same, but minor changes were made in response to findings from the process evaluation. Minor changes included:
- Removal of questionnaires and study e-mails/texts.
- We combined ‘ECO for families’ and ‘ECO for young people’ so you now choose if you are there ‘for my child’ or ‘for myself’ when you first enter, and the content is tailored accordingly.
- The website was built using the latest adaptive technology to ensure it can be accessed from a range of devices.
- Adding information about ‘topical steroid withdrawal’.
- Redesign of the navigation/layout to improve usability.
- Translate website content into Welsh.
The intervention is freely available in both English and Welsh at www.eczemacareonline.org.uk.
Workstream 5.2: implementation and engagement planning
This work has been published:
https://doi.org/10.1136/bmj.o2973.76
Our ambition has always been for ECO to improve the lives of people living with eczema. However, innovations and ideas from research do not always easily translate into changes in healthcare systems outside of the research setting.77–79 Therefore, from the outset of the programme, we have given thought to how ECO, and more widely the knowledge that it aims to share, can reach people with eczema.
Normalisation process theory is an implementation theory that explains what processes are required for an intervention to be taken up in practice.79 NPT has guided our implementation strategy. Our aim was to develop an implementation strategy for ECO; to do so, we needed to understand barriers and facilitators to implementation. The implementation strategy will allow us to understand the best mode of delivery for ECO beyond a trial setting, understand how ECO fits within the marketplace for online information about eczema, and identify the target audience, key stakeholders and target avenues to help prioritise implementation efforts.
Methods
We used the following sources of information to inform our implementation strategy:
- We mapped the process evaluation interview data (reported in workstream 5.1) to the NPT constructs, to help identify barriers and facilitators to implementation.
- We undertook a series of stakeholder consultation meetings throughout the programme. See Patient, public and practitioner involvement in the programme and Workstream 2.4: creation of knowledge tools to support patients and healthcare professionals for details. We mapped the insight from these stakeholders to the NPT constructs, to help identify barriers and facilitators to implementation.
- We used data on how individuals within the RCTs used the interventions (reported in workstream 5.1) to inform the layout and emphasis we put onto sections of the intervention when re-designing ECO for implementation.
We used the process evaluation interviews and stakeholder feedback to help inform our value propositions (i.e. what is the unique value of ECO). We also examined how our value propositions compared to how other online sources of information for eczema were presented.
Eczema Care Online implementation strategy
Our implementation strategy is outlined in the Table 1 below. We worked with professional networks UK Dermatology Clinical Trials Network, British Association of Dermatologists, British Dermatological Nursing Group, Primary Care Dermatology Society and Allied Health Sciences Network to inform and implement our strategy.
| Strategy outline | Summary of rationale | Actions to date |
|---|---|---|
| Delivery: redesign the two interventions into one mobile adaptive website | To improve user experience and accessibility.To simplify maintenance.To meet different user access and preferences (desktops and mobile phones).To include a Welsh language version to enhance accessibility in Wales. | Created and launched a newly developed website: www.eczemacareonline.org.uk |
| Marketplace: ECO value propositions: proven to help people with eczemacreated in partnershipindependentcomprehensiveevidence based and theory informed, andaccessible and appropriate to a UK audience. | Comparison to other websites shows some value propositions are unique to ECO (proven to help people with eczema) and others are comparable with other websites (accessible and appropriate to a UK audience).Each has been emphasised as important by different stakeholders, so all are to be maintained, but certain ones may be more relevant to highlight in different contexts. | Emphasising value propositions in all our communications. Examples include:A short animation that describes the value propositions: https://youtu.be/B99_BKaoncs (see Figure 5)Social media advertisements emphasising value propositions (see Figure 6)E-mails to key stakeholders emphasising the value propositions.Summary document that describes the value propositions and evidence for ECO to share with organisations interested in adopting the resource (Report Supplementary Material 3) |
| Target audience: anyone with eczema, but mainly people with newly diagnosed eczema or people managing their own eczema for the first time. | Stakeholder consultation and interviews with potential users suggest newly diagnosed may experience the most benefit. | Communications with all stakeholders and advertisements have all emphasised the relevance to anyone with eczema or caring for a child with eczema. |
| Website redesign allows for use for either myself or my child. | ||
| Key stakeholders: patient organisations, healthcare professional organisations and healthcare delivery organisations. | Raising awareness among organisations with influence to reach wider numbers of both healthcare professionals who may share the website with patients and individuals who may use the website. | Published academic papers, press releases, and presented at academic conferences that will reach some key stakeholders. |
| Contacted some key stakeholders directly via e-mail explaining how they might benefit from ECO. | ||
| In discussions with both the two UK leading charities and an Australian eczema charity about how they can make use of the resources. | ||
| Avenues for implementation: a wide variety of avenues to be explored, but efforts prioritised on embedding weblinks to the interventions into resources that support clinical practice and training, as well as working with eczema charities to embed and promote the resource. We are also exploring direct promotion through social media advertising. | Eczema charity websites are engaged with the intervention and a less resource-intensive avenue than others. | Provided materials and information to eczema charities so they could promote the website. |
| Integration into primary care electronic systems and resources has more barriers to access, but stakeholder consultation and interviews with users suggest could be the most successful route once embedded. | Contacted multiple training and clinical practice organisations to ask them to embed ECO into systems or resources. | |
| Secondary care users are receptive to the intervention and can be key influencers for primary care via advice and guidance. | Written articles about ECO for organisations to communicate with their network/members. | |
| Pharmacy involvement growing in eczema management, so a longer-term goal is to improve pharmacy implementation avenues. | Key clinical champions promoting ECO at national and international meetings, on podcasts, and among their networks. | |
| Social media advertisements are relatively inexpensive and may help increase traffic and reach a different target audience to other routes. | Created ‘two treatments used well pads’ and business cards that were shared with all practices participating in the trial, some pharmacies and some secondary care departments. | |
| A targeted social media campaign was initiated in November 2022 including paid advertising on Facebook and Instagram, and collaboration with charities and support groups. |
Further work is required to fully action the strategy and to evaluate its effectiveness, but the section below outlines key implementation successes to date.
FIGURE 5 Screenshot of the ECO YouTube animation video. (Figure not reproduced here; see the original article.)
FIGURE 6 Example of materials used in social media advertisements emphasising value propositions ‘comprehensive’ and ‘accessible’. (Figure not reproduced here; see the original article.)
Implementation successes to date
Since launch of the website in June 2022, the website has been accessed by over 13,450 unique visitors from 138 countries. At least 12 organisations have agreed to share in communications with their network, embed into resources or clinical systems, or include in training courses.
Feedback from a range of stakeholders indicates that ECO has been well received:
Very helpful resources indeed. Since it has become available, I have tried to include it in every Advice & Guidance that I do!
I have been disseminating around my clinical networks as this will be a really useful, national, harmonised resource for people affected by eczema.
Our Family Workers regularly signpost families to ECO so hopefully there is a steady stream of families benefiting from the information.
Looked brilliant because you can just send it away to [patients] and say look, this is what it’s all about
Challenges for implementation of our strategy
National Health Service policy is to expand NHS-accredited health apps/websites to support people in managing their own health, but progress on accrediting apps/websites has been slow and online infrastructure is needed to signpost to reliable resources. The Digital Technology Assessment Criteria for health apps produced to give staff and patients confidence that digital health tools meet clinical safety, data protection, accessibility and cyber security standards has been very difficult for research teams to undertake, requiring substantial additional investment of time that is often beyond the scope of the original funding. This is in addition to licensing and technology transfer arising at the end of a programme where contracts may be ending and research teams therefore having difficulty finding capacity to complete these tasks. Costs of hosting the intervention over the coming 5 years have been secured, but longer-term sustainability that allows content, digital infrastructure and format to be kept up to date remains uncertain. We have explored private, public and charity sector partnerships. Participants in the ECO trials reported that knowing the intervention is free from commercial influence was important to them in trusting messages and public contributors have also pushed us to maintain a focus on independence and keeping the interventions free to use. We would therefore like to maintain ‘ownership’ (i.e. content control) of the ECO interventions with a commitment to making sure content is evidence-based, while also disseminating in partnership with eczema charities to raise awareness. NIHR funding enabled us to co-produce engaging and effective interventions, but it is not completely clear yet how these will remain free to use for the NHS in the long term.
Conclusion
Focusing on the barriers and facilitators to implementation of the ECO website throughout this programme has ensured that our product is market-ready and able to be implemented widely, both in the UK (as our primary target audience) and internationally.
The website will be free to access for 5 years; it is free from commercial influence and does not require registration. It has been extremely well received by charity partners, healthcare professionals and people with eczema. To ensure longer-term sustainability, we are exploring various options for partnerships but are mindful of the impact that commercial influence or changes to the content may have on effectiveness.