Discussion

Summary of findings

We successfully developed an intervention and refined it during a feasibility study. In a full-scale RCT of 608 PWCF, there was no statistically significant reduction in the primary outcome of incidence rate of pulmonary exacerbations over a 12-month follow-up period. However, the RCT demonstrated a clinically and statistically significant improvement in the secondary outcome of normative adherence for PWCF. The magnitude of the increase in adherence, at 10% on average, may not have been large enough to affect exacerbations. No significant difference was found between groups in lung function at 12 months. There were statistically significant improvements in secondary outcomes that supported the increase in adherence found, namely improvements in necessities and concerns, habit formation and treatment effort. Only one of the eight domains of quality of life showed statistically significant improvement: there was a reduction in treatment burden. There was a small average weight gain (typically a positive outcome in CF).

The process evaluation identified that the level of implementation fidelity was high and that some of the key expected mechanisms of action were in operation, including self-monitoring. PWCF in the RCT valued the objective adherence data and the relationship they built with the interventionists, which they perceived as helping them to change their behaviour. The process evaluation suggested that adherence improvement might occur more in the baseline middle adherence level groups than the high adherence level groups and the low adherence level groups. A statistical subgroup analysis supported these findings.

The health economic model suggested that the intervention is expected to generate additional health gains of 0.17 QALYs and cost-savings of £1790 over the patient’s remaining lifetime. In this modelling the intervention is assumed to be given for 10 years and that treatment effects and costs accrue only in the years the intervention is given.

Successes of the programme grant

The ACtiF programme set out to develop an intervention to help PWCF to increase adherence to nebuliser adherence and subsequently reduce pulmonary exacerbations. Between February 2015 and December 2019 objectives were achieved through a series of WPs. Each work package was delivered on time and to plan. The intervention was evaluated in what was, to our knowledge, the largest CF RCT conducted in the UK, which exceeded its participant recruitment targets and maintained a sufficient sample size at follow-up. This contrasts with similar RCTs conducted in the CF population that have failed to meet sample size requirements owing to research and intervention procedure difficulties at recruitment and over the duration of the study.73–75 Extensive intervention development and feasibility testing allowed the programme to develop a professional, theory-based, person-based intervention that is both feasible and acceptable. Prior to this study there was insufficient evidence to support the use of any individual self-management support strategy.76,77 A recent, large, well-conducted RCT of strategies to improve adherence in CF had null findings.73 The legacy of the programme is a digital platform for measuring objective adherence in PWCF that is being used in the NHS now, and a manualised set of procedures for training and delivery of behaviour change techniques that improves adherence to nebuliser treatments but does not reduce exacerbations.

Challenges and methodological limitations

A major challenge for the whole programme was obtaining treatment costs for the RCT. Negotiating treatment costs in the context of UK clinical research is a well-documented problem.78

The National Institute for Health Research (NIHR) funding panel members were keen that the RCT in this programme focused on improvements in patient health. The primary outcome in the RCT was a reduction in the number of exacerbations experienced by patients. The number of exacerbations is a valid and sensitive outcome for the detection of change in blinded drug trials.79 However, we identified problems with its use as a primary outcome in this open-label RCT of a complex intervention based in routine care settings. The use of IVAB in clinical practice is essentially discretionary: about half of those meeting objective criteria, and < 10% of people suffering acutely poor FEV₁, decline to receive them.80,81 Conversely, uptake in i.v. rescue antibiotics appears to increase as an individual becomes more adherent with medication, with increased clinician contact allowing more exacerbations to be detected.82,83 It is possible that our intervention and usual-care arms were subject to different levels of surveillance, with the intervention arm experiencing greater contact time and perhaps being more likely to self-present during exacerbations. Therefore, the estimated event rates may have been subject to ascertainment bias, with under-reporting of exacerbations in the usual-care arm.

Improvement in normative adherence was a key secondary outcome on the proposed causal pathway to reductions in exacerbations (see Figure 3). Some RCTs of adherence interventions use this as the primary outcome, but our goal was more challenging. There was an improvement of 10% in this outcome. It is clear (see Figure 6) that adherence rates were high in the first week of the RCT. Indeed, our baseline adherence level was high (≈ 45%) compared with the average observed adherence in UK CF centres (≈ 35%).16,84 This reduced in both arms for a few weeks before increasing in the intervention arm. It was not possible to measure true baseline adherence; baseline adherence was measured when PWCF entered the RCT and it is likely that the Hawthorne effect occurred in the first week of the RCT as both intervention and usual-care arms reacted to being monitored in the RCT by trying to adhere to their medication. It is possible that this was followed by a return to normal adherence before the intervention began to help PWCF to improve their adherence.

The magnitude of the increase in adherence, at 10% on average and 15% in the low- to moderate-level adherers, may not have been large enough to affect exacerbations and FEV₁. Alternatively, detection of change in exacerbations and FEV₁ may be challenging within the time constraints of a 12-month follow-up period. As depicted in the CFHealthHub logic model (see Figure 3), compared with increased necessity, increased habit, decreased concern (immediate outcomes) and increased adherence (intermediate outcome), improvements in exacerbation rates and lung function (FEV₁) are longer-term outcomes. It may be that further health benefits might be observed over longer periods of time, given the window over which habits are thought to develop,85 although our extra analysis of longer-term follow-up over 21 months for some RCT participants did not support this proposition. It is also possible that the number of exacerbations is not a good indicator of patient health, and that quality of life may be a better outcome in future studies, although other secondary outcomes such as quality of life also showed little or no improvement.

A key limitation of the economic analysis is that the data collection mechanisms in the CFHealthHub trial did not allow us to quantify the impact of the intervention on the number of packs of medicines prescribed. As a consequence, there remains uncertainty regarding the relationship between improving adherence on overall treatment costs incurred by the NHS. We undertook sensitivity analyses to explore the potential impact of this factor on the cost-effectiveness of the intervention.

Implications for practice and lessons learned

Effective chronic disease management with accurate diagnosis and medicines optimisation is reliant on accurate adherence measures.15 Using state-of-the-art measurement tools to collect time- and date-stamped inhalation data, this study has evidenced that level of objective adherence to nebulised medications is lower than the patient themselves often realise.16 Poor calibration between subjective and objective adherence may lead to inefficient clinical care for patients and consequent deteriorating health. Clinicians may be unable to distinguish between patients who are deteriorating because of untreated CF and patients whose CF is treated but who have complications requiring a change in treatment. The digital platform component of the intervention provides objective adherence data that may be useful in this context. A total of 19 UK CF centres now have access to the CFHealthHub platform. The joint Chief Investigator Martin Wildman has established the CFHealthHub Data Observatory and Improvement Collaborative (registered as ISRCTN14464661), which is using the infrastructure and learning from the ACTIF programme to enable access to the CFHealthHub platform for all PWCF and members of the MDT in routine care.

If used in practice, the expectation would be that the real-time measurement of adherence would be continuous and that the interventionist-delivered part of the intervention would be used once or more than once when someone’s adherence drops for any reason over their lifetime.

Recommendations for future research (in order of priority)

• Given the non-significant difference in the primary outcome, further research is required to explore why an increase in objective normative adherence did not reduce exacerbations, and to develop interventions that reduce exacerbations.
• The existing intervention could be adapted or tailored to address the needs of PWCF with different levels of baseline adherence, including those with low levels of baseline adherence who often have complex problems. That is, PWCF with low levels of motivation, who are disengaged from their CF team or who inhabit adverse social situations as indicated by high self-reported life chaos and low income. An enhanced, health and social care professional-led intervention to address the wider complexity of individuals’ lives, as a precursor to adherence support, could be developed.
• The economic modelling indicates that the intervention is expected to dominate current care. Future research would be valuable to better understand whether or not and how the effects of the intervention change over time and whether or not improving adherence leads to increases in prescribed CF medicines.

Conclusions

The CFHealthHub was successfully developed. In the full-scale RCT there was no statistically significant reduction in the primary outcome of the number of pulmonary exacerbations at 12 months. Clinically and statistically significant improvements in the key secondary outcome of normative adherence were observed. The magnitude of the increase in adherence, at 10% on average, may not have been large enough to affect exacerbations. The intervention was delivered with fidelity and key mechanisms of action, including self-monitoring, were observed. The health economic model suggests that the intervention is expected to generate additional health gains of 0.17 QALYs and cost-savings of £1790 over the patient’s remaining lifetime. This finding is dependent on assumptions regarding the duration over which costs and effects of the intervention apply, the impact of the intervention on FEV₁% predicted and the relationship between increased adherence and drug-prescribing levels.