Workstream 3: randomised controlled trial

Aim

Evaluate whether the Embedding Package improves SSME uptake for people with T2DM compared with usual care.

Conduct a randomised controlled trial

Methods

An overview follows, with detail elsewhere25 and study materials available in Report Supplementary Material 2.

Study design

The Embedding Package was compared with usual care (practices referred patients to SSME following their current standards) using a wait-list cluster randomised design (Figure 3), with one-to-one practice-level randomisation stratified by the CCG (May 2018–August 2020). All practices provided usual care for 3 months (months −3–0; baseline) and were then randomised by an independent statistician (study team informed practices of their allocation) to receive the intervention for steps one and two (months 1–18; immediate group) or step two only (months 10–18; wait-list group). Patient-level data were extracted for all patients with T2DM registered at a participating practice. The trial ran to completion, although all embedder activities stopped prematurely due to the COVID-19 pandemic, when all practices were at least 4 months into step two.

Outcomes

The primary outcome was patient-level HbA1c compared between control and intervention conditions. HbA1c was a proxy for SSME attendance, which was the intervention’s focus. SSME attendance is a terminal outcome because most people only attend SSME once, so if a person attended before the study started or during a control condition then it would not be possible for them to attend during the intervention condition. HbA1c was selected as it can change continually and SSME attendance is related to reduced HbA1c. SSME referrals and attendance were the main secondary outcomes (binary outcomes coded 1 if the participant was referred/attended, 0 otherwise), with sensitivity analyses to account for their terminal nature. See Davies et al. for other secondary outcomes, including quality of attendance data and patient-reported outcome measures.26

Data sources

The main source was patient-level data from routine primary care records (extracted data). Notably, primary care SSME data only cover traditional modalities (or virtual equivalents of these post-COVID-19). There is no standard mechanism to feedback uptake of digital SSME (e.g. MyDESMOND) into primary care records, so primary care SSME data do not generally include digital SSME.

Referral and attendance data were additionally available from postal questionnaires completed by patients in step one (self-reported data) and SSME providers.

Sample size

Recruiting 58 practices (29 per group) would give over 90% power to detect a population-level difference in HbA1c of 1.1 mmol/mol (0.1%), which could represent a 2.1% decrease in micro and macrovascular events.27 This assumed a standard deviation (SD) of 16.4 mmol/mol (1.5%),27 an intracluster correlation coefficient (ICC) of 0.05 and one baseline HbA1c measurement with one more HbA1c measurement at each randomisation point. This calculation does not account for potential variation in cluster size, as it has been shown that the power of wait-list studies is robust to cluster size variations. Sensitivity of this calculation to changes in average cluster size was assessed. There would be 80% power to detect a difference as small as 0.7 mmol/mol (0.06%). If the average cluster size was larger than 174, there would be 80% power to detect a 1.1 mmol/mol (0.1%) difference. The recruitment target was 66 practices allowing for 10% cluster drop-out.

Statistical analysis

See Davies et al.26

Results

Davies et al.26 gives the full results, with key findings summarised here.

Population

The intention-to-treat (ITT; randomised practices with baseline data) and complete-case populations included 64 (31 wait-list; 33 immediate) and 57 (29 wait-list; 28 immediate) practices, respectively, from 10 CCGs across England (Figure 4). The ITT practices used eight SSME providers, covering four SSME programmes (DESMOND, Diabetes 2gether/Diabetes 4ward, Spotlight, Xpert Health).

At baseline, 29,849 eligible patients were registered with ITT practices (wait-list: 15,527 total, 501 mean per practice; immediate: 14,322 total, 434 mean per practice). Patient populations were very similar between the two randomisation groups.26 The exception was that, at baseline, 19% of wait-list group patients had not been referred to SSME, 16% had been referred but not attended and 64% had attended, while in the immediate group, non-referral (42%) and referral without attendance (20%) were higher, with lower attendance (38%).

Primary outcome

There was no significant difference (p = 0.503) in HbA1c between the intervention (immediate: steps one and two; wait-list: step two) and control (immediate: baseline; wait-list: baseline and step one) conditions after accounting for clustering and time effects. The mean difference was −0.10 mmol/mol [95% confidence interval (CI) −0.38 to 0.18 mmol/mol; HbA1c units −0.01%, 95% CI −0.03 to 0.02], so the 95% CI was wholly below the minimum clinically important difference (Table 1; practice-level ICC 0.02).

There was a statistically significant interaction with ethnicity (p = 0.001). Among ethnic minority individuals, mean HbA1c was 59.13 mmol/mol (SD 17.77; 7.6%, SD 1.6%) in the control condition and 59.06 mmol/mol (SD 16.98; 7.6%, SD 1.6%) in the intervention condition; the adjusted average difference was −0.64 mmol/mol (95% CI −1.08 to −0.20; HbA1c units −0.06%; 95% CI −0.10% to −0.02%). This does not meet the minimum clinically significant threshold in the sample size calculation and SSME attendance did not increase in these ethnic groups.26 HbA1c did not significantly differ between the treatment conditions among white individuals (mean difference −0.02 mmol/mol, 95% CI −0.33 to 0.29; 0.00%, 95% CI −0.03% to 0.03%) or in any of the sensitivity analyses.26 including subgroup analyses by type of SSME programme or local MyDESMOND availability.

Secondary outcomes

The main secondary outcomes were patient-level referral to, and attendance at, SSME. Both were lower or similar during the intervention than control conditions, with an odds ratio (OR; 95% CI) of 0.85 (0.73 to 0.99; p = 0.038) for referrals and 0.82 (0.66 to 1.01; p = 0.063) for attendance (see Table 1).

During the intervention condition, total cholesterol was lower by 0.12 mmol/l (95% CI 0.08 to 0.15) on average, while there were more current smokers (OR 1.18, 95% CI 1.09 to 1.28; p < 0.001) compared with the control condition. There were no other important differences (see Table 1).

Quality of referral/attendance data

Agreement between self-report, practice and provider data ranged from 70% (self-report vs. practice) to 79% (practice vs. provider) for referrals, and from 60% (self-report vs. practice) to 77% (self-report vs. provider) for attendance. Combined, the sources were approximately 69% and 77% complete for referral and attendance, respectively.

Patient-reported outcomes

There was no statistically significant difference in Patient Activation Measure^® 13 (Insignia Health, Norwich, UK) W-BQ12 well-being questionnaire or Problem Areas in Diabetes questionnaire scores between the groups.

Conclusions

Overall, the intervention did not improve the primary or most secondary outcomes. While the intervention was associated with a slightly reduced HbA1c in a subgroup analysis of people in ethnic minority groups, SSME attendance did not appear to increase. This group may have increased SSME uptake through MyDESMOND, which was made available over the study period; however, use of digital SSME programmes is not routinely recorded in primary care.

There were statistical issues with the binary outcome analyses (e.g. SSME referral, SSME attendance, smoking), as models were unstable and may not fit the data well, so binary outcomes should be interpreted with caution.

Baseline SSME attendance and referrals were imbalanced between the study groups, with very high attendance in the wait-list group (64%) meaning that attendance under the control condition was high with little room for improvement. This could partly explain the negative trial results. Baseline attendance was 38% in the immediate group in primary care data and 41% overall in self-reported data, aligning with previous self-reports showing 49% of people with T2DM have attended SSME.14 These data suggest that the 11% attendance within 12 months of diagnosis reported by the National Diabetes Audit only shows part of the picture around SSME attendance.13

Conduct an ethnography study

Context

Informed by NPT,28,29 WS1 and WS2, this component focused on understanding implementation at patient, practice, CCG and provider levels, assessing sustainability of the Embedding Package and contextualising the implementation process, sustainability of the change and the ‘fit’ of the intervention within routine practice.

Methods

Details of the methods are provided in Davies et al., including a description of the ethnographic sample, example extracts from the tracker and an example draft press release.26 Briefly, in-person interviews and observations were planned at 12 sites, including primary care practices, CCGs and SSME providers. Additionally, documentary evidence would provide context around meetings, events and virtual communications. However, study-level factors (including limited intervention engagement) combined with the COVID-19 pandemic severely curtailed face-to-face data collection. Data collection was therefore redesigned to use a larger range of documentary and digital data sources:

• E-mail communications between embedders and practices/providers (2051 individual items).
• Intervention ‘tracker’: database recording embedder actions and time spent on each activity (928 entries).
• Embedder-generated documents (e.g. practice action plans; n = 49).
• Series of debrief interviews with embedders (seven interviews; two embedders) and one interview with an SSME provider.
• Observations of meetings between embedders and practices.

The analysis was informed by interpretive thematic analysis30 and NPT.

Findings

Findings covered three themes:

• Making and sustaining contact with practices and providers.
• Interaction with the toolkit.
• The role of MyDESMOND.

A summary follows, with detail provided in Davies et al.26

Making and sustaining contact: interactions with embedders

Embedders faced obstacles in initiating and sustaining contact with providers and practices, especially a lack of, or substantially delayed, responses to correspondence and conducting agreed actions. While face-to-face meetings evoked interest, enthusiasm and buy-in, once the embedder left, often little action followed from participating organisations.

Interviews suggested that sporadic and delayed communication from practices and providers was due to lack of capacity, staff leaving post or being on (extended) leave and poor or no role handover (so embedders had to restart with newly appointed personnel). Providers and staff were perceived to be anxious about additional workload, therefore seeming hesitant to engage in correspondence. Another reflection from the interviews was a potential lack of clarity in early communications with practices, especially regarding their required capacity and commitment.

These challenges were addressed by consistent chasing e-mails, reminders of actions and continued proactivity in communication by embedders. Prior contact with key individuals in localities appeared to affect the reception of the intervention. However, trying to achieve engagement consumed a substantial amount of time, with very mixed success across CCG localities. In particular, a common feature across several localities was uncertainty about the intervention costs and benefits. Put simply, practices and providers were often unsure about what the intervention would do for their provision locally in terms of potential increase in provision of spaces on education groups, and about the level of input it would require from busy individuals. With the onset of the COVID-19 pandemic, communication with CCG localities became even more challenging.

Interaction with the toolkit

Providers and practices were concerned that they lacked time to implement and use the toolkit. Embedders offered diverse initiatives to address these concerns (e.g. structuring action plans; tailoring promotional print material to local needs; drafting press releases; preparing digital material; offering mentoring).

Analyses highlighted a proactive approach to facilitating local activity, especially when there was a delay in, or lack of, response to embedder’s communication.

Challenges with toolkit implementation were more frequently encountered than successes. A great deal of time and effort went into tailoring initiatives to local need, and this fell significantly on embedders, not just local stakeholders.

Also apparent was how divergent the participating localities were regarding existing provision and infrastructure for SSME, and how consequential these differences could be. While the intervention was designed to account for such differences, local stakeholders were perceived to lack the time and willingness to lead on tailoring activities and, for most CCG localities, embedders lacked specific local knowledge to navigate them.

Providers were perceived as often reluctant to interact with the toolkit in their day-to-day work, despite encouragement to do so. While some could see the toolkit’s merit in discussions with the team, this did not translate into further engagement, let alone embedding into routine practice.

The first COVID-19 lockdown saw an increase in social media use, particularly Twitter^® (now X^®; X Corp., San Fransisco, CA, USA) to advertise and encourage toolkit adoption. Functionality, such as hashtags and retweets, seemed important in Twitter’s popularity among providers and practices, as it made the interaction straightforward. However, while there was some apparent engagement with social media among practices and providers, this did not clearly result in greater toolkit use.

In short, time and capacity constraints, along with behaviours/procedures associated with existing infrastructure, limited toolkit interaction. The embedders perceived that the toolkit alone would be insufficient to bring about the desired change, especially given the challenges involved in incorporating them into existing behaviours and processes. Rather, a fundamental change in attitudes and individuals’ behaviours was needed.

Role of MyDESMOND

In some locations, MyDESMOND was used as an additional modality via which patients could access SSME. While 4 of the 10 CCG localities had MyDESMOND available (with a further 2 localities using it when the COVID-19 pandemic started), other localities did not use it, partly due to its compatibility and workability with existing infrastructure.

Where CCG localities took up MyDESMOND, practical difficulties could confound its inclusion in routine care. Interviews showed that, for areas already running DESMOND, MYDESMOND expanded their offer to reach underserved populations.

Yet, while MyDESMOND supported people who struggled to access in-person sessions, some rural areas lacked broadband to access the digital sessions. Moreover, compared with in-person approaches, MyDESMOND was felt to rely more heavily on the user navigating their way through SSME without an educator’s assistance. Not all patients appeared comfortable with this approach.

Conclusions

Although data collection was limited by access and participant engagement challenges, combined with the COVID-19 pandemic, qualitative data offered useful understanding around the intervention implementation process and some context for the negative RCT results.

First, despite efforts to develop an intervention that would resonate with, and be adaptable to, local SSME challenges, the attractiveness and usefulness of the package were not self-evident to all practices and providers. There was initial engagement and enthusiasm among local leads, but in many CCG localities this was short-lived, and embedders required much effort to maintain communication and encourage uptake following initial meetings. While this might be partly attributed to time constraints of local staff, the package (especially the toolkit) clearly did not have sufficient immediate appeal, regarding purpose or usability, to become a priority.

Consequently, a very uneven process of cognitive participation was evident across CCG localities, with much activity led by embedders and less evidence of active work from local CCG, practice or provider stakeholders. Indeed, in some areas, some important groups were not involved in intervention implementation, for example due to scepticism about its added value, the presence of other potentially competing practices (e.g. alternative online programmes and recruitment approaches) or ongoing local disagreements about issues such as eligibility and reimbursement. Accordingly, operationalising the intervention locally (orchestrating collective action to implement it) proved challenging in several CCG localities. A good deal of this work related to tailoring the intervention and its materials (e.g. pathways, publicity) to the local context. Often, limited stakeholder engagement meant that this fell primarily to embedders, but it was not clear that an embedder, at least a centrally located one, was best placed to lead this work, as they were not integrated into provider teams and lacked local knowledge.

Overall, the high volume and nature of work by embedders, and sometimes local stakeholders, indicate the importance of adapting the intervention to local contexts, and how divergent those local contexts could be. While much work went into developing an intervention that would be malleable and adaptable and would build on existing infrastructure to encourage SSME uptake, localities differed greatly in existing approaches to provision and in the maturity and sophistication of existing infrastructure for identification, referral and monitoring. Consequently, the work of translating the intervention into divergent local contexts was significant: as Harvey and Kitson have noted, ‘the success or otherwise of implementation [rests] upon the ability of the facilitator and the facilitation process to enable recipients within their particular context to adopt and apply the innovation by tailoring their intervention appropriately’.31 However, our findings suggest that this work was not easily undertaken by an embedder covering multiple CCG localities and lacking detailed local knowledge of infrastructure, relationships and politics.

Refine the intervention

It was anticipated that intervention refinement may be needed during the RCT based on ongoing learnings from intervention delivery and the ethnographic study. Only two minor amendments were made. The self-referral article was updated with figures from the feasibility work. Further details about ensuring accessibility were added around having venues for groups aligned to larger practices/health centres.

Changes from workstream 3 original plan

Ethnography methodology was adapted as described above.

It was planned that embedders would be local to participating CCGs. However, this was not possible due to sustainability and transformation partnerships being introduced (see Challenges), meaning that CCGs appointed staff into roles similar to the embedder. Furthermore, the CCG pool was increased to 10 from 2 to 3 to aid practice recruitment. Centralised roles were therefore more feasible from a study perspective and to try to join up activity by the sustainability and transformation partnerships roles.

All practices were expected to be recruited before randomisation and therefore randomised simultaneously. However, practice recruitment took longer than expected, so practices were randomised in batches with staggered study start times.

Data extraction was planned to take place at the end of each step. However, this was difficult to arrange with some practices and so data extraction took place over several months. Data were extracted retrospectively, so this did not affect the results.

Self-report questionnaires were going to be mailed out immediately after practice randomisation, but for practical reasons this mail-out happened over the duration of the first step.