Results: cluster randomised trial

Recruitment and baseline characteristics

The trial recruited 80 (25%) general practices to the trial, of which one withdrew from the CPRD before the start of the intervention and the remaining 79 were included in the intention-to-treat analysis. Figure 4 provides a flow chart showing the progress of practices and participants through the trial.

FIGURE 4 A flow chart showing trial practices and registered populations. (Figure not reproduced here; see the original article.)

The trial included general practices from throughout the UK, including 19 practices from Scotland, 15 from Wales, nine from Northern Ireland and the remaining 36 from regions in England (Table 7). There were 18 practices allocated in November 2015, 31 in January and February 2016 and 30 in June–August 2016. The registered population included participants of all ages, including approximately 17% aged under 15 years and approximately 8% aged ≥ 75 years. General practice list sizes were generally slightly higher in intervention trial arm practices, but the range of list sizes was similar across trial arms. There were 17.1% of participants with comorbidity in intervention trial arm practices and 14.5% at control trial arm practices. In the baseline period, the point estimates for RTI consultation rate and antibiotic-prescribing rate were slightly higher at control trial arm practices, but in the context of the wide range of variation among practices (Table 7).

Main results

Initial analyses were conducted using cluster-level statistics. Figure 5 presents a plot of the age-standardised estimates for each general practice for the intervention phase of the trial. The baseline antibiotic-prescribing rate is plotted on the x-axis and the change in antibiotic-prescribing rate (trial period – baseline) on the y-axis. Linear models, weighted for practice size, are plotted by trial arm, together with 95% CIs. There was a wide range of antibiotic-prescribing rates. Both intervention and control trial arm practices showed lower mean antibiotic-prescribing rates during the trial than with baseline, with no clear difference between trial arms. However, changes in antibiotic-prescribing rates showed wide variation at any level of baseline antibiotic prescribing. The data appear to be overdispersed, with several extreme results possibly having undue influence in this representation of the data. In an analysis of covariance, weighted for practice size, the adjusted mean difference between the intervention and control trial arms for the antibiotic-prescribing rate for RTI was –0.5 per 1000 patient-years (95% CI –8.2 to 7.2 per 1000 patient-years).

FIGURE 5 Scatterplot showing change (trial period – baseline) in age-standardised antibiotic-prescribing rate for each trial practice plotted against the baseline antibiotic-prescribing rate. (Figure not reproduced here; see the original article.)

Figure 6 presents a density plot for the distribution of changes in antibiotic-prescribing rate from baseline for the intervention and control trial arms separately. Although the trial analysis was not based on changes from baseline, in a cautious interpretation of Figure 6 there appears to be substantial overlap in the distributions for the two trial arms, although the curve for intervention trial arm practices in green is generally shifted to the left in comparison with control trial arm practices in black. Both Figures 5 and 6 illustrate wide variation among trial general practices, which might make it difficult to draw clear conclusions from the trial.

FIGURE 6 Density plot showing distribution of change from baseline in antibiotic prescribing at ages for intervention (green) and control (black) trial arms. (Figure not reproduced here; see the original article.)

Table 8 presents the results of the primary analysis. There were 31,907 antibiotic prescriptions for RTIs during 323,155 patient-years at 41 intervention trial arm practices and 27,923 antibiotic prescriptions during 259,520 person-years at 38 control trial arm practices. There were 98.7 antibiotic prescriptions per 1000 patient-years in the intervention trial arm and 107.6 per 1000 in the control arm, giving an unadjusted RR of 0.92. The RR for antibiotic prescribing for RTIs was 0.88 (95% CI 0.78 to 0.99; p = 0.040). Point estimates for the RR for RTI consultations (adjusted RR 0.94, 0.86 to 1.03; p = 0.186) and the proportion of antibiotic consultations with antibiotics prescribed (adjusted RR 0.96, 0.89 to 1.03; p = 0.253) were both lower than unity. However, the CIs were wide and the p-values were large, which was consistent with an insufficiently precise estimate being obtained given the size of the trial and the substantial level of variation in these measures.

Intervention effect by the subgroup of age

Respiratory tract infection consultations and antibiotic prescribing were strongly associated with age. In the control trial arm, children aged 0–14 years accounted for 18% of all person-time, but 31% of all RTI consultations and 23% of antibiotic prescriptions for RTIs. People aged 45–54 years accounted for 15% of all person-time, but 11% of all RTI consultations and 12% of all antibiotic prescriptions. People aged 65–74 years accounted for 9% of all person-time, but 12% of antibiotic prescriptions. These figures are presented as per cent of the total person-time in each trial arm and an effect of intervention is not intended to be demonstrated.

There was strong evidence that the effect of the intervention varied by age group. A Wald test of the trial arm by age–group interaction terms gave a p-value of < 0.001. The results of a prespecified subgroup analysis, by age, are shown in Figure 7. There was no evidence of an effect of intervention in children aged < 15 years (adjusted RR 0.96, 0.82 to 1.12) or in people aged ≥ 85 years (adjusted RR 0.97, 95% CI 0.79 to 1.18). At intermediate ages, antibiotic prescribing was lower in the intervention trial arm. Adjusted RRs were 0.84 (95% CI 0.73 to 0.96) for people aged 45–54 years; 0.79 (95% CI 0.69 to 0.91) for 55- to 64-year-olds; 0.80 (95% CI 0.70 to 0.91) for 65- to 74-year-olds; and 0.79 (95% CI 0.69 to 0.91) for 75- to 84-year-olds. Antibiotic prescribing was generally similar across categories of gender and comorbidity.

FIGURE 7 Effect of intervention for primary outcome, by age group. (a) All ages; and (b) 15- to 84-year-olds. RRs were adjusted for the random effect of general practice and fixed effects of gender, comorbidity, region, quarter in study, practice-specific baseline rate and interaction with period of randomisation. LL, lower limit of CI; UL, upper limit of CI. (Figure not reproduced here; see the original article.)

Effect modification was summarised by age by comparing effect measures in children, adults aged 15–84 years and senior elderly aged ≥ 85 years (Table 9). The intervention was associated with lower antibiotic prescribing for RTIs in adults aged 15–84 years (adjusted RR 0.84, 95% CI 0.75 to 0.95). During the intervention period the mean rate of antibiotic prescribing for RTIs in adults aged 15–84 years at control trial arm practices was 100.2 per 1000 patient-years. The relative risk reduction was used to estimate an absolute risk reduction of 16.0 (95% CI 5.0 to 25.1) antibiotic prescriptions per 1000 patient-years. In adults, one antibiotic prescription was avoided for every 62 (95% CI 40 to 200) registered patients aged 15–84 years per year. Figure 8 presents a density plot of changes in antibiotic prescribing for adults aged 15–84 years. This reveals a higher proportion of general practices showing reductions in antibiotic prescribing in the intervention trial arm, providing visual confirmation of the estimates provided from statistical modelling.

FIGURE 8 Density plot showing distribution of change from baseline in antibiotic prescribing at ages 15–84 years for the intervention (green) and control (black) trial arms. (Figure not reproduced here; see the original article.)

For adults aged 15–84 years, the relative risk estimate was lower than unity for the proportion of RTI consultations with antibiotics prescribed (adjusted RR 0.95, 95% CI 0.89 to 1.03; p = 0.209), but the estimate was imprecise and the p-value was large, which is consistent with a lack of evidence of an effect on this measure (Table 9). There was evidence that, for adults aged 15–84 years, the rate of RTI consultations was lower in the intervention trial arm (adjusted RR 0.90, 95% CI 0.82 to 0.98). These findings suggest that the effect observed for the primary outcome might be more strongly associated with differences in respiratory consultation rates rather than changes in the proportion of RTI consultations with antibiotics prescribed. Figure 9 shows a series of density plots comparing changes in age-standardised general practice-specific values for RTI consultation rates, antibiotic-prescribing rates and proportion of consultations with antibiotics prescribed, for all ages and for the age range 15–84 years. This series shows that RTI consultation rates were generally lower during the trial period than in the baseline period, with similar reductions being observed in both trial arms. Antibiotic-prescribing rates, and the proportion of RTI consultations with antibiotics prescribed, showed substantial overlap, but any reductions tended to be slightly greater in the intervention trial arm. There were several outlying values. This provided visual confirmation of the statistical modelling results.

FIGURE 9 Density plots showing distribution of change from baseline for RTI consultation rates, antibiotic-prescribing rates and proportion of RTI consultations with antibiotics prescribed at all ages and for ages 15–84 years for the intervention (green) and control (black) trial arms. (Figure not reproduced here; see the original article.)

Additional subgroup analyses

The intervention effects for the primary outcome (Figure 10) were generally similar for men and women and for comorbidity categories. Estimated effects varied widely among regions, but caution must be observed when drawing firm conclusions because of the small numbers of general practices included in each region. When the primary outcome was analysed by subgroups of baseline antibiotic-prescribing rate quartile, there was no evidence for any consistent pattern of effect. Adjusted RRs were 0.94 (95% CI 0.66 to 1.34) for the lowest quartile, 0.79 (95% CI 0.57 to 1.09) for the second quartile, 0.99 (95% CI 0.81 to 1.20) for the third quartile and 0.99 (95% CI 0.89 to 1.10) for the highest quartile.

FIGURE 10 Forest plot showing RRs for antibiotic prescribing, by the subgroups of gender, age, comorbidity and region. Estimates were adjusted for the random effect of general practice and the fixed effects of quarter in study, practice-specific baseline rate and interaction with period, as well as each of the variables shown. LL, lower limit of CI; UL, upper limit of CI. (Figure not reproduced here; see the original article.)

Estimated effects for the proportion of RTI consultations with antibiotics prescribed (Figure 11) showed consistent results for men and women and for comorbidity categories. Estimates by age group were generally consistent with the overall effect, but there was no evidence of a clear pattern of variation by age, though the estimate for people aged ≥ 85 years was consistent with absence of effect. It is relevant to note that the analyses for the antibiotic-prescribing proportion uses a different denominator, that is, the number of RTI consultations, as compared with the analyses for antibiotic-prescribing rate and RTI consultation rate, which both incorporate person-time as denominator.

FIGURE 11 Forest plot showing RRs for the antibiotic-prescribing proportion, by the subgroups of gender, age, comorbidity and region. Estimates were adjusted for the random effect of general practice and the fixed effects of quarter in study, practice-specific baseline rate and interaction with period, as well as each of the variables shown. LL, lower limit of CI; UL, upper limit of CI. (Figure not reproduced here; see the original article.)

Sensitivity analyses

Sensitivity analyses were conducted to evaluate whether or not the use of potentially better-fitting models might influence conclusions (Table 10). The rate of respiratory consultations was adjusted for in the trial period. In addition, interactions of covariates with age group was allowed for. Estimates and p-values from these models were generally consistent with the findings from the base-case model. DHGLMs were fitted to address possible overdispersion of the general practice-derived data. In DHGLMs, random effects may be fitted in the dispersion part of the model, as well as in the mean model for the fixed effects. In the DHGLM framework, an overdispersed Poisson model yielded a similar estimate to the base case. In addition, results were compared using EQL and EQL1 estimation, as the former is known to give biased estimates in some data configurations, but the same estimates were obtained with either option.

Total antibiotic prescribing

Table 11 presents data for antibiotic prescribing for all indications during the trial. There were 185,924 antibiotic prescriptions in intervention trial arm practices and 150,539 in control trial arm practices. Crude rates were 575.3 and 581.0 per 1000 patient-years, respectively. The adjusted RR was 0.93 (95% CI 0.83 to 1.04; p = 0.184). There was no evidence that total antibiotic prescribing was reduced by this intervention.

Antibiotic prescribing by type of respiratory tract infection

Table 12 presents data disaggregated by type of RTI. Although the proportion of consultations with antibiotics prescribed varied considerably by type of RTI, there was no evidence that the intervention might selectively impact on prescribing for any single subgroup of RTI. Adjusted RRs were all lower than unity, but there was no evidence that the intervention could be judged to be effective in any subgroup of RTI considered separately.

Deferred prescriptions

Table 13 shows the number of deferred prescriptions by trial arm and period. During the intervention period of the trial, 20 control practices and 22 intervention trial arm practices recorded no deferred antibiotic prescriptions. There were three control practices and six intervention trial arm practices that recorded > 50 deferred antibiotic prescriptions. Across all practices, the rate of deferred antibiotic prescriptions was similar in the intervention and control trial arms and did not increase after the introduction of the intervention.

Safety outcomes

The safety outcomes for the trial were those agreed by the Data Monitoring Committee and the study team. Table 14 presents the number of safety outcomes and rates per 100,000 patient-years by trial arm. Figure 12 presents a forest plot showing relative incidence rates for each safety outcome. There was no evidence to suggest that any of these outcomes were more frequent in the intervention trial arm. Estimates were imprecise for rare outcome events. Figure 13 shows the distribution of counts of safety events by age group and trial arm. Scarlet fever was the most frequently recorded outcome in children; pyelonephritis and peritonsillar abscess were most frequent at intermediate ages, whereas pneumonia was the most frequent outcome at older ages. It may be noted that the denominator population is slightly larger for the intervention trial arm. There was weak evidence that scarlet fever might have been more frequent in the control trial arm, but this might have been a chance finding.

FIGURE 12 Forest plot showing RRs (95% CI) of safety outcomes in the intervention trial arm compared with the control trial arm as reference. Estimates were from a Poisson model adjusted for age group, gender and comorbidity. Analyses for pneumonia were adjusted for the random effect of general practice. LL, lower limit of CI; UL, upper limit of CI. (Figure not reproduced here; see the original article.)

FIGURE 13 The number of safety outcomes, by trial arm and age group. (Figure not reproduced here; see the original article.)

Process evaluation

The trial outcome measures were evaluated in relation to the level of utilisation of DSTs at the intervention trial arm practices. In the lowest quartile of utilisation, DSTs were viewed at < 1% of RTI consultations. In the highest quartile, up to 28% of RTI consultations were associated with DST viewing (Table 15). In adults aged 15–84 years, there was evidence of a linear trend across quartiles of DST utilisation (adjusted RR 0.96, 95% CI 0.93 to 0.99). This association was not evident for children (adjusted RR 0.98, 95% CI 0.94 to 1.03), the senior elderly (adjusted RR 0.99, 95% CI 0.94 to 1.05) and, only weakly, for the sample as a whole (adjusted RR 0.97, 95% CI 0.93 to 1.00; p = 0.043).

Table 16 shows the responses of GPs to the process evaluation questionnaire items. There were 51 respondents from 31 out of 41 (76%) intervention trial arm practices. Respondents generally gave positive responses to items concerning the monthly antibiotic-prescribing reports. Respondents were pleased to receive the reports and found that they provided credible data, which were was easy to read and understand. Respondents discussed these with colleagues and found that these data were useful for their practice. However, items concerning the possible impact on antibiotic prescribing were generally slightly less favourably addressed, with < 80% of respondents agreeing that the reports encouraged them to reduce antibiotic prescribing or had impact on the prescribing of antibiotics at the practice. The webinar was generally favourably received. The DSTs tended to be slightly less favourably received than the prescribing reports, with nearly one-third of respondents not affirming that the tools might support their reduction in antibiotic prescribing.

Cost analysis

The total costs of health-care utilisation were estimated for patients who made one or more consultations for RTIs during the trial intervention period. Costs were evaluated from the date of the index consultation until the end of the trial and were scaled to annual costs using the person-time available for analysis. During the trial there were 144,438 RTI consultations by 106,219 patients. Costs of health-care utilisation were estimated for 105,389 (99%) patients. The overall distribution of health-care costs per patient-year are shown by trial arm in the Table 17.

The coefficient associated with intervention trial arm was –0.081 (standard error 0.078); after exponentiation this may be interpreted as a multiplicative factor, showing that costs in the intervention trial arm were 92% of those observed in the control trial arm. The 95% CI was from 79% to 107%, with a p-value of 0.298. Therefore, the trial provided no evidence that the total costs of health-care utilisation might differ as a result of intervention, at least during the time horizon of the trial.