Results and discussion

Organisation of results and discussion section

The results and discussion section is organised into two parts, as illustrated in Table 2. We have organised the chapter in this way because the choice of analysis in the second part was determined by content in both the results and the discussion of analysis in the first part. Therefore, the most logical way to present the results and discussion was a single chapter that is split into two parts (see Table 2).

Results and discussion part 1: analysis of chronic obstructive pulmonary disease treatment effects in a TORCH trial-analogous3 cohort (objective 1)

Results

Participants

FP-SAL exposed compared with FP-SAL unexposed

Between 1 January 2004 and 1 January 2017 there were 125,671 people in the CPRD with a diagnosis of COPD, 73,889 (59%) of whom were from HES-linked CPRD practices (Figure 3). Application of TORCH trial3 inclusion criteria reduced this to 18,715 people, contributing 35,746 unexposed-to-FP-SAL time periods and 26,390 exposed-to-FP-SAL time periods. After applying TORCH trial3 exclusion criteria, dropping records with missing covariate data and matching the unexposed patients to TORCH trial3 participants, there were 4196 unexposed patients available for propensity score matching to 10,463 FP-SAL-exposed time periods. The final propensity score-matched cohorts included 2652 patients in each exposure group for the exacerbations analysis, 2708 patients in each exposure group for mortality and 2779 patients in each exposure group for pneumonia.

FIGURE 3 Flow of number of individuals included in the exposed to FP-SAL vs. unexposed to FP-SAL cohort analysis. Note that the current/previous use of COPD drugs relates to any of the drugs studied in the TORCH trial,3 long-acting bronchodilators and OCSs (see Box 1). SES, socioeconomic status. (Figure not reproduced here; see the original article.)

FP-SAL exposed compared with SAL exposed

For the FP-SAL compared with SAL analysis, there were 154,785 people with a diagnosis of COPD in the CPRD between 1 January 2000 and 1 January 2017, 91,733 (59%) of whom were from HES-linked CPRD practices (Figure 4). A total of 1146 SAL-exposed patients were available for propensity score matching to 11,235 FP-SAL-exposed periods. The final propensity score-matched cohorts included 991 (exacerbations), 432 (mortality), 935 (pneumonia) and 996 (treatment discontinuation) patients per exposure group.

FIGURE 4 Flow of number of individuals included in the exposed to FP-SAL vs. exposed to SAL cohort analysis. SES, socioeconomic status. (Figure not reproduced here; see the original article.)

Application of TORCH trial3 inclusion/exclusion criteria and matching to the TORCH trial3

Applying the TORCH trial3 inclusion/exclusion criteria and matching to the TORCH trial3 resulted in cohorts that were much more similar to those recruited to the TORCH trial3 (e.g. FEV₁% of predicted for the FP-SAL vs. unexposed to FP-SAL analysis was 66.3 in the CPRD before applying any criteria or matching, compared to 47.2 after these steps, compared to a TORCH3 placebo group value of 44.2) (see Table 3). The largest residual difference to the TORCH trial3 placebo group was for prior cardiovascular disease for both comparisons (Tables 3 and 4).

Propensity score matching of Clinical Practice Research Datalink cohorts

Details of the variables included in the final propensity score models are provided in Table 5.

FP-SAL exposed compared with FP-SAL unexposed

Prior to propensity score matching, for the exacerbations, analysis differences by exposure status were noted for sex, FEV₁, BMI, prior exacerbations, coronary heart disease, peripheral vascular disease, cerebrovascular disease, prescriptions for aspirin, COPD medications, number of general practitioner (GP) consultations and number of distinct medications (Table 6). After propensity score matching, only the differences with respect to coronary heart disease, peripheral vascular disease and LABA persisted (see Table 6). Plots of propensity score distributions indicated close propensity score matching for exacerbations and all other outcomes under study (Figure 5).

FIGURE 5 Propensity score distributions before and after matching. (a) FP-SAL exposed (n = 10,926 before matching) vs. FP-SAL unexposed (n = 4391 before matching); and (b) FP-SAL exposed (n = 11,235 before matching) vs. SAL exposed (n = 1146 before matching). Treatment discontinuation not included for these analysis as only one of the exposure groups was receiving treatment. (Figure not reproduced here; see the original article.)

FP-SAL exposed compared with salmeterol exposed

For the FP-SAL compared with SAL exacerbations analysis, after propensity score matching, there were notable imbalances in prior prescriptions for a LABA or an ICS and frequency of consultations, with smaller imbalances for lung function, BMI, coronary heart disease, statin prescription, aspirin prescription, LAMA, ICS plus LABA and prior GP consultations (Table 7). Plots of propensity score distribution indicated that, overall, groups were well matched on propensity score for each outcome (see Figure 5).

Main results

FP-SAL exposed compared with FP-SAL unexposed

For the exacerbations analysis, the rate ratio in the propensity score-matched groups was 1.30 (95% CI 1.19 to 1.42) (Table 8). According to our prespecified protocol, this (harmful) association was not considered to be consistent with the (protective) TORCH trial3 placebo-controlled result for the same outcome (0.75, 95% CI 0.69 to 0.81).1 Similarly, our result for the mortality outcome [hazard ratio (HR) 1.11, 95% CI 0.95 to 1.26] was in the opposite direction to the TORCH trial3 placebo-controlled result (HR 0.83, 95% CI 0.68 to 1.00). For the pneumonia analysis, we found weak evidence for a 14% increased risk associated with FP-SAL [risk ratio (RR) 1.14, 95% CI 0.96 to 1.34], which was not consistent with the stronger harmful association found by the TORCH trial3 placebo-controlled analysis (RR 1.59, 95% CI 1.35 to 1.88).

FP-SAL exposed compared with salmeterol exposed

For the exacerbations analysis, we obtained a propensity score-matched rate ratio of 0.85 (95% CI 0.74 to 0.97). According to our prespecified protocol, this (protective) effect was considered to be consistent with the TORCH trial3 FP-SAL compared with SAL result for the same outcome (rate ratio 0.88, 95% CI 0.81 to 0.95) (Table 9).1 Similarly, our result for the mortality outcome (HR 0.93, 95% CI 0.65 to 1.32) was consistent with the TORCH trial3 FP-SAL compared with SAL result (HR 0.93, 95% CI 0.77 to 1.13). For the pneumonia analysis, we found evidence for a 39% increased risk associated with FP-SAL (RR 1.39, 95% CI 1.04 to 1.87), which was also consistent with the harmful association found by the TORCH trial3 FP-SAL compared with SAL analysis (RR 1.47, 95% CI 1.25 to 1.73). For the time to treatment discontinuation analysis, the effect was apparently much stronger outside the trial setting (non-interventional HR 0.23, 95% CI 0.20 to 0.27 vs. TORCH trial3 non-interventional HR 0.89, 95% CI 0.79 to 0.99).

Analysis of impact of (1) TORCH trial3 matching and (2) TORCH trial3 criteria (post hoc analysis)

Repeating the FP-SAL compared with SAL analysis and omitting the TORCH trial-matching3 step led to an exacerbations rate ratio of 0.87 (95% CI 0.81 to 0.94) (Table 10), which is very similar to both the main analysis and the TORCH trial3 result. By contrast, neither applying the TORCH trial3 criteria nor matching led to a completely different effect estimate (rate ratio 1.64, 95% CI 1.52 to 1.77).

Discussion

We have demonstrated that methods applied to non-interventional data can generate results comparable to active comparator trials for COPD treatment effects. By contrast, we found that the same methods were unable to replicate placebo-controlled trial results.

Comparison with previous studies

Previous studies applying similar ‘trial-replication’ approaches

Although a number of papers have compared the designs of observational studies with RCTs21–26 and some studies have generated results similar to an earlier or subsequent trial,27–29 to our knowledge there are very few non-interventional studies that have set out to explicitly replicate a specific trial cohort and its results.

Hernán et al.30 replicated the design and result of the Women’s Health Initiative31 randomised trial on the effect of oestrogen/progestin therapy on coronary heart disease risk. Smeeth et al.32 analysed the effect of statins on a range of health outcomes and replicated the Heart Protection Study33 randomised trial. Fralick et al.34 applied trial criteria and utilised propensity score matching to replicate cardiovascular results from ONTARGET35 (ONgoing Telmisartan Alone and in combination with Ramipril Global Endpoint Trial).

Previous studies of chronic obstructive pulmonary disease drug treatment effects

Results of five (LABA/ICS vs. LABA) interventional studies (including the TORCH trial3) were summarised in a Cochrane review (rate ratio 0.76, 95% CI 0.68 to 0.84).36 Three out of these five studies estimated effect sizes that were considerably greater than effect sizes reported in the TORCH trial.3 As this study mirrored the TORCH trial,3 our results aligned most closely to those of the TORCH trial.3

A number of studies have found strong survival benefits of ICS therapy after hospital discharge.36–38 After accounting for likely time-related biases that have an impact on these studies, a null effect was obtained (rate ratio 0.94, 95% CI 0.81 to 1.09).39 The methodology we applied obtained a mortality effect estimate comparable to the analysis designed to account for time-related biases (0.93, 95% CI 0.65 to 1.32).

In line with the TORCH trial,3 previous studies have found an increased risk of pneumonia associated with ICS-containing treatments for COPD.36,40,41 Our result (RR 1.39, 95% CI 1.04 to 1.87) was consistent with results of a meta-analysis of trials comparing LABA/ICS with LABA formulations (OR 1.55, 95% CI 1.20 to 2.01)36 and very similar to a recent non-interventional study comparing LABA/ICS with LAMA formulations (HR 1.37, 95% CI 1.17 to 1.60).42

Our 3-year probability of treatment discontinuation for FP-SAL (28%) is comparable to non-adherence figures from previous non-interventional real-world data studies (49% and 43%).43,44 The probability of discontinuation of SAL that we observed (77%) was higher than these two previous non-interventional studies, leading to the discrepancy with the TORCH trial.3 We hypothesised that during our study period a large proportion of the patients who would have been initially prescribed SAL would have been likely to switch to FP-SAL because of prescribing decisions in primary care. A post hoc analysis found that 43% of people prescribed SAL switched to FP-SAL during follow-up (vs. only 2% switching from FP-SAL to SAL).

Implications and further work

When studying COPD treatment effects, if (1) the analysis is of active comparators, (2) trial exclusion and inclusion criteria are applied and (3) the propensity score models that we developed for each outcome are applied to balance exposure groups, then the results of studies carried out in routinely collected non-interventional data can be considered robust in the sense that they will be highly comparable to trial results. This now provides a methodological framework for being able to analyse COPD drug treatment effects in real-world data, focusing on groups that were either not included or under-represented in trials.1

Our inability to replicate placebo-controlled analyses suggests uncontrolled confounding by indication, a well-known bias in pharmacoepidemiology that is highly likely to be present when performing a comparison between people prescribed a drug and people not prescribed a drug.45–48 An established design approach for addressing this bias is to perform an active comparator analysis (i.e. comparing the effects of one medicine with another, rather than one medicine with no treatment).45–48 Based on the likelihood of confounding by indication having an impact on our results compared with the results of no treatment, we proceeded to perform the active comparator analysis and obtained results very similar to the trial, which indicates that confounding by indication is highly likely to be the reason for being unable to replicate placebo-controlled analyses in this setting.

One possibility for how this confounding by indication may be manifesting relates to an aspect of our study design that allowed people to be included in both the exposed and unexposed cohorts (i.e. the result we obtained could be strongly influenced by people initially in the unexposed group who are relatively healthy but then get more sick over time and require FP-SAL treatment and end up in the exposed group). However, in a post hoc analysis in which we dropped the 730 people (out of a total of 2652 per group) who appeared in both cohorts, our effect estimate was nearly identical (RR 1.33, 95% CI 1.20 to 1.47). We do consider, however, that because COPD treatment is based on a step-up approach, it is highly likely that patients not exposed to FP-SAL in routine primary care are generally likely to be those with milder COPD.

An additional point that further explains our inability to replicate the placebo-controlled analysis relates to the large difference in incidence rate between the TORCH trial3 placebo group (1.13 exacerbations/person/year) and our FP-SAL-unexposed group (0.53 exacerbations/person/year). To investigate underlying reasons for this discrepancy, we performed a post hoc analysis in which we compared the characteristics of the 1753 people from TORCH trial3 who were not able to be matched to our unexposed-to-FP-SAL population in step 3 with those who were successfully matched. We found that those not matched were younger (mean age 60.7 vs. 65.8 years), more sick (history of cardiovascular disease: 93% vs. 46%), had worse lung function (FEV₁ 34.9 vs. 45.9) and included a higher proportion of people recruited from Eastern European trial sites (27% vs. 17%). People with these characteristics may have been highly suitable for recruitment to the TORCH clinical trial,3 but are very difficult to find in UK primary care, providing another reason why we were not able to replicate placebo-controlled analyses. Furthermore, both TORCH trial3 placebo-assigned patients and patients in our own cohort were permitted to use other COPD treatments during follow-up, but given the time and setting differences between our FP-SAL-untreated group and the TORCH trial3 placebo group, people in our placebo group are much more likely to have been prescribed, for example, an ICS during follow-up. More generally, it is also likely to be challenging to obtain comparable absolute rates in emulated cohorts within a single country based on historical international trials for reasons such as this.

Previous authors have gone as far to specifically recommend that when trying to emulate trial results it is important to choose an active comparator trial.34 There are, however, examples in which placebo-controlled analyses have been successfully replicated.30,32 One possibility is that replication of placebo-controlled results works better when the drug studied is (1) preventative and (2) used in a generally healthy cohort (e.g. the cited studies were of statins and of postmenopausal hormone therapy both prescribed, in some instances, to people without a specific underlying chronic disease, in contrast to the patients with COPD who received therapy in our study). We consider that further avenues of research could be followed to understand if there remains a possibility of replicating placebo-controlled studies within a non-interventional setting for COPD therapies. These could include the application of high-dimensional propensity scores or the use of instrumental variables. Our work also suggests that treatment discontinuation in the setting of non-interventional data may be driven by very different factors to those seen in trials and, at least in the setting of COPD, may not be a useful outcome to study. Interpretation of treatment discontinuation in routine data is challenging. For example, it is difficult to establish from routinely collected data whether a patient has truly stopped taking their medicine or is just taking the medicine differently than prescribed (e.g. is taking less than has been prescribed over a longer period), a point that was emphasised by our patient and public involvement representative.

Finally, in our post hoc analysis, we found that the application of the trial-matching step did not confer any advantage over the application of trial criteria alone in this setting. This suggests that treatment–covariate interactions are not as critical as we initially thought in this therapeutic area.

Limitations

Some of the TORCH trial3 inclusion criteria were not fully assessable using CPRD data, meaning that the inclusion/exclusion criteria are analogous with TORCH trial3 criteria, but we acknowledge that they are not identical. For example, TORCH trial3 inclusion criteria included negative reversibility spirometry criteria, but it was not possible to replicate this in the CPRD data. In addition, at entry to the TORCH trial3 2-week run-in period all ICSs and inhaled long-acting bronchodilators were discontinued, but it was not possible to ensure that all patients selected from the CPRD had discontinued these at the start of the run-in period that we applied.

We originally planned to apply frequency of COPD therapy prescriptions in the previous year as a matching character/criterion. In practice, this was not feasible. However, it appears that matching at this level of detail was not required to be able to replicate trial results for active comparator analysis. Furthermore, it is clear from Figures 3 and 4 that we had to drop around one-fifth of patients as they did not have spirometry measurements recorded in the CPRD. This does mean that our initial study population may be missing key groups of people who tend not to have their spirometry measured (e.g. people with COPD who have the least contact with the health services). This is a problem for any COPD research performed using routinely collected primary care data, and although we acknowledge the issue we think that in this work it is likely to have minimal impact, as our aim was to create trial-analogous exposure groups that were highly comparable to each other, not to select a highly generalisable sample.

Finally, within the TORCH trial,3 the dose of the fixed combination product FP-SAL was specified as 500 µg of FP and 50 µg of SAL (500/50) and the dose of SAL alone as 50 µg, whereas in our study we did not limit to a specific dose. The reason for this is that dosage information is incompletely captured in the CPRD. These are the only approved doses of FP-SAL and of SAL for COPD in the UK and so we consider the doses that people were prescribed in our study would have been generally similar to that administered in the TORCH trial,3 although we do acknowledge that there are likely to be some differences in dosing between the TORCH trial3 and our cohort because of the long-term management of patients with varying disease severity and varying concomitant conditions.

Conclusions

By replicating the COPD TORCH trial3 selection procedures and inclusion/exclusion criteria in real-world data and developing propensity score models to account for any remaining differences between groups, we were able to obtain highly comparable relative effect estimates to the TORCH trial3 active comparator analysis for exacerbations, mortality and pneumonia.

Replication of placebo-controlled analyses was not possible. This is a not entirely unexpected result because of the well-established fact that when comparing outcomes in people on treatment with people not on treatment using non-interventional data, confounding relating to the difference in the underlying health status of the two groups is usually too severe to address. We also found that trial placebo groups in this therapeutic area may be much less healthy than people attending UK primary care, having a further impact on the ability to replicate no-treatment comparisons. Further work to investigate whether or not confounding by indication for no treatment comparisons can ever be accounted for in this therapeutic area is warranted.

Performing active comparator analyses is a well-established design approach for minimising confounding by indication. In addition, obtaining such similar results to the TORCH trial3 when comparing active comparator results provides confidence that our methods will provide valid results when used in other active comparator analyses within this therapeutic area. Application of the same selection procedures and propensity score models developed here to active comparator analyses of COPD drug treatment effects in groups under-represented or excluded from trials provides a practical way for key evidence gaps to be filled in relation to whether or not one treatment is more effective than another.

Results and discussion part 2: analysis of chronic obstructive pulmonary disease treatment effects in (1) patients excluded from trials and (2) patients with milder chronic obstructive pulmonary disease (objectives 2 and 3)

Note on results presented in results and discussion part 2

As presented and discussed in part 1 of this chapter (covering the primary analysis of objective 1), only active comparator analyses (FP-SAL vs. SAL) obtained valid results for the analysis of treatment effects (i.e. comparable to the TORCH trial3) using our methods. All of the primary analyses for objectives 2 and 3 (which are presented and discussed in this part of the results and discussion chapter) were, therefore, also performed using the FP-SAL compared with SAL comparison. Of the original analyses planned in the Chapter 2, Objective 1: validation of methods for measuring chronic obstructive pulmonary disease medication effectiveness in electronic health record data by comparing with trial results and Objective 2: measurement of chronic obstructive pulmonary disease treatment effects in patients excluded from trials, there were insufficient numbers of people exposed to SAL for performing analyses of cohorts that were excluded from the TORCH trial3 because of a history of lung surgery (n < 60 prior to propensity score matching), history of long-term oxygen therapy (n < 29), substantial comorbidity (i.e. a serious uncontrolled disease with high likelihood of causing death within 3 years) (n < 60) or evidence of drug/alcohol abuse (n < 30). Results are, therefore, presented for the analysis of the effect of FP-SAL compared with SAL on exacerbations, mortality and pneumonia for the following groups that would have been excluded from the TORCH trial:3

• people aged > 80 years
• people with an asthma diagnosis within the 5 years prior to inclusion
• people with milder COPD based on spirometry measurements (i.e. those who did not meet either or both of the TORCH3 requirements of a FEV₁ of < 60% predicted or a FEV₁/FVC ratio of < 70%).

Results

Participants

After applying the TORCH trial3 exclusion criteria, as outlined in part 1, to the 91,733 people from HES-linked CPRD practices with a diagnosis for COPD between 1 January 2000 and 1 January 2017 but with exclusion criteria altered relating to (1) aged > 80 years, (2) presence of asthma or (3) selection of people with milder COPD based on spirometry, the final cohort sizes before and after propensity score matching were as shown in Table 11.

Propensity score matching of cohorts of people who would have been excluded from the TORCH trial3

Details of the variables included in the final propensity score models are provided in Table 12. As described in Chapter 2, Propensity score for addressing confounding, variables from the pool of initial variables that were not associated with the specific outcome under study were not included in the multivariable logistic regression model used to generate the propensity score. This, therefore, meant that the final propensity score models for each of the outcomes under study could contain a different set of variables, depending on the outcome being studied. For example, for people with milder COPD, the mortality propensity score included sex and age, but these variables did not end up in the final propensity score for exacerbations or pneumonia. Although sex and age were associated with each of these outcomes in crude analysis, exacerbations and pneumonia were dropped from the subsequent multivariable logistic regression models that had drug exposure as an outcome (specified in Chapter 2, Propensity score for addressing confounding) because of the lack of association with exposure after multivariable adjustments.

People aged over 80 years

For the analysis of COPD exacerbations, prior to propensity score matching, differences by exposure status were noted for sex, prior exacerbations, coronary heart disease, peripheral vascular disease, cerebrovascular disease, prescriptions for COPD medications, chronic kidney disease and all measures of health-care utilisation (Table 13). After propensity score matching, many differences persisted or became apparent. Variables with substantial imbalance were age, sex, BMI, lung function, coronary heart disease, peripheral vascular disease, statin use, prior LABA or LAMA use, type 2 diabetes, chronic kidney disease and most measures of health-care utilisation.

Tables 17 and 18 in Appendix 1 show the balance of baseline variables for the analyses of mortality and pneumonia outcomes, respectively. For both outcomes, the pattern of imbalances before and after propensity score matching was very similar to that seen for COPD exacerbations.

People with an asthma diagnosis within 5 years prior to study entry

For the analysis of COPD exacerbations, prior to propensity score matching, differences by exposure status were noted for age, BMI, prescriptions for COPD medications, number of GP consultations and number of distinct medications prescribed (Table 14). After propensity score matching, many differences were minimised, although imbalances are notable for peripheral vascular disease, cerebrovascular disease, prior use of ICS and type 2 diabetes.

Tables 19 and 20 in Appendix 1 show the balance of baseline variables for the analyses of mortality and pneumonia outcomes, respectively. After propensity score matching, imbalances were similarly reduced for both outcomes. Minor differences from the exacerbations analysis were noted. For the mortality analysis, post-matching imbalances were noted for FEV₁ and prior use of LABA or ICS plus LABA (see Appendix 1, Table 19). For pneumonia, post-matching imbalances were seen for LABA rather than ICS use, BMI, ICS plus LABA and number of hospital procedures (see Appendix 1, Table 20).

People with mild chronic obstructive pulmonary disease (based on spirometry)

For the analysis of COPD exacerbations, prior to propensity score matching, differences by exposure status were noted for FEV₁ : FVC, prior exacerbations, coronary heart disease, peripheral vascular disease, cerebrovascular disease, statin use, use of other COPD medications and most measures of health-care utilisation (Table 15). After propensity score matching, most differences were minimised, although small differences can be seen for number of distinct medications and number of hospitalisations.

Tables 21 and 22 in Appendix 1 show the balance of baseline variables for the analyses of mortality and pneumonia outcomes, respectively. Similar imbalances to those seen for COPD exacerbations were seen before propensity score matching. After propensity score matching, for the mortality outcome, imbalances remained for FEV₁ : FVC, prior use of LABA or ICS, history of cancer, number of GP consultations and number of distinct medications (see Appendix 1, Table 21). For the pneumonia outcome, post-matching imbalances can be seen for FEV₁ : FVC, coronary heart disease and prior use of ICS (see Appendix 1, Table 22).

Main results

Table 16 shows the results for the association between FP-SAL and COPD exacerbations, mortality and pneumonia in the patient populations aged > 80 years, those with asthma and those with mild COPD. Crude associations and associations from propensity score-matched and propensity score-adjusted analyses are shown.

People aged over 80 years

For the exacerbations analysis, we obtained a propensity score-matched rate ratio of 0.59 (95% CI 0.36 to 0.95) and a propensity score-adjusted rate ratio of 0.83 (95% CI 0.60 to 1.14), which is consistent with the association measured in the TORCH trial-analogous3 CPRD population (0.85, 95% CI 0.74 to 0.97). For the mortality outcome, we obtained a propensity score-matched HR of 0.99 (95% CI 0.56 to 1.74) and a propensity score-adjusted HR of 1.29 (95% CI 0.84 to 2.00), which, again, is consistent with the TORCH trial-analogous3 CPRD result (0.93, 95% CI 0.65 to 1.32). For the pneumonia analysis, we found no evidence for an increased risk associated with FP-SAL, with a propensity score-matched rate ratio of 0.82 (95% CI 0.44 to 1.53) and a propensity score-adjusted rate ratio of 0.88 (95% CI 0.54 to 1.42).

People with an asthma diagnosis within 5 years prior to study entry

For the exacerbations analysis, we found a propensity score-matched rate ratio of 0.74 (95% CI 0.62 to 0.89) and a propensity score-adjusted rate ratio of 0.67 (95% CI 0.59 to 0.78), which is consistent with the association measured in the TORCH trial-analogous3 CPRD population (0.85, 95% CI 0.74 to 0.97). For the mortality outcome, we obtained a propensity score-matched HR of 1.49 (95% CI 1.21 to 1.85) and propensity score-adjusted HR of 1.20 (95% CI 1.04 to 1.40), contrary to the null findings with the TORCH trial-analogous3 CPRD result (0.93, 95% CI 0.65 to 1.32). For the pneumonia analysis, we found no evidence for an increased risk associated with FP-SAL, with a propensity score-matched rate ratio of 1.09 (95% CI 0.74 to 1.63) and a propensity score-adjusted rate ratio of 1.04 (95% CI 0.79 to 1.37).

People with mild chronic obstructive pulmonary disease (based on spirometry)

For the exacerbations analysis, we found a propensity score-matched rate ratio of 0.56 (95% CI 0.46 to 0.70) and a propensity score-adjusted rate ratio of 0.52 (95% CI 0.45 to 0.61), which suggests a stronger protective association than that measured in the TORCH trial-analogous3 CPRD population (0.85, 95% CI 0.74 to 0.97). Notably, however, the crude association in those with mild COPD was also strongly protective, unlike in the TORCH trial-analogous3 population. For the mortality outcome, we obtained a propensity score-matched HR of 0.98 (95% CI 0.67 to 1.45) and a propensity score-adjusted HR of 0.84 (95% CI 0.66 to 1.08). Again, this is consistent with the TORCH trial-analogous3 CPRD result (0.93, 95% CI 0.65 to 1.32). For the pneumonia analysis, we found no evidence for an increased risk associated with FP-SAL, with a propensity score-matched rate ratio of 0.78 (95% CI 0.45 to 1.35) and a propensity score-adjusted rate ratio of 1.08 (95% CI 0.74 to 1.57).

Discussion

In the second part of the project, we applied the observational methods that we previously validated against the TORCH trial3 to the study of three groups of people who would have been excluded from the TORCH trial:3 (1) people aged > 80 years, (2) people with a diagnosis of asthma in the preceding 5 years and (3) people with milder COPD, as determined by lung function. When analysing exacerbations, our results were consistent with our analysis of the TORCH trial-analogous3 CPRD cohort (and, therefore, with the TORCH trial3 itself), with the exception of people with milder COPD, for whom we observed a stronger protective association with FP-SAL. For the analysis of mortality, as for the TORCH trial-analogous3 cohort, we saw a lack of association between being prescribed FP-SAL (vs. being prescribed SAL), with the exception of those with prior asthma, for whom we observed an increase in mortality. Finally, for the analysis of pneumonia, although we found an increase in those prescribed FP-SAL in our TORCH trial-analogous3 cohort, we found no evidence of an association in any of our TORCH trial-excluded3 cohorts.

Comparison with previous studies

Previous studies that have extended ‘trial-replication’ approaches to understudied groups

In Comparison with previous studies, we highlighted a number of other studies in which the aim was to replicate specific trials (and their results).30,32,34 None of these studies specified that further work was to include the use of the methodology that had been applied to specific study groups excluded or under-represented in trials. The authors of one of these studies34 (one of whom is a co-investigator on this grant) are working on the RCT-DUPLICATE (Randomized, Controlled Trials Duplicated Using Prospective Longitudinal Insurance Claims: Applying Techniques of Epidemiology) project,49 which aims to use similar trial-replication approaches to those we have used here to determine which types of clinical questions and real-world data analyses can be conducted with confidence with real-world data. The work we have performed here contrasts and compliments the RCT-DUPLICATE project,49 as it is specifically concerned with answering questions about populations that would not be included in trials.

Previous studies of chronic obstructive pulmonary disease drug treatment effects in groups excluded from trials

Although it is recognised that treatment of COPD in the elderly population is a particular challenge for both clinicians and patients,50 there is a lack of studies of treatment effects. Pooled analyses of trial data have been performed to address this knowledge gap and have demonstrated that effects obtained are similar to those obtained in overall trial populations.51 In contrast to our methodology, these analyses may require access to trial data from a large number of trials to achieve sufficient power, rely on the trials being well performed to produce valid results and can be performed only when these data exist already.52

There is a lack of studies focusing on the effects of COPD treatments on people who have both asthma and COPD. The ‘recommendations for research’ section of the current NICE guidance on COPD treatment includes ‘inhaled therapies for people with COPD and asthma’.20 Emerging evidence suggests that this group of people is a group that is more sick, with higher levels of inflammation and a worse prognosis.53,54

In a recent non-systematic review55 of the evidence for COPD treatment effects in people with mild COPD, the authors commented on the concerning lack of evidence for commonly used maintenance treatments (e.g. LABA/ICS combinations) in this population. This review highlights only two studies56,57 that focused on any of the outcomes we studied in mild COPD patients, both focusing on exacerbations and both only studying monotherapy. The first study was a systematic review and meta-analysis of ICS compared with placebo and found a lack of effect on exacerbations, although the size of the analysis was small (n = 191 from three RCTs) and the result had wide CIs (RR 0.92, 95% CI 0.55 to 1.53).56 In the second study (a RCT), there was no evidence of a difference between exacerbation rate in 391 patients with mild COPD taking the LABA FP compared with placebo.57

Implications and further work

Our propensity score-matched results for the > 80-year-old cohort suggest that the effect of FP-SAL (vs. SAL) on exacerbations is of a similar protective magnitude as it is in a cohort of people with similar characteristics to those in the TORCH trial3 (RR 0.59, 95% CI 0.36 to 0.95 in the > 80-year-old group compared with RR 0.85, 95% CI 0.74 to 0.97 in our TORCH trial-analogous3 cohort). This provides some reassurance of the effectiveness of this treatment in this population in a UK primary care setting. Further work could be to extend this analysis to other drug treatments (e.g. looking at inhaled triple combination therapy LAMA + LABA + ICS, as recently added to NICE guidance5) and also assessing whether or not the same effect is seen when using the same methods applied in a different setting (e.g. an EHR database from another country). For the mortality and pneumonia effects, although both our propensity score-matched and our propensity score-adjusted associations were consistent with those found in our TORCH trial-analogous3 analysis, there were wide CIs. One possibility could be to repeat these analyses and include the additional EHRs now available in CPRD Aurum.58

For our asthma cohort, the increased mortality observed in the FP-SAL group (in contrast with the null effect found in our TORCH trial3 analysis) could be a spurious result, possibly due to imbalances in the cohort that were introduced by our propensity score method (although from Appendix 1, Table 19, the only observed imbalance introduced to matching is a small difference in FEV₁). Further work is, therefore, needed to investigate this finding. However, as mentioned previously, people with both asthma and COPD have a poorer prognosis than those with just COPD, and how this has a selective impact on those on FP-SAL compared with those on SAL while having a protective effect on exacerbations and a null impact on pneumonia needs further consideration. Future work (in both CPRD GOLD and other data sources) should focus on identifying any patterns in cause-specific mortality and whether or not this represents any possible causal association with treatment or unadjusted confounding due to underlying differences between treatment groups.

Finally, for the cohort of people with mild COPD, our results indicate that there is a stronger protective effect on exacerbations than those people in the TORCH trial-analogous3 cohort. This is a potentially reassuring result; however, as it has not been seen before, it would be advisable to try to replicate this finding in a completely separate setting using this methodology (e.g. an EHR database containing a completely distinct population to the CPRD).

Limitations

Some of our analyses had very small numbers. In particular, the propensity score-matched groups for those aged > 80 years were all < 100 (see Table 11), despite using one of the largest data sets available for these kinds of analyses. To help address this, we also presented results that were adjusted for the propensity score rather than matched. Although some of the point estimates differed when comparing matched with adjusted estimates, none of our conclusions changed for this group when considering the propensity score-matched cohort with the propensity score-adjusted cohort. Nonetheless, the more recent availability of even larger quantities of EHR data (i.e. CPRD Aurum) will allow for more precise estimates in future and would likely mean that any residual discrepancy between matched and adjusted estimates would be accounted for.

A key limitation of this part of the analysis (that also effects all observational studies) is that we cannot have the same level of certainty regarding causal associations as would be possible in a well-conducted randomised trial. Owing to the nature of the research questions in this part of the project, we cannot know if our results are valid and would replicate those of a reference trial, as for this stage we were not matching our cohorts to actual trial participants or comparing the results to actual clinical trial results. However, as discussed in Analysis of impact of (1) TORCH trial3 matching and (2) TORCH trial3 criteria (post hoc analysis), it was the application of the trial inclusion/exclusion criteria when selecting cohorts that led to trial-comparable results not matching to the individual trial participants, and we did apply the same rigorous trial selection-type approach (described in Chapter 2, Selection of participants) to the selection of people for inclusion in all of the cohorts in this section. Furthermore, our methods for preparing the propensity score models were identical to those used in the TORCH trial-analogous3 analysis (described in Chapter 2, Propensity score for addressing confounding).

Conclusions

Analyses of the association between FP-SAL and COPD exacerbations, mortality and pneumonia compared with SAL are largely consistent with the results from the TORCH trial3 when we extend to patients excluded from or under-represented in the TORCH trial.3 This is largely reassuring, but two potential differences were noted. First, we observed a larger protective association than expected for COPD exacerbations in people with mild COPD. Second, and more concerningly, we observed a small increased risk of death in patients with evidence of both COPD and asthma. Both these signals warrant further investigation in CPRD GOLD and in other data sets.