Aims
We aimed to investigate the effect of prescribing varenicline on smoking abstinence, all-cause and cause-specific mortality and hospitalisation, a broad range of health outcomes and whether or not the effects of varenicline differed by SEP.
Summary of the risks and the benefits
Effectiveness
Randomised controlled trials have demonstrated that varenicline is more efficacious than NRT for smoking cessation. Our results demonstrate that varenicline is more effective in the general population when prescribed as part of patient usual care. Our estimates suggested that 4.76 (95% CI 2.77 to 6.74) additional patients would be expected to quit within 2 years per 100 patients prescribed varenicline rather than NRT. These estimates are highly consistent with results from RCTs.10 There was little evidence of large differences in effectiveness by SEP.
Health outcomes
Patients prescribed varenicline had lower all-cause and cause-specific mortality; however, they were healthier than those prescribed NRT in almost all ways we could measure at baseline. Therefore, the multivariable-adjusted results are likely to suffer from a degree of residual confounding. We addressed this using instrumental variable analysis. Our instrumental variable results found little evidence that varenicline caused large reductions in mortality. Similarly, smokers who were prescribed varenicline had lower rates of all-cause and most types of cause-specific hospital admission than those prescribed NRT. However, our instrumental variable results for hospitalisation were imprecise and consistent with the null and the multivariable-adjusted results. All analytic methods suggested that, on average, patients prescribed varenicline gained around 1 kg in weight and attended primary care less frequently. There was some evidence from the instrumental variable analysis that prescribing varenicline increased the risk of diabetes-related mortality.
Public and patient involvement
Aim
To get feedback on our study from the patients and the public.
Methods
On 30 September 2015, we presented our plans for the research project to the public advisory panel of the Elizabeth Blackwell Institute. The panel members gave feedback about outcomes of most interest to the public and suggested possible avenues for dissemination. On 6 October 2015, we presented our plans for the research project to the UK Centre for Tobacco and Alcohol Studies Smokers Panel in Nottingham. This interactive session involved a group of around 30 smokers, ex-smokers and health-care professionals. We presented details of the outcomes we planned to study, the data we planned to use and the analysis we expected to run. This presentation provided an opportunity to discuss with smokers what they knew about varenicline and where they got information on side effects. The panel was interested in more evidence about the effects of varenicline in everyday clinical care, and possible side effects of varenicline. On 5 June 2016, we presented our findings to the UK Centre for Tobacco and Alcohol Studies panel.
Study results
These presentations had a major impact on how we presented our results in the published papers. In particular, it was clear that it was difficult for panel members to interpret multiple estimates from the same study. Furthermore, providing the results on an intuitive scale helped make the results from the study more accessible. We integrated this feedback into our presentation of the results in the published papers.
Discussion and conclusions
The findings from the patient and public involvement helped inform our study and changed the way we presented results, particularly around risk.
Strengths and limitations
A strength of our study is that we used the CPRD, which includes data from > 4.4 million patients actively registered at general practices across the UK.15 These patients are likely to be representative of patients prescribed smoking cessation medications in the UK. These patients are also more likely to be representative of the population than participants recruited to RCTs. Furthermore, compliance with treatment regimen is likely to be more representative of real-world compliance seen in primary care. This fact means that our estimates of the effects of prescribing are likely to better reflect the effects of prescribing smoking cessation medications in primary care. A drawback of this approach is that they are likely to underestimate the effect of complying with treatment. The parameter is perhaps of most interest to patients, who have the largest effect on compliance. Physicians, who have less influence over compliance, may be more interested in the effect of issuing prescriptions for varenicline or NRT.
Where possible, we have used validated code lists to define the events used in this study. These lists ensure that the events are well defined and have high sensitivity and specificity for detecting events. For other outcomes, we constructed the code lists by using expert clinical review of search terms and Read terms. We have uploaded the code lists used in this study to a repository for transparency and to enable other researchers to reuse our code lists (https://github.com/nmdavies/varenicline-effectiveness).
The CPRD accurately records smoking status and GPs had financial incentives to record the smoking status of their patients. Furthermore, they were incentivised to repeatedly ask smokers and former smokers about their current smoking status. We used these records to determine patients’ smoking status in the years after their first prescription.
Most of the data were complete. However, there were missing values of IMD and BMI. We addressed this using multiple imputation for the multivariable-adjusted and propensity score regression models. The imputation suggested that missing values of these covariates had little impact on our findings. This similarity is partially because most of our covariate data and all of our prescription data were complete. In a sensitivity analysis, we also investigated whether or not our results were robust to using a multiple imputation model for smoking status after the first prescription. We found little evidence that our results differed when we used multiple imputation.
We retrospectively analysed patients’ electronic medical records. This approach means that we had a substantial follow-up period for each patient, both before they first received a smoking cessation prescription (which we used to define baseline covariates) and after their first prescription (which we used to define outcomes).
A major limitation of observational studies is residual confounding. Confounding can occur if there are baseline differences between patients given different treatments, in this case varenicline and NRT. Patients who were prescribed varenicline were healthier in almost every way that we could measure, although they were more likely to be male. We used multivariable-adjusted regression to control for these differences. This approach assumes that the measured differences account for all differences between patients given one treatment and another. This assumption is strong: that conditional on the measured covariates, the patients were as good as randomised. Although we used a comprehensive set of covariates, these variables are unlikely to have measured all differences between patients prescribed NRT and varenicline. We addressed this limitation using propensity score matching and instrumental variable analysis. The propensity score matching depends on similar assumptions to multivariable regression (i.e. no residual confounding) but can help address imbalance where there are groups of patients who were very unlikely to be given one of the treatments. Instrumental variable analysis can estimate the causal effect in the presence of unmeasured confounding of the treatment outcome association if the instrumental variable assumptions hold. The multivariable-adjusted and instrumental variable estimates for the effectiveness analysis were similar, suggesting that they are unlikely to be due to residual confounding. In contrast, for a number of the safety outcomes, there were substantial differences between the multivariable-adjusted and instrumental variable analysis. These differences in the results may indicate residual confounding.
We drew our sample from the general population of patients who were prescribed smoking cessation medications in primary care. We included all smoking cessation treatments except bupropion. However, very few patients received prescriptions for bupropion so this is unlikely to have caused selection bias in our results.35 Furthermore, studies have found that bupropion is less effective than other treatments and it is increasingly rarely prescribed.7
A significant limitation of our study is that we had little information on medication adherence or over-the-counter use of NRT. We did not model treatment adherence or treatment duration. In this study, we were aiming to replicate as closely as possible the intention-to-treat estimate in a RCT of prescribing NRT or varenicline. Treatment adherence decisions are likely to be highly non-random and may be related to the strength of addiction. This issue makes it difficult to estimate the causal effect of adhering to treatment. Our instrumental variable affected prescribing decisions rather than adherence. Varenicline is a prescription-only medicine. Therefore, we are likely to have complete information about the amount of varenicline issued to our sample. However, individuals in our sample are likely to have previously attempted to quit smoking using over-the-counter NRT. This could bias our multivariable-adjusted estimate, for example if individuals prescribed varenicline were more strongly addicted and had previously attempted to quit more frequently. Our instrumental variable estimates could be biased, for example, if individuals who attend physicians preferring varenicline have attempted to quit more frequently.
Finally, we estimated the effect of issuing prescriptions for NRT or varenicline, not the effect of actually taking NRT or varenicline. This estimate is perhaps most relevant for clinicians who have limited influence over treatment adherence. The effect of actually taking the medications is the estimate most of interest to patients. However, we cannot directly address this hypothesis using the models described in this project.
Areas of uncertainty
Our results provide relatively little evidence about the effects of smoking cessation medication on outcomes after 4 years. Our follow-up is much longer than RCTs, which typically follow patients for 1 year. However, varenicline became available only in 2006, so the maximum possible follow-up in our study was around 9 years. The follow-up for the vast majority of patients, who were prescribed varenicline after 2006, is much less than this. Future studies could use (1) a new extract of CPRD data and (2) instrumental variable survival models to make more efficient use of these data.27
We had limited power to investigate whether or not the effects of these medications differed in subgroups (e.g. older patients, patients with type 2 diabetes or patients diagnosed with cancer). A larger sample of data could provide far more precise estimates of the effects of these drugs. The CPRD covers only around 12% of the population. Therefore, a population sample could potentially provide results that were 2.7 times as precise as the estimates presented in this study. We had limited information about cancer stage and type. The information in the cancer registry could provide valuable evidence about the effects of smoking cessation by cancer type and stage.
However, it is reassuring that large experimental studies and real-world observational studies suggest that varenicline is both efficacious and effective in everyday clinical care. There was little evidence of treatment effect heterogeneity.
We had limited power to detect effects on important clinical outcomes, particularly with the instrumental variable analysis. The multivariable-adjusted analysis suggested that patients prescribed varenicline had better outcomes very shortly after their first prescription. However, the instrumental variable estimates suggested that residual confounding may drive these apparent benefits on clinical outcomes. There is little experimental data on the long-term effects of smoking cessation. Therefore, most evidence about the long- and medium-term effects of smoking cessation comes from observational studies, most of which use methods similar to multivariable-adjusted regression.1 Our results suggest that multivariable-adjusted regression may suffer from bias. Future studies could increase the precision of the results by using a larger sample, or survival analysis to make more efficient use of the data.