Methods

Material throughout the report has been adapted from the study protocol [see NIHR Journals Library www.fundingawards.nihr.ac.uk/award/16/57/01 (accessed 19 December 2022)].

Research questions

Our primary aim was to investigate the role of e-cigarette regulation, via the TPD, in influencing trajectories in young people’s use of e-cigarettes. We address the following research questions in relation to this aim:

• Did increased regulation of e-cigarettes interrupt prior growth of young people’s e-cigarette use?
• How did young people perceive risks and social norms surrounding e-cigarettes (and how did perceptions change over time as products become TPD compliant):
  • as a product in their own right?
  • relative to tobacco?
• How did young people interpret and respond to the presence or absence of health warnings on e-cigarette packets?
• To what extent, and in what ways, did young people continue to interact with e-cigarette marketing after the prohibition of cross-border advertising?

As a secondary aim, we also examine trends in young people’s smoking behaviour over time. This allows us to test the theoretical basis for much e-cigarette regulation, including that via the TPD, which centres on assumptions that e-cigarettes renormalise smoking. In addition, it enables us to estimate if the suite of regulation introduced in 2016 has maintained or increased the downwards trend in young people’s smoking uptake. We will address the following questions:

• Have trajectories in young people’s ever and current smoking been significantly interrupted (positively or negatively) by growing prevalence of e-cigarettes?
• Did the rate of decline in young people’s smoking change after additional regulation of tobacco and e-cigarettes in May 2016 (including TPD and plain packaging)?

Finally, as additional secondary aims, we explore the implementation and context of TPD regulation, including:

• To what extent was compliance with TPD in product sales achieved, and what are the barriers to and facilitators and unintended consequences of implementation?
• To what extent, and in what ways, did variations between UK countries in e-cigarette policy emerge during the study period?
• What other changes to the regulatory context of tobacco and e-cigarettes occur during the study period in the UK and across individual UK countries?

Study design

This study used a mixed-method natural experimental evaluation design to address the above research questions. The quantitative study elements drew on repeated cross-sectional secondary data from routinely undertaken surveys in Wales, Scotland and England. Study populations were nationally representative samples of secondary school-aged young people aged 13/15 years (or equivalent school years, i.e. years 9 and 11 in Wales and England and S2 and S4 in Scotland). These data were analysed alongside qualitative data from young people, policy representatives, TSOs and retailers of e-cigarettes, and retailer audits. Our overall evaluation and integration framework is presented in Figure 2, which provides an overview of how data sources are mapped onto subsequent findings chapters. Our evaluation and integration framework draws on the Medical Research Council (MRC) guidance for process evaluation90 and on the more recent context and implementation of complex interventions framework91 in framing context and implementation processes.

FIGURE 2 Evaluation and integration framework for TPD evaluation. CI-CI, content and implementation of complex interventions. Adapted with permission from Moore et al.90 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by/4.0/. The figure includes minor additions and formatting changes to the original text. (Figure not reproduced here; see the original article.)

Secondary data analysis

Data sources, collection and handling

We obtained data from routinely collected school-based surveys in each UK nation. Although some of these surveys go back to the 1980s, we made an a priori decision to restrict our analyses to data sets from 1998 to 2019 because published estimates indicate that smoking rates climbed up to this point, but have followed a prolonged period of decline ever since.

Data from Wales were obtained from the Welsh Health Behaviour in School-aged Children (HBSC) survey, conducted as part of an international World Health Organization collaboration every 2–4 years,92 and the biennial School Health Research Network (SHRN) survey.93 The SHRN survey was introduced in 2015 and was modelled on the HBSC survey, and from 2017 the SHRN survey incorporated the smaller HBSC survey as a component of it. Hence, HBSC survey data were obtained from 1998 to 2013, with SHRN survey data from 2015 to 2019. Data for Wales before 2013 were provided by the Welsh Government. SHRN and HBSC data from 2013 onwards are held by the principal investigator’s research group.

Although HBSC surveys are also conducted in England and Scotland, neither country included measures of e-cigarette use prior to TPD implementation, and these surveys are conducted only once every 4 years. Hence, in Scotland and England, we used the larger and more frequent Scottish Schools Adolescent Lifestyle and Substance Use Survey (SALSUS)94 and Smoking, Drinking and Drug Use (SDDU) survey in England.95 In Scotland, the SALSUS is a bi-ennial, school-based survey undertaken in local authority and independent schools. E-cigarette use questions were first incorporated into the SALSUS in 2013. The UK data service includes a time series data set that provides ready-pooled data through to 2013. Our original intention was to use the SALSUS planned for 2017 and 2019 as post-TPD data sets in Scotland; however, SALSUS 2017 was postponed to 2018,and no data were collected in 2019. Hence, we integrated measures from the 2015 and 2018 data sets into the time series data set, providing only one post-TPD survey for Scotland. In England, the SDDU survey was conducted annually until 2014, but then moved to a bi-ennial cycle. Measures of e-cigarette use were included in the SDDU survey from 2014. The SDDU survey shares substantial overlap with the SALSUS, including use of similar measures of smoking and e-cigarette use. Data sets from 1998 to 2018 were obtained via the UK data service. Data files were combined into a single data set and stored on a secured network at Cardiff University. The Cardiff University-owned Welsh data will be retained, and models updated with newer data following completion of this study. Terms of use for the third party-owned data sets require these data to be deleted following completion of the use period, currently the end of 2022.

Sample sizes and response rates

Table 1 provides sample sizes by country and survey year, overall and for the target age group. Survey response rates, where available, are presented in Appendix 1, Table 19. These data indicate a gradual decline in response rates over time in Scotland (from 70% in 1998 to 52% in 2018), and a much larger decline in England (from 70% to 22%). However, there was recent growth in response rates in Wales, with the 2019 SHRN survey achieving a response rate of 77%.

Study measures

E-cigarette use

In Wales, young people were asked if they had ever tried e-cigarettes, with options of never, once or more than once. Young people who selected ‘more than once’ were asked how often they currently use e-cigarettes (with options of, I don’t, less than weekly, at least weekly or daily). In Scotland and England, young people were asked to select a single option from a list of statements, which included they had never tried e-cigarettes, had used e-cigarettes once or twice, have used them in the past but don’t now, use e-cigarettes now but less than once a week, or use e-cigarettes now at least once a week. In all cases, a binary variable of ever use was created, which compared ‘never’ responses with all others. A regular (at least weekly) use measure was derived by classing all young people who selected options of weekly or more as regular users.

Smoking

In Wales, young people were asked at what age they first engaged in a range of risk behaviours (including smoke a cigarette) and were instructed to select the never option if there was something they had never done. Options included ‘never’ and ages from 11 to 16 years. A second question asked young people how often they currently smoked (with options of I don’t, less than weekly, at least weekly or daily). In Scotland and England, young people were asked to select a single option from a list of statements, which included they had never tried smoking, had tried smoking only once, used to smoke but don’t now, smoke but less than once a week, smoke between one and six cigarettes a week or smoke more than six cigarettes a week. In all cases, a binary variable of ever use compared ‘never’ responses with all others. A regular (at least weekly) use measure was derived by classing all young people who selected options of at least weekly or more as regular users.

Smoking attitudes

The acceptability of smoking is measured in both the SALSUS and SDDU survey via a question asking young people whether or not they think it is OK for someone their age to try smoking a cigarette to see what it is like. In the SDDU survey, young people were also asked whether or not it was OK for someone their age to smoke cigarettes once a week. Response options for both questions were: ‘it’s OK’, ‘it’s not OK’ and ‘I don’t know’. Responses of ‘I don’t know’ were combined with ‘it’s not OK’ to create a binary variable (sensitivity analyses combining ‘don’t know’ with ‘it’s OK’ showed similar trends over time). This item was used as an indicator of the plausibility of change in trend for smoking being caused by the mechanism of smoking renormalisation (see Statistical analysis of survey data).

Cannabis use

In SALSUS, young people were presented with a grid listing a range of drugs, including cannabis, and were asked which, if any, they have ever used. In the HBSC survey, young people were asked how many times they have used cannabis in their lifetime (response options: never, once or twice, three to five times, six to nine times, 10–19 times, 20–39 times and ≥40 times). In the SDDU survey, young people were asked if they had ever tried cannabis, with response options of yes or no. A binary variable indicating whether or not young people had tried cannabis was derived, with any response other than never (or no) classed as ever use. This item is used to examine whether or not any change in trend observed for tobacco use is specific to tobacco use (see Statistical analysis of survey data).

Alcohol use

Both the SALSUS and SDDU survey asked young people whether or not they had ever had a proper alcoholic drink. Since 2002, the HBSC/SHRN surveys asked young people at what age they first did a list of things, including drinking alcohol, with response options of never, and ages from 11 to 16 years. All responses other than never were classed as indicative of ever drinking. A binary variable was derived to distinguish between ever and never use of alcohol. This item is used to examine whether or not any change in trend observed for tobacco use is specific to tobacco use (see Statistical analysis of survey data).

Energy drinks

From 2013, in the HBSC survey, students in Wales were asked how often they drank energy drinks (response options: ‘never’, ‘less once a week’, ‘once a week’, ‘2–4 days a week’, ‘5–6 days a week’, ‘once daily’ or ‘more than once daily’). A binary variable was created to distinguish students who ever used energy drinks from students who never used energy drinks. This item is used to examine whether or not any change in trend observed for e-cigarette use is specific to e-cigarette use (see Statistical analysis of survey data).

Time and intervention variables

Time and intervention variables differed across analyses because of differences in data availability and granularity. In each case, time was coded as a continuous variable, starting at 0 at the start of the time series and increasing by 1 point per unit of time (using ‘month’ as unit of time in our primary statistical analysis of e-cigarette use in Wales, and ‘year’ as unit of time in three country analyses of smoking trends over time). Quadratic terms were generated by squaring time variables. For analyses of change in smoking outcomes following the emergence of e-cigarettes, we use a ‘level’ variable coded 0 for the baseline period 1998–2010, and 1 within the period from 2011 to 2015. Although not an ‘intervention’, the analytical approach of examining change in level and trend following the emergence of e-cigarettes follows a similar format to the evaluation of our intervention of interest (i.e. TPD). The period of 2011–15 was chosen because survey measures in the UK began to identify e-cigarette use among adult smokers from 2011, which grew rapidly from 2011 onward.16 We used a ‘post-slope’ variable coded 0 through the baseline period and sequentially from 1 to 5 through the period 2011–15. For our analysis of e-cigarette use, using Welsh data that were broken down by month, we used a ‘level’ term coded 0 prior to May 2016, coded 1 from May 2016 onwards and a ‘post-slope’ term coded 0 prior to May 2016, increasing by 1 point per month from the intervention point onwards. For analyses of post-TPD changes in tobacco smoking, level and post-slope variables were coded 0 prior to 2016 and 1 thereafter, and sequentially coded 1 onwards from 2016 to 2019. In simpler analyses of data for which few time points were available, time is analysed as a categorical variable (see Statistical analysis of survey data).

Sociodemographic information

In all surveys, young people were asked to indicate their gender. Historically, this question has been asked as a binary variable, which asks young people to indicate whether they consider themselves to be a boy or a girl. In SHRN in 2019, following consultations with policy and practice stakeholders and young people, a response option of ‘neither option describes me’ was provided. However, as data on young people identifying as neither male nor female are available only at one time point, our analysis of change over time is limited to young people identifying as a boy or girl. The percentage of students selecting ‘I do not want to answer’ to the question on gender was reduced (relative to 2017) by an amount equivalent to the percentage of young people selecting the new ‘neither option describes me’, suggesting that young people selecting this option were more likely to previously have declined to answer.96

School year was used as a proxy for age, with years 9 and 11 in England and Wales and S2 and S4 in Scotland used to represent young people aged 13 and 15 years, respectively. However, from 2016, the SDDU data set removed pupil year group because of concerns regarding potential deductive identification of individual children outside the expected school year for their age. Pre-legislation analyses were complete prior to this data set being obtained. Hence, in analyses using data from 2016 onward only, a variable for age rather than school year is used throughout the time series in England (i.e. ages 13 and 15 years).

For Wales, socioeconomic status (SES) was measured using the Family Affluence Scale (FAS),97 which, from 2013, comprised six items measuring car, computer and dishwasher ownership, bedroom occupancy, number of household bathrooms and prevalence of family holidays. An overall measure of family affluence is computed via summation of individual item scores (with higher scores reflecting greater material affluence). However, although consistent from 2013 to 2019 (i.e. the time period for our primary statistical analysis of change in e-cigarette use), this scale has changed over time as material products lose their ability to differentiate between socioeconomic groups and is problematic as an indicator of affluence over time for the period 1998–2019. To help assess inequality over time with HBSC/SHRN data in analyses beginning prior to 2013, a relative measure of SES was used, whereby the sample was divided into ‘high’ and ‘low’ affluence, regardless of the content of the FAS in any given year. SES was measured by free school meals entitlement in England from 1999 to 2014 (the SDDU survey also switched to use of FAS from 2016 onward) and in Scotland from 2006 to 2013. For Scotland, a measure of SES indicated by the Scottish Index of Multiple Deprivation (reported in quintiles) was available between 2006 and 2018. Items on ethnicity were introduced into HBSC from 2009/10. The SDDU survey provided data on ethnicity for most survey years, although no measures of ethnicity were included in most SALSUS data sets.

Statistical analysis of survey data

Analyses were based on a statistical analysis plan agreed with our Study Steering Committee in advance of data sets being combined for analysis. Analyses completed at each stage of the study were presented to the committee prior to submission for publication. Although our primary aim was to evaluate impacts of TPD, analyses are described (and presented) in the order in which they were undertaken, in line with our overall evaluation and integration framework (see Figure 2), including analyses of:

• trajectories in young people’s smoking before and after the emergence of e-cigarettes (but prior to TPD)
• change in young people’s e-cigarette use following the implementation of TPD
• change in trend for young people’s smoking following the 2016 suite of tobacco control legislation.

As described in our evaluation and integration framework in Figure 2, our analysis of trends in young people’s smoking prior to TPD is conceived as an element of our process evaluation, in that this provides an understanding of the epidemiological context into which intervention was introduced. This analysis is, however, described in this section to limit repetition due to methodological similarity with our primary statistical analysis.

Young people’s tobacco cigarette smoking before and after the emergence of e-cigarettes

Percentages and confidence intervals (CIs) of ever and regular smoking are presented for each outcome by year, country, age and sex. Binary logistic segmented regression analyses98 were then used to estimate change in smoking level and trends (post slope) during the emergence of e-cigarettes, relative to secular baseline trends, using the time, level and post-slope terms described in Study measures. The analysis was modelled on a similar approach used by Katikireddi et al.99 to examine change in trend for young people’s smoking following smoke-free legislation (using many of the same data sets), with two main differences. Although Katikireddi et al.99 conducted linear regression analyses of aggregated estimates, we had access to, and hence analysed, individual participant survey data. Furthermore, although Katikireddi et al.99 used estimates dating back to the 1980s (see Katikireddi et al.99 for a full list of historical data sets earlier than the cut-off point selected for this study), including a quadratic term to account for non-linearity, given that smoking rose until the mid-1990s, we limited our analyses to 1998 onward because of the continued decline in young people’s smoking from this date. Individual survey data on smoking from 1998 to 2015 for each of the three countries were used, with 2010 treated as the end of the baseline period and compared with the period 2011–15 (see Study measures). Country was included as a covariate, modelled as a set of dummy variables, and analyses were adjusted for gender and age. These analyses were conducted, presented to our Study Steering Committee and submitted for publication10 prior to obtaining post-legislation data sets.

Models were repeated for a secondary outcome of attitudes towards smoking (available from Scotland and England only). According to theories of renormalisation, e-cigarettes drive re-growth in young people’s smoking by increasing the extent to which young people see smoking as a normative behaviour. Analyses of attitudes towards smoking (i.e. the extent to which young people thought it was ‘OK’ for young people to smoke) provided evidence on plausibility of any change in trend (post slope) for smoking being causally driven by this hypothesised mechanism. For our primary statistical analysis, we were asked by the Public Health Research (PHR) Funding Committee to include falsifiability analyses, in line with the Bradford Hill criterion for causal inference of ‘specificity’.100 The falsifiability analyses involve modelling the same time series for related variables, which would not be expected to be altered by the exposure. Hence, following discussions with our Study Steering Committee prior to undertaking analyses, changes in trend for ever alcohol and cannabis use were also modelled to test whether any break in trend observed for tobacco use was unique to smoking or reflective of wider trends in young people’s substance use behaviours.

Although our a priori analysis plans for trends in smoking assumed linearity in the relationship between time and smoking behaviours, visual inspection indicated some evidence of a quadratic trend. Hence, consistent with similar analyses on change in young people’s smoking following smoke-free legislation,99 a quadratic time variable was included to account for non-linearity in the data. Both linear and quadratic models are presented, and models including the term treated as the final model. We also handled non-linearity through a post-hoc sensitivity analysis, which limited the baseline period to 2001–10, a period during which there was no statistical evidence of departure from a linear trend for smoking outcomes, for our main models. As additional sensitivity analyses, we ran our pre-legislation models (1) for England only (the country with the largest number of data points for pre-legislation analysis) and (2) excluding SALSUS rounds conducted at a different time of year to the rest of the data series (i.e. 2002–6). Given that later analyses also switched to use of age rather than year group for England, as described, we also re-ran our main pre-legislation models using this new age classification.

To estimate change in young people’s e-cigarette use following Tobacco Products Directive regulations

Percentages and 95% CIs for ever and regular e-cigarette use are presented by year, school year/age, gender and country. Trends are presented graphically, with estimates for each country plotted on a single graph to enable comparison of trends over time by nation. Segmented regression analyses were then used to formally test changes in the prevalence of ever e-cigarette following TPD implementation, using monthly measures of time, level and post slope (see Study measures). Models were adjusted for age and gender. Quadratic terms were not significant in models for e-cigarette use and so were not included. As TPD regulations were brought in gradually with a 1-year transitional phase, we anticipated that change in trend is more likely than immediate change in level. Indeed, for a public health problem with a rising baseline trajectory, whether or not an intervention is able to reverse, or at least slow, growth in young people’s e-cigarette use is of greater interest than whether or not an intervention causes a stepped disruption followed by a return to growth. Hence, we present changes in level and trend (post slope), focusing primarily on the latter.

In segmented regression analyses, there are trade-offs between long- and short-term analyses, in that longer-term analyses will have more power, but passage of time also increases the likelihood of changes being caused by other events.98 Hence, we model both short- (to 2017) and long-term (to 2019) changes in level and trend, and attend to similarities and differences between short- and long-term models. Models were implemented for the short-term post-implementation data (2013–17), presented to our Study Steering Committee in June 2019 and were submitted for publication before long-term data were available. Following peer review and publication of short-term effect models,101 these models were replicated and extended using the same syntax once longer-term data were available, and were presented to our Study Steering Committee in November 2020. Short-term models were constructed by a second analyst who covered a period of maternity leave for the study’s lead analyst and, hence, the lead analyst fully reproduced short-term models prior to extending them.

This analytical procedure was replicated for energy drink use in Wales to evaluate whether or not any change in level or trend was specific to e-cigarettes or replicated on an unrelated outcome. Energy drink use was selected as an example of another psychoactive substance that, like e-cigarette use, was relatively new and not following the secular decline observed among other substances, such as tobacco and alcohol use.

Although data sets in Scotland and England provided insufficient data points for more fine-grained segmented regression analyses, we use the data sets to evaluate whether trends observed within our primary statistical analyses are specific to the Welsh data or are mirrored in the data from other nations. Data from England provided one pre-TPD time point (2014) and two post-implementation time points (2016 and 2018), whereas Scottish data provided two pre-implementation time points (2013 and 2015) and one post-implementation time point (2018). Binary logistic regression models were constructed in each nation examining change over time in ever use of e-cigarettes, using the time point closest to TPD implementation as the reference category. These analyses were also conducted for our secondary outcome of regular e-cigarette use (including in Wales, where regular use was not measured until 2015).

Models using only Welsh data were adjusted for school-level clustering, but this was not possible in other data sets, as most data sets did not include school identifiers.

For our primary statistical analyses of ever e-cigarette use in Wales, we observed that in models stratified by ever smoking status, changes in trend were both of greater magnitude than for the whole group models (i.e. models including all eligible participants). We hypothesised that this may have been because ever smoking acted as a time-varying confounder. Hence, as an additional data-led post-hoc analysis, we included a term for ever smoking in whole group models, which increased the estimate of change in trend. As this was an unplanned post-hoc analysis, we refer to models without this term as our primary statistical analyses, but report the model with additional adjustment. Although our analyses were limited to 13- and 15-year-olds for comparability with nations where only these year groups are surveyed, we also ran our analysis of ever e-cigarette use for the whole Welsh sample (i.e. for 11- to 16-year-olds) as a sensitivity analysis.

To estimate post-intervention change in young people’s smoking uptake

Given the longer time series available for the secondary outcomes of ever and regular tobacco smoking, analyses involved repeating and extending models constructed to estimate pre-legislation changes in trend, with additional level and post-slope terms with 2016 as the intervention point (see Study measures). Although we aimed to analyse the same outcomes as included in pre-legislation analysis, changes in data collection schedules for some surveys and changes to question wording for some outcomes meant that fewer data were available for this analysis than had been hoped. Analyses are, therefore, limited to outcomes present in all five post-legislation data sets.

Additional analyses

Whole sample models were visually represented to aid interpretation, using plots of predicted probabilities over time, estimated following model execution in Stata^® (see Report Supplementary Material 1, Figures 1 and 2). For each objective where data were available across data sets, subgroup effects were examined for gender, age/school year, SES, ethnicity and smoking status. The subgroup effects differ between analyses because of data availability. An additional data source held by the research team was the Youth Tobacco Policy Survey (YTPS) (principal investigator: AMM). These data were not integrated into our main analyses because of small numbers in the target age groups and the limited number of data points. However, the YTPS data provide an additional source of external data for triangulation of our findings. Time trends in the prevalence of our main study outcomes in the YTPS from 2002 to 2016 are presented in Appendix 2, Figures 10 and 11.

Procedure for dealing with missing data

The main outcomes for each of our objectives were available for ≥95% of respondents. For our overall primary outcome (i.e. ever e-cigarette use), data were available for 95% of young people, whereas 97–98% of young people were included in the final analytical samples for tobacco use. Hence, no imputation was undertaken, and analyses are based on complete cases.

Weighting

There is inconsistency within surveys over time in whether or not weighting is used to ensure national representativeness. Weights were included in data sets from 2006 in Scotland, 2009 in Wales and 2010 in England. In Wales, following movement from a sample to a whole population survey, weights were no longer used after 2013. Weights are calculated on the basis of the composition of the samples as a whole; however, in England and Wales, our analyses focuses on subsets of available data to harmonise age groups across countries. In plots of smoking prevalence trends over time using weighted (where available) and unweighted data, differences between these were barely discernible. Hence, for tobacco use models, we ran analyses using unweighted data, re-running whole group models with weights applied where available, as a sensitivity analyses. Survey weights were available in Scotland and England (but not Wales) for all surveys since first measurement of e-cigarette use. Hence, whole group and subgroup analyses for e-cigarette use in England and Scotland are presented with and without weights.

Software

All statistical analyses were undertaken in Stata/SE 15.0.

Process evaluation

Drawing on MRC guidance for process evaluation,90 we conducted an in-depth process evaluation focused on key uncertainties in causal logic for the TPD regulations, in relation to implementation, mechanisms and context. The evaluation included (1) qualitative interviews with young people aged 14–15 years, during the transitional phase and after full implementation of TPD regulations; (2) observation of compliance with TPD regulations in e-cigarette retailers, conducted at the same time as pupil interviews; and (3) interviews with policy stakeholders, TSOs and retailers on the context, implementation and perceived unintended consequences of the TPD regulations. In consultation with our patient and public involvement group Advice Leading to Public Health Action (ALPHA), a small number of items on potential mechanisms through which TPD might impact young people’s e-cigarette use were developed for inclusion in 2017 and 2019 SHRN surveys to enable triangulation of qualitative data on post-legislation change in these mechanisms. Process evaluation data collection materials are provided in Appendix 3.

Qualitative interviews with young people

School sampling

Although it was not possible to interview young people prior to legislation, which came into force before the study began, we aimed to interview young people as early as possible before the date of full compliance and then again 1 year later. Collecting data both during the transitional period and after full implementation of the TPD regulations enabled us to understand perceptions in relation to a context where unregulated e-cigarettes were, or were not, legally available for sale on the UK market. Our aim was to recruit 12 schools overall to provide representation of (1) schools in each of the three countries, (2) high, low and medium SES schools (as indicated by free school meal entitlement) and (3) urban and rural locations (which was a larger number than the six we had originally proposed, but followed a request from the PHR Funding Committee to expand this). However, contracting and ethics approval were not in place until March 2017, limiting time for recruitment. Hence, nine schools agreed to participate and data collection was completed within seven schools, including a total of 76 young people. The risk of low recruitment was anticipated in our protocol, given the time frame in which these were undertaken, and we indicated that should our intended level of recruitment not be achieved in the first round of data collection, then further schools would be recruited for post-legislation interviews in 2018 to provide a broader set of perspectives. Hence, in 2018, we returned to schools that participated in the first round and recruited a number of additional schools for post-implementation interviews only to provide a broader range of perspectives. We conducted post-implementation interviews in 11 schools. One original school was unable to take part in phase 2, with 62 of the same young people within six of these schools completing follow-up interviews in 2018. A further 86 young people participated in interviews at phase 2 only. Our intention was to complete follow-up interviews by Easter 2018 to reduce conflict with exams, but our planned data collection period coincided with university strikes and a period of extreme weather, including closures of schools (and in some cases universities) due to snow occurring in all three sites simultaneously, leading to challenges rescheduling school visits. Hence, data were collected between February and May 2018. Schools received a payment of £100 per data collection as compensation for administrative time supporting data collection.

Pupil sampling

We aimed to conduct four group interviews in each participating school, with three to five young people (aged 14–15 years when recruited, corresponding to Year 10 in England and Wales, and S3 in Scotland) in each group. To maximise rapport and interaction between young people within groups, we worked with school staff to identify groups of friends for interviews. Although smoking rates have historically been higher among girls, with convergence in genders in recent years, the opposite was true of e-cigarettes, which were becoming more popular among boys. Hence, we conducted single-gender group interviews, and sampled young people from higher- and lower-ability classes within secondary schools. Given that normalisation processes are driven as much by the reactions and behaviours of the majority who do not engage in a given behaviour as by the minority who do, we did not explicitly attempt to recruit young people who did smoke, or who were at high risk of smoking, or used e-cigarettes. We advised teachers of this in advance so that they did not use pupil smoking and vaping behaviours as criteria for selection.

Interview schedules

Semistructured interview schedules explored young people’s perceptions of e-cigarettes, tobacco and the inter-relationship between the two, with an emphasis on normalisation and elements of TPD theorised as likely to influence young people’s perceptions, including marketing and product-labelling. Interviews focused primarily on perceptions of tobacco and e-cigarettes among their peer group, rather than young people’s own use of e-cigarette and tobacco use. Topics included perceptions of social norms for e-cigarettes and tobacco use, including norms among their peer group and perceived parental reactions to tobacco and e-cigarettes, exposure to and perceived responses to advertising for e-cigarettes, and risk perceptions for tobacco and e-cigarettes, including views on elements of the TPD regulations, such as warnings about nicotine.

Data collection

Interviews were held on school premises during school time. Data collection was undertaken by research staff with prior experience of data collection with young people in education contexts. Researchers without this prior experience were accompanied by experienced staff in initial data collections. Interviews were recorded on a handheld digital recorder and uploaded to a secure university server at the earliest opportunity before being deleted from the recorder.

Retailer audits and interviews with professional groups

Retailer audits

We audited approximately 10 e-cigarette retailers in each country on two occasions to assess the availability of TPD-compliant/non-compliant products during and after the transitional phase. Audits were conducted at times coinciding with qualitative pupil interviews to put in context the extent to which unregulated products remain available during initial interviews conducted during the transitional phase, where non-compliant products could still legally be sold, and the extent to which fuller compliance was achieved by follow-up interviews after full compliance is expected. Our intention was that two observers independently estimate the proportion of e-cigarettes on sale in each location that have compliant labelling at each time point to enable assessment of interobserver agreement; however, because of a miscommunication in one nation, only one observer undertook each observation. Hence, estimates of interobserver agreement are based on the remaining two nations. Locations were sampled purposively to include large and small mixed retailers (e.g. supermarkets and newsagents), specialist e-cigarette shops and street vendors. Retailers sampled included any near the schools in which qualitative interviews were undertaken, and included supermarkets, specialist e-cigarette vendors, convenience store/newsagents and garages. Observations were ‘covert’ in so far as researchers did not voluntarily identify themselves as researchers to the store staff. However, researchers carried an information sheet with contact details for the study team, which was made available to any staff who asked the researchers what they were doing. Researchers were to leave the store if asked to do so.

Interviews with retailers, policy stakeholders and trading standards officers

We conducted interviews with 27 e-cigarette retailers after the final date for full compliance. Although audits included an approximately even split of retailer types, we anticipated that specialist e-cigarette retailers would provide more rich information on the implementation of TPD and regulation of e-cigarettes. Hence, two-thirds of interviewees were specialist retailers and one-third were non-specialist retailers. Interviews were conducted either by telephone or at the interviewee’s place of work. Interviews were conducted between June and November 2018, approximately 2 years after the initial implementation date for the TPD regulations (and 1 year after the date for full compliance).

We conducted interviews with 12 policy representatives and 13 TSOs across the three nations to explore perspectives on e-cigarettes, as well as barriers to and facilitators of implementing the legislation. Policy representatives were recruited through the team’s existing links with UK governments, public health agencies, non-governmental organisations and other organisations involved in the policy-making process. Interviews focused on perceived roles of e-cigarettes in movement into and out of smoking, policy implementation and compliance, local contextual factors and variations in e-cigarette policy, and theorised mechanisms of the TPD legislation in relation to e-cigarettes. Topic guides allowed flexibility in response to the roles of interviewees, for example with less emphasis on policy development and more emphasis on enforcement activity when speaking with TSOs. Interviews were conducted between June and November 2018. Interviews with policy representatives included a focus on perspectives regarding the renormalisation of smoking and were completed before our own findings relating to this were in the public domain.

Quantitative process indicators

For the 2017 SHRN survey, a number of questions were added in consultation with our patient and public involvement group to provide a snapshot of the prevalence of potential mechanisms through which TPD may impact pupil’s use of e-cigarettes, and were linked to research questions on young people’s perceptions of risks and norms for e-cigarette use and tobacco (see Table 2). In our protocol, in response to PHR programme feedback, we indicated that we would request to add questions on where young people obtained e-cigarettes from and whether or not e-cigarettes used by young people contained nicotine. In addition, we included measures of risk perceptions for e-cigarettes compare with tobacco, exposure to marketing and perceived parental attitudes, mapping onto themes arising from our qualitative analysis. Most items were repeated in 2019, allowing examination of post-legislation change over time. In both cases, owing to pressures on the survey, items were asked of a random subsample of participants, rather than all survey participants.

Analysis of process evaluation data

All qualitative interviews were recorded and transcribed verbatim, with identifiable details (e.g. names of individuals and organisations) removed during and following transcription. The transcribed data were subjected to thematic analyses using Braun and Clarke’s102 six-step approach, which included (1) data familiarisation, (2) generating initial codes, (3) searching for themes, (4) reviewing themes, (5) defining themes and (6) writing up the account. Data analysis and data collection were conducted in parallel. Philosophically, the analyses were conducted through an abductive critical realist lens, beginning with exploration of predefined themes derived from the intervention logic model, and pursuing themes that emerged inductively from earlier interviews in later ones. A first researcher developed draft coding frames, with a sample of interviews second coded by a second researcher. Inconsistencies and disagreements were resolved through discussion with other members of the process evaluation subteam where necessary. For pupil interviews, which included a longitudinal cohort of young people and a cross-sectional sample of pupils recruited at follow-up only, the longitudinal and cross-sectional data were analysed as separate data sets prior to synthesis of themes across data sets. For structured observations, percentage interobserver agreement is presented for each item. One observer was a priori assigned as the primary observer, with percentages of retailers selling TPD-compliant products at each wave presented by retailer type, according to the primary coder. For quantitative process indicators, simple descriptive statistics are presented graphically by year the questions were asked (i.e. 2017 and 2019). These statistics are presented alongside the themes from the qualitative analyses of pupil data to examine the extent to which views expressed in qualitative interviews concur with, or contradict, views expressed in the national surveys.

Triangulation and integration

Combining data sources

Our sampling methods for qualitative process evaluation components were directed towards obtaining views from a diverse range of participants, rather than representativeness. By contrast, quantitative surveys aimed for representative of participants’ respective countries. This combination enables us to advance nuanced explanations for quantitative trends, and examine the extent to which views expressed within our qualitative interviews concur with, contradict or build on survey fundings. At each phase, quantitative and qualitative data analyses were conducted in parallel, but converged following analysis. Qualitative and quantitative analyses were undertaken by different subgroups, with integration of quantitative and qualitative data subsequently focused on areas in which data sources aligned, challenged one another’s conclusions or added further nuance and explanation. Study components were synthesised into a whole using an evaluation and integration framework based on MRC guidance for process evaluation90 (see Figure 2), with data sources organised and presented chronologically rather than by method. Although we considered a more traditional presentation by method, with effects data followed by process evaluation data, this would involve a potentially confusing rotation between a range of different time frames. Hence, the most coherent means of narrating the current study as a whole was to position the impact of TPD as the ending of the story. Hence, Chapter 3 presents data on the context into which TPD was introduced (i.e. pre-implementation trends in smoking and young people’s perceptions of vaping), Chapter 4 presents data from policy stakeholders, TSOs and retailers on implementation of legislation, before Chapter 5 presents data on mechanisms of change and post-legislation vaping and smoking outcomes.

Owing to significant policy interest in findings and the fast-moving nature of this evidence base, we chose to conduct analyses in a sequential manner and publish these as they were completed, rather than waiting for the final report. Our analyses of smoking trends during the emergence of e-cigarettes was, for example, one of the first of its type (and formed part of an evidence review on e-cigarettes and smoking uptake within National Institute for Health and Care Excellence guideline development immediately after publication);103 however, a number of similar analyses have since been published internationally, as described in Chapter 1. All findings have been published under Creative Commons Attribution licenses that allow adaptation and re-use of material for any purpose so long as the original work is properly cited. The original contribution of this report relates to the synthesis of previously published components, and additional unpublished data, to tell the story of the study as a whole. We indicate at the beginning of each chapter which elements draw on previously published sources, and which are new.

Cross-country integration

The quantitative elements of the study involve pooling of data sets collected at differing time points, using slightly different questions in each nation. The most robust analysis available to test the impact of TPD on vaping rates is the Welsh data, which (1) is available in a monthly format, therefore, providing multiple time points within the same survey year, and (2) includes data on ever e-cigarette use since 2013, enabling a segmented regression approach. However, this is supplemented by more crude before-and-after analyses of e-cigarette and tobacco use rates across all three countries, using an integrated three-country data set to examine the transferability or context specificity of findings relating to change in trend in our primary statistical analysis in Wales to other nations in Great Britain. Analyses of tobacco use draw on an integrated three-country data set, with adjustment for country. The process evaluation is conducted using harmonised methods across countries and was analysed as a single data set.

Management and governance

Ethics and consent

Ethics approval was provided by the Cardiff University School of Social Sciences Research Ethics Committee (reference SREC/2188). Surveys included in secondary analyses used opt-out parental consent approaches, in which parents are informed about the study and provided the opportunity to opt their child out of the research. Opt-out approaches are preferred in low-risk school-based research with adolescents, as requiring active parental consent tends to lead to exclusion of the voices of higher-risk groups from research.104 The consent process for pupil interviews comprised the following three stages: (1) school-level consent for the young people to be invited, (2) letters for parents describing the study and (3) consent from selected young people. School staff identified groups of young people who were interested in taking part and gave students information and consent materials to discuss with their parents 2 weeks prior to interviews. A summary of the study information was provided to young people on the day of interviews and a consent form completed. At phase 2, schools were asked to remind parents of the study and young people were reminded that they were under no obligation to take part again if they did not wish to, and consent repeated. Our original intention, as reflected within the proposal approved by the PHR Funding Committee, was to use a more typical opt-out consent process for interviews, which has been used in other similar parallel interview research about e-cigarettes with children as young as 7 years.62 However, our ethics committee initially asked us to use an opt-in method for our first round of interviews. Prior to 2018 interviews, this decision was appealed and reversed with the support of our Study Steering Committee and the National Institute for Health and Care Research (NIHR) Research Manager. Opt-out consent methods were, therefore, used for follow-up interviews, with the exception of one school within a local authority in Scotland, which required that parents provide written consent for their child to take part. For adult interviewees, information sheets were provided in advance of an interview taking place, and a consent form completed prior to the interview.

Patient and public involvement

DECIPHer’s (the Centre for Development, Evaluation, Complexity and Implementation in Public Health Improvement’s) young people’s advisory group ALPHA were consulted in the development of the original grant application. Subsequently, ALPHA advised on study materials for qualitative interviews, and the selection of quantitative process indicators included in the 2017 and 2019 SHRN surveys. ALPHA also contributed to the drafting of the lay summary for this report via group discussion of a draft and provision of comments. Via the SHRN, we also discussed the study with school staff and education stakeholders in several webinars. Research user perspectives at the level of policy and third-sector representation were included via our Study Steering Committee.

Study management

The outcomes and process evaluations were managed by separate individuals, with integration of study components led by the principal investigator. Rachel Brown managed the process evaluation and oversaw collection and analysis of primary data throughout the study. Britt Hallingberg managed the quantitative outcomes evaluation, including the development and implementation of the statistical analysis plan, with oversight and guidance from senior statistician Linsay Gray. Nicholas Page covered Britt Hallingberg’s role during a period of maternity leave in late 2018/early 2019, and implemented short-term effect analyses during this time. Britt Hallingberg moved from Cardiff University to Cardiff Metropolitan University in 2019, but was appointed on an Honorary contract with Cardiff University and continued to manage the secondary data analysis. Marcus Munafò led the Bristol subcontract and was responsible for delegation of responsibilities to research staff in the Bristol study arm, with Linda Bauld fulfilling this role in Stirling. Linda Bauld left the University of Stirling for a post in University of Edinburgh on completion of the primary research in late 2018, and responsibility for the Stirling arm was assumed by Jennifer McKell, supported by Anne-Marie Mackintosh. Monthly or bi-monthly Study Management Group meetings, including co-investigators and employed staff, were held throughout the study and were chaired by the principal investigator. Additional small group operational meetings were held during periods of data collection. The study had a planned pause period from April 2019 to October 2020 while longer-term data were awaited for quantitative analyses and during which no process data were collected. The Study Management Team met every 4–6 months during this break period.

The study had an independent Study Steering Committee whose members included Amanda Amos (University of Edinburgh), Matt Egan (London School of Hygiene & Tropical Medicine), Chris Roberts (Welsh Government), David Humphreys (Oxford University), Sheila Duffy (ASH Scotland) and Ashley Gould (Public Health Wales). Amanda Amos chaired the group initially. All members declared no conflicts of interest at the first meeting, and were asked to declare any new conflicts at the start of each meeting. As Linda Bauld moved to University of Edinburgh in 2018 and remained a co-investigator on the study, this meant that a co-investigator was now in the same department as the chairperson. This new conflict of interest was declared to NIHR, which advised that Amanda Amos no longer met the criteria for independence and was, hence, no longer eligible to chair the committee. Matt Egan took over as chairperson. Sharon Cox (University College London) joined the committee at this stage as an additional independent member. The committee met approximately 6-monthly during the live study periods. Analyses reported in each journal article were presented to the group prior to submission. Our primary statistical analyses of long-term change in e-cigarette use were presented to the Study Steering Committee at a final meeting in November 2020.

Deviations from protocol

Our original protocol included an examination of social media mentions of e-cigarettes, including sites such as Facebook (URL: www.facebook.com; Meta Platforms, Inc., Menlo Park, CA, USA) and Twitter (URL: www.twitter.com; Twitter, Inc., San Francisco, CA, USA). However, our qualitative data indicated that the kinds of media covered by our proposed methods were less relevant to young people than other forms of media, particularly video-sharing websites, such as YouTube (URL: www.youtube.com; YouTube, LLC, San Bruno, CA, USA), which would not be captured by our proposed methods. Following discussion with our Study Steering Committee, this small element of the process evaluation was removed from our protocol. Owing to a miscommunication within the process evaluation subteam, in one nation, two observers each observed different premises rather than two observers per premises. Hence, inter-rater reliability assessments are based on two rather than three countries. Remaining variations from protocol largely related to data availability changes. We originally aimed to use SALSUS data from 2017 and 2019, assuming a typical 2-year cycle. However, the 2017 survey was postponed to 2018 and the 2019 survey did not take place. Our updated protocol indicated that we would seek to use data from the Scottish Health and Wellbeing census planned for 2019 if access could be negotiated; however, this also did not take place, with its launch postponed to 2021. In England, at the time of writing the protocol, funding was confirmed for the 2016 and 2018 SDDU surveys; however, with SDDU moving from an annual cycle to a biennial cycle, funding was being sought for 2017 and 2019 surveys. This funding was not obtained and, hence, the additional surveys we said that we would use if they were conducted were not. There were significant delays in the 2018 SDDU data reaching the UK data service. We sought to present all analyses to our Study Steering Committee prior to submitting for publication. At our final meeting in November 2020, we presented all analyses other than those requiring this data set, and were advised to submit the report without these analyses. However, the editorial team at NIHR asked us to wait until these were analyses available. As this final data set became available in only November 2021, the analyses using these data were finalised immediately before report submission in December 2021. Some intended subgroup analyses could not be conducted because of changes in survey methods. In England, for example, SES was measured by asking young people if they were entitled to free school meals until 2014; however, from 2016, SES was measured via the FAS.