Generalisability

Introduction

Although the numbers of adults achieving the recommended PA levels are generally low,9 there are marked differences between groups, with lower PA levels in women, older people and those from socioeconomically deprived areas and of Asian ethnicity.9,143 Walking interventions aiming to increase PA levels ideally need to try to ensure that these groups are well represented. When participation rates are low,144–146 there may be systematic differences between those who participate and those who do not, but to whom the intervention could reasonably be applied. Failure to include certain groups of people for whom PA levels are lower may lead to the implementation of interventions that are likely to increase health inequalities. When there are differences between participants and non-participants, exploring the reasons for non-participation using a qualitative approach can be instructive.

The population-based sampling frame used in this study provided an opportunity to assess whether or not there were differences in terms of age, sex, ethnicity and area-level deprivation between general practice patients who replied to the invitation letter compared with those who did not. We also compared the health, lifestyle, education and social factors of those who agreed to participate in the trial with those who agreed to complete a questionnaire, but who did not wish to participate (see Appendix 4 for the non-participant questionnaire). These findings are now published.147 Qualitative interviews with a sample of this latter group allowed us to investigate the reasons for non-participation in the trial, and these findings are also published.148

Methods

Data collection for quantitative comparisons

The sex, age and IMD score of all those invited were collected from general practice records. The IMD score is an anonymised measure of deprivation based on postcode.71 To avoid the possibility of individuals being identified, aggregated practice-recorded ethnicity was exported from the practice in 10-year age bands for all batches where everyone was mailed, less exclusions (n = 10,155). Those not wanting to participate in the trial were asked in their invitation letter if they were willing to complete a shortened version of the trial baseline questionnaire, including questions on demographics, health, PA levels and a question on reasons for not participating. The following categories were offered, based on previous research, with space to add other reasons for trial non-participation: (1) I do not have time, (2) I cannot increase my PA, (3) I am not interested in increasing my PA, (4) I am already very physically active, (5) I am not interested in research and (6) I do not want to be put in a group by chance.

Comparison groups

Individuals whose invitation letters were ‘returned to sender’ were excluded from the analyses before calculating response rates. ‘Responders’ are defined as those who replied to the invitation letter, regardless of whether or not they wanted to take part. Individuals could respond by post, e-mail or telephone. ‘Participants’ are those who completed the baseline assessment, although not all were randomised as some provided inadequate accelerometry data. ‘Non-participants’ are those who completed a questionnaire but did not wish to participate in the trial (Figure 10).

FIGURE 10 Flow chart to show the recruitment process in the PACE-UP trial. All percentages are out of all of those whose age and sex were matched with GP records (n = 10,927). (Figure not reproduced here; see the original article.)

As the PACE-UP trial targeted inactive adults, participants who attended a baseline appointment were selected on the basis of their low PA levels. Non-participants were not selected in this way. In order to minimise selection bias, the quantitative analysis of participants and non-participants was therefore restricted to those categorised as ‘not active’ according to a self-reported primary care PA questionnaire, the GPPAQ,73 which was the only PA measure available for both groups.

Statistical analysis

Age- and sex-standardised rates were used to compare IMD quintiles for responders. Similarly, sex-standardised rates were used to compare age groups and age-standardised rates were used to compare sexes. The full population of invitees was used as a standard population throughout. No further analysis on non-responders was possible because they did not provide any questionnaire data on ethnicity or other factors.

Practice ethnicity data were available in 10-year age bands for 10,155 invitees, which was effectively a random sample of the 11,015 people invited to participate. The proportion of patients belonging to each ethnicity category within each age band and within each practice was calculated, and the number of invitees in each ethnicity in each practice and age band was estimated. Overall, 1903 invitees had ethnicity recorded as ‘unknown’. These are assumed to be missing at random in the main results, but sensitivity analyses were performed assuming that these were all white people or all non-white people. Age-standardised participation rates for not-active participants and non-participants completing questionnaires were calculated, assuming that invitees gave the same ethnicity on the questionnaire as was recorded in their practice records. Participation rates by age, sex and IMD score were calculated for not-active participants versus not-active non-participants completing questionnaires, as in the analysis of responders.

Not-active participants and non-participants completing the questionnaire were compared for additional demographic and social characteristics, and health and lifestyle factors, using logistic regression. All data came from the questionnaires. Models were adjusted for clustering by practice and household, by including fixed effects for practice and using robust standard errors for household.

Methods for the interview study of non-participants

This is fully described elsewhere.148 Non-participants completing the questionnaire were asked if they could be contacted to discuss their reasons in more detail. A purposive sample of those willing to be contacted was selected to provide men and women of varying ages, ethnicities and employment status from the initial six participating practices. To maximise participation, we used focused telephone interviews; permission was gained for interviews to be audio-recorded. The topic guide was developed from the literature, from the previous PACE-LIFT trial qualitative evaluation21,149 and discussions between the authors, and is published148 and provided in Appendix 4. Approximately 30 interviews were planned, with recruitment continuing until no new themes were identified and a demographically balanced sample had been achieved. Open questions were asked about what influenced their decision not to participate and their opinions on the trial information received. Responses given on the completed questionnaires were used as a starting point to further explore their reasons. They were asked general questions about the perception of the trial design and were invited to make any final comments.

Methods for analysing interviews

Audio-recordings were transcribed verbatim and checked for accuracy. After 10 interviews, researchers (Rebecca Holmes, TH and CV) read the transcripts and discussed the interviews. The interview technique was then modified slightly to ensure that interviewees understood the trial randomisation process, as several participants had appeared not to understand the question about whether or not being put in a group by chance had influenced their decision not to participate. On completion of the interviewing, the transcripts were read and re-read for familiarisation by the researchers, who assigned codes (RN and TH), before a thematic framework was produced.150 Coding discrepancies between researchers were resolved by discussion. The framework produced was informed both by a priori issues, mostly related to trial design, and by emerging themes. Themes were refined further by discussion between authors (Rebecca Holmes, TH, RN and CV), and broader categories, encompassing several subthemes, were generated. Reasons for declining given by all non-participants were also compared with those given at interview, to put our findings in a wider context and assess the generalisability to all of those actively declining.

Results

Results from quantitative comparisons

Of the 12,625 individuals selected for screening (see Figure 10), 1421 (11.3%) were excluded by practice staff and 189 (1.5%) had invitation letters that were returned, as they had moved away; both of these groups were classified as ‘not invited’. In the 44 households where one person refused the invitation and the other did not respond, it was impossible to match the response to individual invitees within the household, so age and sex are unknown. These 88 people have been excluded from further analyses. Of the remaining 10,927, 4572 (42%) responded to the invitation letter, mainly by post, and 1150 (11%) completed the baseline assessments.

Of all invitees, 5229 (48%) were aged 45–54 years. Although all quintiles of deprivation were represented, only 7% were in the most deprived quintile. Response rates were higher in older people, women and those living in less deprived areas (Table 12). As individual ethnicity was available only for the participants and non-participants who completed a questionnaire, it was not possible to estimate response rates by ethnicity for all responders.

Although the GPPAQ was not used to assess PA levels for trial inclusion, it was the only PA measure available for both participants and non-participants. A total of 118 participants and 388 non-participants were classified as active by the GPPAQ, and 134 people did not complete the GPPAQ. These people were excluded from further analysis, leaving 924 participants and 715 non-participants.

Similar to the response rates, participation rates were higher in older people, women and those living in less deprived areas. (Table 13). Ethnicity was extracted from the practice for 10,155 invitees. Of these, 5991 were recorded as white (59%), 893 (9%) were recorded as Asian or British Asian and 915 (9%) were recorded as black Caribbean, black British or black African. A total of 1903 (18.7%) were recorded as ‘unknown’. The percentage of people with ‘unknown’ ethnicity varied by practice from 3% to 48%. Table 13 shows the estimated number in each ethnicity category for all 10,927 invitees, assuming that those for whom ethnicity was recorded as ‘unknown’ and those for whom we were not able to collect ethnicity have the same ethnicity distribution as the group with known ethnicity.

Of the white invitees, 709 (8.7%) agreed to participate in the trial and were not active, and a further 638 (7.9%) completed a non-participant questionnaire and were not active. Both Asian and black invitees had very low non-participant questionnaire completion (2.4% and 1.9%), but black invitees were as willing to participate as white invitees (8.5% vs. 8.7%), whereas only 5.4% of Asian invitees participated. The sensitivity analyses assuming that all ethnicities recorded as ‘unknown’ were white or non-white people showed similar results, and the same patterns were also seen in practices with nearly complete ethnicity coding.

Those providing questionnaire data were more likely to be working part-time, married or living with a partner, and less likely to have finished their education aged ≤ 16 years (Table 14). Participation was associated with recent primary care contact and with some degree of health problems (general health, long-standing illness and comorbidities), although those more severely affected were less likely to participate (see Table 14). This is consistent with the EQ-5D-5L (health-related quality-of-life) domains, whereby participants were more likely to have problems with pain and mobility, but less likely to have problems with self-care, which is likely to indicate greater disability. Forty-five per cent of the sample gave insufficient time (n = 327) or already being physically active (n = 325) as reasons for non-participation, even though those classified on the GPPAQ as active were excluded from this analysis. Less commonly, 152 (21%) people answered that they could not or were not interested in (n = 122, 17%) increasing their PA. Randomisation was cited as a reason for non-participation only by 88 respondents (12%).

Results from qualitative comparisons

Fifty-five trial non-participants were telephoned from March to July 2013: 21 could not be contacted and four declined to be interviewed. Thirty trial non-participants representing the six initial participating practices were interviewed. Data saturation was achieved prior to completing 30 interviews, but we continued to 30 to ensure a more ethnically diverse sample. The demographic details of interviewees and their main reasons for non-participation are published.141 Those interviewed were not selected on the basis of inactivity, and a slightly higher percentage (67%; 20/30) reported being too active as a reason for non-participation compared with those who were included in the main quantitative analysis.

Most interviewees gave one primary reason for declining participation, consistent across sex, ethnicity and age groups. The majority (n = 18) said that they were too active either because they felt that their activity exceeded the trial’s target levels or because others would benefit more. Less frequently cited main reasons included existing medical problems (n = 4), travel from home (n = 3), work/other commitments (n = 3), concerns about potential equipment problems (n = 1) and reluctance to be randomised (n = 1). To further understand the reasons for non-participation, we categorised the emerging themes into three domains: internal, external and trial related. Short quotations illustrating all of these reasons are shown in Table 15 and more detailed quotations are given in our published paper.148

Internal reasons for non-participation included already being active, medical problems (pain, heart conditions, stroke and multimorbidity), no wish to increase activity, no interest in walking, feeling incorrectly ‘targeted’ and altruistic reasons. The dominant reason in this category was a belief in already being sufficiently active. When explored in more depth, it seemed that, on self-report, many were achieving, with some significantly exceeding, the recommendations. This is supported by the finding that non-participants were found to be more active on the GPPAQ. Of those citing medical reasons, it was less clear whether or not these problems constituted a definite contraindication, especially as those with predefined medical conditions contraindicating an increase in walking should have been excluded. A small number of people suggested that they did not enjoy PA, were not interested in walking or suggested a different activity or a team sport.

The external theme related to factors ‘external’ to the potential participant, including work and other commitments, travel problems, being a carer and advice from others. Work and work-related travel were frequently given as reasons for not participating and many feared that they could not commit to the trial. Family and home life commitments were also important barriers to increasing activity, including being a full-time carer for family members. We were interested in finding out whether or not advice from friends or family affected the decision not to participate. Very few interviewees discussed participation; for those who did, it did not influence their decision, except for one interviewee whose daughter strongly agreed that she should decline.

Reasons related to trial design included programme length, trial material, equipment problems, being randomised, the venue, the walking environment, the nurse interaction and the overall trial design. For some interviewees, the trial duration, at 3 months, was too long and it was difficult to commit for this period. One interviewee reported a previous negative experience with pedometers as her primary reason for declining. Several felt that not being able to choose their allocated group was a disadvantage. Two interviewees expressed concern about the trial promoting walking as an exercise, because the local walking environment was ‘boring’, and another stated that it was ‘the wrong season’ for walking outdoors; however, most interviewees thought that walking was an appropriate and inclusive activity. Some expressed interest in a group intervention rather than one to one with a nurse, feeling that this would improve motivation and sociability; however, others felt that interacting directly with a nurse was preferable to being in a group. Most interviewees approved of the choice of their GP surgery as the location for a PA intervention, describing their surgery as ‘lovely’, ‘pleasant’, ‘convenient’ and ‘appropriate’. Many interviewees expressed a positive attitude towards PA and research, and regretted not being able to participate.

Summary of the findings

The PACE-UP trial recruited 11% of patients aged 45–75 years who were invited by post from their registered general practice, although not all were randomised because of failure to provide adequate accelerometry data. Those not replying were younger and more likely to be male and from deprived postcode areas. Asian patients were less likely to participate. Participation was associated with mild or moderate health impairment, although those with more severe problems were less likely to participate. Not having enough time and being already physically active were the most common reasons for non-participation, even among patients who were classified as not active. Interview findings supported the questionnaire findings and gave more detail about the reasons behind the lack of time (work, travel, family, caring commitments, etc.) and the type of health impairments that stopped people from taking part. Despite not wanting to participate, almost all interviewees were positive about the trial, aware of the benefits of PA and the importance of research, and supported primary care as a venue for such programmes. The design of the trial and intervention was not stated as a key reason for declining to participate.

Strengths and limitations

The PACE-UP trial is a large trial recruiting from a clearly defined invited population, based on GP lists, enabling us to assess the potential reach of the intervention in terms of age, sex and deprivation. Our estimate of 11% participation may underestimate the true rate, particularly in areas of high mobility where patients may have moved away and not informed the practice, inflating the number of patients counted as invited. Although based on limited data, the PACE-UP trial offers a rare opportunity to examine demographic differences between participants and non-participants. We were able to estimate participation within different ethnicities using pooled data. However, we were not able to match at an individual level and some participants may have categorised themselves in a different ethnic group to that on the GP register. Ethnicity was also poorly recorded in some practices and we needed to make assumptions about whether or not those with ‘unknown’ ethnicity were similar to those with recorded ethnicity. In a sensitivity analysis, even under extreme assumptions, the difference between Asian and black ethnic groups persisted. The trial excluded individuals who self-reported being active, but the non-participants were not selected in this way. Our quantitative analysis attempted to mitigate this difference by restricting analysis to all those who self-reported as not being active. However, some residual bias may remain.

The interview study represents an innovative attempt to systemically explore the reasons for non-participation with a purposive sample of those who were invited but declined. Our aim was to further understand the perceived barriers to participation to enhance recruitment to future trials and exercise programmes. We were also able to explore non-participants’ perception of the trial design and research in general. This sample spanned six out of seven of the practices involved and included both sexes, a range of ages, ethnicities, employment and educational backgrounds. The telephone interviews allowed in-depth exploration of the barriers to participation not possible from a questionnaire alone and allowed us to compare the interview findings with the non-participant questionnaire responses. The main limitation of the interview study is that this was based on a self-selected group of people who both returned the non-participant questionnaire and agreed to be interviewed. It is possible that some of our sample would have been excluded in any case on the basis of their pre-existing activity levels and, therefore, their decision to decline may have been entirely appropriate. In addition, despite our attempt to sample interviewees from non-white British backgrounds, these groups were under-represented when compared with the ethnic diversity of the population invited.

Comparisons with previous work

A systematic review of 47 studies of walking interventions151 showed that recruitment methods and participation rates were poorly reported. Of 25 RCTs, participation rates could be calculated for only five. We recruited by post to reduce the burden on practice staff and to obtain response rate data. Postal invitations are used in primary care for other preventative activities, making this a pragmatic approach.152 Other walking intervention trials using postal invitations in primary care33,48,144,153 had similar response rates of 10–20%; those with higher rates (37% and 39%)154,155 recruited individuals who were older and more frequent attenders in primary care, and invited individuals in the primary care consultation, as well as by post. Our previous trial21 used similar recruitment strategies to the PACE-UP trial, but, unlike other trials, included active people. This trial had a recruitment rate of 30%, but was conducted among older people in an affluent setting with few non-white residents.

Non-responders were followed up with one reminder letter, but because of data protection constraints we were unable to contact patients by telephone. Although only 1% of invitation letters were returned to sender, this may underestimate those who did not receive the letter as we did not use registered post. A previous London study using registered post found that 26% of letters were not delivered.156 Our findings of greater participation in women,157,158 older people158 and those in affluent areas157 are supported by other studies. Attwood et al.158 found no association with deprivation or ethnicity, but this trial was based in an area of high deprivation with few non-white patients. Among Asian patients, our response rate was similar to postal invitations in the PODOSA (Prevention of Diabetes and Obesity in South Asians) trial (5.2%), in which community-based approaches,159 through partnership with local South Asian groups, were found to be more effective. Wilbur et al.160 found that social networking was the most effective method for recruiting African American women from low-income areas.

The finding that declining participation in this PA trial was attributable to interviewees considering themselves to be already sufficiently active is in line with other literature.157,161–163 Importantly, objective measurement of PA reveals that most people overestimate their activity levels,9 and that their assessment of their personal activity levels is likely to be influenced by a social context.161 However, this interview series allowed activity levels to be explored in more detail and revealed that, at least on self-report, this was a relatively active cohort for some of whom the trial may not have been appropriate.

Our finding that participation was associated with some degree of health problems, but that severe impairment reduced participation, is more nuanced than previous work, which has suggested that declining participation in PA programmes or trials is attributable to medical problems, including pain,157,161,162,164 particularly in studies with older participants.165 Lack of time because of work and other commitments has also been identified as an important barrier,157,161–163,166,167 particularly in younger and middle-aged participants.165 A lack of interest in PA has also been reported in the literature,157,161–163,166 but travel away from home has not been reported prominently. This may reflect the high proportion of our interviewees still in full- or part-time work and the seasonal migration of the diverse south-west London population.

Implications

Guidelines published by NICE36 concluded that more research was needed to determine which interventions are effective and cost-effective in increasing activity levels among lower socioeconomic and high-risk groups, and that there is little evidence on differential effects of interventions. In our trial, those groups for which more evidence is required tended to be those with the lowest recruitment rates, such as Asians and those in more deprived areas.143 It has been suggested168 that specific cultural groups may respond better to interventions directly targeted at their needs, rather than to universal interventions. Reasons for non-participation were often related to individuals not wanting to increase activity or feeling that they were sufficiently active. It is likely that such resistance will similarly apply to any intervention roll-out and may apply more widely to other public health interventions. Low participation rates mean that policy-makers should be cautious about the intervention’s potential reach and the possibility that it could increase activity inequalities, but these are not a reason not to implement an intervention shown to be effective in 11%93 of the population. We were successful in recruiting older people, women and those with comorbidities or some degree of health limitation. These groups have lower PA levels and are likely to benefit more from increased PA. However, those with more severe disability, people who have had falls and those with a fear of falling were not over-represented, indicating a rational choice by individuals.

Only 12% of non-participants cited randomisation as a factor for not participating, whereas 45% cited time constraints. The nurse intervention required three additional visits to the practice on top of the three data collection visits, which may deter working people or those with child-care and other commitments. However, the PACE-UP trial showed that both the nurse-support and postal delivery groups performed similarly at the main 12-month outcome.93 An intervention offering pedometers with brief advice, without the need to provide research data, may be more acceptable.

Both the PACE-LIFT and the PACE-UP trials recruited to target, achieved follow-up rates of over 90% and demonstrated that the interventions were effective in increasing PA levels.21,93 However, considerably more research effort was required (e.g. more contacts from research assistants) per randomised participant in the PACE-UP trial than in the PACE-LIFT trial, resulting from a lower uptake rate. In spite of the effort, we still had limited power to investigate ethnic and socioeconomic subgroups. Trials with greater reach are likely to be more expensive in terms of recruitment, and gains in generalisability need to be balanced with greater costs. Our findings have important implications for those planning PA trials, as well as for those commissioning community PA programmes. As the cohort we interviewed appeared to be relatively physically active, it may be necessary to tailor some interventions to maintaining, rather than increasing, activity; this may also be important to mitigate the decrease in PA levels that often occurs with ageing. Equally, education about the levels of activity that optimise health gain may prevent potential participants from declining because of overestimation of their actual levels of activity. Measurements using pedometers or accelerometers provide an easy approach to validating PA levels.

Lack of time was an important barrier, so it may be helpful to reiterate that activity can be broken up into 10-minute bouts throughout the day (this can also help those individuals limited by pain or disability). Tailoring interventions to an individual’s travel and work commitments and for their specific health problems may also increase uptake. Promotional material should be inclusive and explicitly state that pre-existing medical conditions do not necessarily prevent participation and dispel myths about the risks of moderate-intensity exercise. Information about the value of PA, particularly walking, for many different health conditions1 should be emphasised in the invitation to participate. RCTs inevitably involve randomisation, but emphasising that, in some trials (including the PACE-UP trial), the control group can receive the intervention after the trial may help recruitment. Most interviewees felt that primary care was an appropriate, convenient location for delivering a walking-based PA intervention, indicating that further primary care-based trials and programmes are likely to be well received.

Conclusions

Participation in an effective PA trial among adults and older adults in a socially and ethnically diverse population was only 11%, with lower rates in more deprived and Asian subgroups, limiting the trial’s ability to investigate differential effects in these important subgroups. Trials with greater reach are likely to be more expensive in terms of recruitment, and gains in generalisability need to be balanced with greater costs. Differential uptake of interventions found to be successful in trials may increase inequalities in PA levels and should be monitored.