Parts of this chapter are reproduced with permission from Lawrie et al.67 and Tunji-Ajayi et al.68 Lawrie et al. is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by/4.0/. Permission to reproduce sections from Tunji-Ajayi et al. in a report was sought from the Copyright Clearance Center RightsLink (license number 5244100770770).
Introduction
The C-GALL trial evaluated very different interventions for the treatment of gallstone disease: laparoscopic cholecystectomy or observation/conservative management. Based on evidence from other surgical versus non-surgical intervention trials, it was anticipated that the trial would face a number of challenges, particularly around informed consent and recruitment, from both the perspective of patients and recruiting clinicians.69–71
An embedded process evaluation was incorporated at the design stage of the trial to identify challenges relating to trial design and or conduct that could be addressed and modified. The process evaluation component was designed to be responsive to the ‘needs’ of the trial, allowing flexibility to address core problems in relation to recruitment and/or retention in a timely manner. The overall structure and methods of data collection within the process evaluation were guided by the QuinteT Recruitment Intervention (QRI) and lent heavily on participant flow data, audio recordings of trial discussions and interviews with trial participants.72 However, a significant change to the traditional QRI method was the application of a behavioural science approach to understand problems of trial recruitment and retention. Clinical trials depend on behaviours: they rely on people (patients, clinicians, trial staff) performing actions (such as receiving or delivering a trial intervention, attending a clinic, returning a questionnaire or approaching eligible participants) that they would not do otherwise. Emerging evidence suggests behavioural science has the potential to add value with regard to improving the conduct of trials.73
Within this process evaluation we applied the theoretical domains framework (TDF) as a method to help inform data collection and analysis. The TDF is an established framework that integrates 33 theories of behaviour into 14 domains that inhibit or enable behaviour (knowledge, skills, social/professional role and identity, beliefs about capabilities, beliefs about consequences, optimism, reinforcement, intentions, goals, memory/attention/decision processes, environmental context and resources, social influences, emotion and behavioural regulation).74 A handful of projects interested in trial recruitment and retention have now successfully applied the TDF as a tool to detail the challenges, from the perspectives of both patients and healthcare professionals, through a behavioural lens.75–78 The TDF was chosen as the framework of choice for components of this process evaluation, as it also lends itself to evidence-based methods for intervention development through mapping onto behaviour change techniques (BCTs) through established methods.79 BCTs are defined as the smallest active ingredient of an intervention such as feedback on behaviour or action planning, and they can be used alone or in combination with other BCTs.80 This process evaluation applied these methods from the science of behaviour change to address problems of recruitment and retention in trial. We developed and implemented behaviour change interventions to target the challenges identified during the behavioural investigation.
The process evaluation team worked with the Co-Chief Investigators, PMG and site staff to identify and address challenges to trial recruitment and retention across two sequential phases:
- Phase 1 focused on identifying and understanding the ‘problem’ for trial recruitment and/or retention using multiple methods and data sources.
- Phase 2 sought to develop solutions to the ‘problem’ identified in Phase 1 through the development and delivery of interventions, with the wider trial team, to target the conduct challenges.
Methods
The process evaluation was approved through the main trial application from NHS North of Scotland Research Ethics Committee (16/NS/0053). Informed consent was obtained from all participants.
Phase 1: identifying and understanding challenges for trial recruitment and retention
Phase 1 focused on the collection and analysis of data from three main components: (1) participant flow data; (2) audio recordings of trial discussions; and (3) interviews with trial participants. Each of these components is described below with regard to data collection and analysis.
Component 1 – Participant flow data
Data collection
Data on the number of participants screened, eligible, approached and randomised were taken from screening logs and the trial website for all recruiting centres. Additional data were collected on: eligibility (including whether they met the protocol inclusion/exclusion criteria, or any reason they were deemed ineligible); whether eligible participants were approached about the trial (if not, why); and finally, whether participants were randomised and if not, why not, and which treatment they chose. Reasons for participants’ declining participation, specifically patient or surgeon preference for surgery or observation/conservative management were elicited. Finally, the number of participants who had requested or had received the treatment to which they had not been randomised (i.e. potential and actual crossovers) were also downloaded from the trial website.
Data analysis
Alongside primary qualitative data, an in-depth analysis of participant flow at each recruiting site was conducted. Analysis of patient recruitment pathways using the Screened, Eligible, Approached, Randomised (SEAR) framework was applied to identify and assess areas of complexity and protocol compliance.81 Simple counting of data collected in SEAR logs can provide useful information about the complexity of the recruitment process; differences between centres or over time can give indications of difficulties that can be investigated further. These data were compared across study sites to illustrate any variation between centres and identify areas of good practice that can be shared. Data were used to guide decisions about prioritisation of feedback by comparing activity across sites to identify core problem areas, for example, sites that had much lower (proportionally) approached to randomised rates, or high crossover rates, etc.
Component 2 – Audio recordings of trial discussions
Sampling and recruitment
All staff at the trial sites involved in discussing the C-GALL trial with potential participants were asked to routinely record consultations in which the C-GALL trial was discussed. To facilitate C-GALL-qualitative (C-GALL-QUAL) audio recording of recruitment discussions, a specific PIL was given to participants, at the same time but before any discussion of the trial was initiated, to explain the purpose of, and the request to, audio-record their consultation which involved discussion of the trial (herein referred to as recruitment consultations). Patients were not obliged to participate in audio recording of recruitment consultations and their decision did not affect their invitation to take part in the C-GALL trial. Following receipt of the audio recording PIL, and after having had the opportunity to ask any questions, willing participants were asked to verbally confirm their consent to the recruitment consultations being audio-recorded. Recruitment consultations were recorded after an initial greeting and introduction to the consultation. If a participant consented, the recording would continue and there would be a record of consent. If a participant declined, the audio recording was stopped, and the file was deleted. With regard to staff consent, a nominated individual at each site (usually the PI) distributed staff information sheets about the recording process and obtained a one-off written consent from all staff involved in audio recording that covered all subsequent recordings captured throughout the study period.
For the analysis of discussions relating to trial retention, sites included in the sampling had to have trial participants who had reached the 9-month follow-up time point to be eligible for inclusion. From the seven eligible sites, we selected sites based on a positive (i.e. high postal questionnaire response rates) and negative (i.e. low postal questionnaire response rates) deviant approach to provide a variety of discussions from sites with varying retention patterns. We aimed to analyse the 10 most recent (assuming that analysing most recent practices would be required if intending to implement a change based on findings) consultations where available.
Data collection
Efforts were made to audio-record recruitment consultations across sites throughout the duration of the trial (September 2016–November 2018) for those participants who consented to the audio recording. Sites were provided with devices for recording conversations and were asked to upload audio files to a secure trial webpage following data capture. The audio files were then deleted from the devices. All conversations within the recordings related to C-GALL trial (where recruiters explain the design and details of the C-GALL trial, and patients decide whether or not to take part), as determined by the researcher, were transcribed for the purpose of analysis that is, targeted transcription. At least 10 consultations per site were required before site-specific analysis was conducted.
Data analysis
All recordings (from both recruitment consultations and interviews) were transcribed verbatim by a professional transcription service, anonymised and labelled with a unique identifier, to ensure confidentiality. Data were initially analysed deductively against a coding framework informed by existing research using audio recordings to improve recruitment processes in RCTs.72 Coding categories included how the trial was introduced, whether the trial or treatment was introduced first, balance of options/risks, patients’ preference, discussion of randomisation, discussion of uncertainty, discussion of crossover and discussion relevant for retention such as in relation to questionnaires or withdrawal. Initial coding was conducted by one researcher with 25% checked by another member of the team and any discrepancies discussed with an arbiter. In addition, analysis took the form of constant comparison alongside case study methods, both within and across sites, to determine problem areas or identify aspects of good practice.
Various information sources were drawn upon when deciding which BCTs to incorporate into the e-mail feedback delivered to sites. Evidence from existing studies that had explored healthcare professionals’ actions reporting challenges to recruitment (in surgical trials) was reviewed to consider how challenges should be mapped against the BCT taxonomy.82,83 Feasibility, with regard to what could be delivered in the mode we had available (i.e. e-mail), was a consideration, alongside input from Co-Chief Investigators. Finally, we requested guidance from the first two sites to receive the feedback regarding positive versus negative framing of messages.
In addition to the more general discussions about trial participation, we were also interested in how trial retention was discussed during consultations. Therefore, in addition to the content analysis described above, qualitative data from the audio recordings were also transformed into quantitative data using the quanti-qualitative appointment timing approach (Q-QAT)84 to quantify time spent on discussions of retention during the trial consultation.
Component 3 – Interviews with trial participants during the trial
Sampling and recruitment
A purposive sample of trial participants who had not returned at least one questionnaire at any time point during the study, and therefore had discontinued their follow-up at least once, were invited to interview. An invitation letter and a qualitative interview PIL were distributed to all potential participants with a reply slip and a prepaid envelope. A researcher then contacted participants who responded to confirm they were willing to be contacted to discuss the qualitative interview study further and book a mutually convenient time for a telephone interview. All invitation packs were distributed from the C-GALL trial office to maintain the confidentiality of potential participants. Two attempts were made to engage with potential participants. A total of 279 invitations were issued.
Patients approached for the C-GALL trial (both consenters and non-consenters) were also invited to interviews to explore outcomes of relevance to patients to inform the development of a COS. This work is reported in Chapter 7.
Data collection
The topic guide was informed by the TDF and refined by the research team. The topic guide was iteratively updated to ensure robustness. Broadly, the topic guide explored participants’ reasons for not returning questionnaires and the behavioural barriers and facilitators to returning their questionnaires. Telephone interviews were conducted by two members of the research team between January and November 2019, with interviews lasting between 25 minutes and 1 hour 28 minutes (median = 36 minutes). Verbal informed consent was sought from all participants. All interviews were audio-recorded and transcribed verbatim. Sufficiency of our sample size was judged against five key aspects of information power: whether the study aim was broad or narrow (with a focused aim requiring a smaller sample); dense or sparse sample specificity (where dense specificity requires a smaller sample); application or not of established theory (theoretical perspectives requiring smaller sample); quality of the dialogue (with strong clear communication requiring less); and finally, whether case or cross-case analysis (with cross-case requiring more participants).85
Data analysis
A TDF coding guide specifying each of the domains, relevant constructs and example quotes relevant to said domains, was developed to facilitate coding. Three interview transcripts were coded independently using the draft coding guide prior to comparing the coding results, with discrepancies resolved through discussion. One researcher coded all participants’ interview responses into the relevant theoretical domains. After coding data into theoretical domains, belief statements were generated which represented similar underlying beliefs held by participants within each domain.79 We used the recommended procedure for TDF analysis to identify the relevant domains that were most likely to influence the behaviour:79 (1) the frequency of belief statements across all domains; (2) the presence and prevalence of conflicting beliefs; and (3) evidence of strong beliefs that influence the behaviour. All three criteria were considered concurrently to judge the relevance of each domain. We developed overarching themes that described the content of related belief statements and domains to effectively summarise the findings. Overarching themes were initially generated by one member of the research team. Themes and belief statements were refined by two other members to ensure that they summarised the data accurately.
Following identification of the TDF domains relevant for PQ return, existing patient-facing documents relevant for retention were analysed for inclusion of BCTs. Existing questionnaire cover letters and newsletters were coded against the BCT Taxonomy to identify pre-existing BCTs and compared to templates of previously developed BCT-informed cover letters to promote questionnaire response.86 These BCTs were then compared against the relevant TDF domains identified in the interviews to ensure all relevant domains were being targeted by pre-existing BCTs. If a TDF domain was not covered, relevant BCTs were added into the text of the letter/newsletter, also, if a relevant domain required enhancing the BCTs were dosed up by inclusion of additional relevant techniques. This process was conducted by one researcher and checked by two others with disagreements resolved through discussion.
Phase 2: development and delivery of interventions to target recruitment and retention challenges
The findings from Phase 1 were developed into strategies to target the recruitment and retention challenges identified. As per the QRI method for generating trial-focused solutions, data generated in Phase 1 was discussed with the PMG and Co-Chief Investigators to directly inform plans.72 Strategies developed as a direct result of the analysis of the participant flow and audio-recorded consultations were at a site level, with (depending on the data collected) opportunities for generic and/or site-personalised feedback. Feedback had originally been planned as face-to-face meetings (which did happen for the one site); however, due to logistical challenges this feedback was moved to e-mail. Face-to-face generic feedback did occur at regional meetings for investigators. Other mechanisms for feedback included a ‘top tips’ recruitment sheet, which was also shared via e-mail. No identifiers of individuals or clinical centres were included in feedback. Findings from the interviews informed the development of strategies to target return of trial questionnaires. These strategies included adaptations to the cover letters accompanying the questionnaire and updating newsletters. Data from the audio recording analysis of discussions of retention also directly informed the trial-specific PIL provided to potential participants when making a decision about participation.
Findings
Phase 1: identifying and understanding challenges for trial recruitment and retention
Recruitment
Twenty-one of the twenty-two C-GALL sites were included in the participant flow-data analysis. One site was not included in this analysis due to inactivity. Of these twenty-one, three had very low recruitment numbers due to them becoming active recruiting centres late in the trial, and one site was closed down due to poor recruitment numbers. As such, these four sites were not progressed for analysis. All further data presented are based on 17 sites. Diagnostic data from the participant flow revealed that for most sites (n = 13; 76%) the main challenge for recruitment existed in converting approached patients to randomisations. The biggest barrier to randomisation was patient preference, largely for surgery (Table 25). A total of 16 sites provided 180 audio recordings for analysis (median number of recordings per site = 10). A further five sites did not audio-record consultations, due to sites not agreeing or forgetting to record. Consultations included consultant surgeons, RNs and patients who were sometimes accompanied by family members (partner or child). Analysis of the transcripts identified four core challenge areas for recruiters discussing the trial:
| Site ID | ‘Problem’ identified from participant flow data | Audio | Example quotesa | Targets for feedback | Feedback |
|---|---|---|---|---|---|
| 1 | Crossovers | Y | Patient: ‘It is the surgery I am thinking I’d rather have’Site staff: ‘We can see what randomisation says and then we can take it from there’ | Positive: site is the second top recruiting site for the whole of the C-GALL trialNegative: site is also the site with the highest rate of patients crossing over | Site specific |
| 2 | Approached to randomised patients who are declining trial participation by expressing a preference for surgery | Y | Not available at time of feedback | Positive: are discussing the trial with 97% of eligible patientsNegative: has the second lowest proportion of approached patients being randomised | Generic |
| 3 | Approached to randomised patients who are declining trial participation by expressing a preference for surgery | Y | Site staff: ‘The trial’s only there if you’re interested. If you feel that you have a preference, then that’s absolutely fine’Patient: ‘I think for my circumstances I think I would opt to just have it taken out and hopefully it would deal with everything’.Site staff: ‘Okay’ | Positive: approaching 80% of eligible patients to talk to them about C-GALLNegative: only 20% of trial discussions go on to lead to randomised participantsWhy: majority have a preference for surgery | Site specific |
| 4 | Crossovers | Y | ‘Let’s do the randomisation right? And see what you are allocated. You might be allocated surgery and that’s happy days. If you are not allocated surgery, you can tell us immediately to swap you over’ ‘So obviously let’s see … what you are randomised into. If you are randomised to surgery, that’s the answer. If you are not randomised to surgery, and then we can have a discussion, and then you are allowed to cross over at any time’ | Self-identified the problem of crossovers | Site specific |
| ‘You are not stuck. You can immediately within the same minute say, “no I want surgery” and we’ll swap you over’ ‘You can cross over to the surgical arm without any problems, or even giving an explanation. This is a completely flexible trial’ ‘If you are randomised to no surgery and you want surgery, you can cross over. You are in the driving seat; we are just trying to observe and learn from your disease process’ | |||||
| 5 | Approached to randomised patients who are declining trial participation by expressing a preference for surgery | Y | Site staff: ‘What’s your initial thoughts on that?’Patient: ‘Yeah, that’s okay’ | Positive: approaching 97% of eligible patients to talk to them about C-GALL | Generic(No audio uploaded at time of feedback. Quote provided as example content of site discussions retrieved post feedback) |
| Y | Site staff: ‘Do you want to go for surgery? Are you wanting to see how things go?’Patient: ‘Just happy’Site staff: ‘Are you interested in the study or what do you think?’Patient: ‘Just happy to get it gone’ | Negative: site had only 22% of participants go on to be randomisedWhy: 63% of those who decline to take part have a preference for surgery | |||
| Site staff: ‘You just want an operation?’Patient: ‘Yeah, just get it gone’ | |||||
| 6 | Approached to randomised patients who are declining trial participation by expressing a preference for surgery | N | Positive: upwards of 85% of eligible patients talked about C-GALLNegative: only 19% of participants go on to be randomisedWhy: 61% of those who decline to take part had a preference for surgery | Generic | |
| 7 | Eligible to approached and crossovers | Y | ‘And if you’re in the study what we do is we allocate you to one of those two arms in the study, and one is that we do the gallbladder operation, and the other arm is we do what’s called watchful waiting, and either way we keep a very close eye on you and see what happens. And if we need to take your gallbladder out during the study, if you’re allocated to watchful waiting for instance and then you suddenly came in with a dreadful attack of gallbladder problems then obviously, we do what’s needed clinically to take your gallbladder out. But maybe from the circumstances you don’t get any further attacks, so that’s the reason it’s difficult to advise you on what we do’ | Positive: randomising 40% of patients approached about the C-GALL trial (average)Negative: only 71% of participants being told about the C-GALL trialNegative: site has the third largest number of patients intending to crossover | Generic(No audio uploaded at time of feedback. Quote provided as example content of site discussions retrieved post feedback) |
| 8 | Approached to randomised patients who are declining trial participation by expressing a preference for surgery | N | Positive: upwards of 80% of eligible patients talked about C-GALLNegative: only 25% of participants go on to be randomisedWhy: 42% of those who decline to take part had a preference for surgery | Generic | |
| 9 | Approached to randomised patients who are declining trial participation by expressing a preference for surgery | Y | ‘I think that having the operation is the best thing to do’ ‘if you don’t ever want the pain again you could have the operation’ | Positive: discussing the trial with 100% of eligible patientsNegative: only 29% of trial discussions lead to randomised participants | Site specific |
| ‘you may go for surgery and be no better off, I suspect you’ll be better off’ | Why: 73% of those who decline to take part had a preference for surgery. Surgery is consistently mentioned to be the best option | ||||
| 10 | Approached to randomised patients who are declining trial participation by expressing a preference for surgery | Y | ‘We know that obviously if we do the operation, it would be done by keyhole, 95% of the time. Good chance of helping your symptoms, but we do know that after having had your gallbladder out, up to a third of patients still have ongoing pain… Obviously if you go down the line of being observed, and things become worse, then we would see you again to look at taking your gallbladder out, should that be required. Obviously if you got put down the line of having your gallbladder out, if you changed your mind, then that’s absolutely fine. You are under no obligation. Would you be happy to be part of the study?’ | Positive: upwards of 87% of eligible patientsNegative: 43% of trial discussions lead to randomised participantsWhy: 49% of those who decline to take part had a preference for surgery | Generic(No audio uploaded at time of feedback. Quote provided as example content of site discussions retrieved post feedback) |
| 11 | Approached to randomised patients who are declining trial participation by expressing a preference for surgery | Y | ‘We find the gallbladder – this is all under general anaesthetic, so you are asleep – find the gallbladder, free it up, and we take the gallbladder with the gallstone out, because you can’t leave any of that behind because you are at the risk of forming more. That’s the best way to stop any further problems (…) There’s a risk of damage to upper abdominal structures – bowel, blood vessels, liver, in that area (…) if you go onto the non-operative arm, you’ll take painkillers if you need it, or antibiotics, or whatever, and then that will be part of your questionnaire when it gets sent out’ | Positive: approaching 93%Negative: 47% of trial discussions lead to randomised participantsWhy: 87% patient preference for surgery | Site specific |
| 12 | Approached to randomised patients who are declining trial participation by expressing a preference for surgery | Y | Site staff: ‘So, you’d rather have it removed?’Patient: ‘I’d rather have it out, yeah’Site staff: ‘Yeah, okay. Well, fair enough’ | Positive: 90% approachedNegative: 43% led to randomisations | Site specific |
| Patient: ‘Is there no other way for the gallbladder if you have stones? Is it just surgery or that’s it?’Site staff: ‘Yes’ | Why: 42% patient preference for surgery | ||||
| 13 | Approached to randomised patients who are declining trial participation by expressing a preference for surgery | Y | Not available at time of feedback | Positive: 97% of eligible approachedNegative: only 40% of participants approached were then randomisedWhy: 90% of patient preference for surgery | Generic |
| 14 | Eligible to approached | Y | Not available at time of feedback | Positive: has highest percentage of participants approached randomisedNegative: lowest amount initially approached, high percentage of patient preference for surgery | Generic |
| 15 | Approached to randomised patients who are declining trial participation by expressing a preference for surgery | Y | ‘The attitude has always been, we do an operation we remove your gallbladder, and you get better when we look back on people, that 20 years ago that’s what we did, we can help 80% of them did, but up to 20% of people did not feel better (…) We can consider an operation … that is option 1, option 2, we do know that increasingly that some people get better by losing weight, having healthier diet (…) we are also looking at doing research’ | Positive: 53% of eligible patients approachedNegative: 12% of those trial discussions lead to randomised participantsWhy: 52% of those who decline to take part had a preference for surgery | Site specific |
| 16 | Approached to randomised | Y | ‘So, if you go down the surgical route, then obviously the advantages – you get rid of the stones. Not only are you taking the gallstones away, we take the gallbladder as well, so they don’t come back, and it gets rid of the source of the pain for good’ | Negative: only 39% of participants approached were randomised | Generic(No audio uploaded at time of feedback. Quote provided as example) |
| 17 | No immediate issues | Y | Not available at time of feedback | One of the highest percentage of participants approached to randomised | Generic |
| 18 | Small overall numbers and no immediate issues | N | Generic | ||
| 19 | Small overall numbers and no immediate issues | Y | Not available at time of feedback | Generic | |
| 20 | Small overall numbers and no Rimmediate issues | N | Generic | ||
| 21 | Site was in process of closing | N | Generic |
- providing a balanced presentation about both treatments
- discussing and exploring preferences
- discussing uncertainty
- discussing crossovers.
Examples of excerpts from the audio consultations are provided in Table 25.
Retention
Audio recordings of trial discussions
A subset of 38 audio recordings from 4 sites were included in the analysis of discussion of retention with 3 sites providing 10 recordings each and the fourth site providing 8. The findings from this aspect of the process evaluation have been published in full elsewhere.68 In brief, most of these consultations (n = 30/38; 79%) did not include any discussion of trial retention. Where retention was discussed, the median proportion of consultation time in which retention was discussed was 3.8%, with the longest discussion of retention lasting 89 seconds and shortest lasting 20 seconds. Presence of discussion of retention also varied by site, with one of the four sites never discussing retention in any of the consultations included in the analysis. Where consultations did discuss retention, some contained inaccuracies (n = 1, 9%), some failed to detail the frequency of follow-up questionnaires (n = 1, 9%) and the majority focused on the participant’s right to withdraw (n = 6, 54%). In some instances, the patient initiated the conversation around retention, with respect to frequency of follow-up. Examples of excerpts from the consultations relevant to trial retention are presented in Table 26.
| Theme | Quote |
|---|---|
| Timing and purpose of questionnaires | ‘So, with the quality of life, I mentioned there was a baseline – and with those forms you’ll get sent one 3 months, 9, 12, and 18 …’. ‘They will be very similar to the baseline ones if you want to go into the study. And the study centre will send those out’ RN Site B |
| Surgeon: ‘Umm I mean you can always say uhh uhh that ah regardless of which treatment group you do – the study allocates to you, for example if it says observation then uh you will uh will require observation in about 3 months’ time. Is that right?’ (Doctor asking RN) | |
| RN: ‘Umm yeah, we just send you the questionnaire by email or by phone – so you will just answer them every couple of months. That’s all we will do. There is nothing where you physically have to come or need to be examined or anything like that’ Site C | |
| ‘Emm what we do is that we send you a questionnaire in the post. 3, 6, 9, 12, and 18 months. So, it’s every 3 months you get questions in the post. Kind of day-to-day thing about how your pain is and things and such’. | |
| ‘It does not involve another visit back for us. Emm, If you do get randomised to surgery – emm, it’s a kind of a standard way to check for that. And then we also send you the questionnaires in the post as well’ Surgeon Site D | |
| Participant’s right to withdraw | ‘Like Mr X said, it is a randomised controlled study – so if somebody agrees to go into the study there is some baseline information that we do when they consent. So there is a consent process that I’d go through. Umm a consent form that you fill in. And you can withdraw your consent at any time. So if you fill in your consent from today, decided you wanted to do it – and then went home, thought about it – and thought, actually no this isn’t right for me; you can withdraw at any time. Umm and again with the follow ups – so the follow up questionnaires you can always decide that when life gets busy and get in the way – if you decide actually, oh I’ve not got time to do these questionnaires, or too much other things going on – again you can choose to withdraw from the follow up as well. So if you decide to go into the study it is not set in stone. You can withdraw at any time alright?’ RN Site B |
| ‘With the follow-up, it’s the same for both arms for the questionnaires. I will show you a copy so you know what is expected of you. This is the baseline. But it is the same for all of them. There is no massive essays. It’s a tick box – and I don’t know if you want to have a look through that. Umm it’s all about the general health, following activities of daily living, discussing pain …’ RN Site B | |
| ‘That’s pretty good, perfect! And the questionnaire don’t forget the questionnaires. In a way that is kind of confirming your ongoing consent’ Surgeon Site B |
Interviews with trial participants during the trial
Nine interviews were conducted with trial participants who had not returned at least one questionnaire during their participation in the C-GALL trial. The median age of these participants was 53 years old (range: 35–75 years) and the majority (n = 7/9; 78%) identified as female. This was comparable to the entire population of C-GALL trial non-responders at the point of data analysis (February 2020; 79% female with a median age of 51 years). The length of interviews ranged from 25 minutes to 1 hour 28 minutes (median = 36 minutes).
Domains and associated belief statements were categorised into overarching themes to summarise the main messages arising from the data (Figure 14). The specific beliefs, the TDF domains they are associated with, and the sample quotes are presented in Table 27. The relationship between TDF domains and the overarching themes is presented in Figure 14. A total of seven themes were identified. These were:
FIGURE 14 Overarching themes and associated domains. ECR, environmental context and resources; SPRI, social professional role and identity; TDF, theoretical domains framework. (Figure not reproduced here; see the original article.)
| Domain (frequency = participants who mention domain/number of total participants) | Specific belief | Sample quote |
|---|---|---|
| Environmental context and resources (9/9) | The format, layout and postal administration of the questionnaire was (not) convenient for me | ‘That’s it, yeah. It comes with an envelope to just pop it straight back in the post. You don’t have to put a stamp on or write the envelope, so I don’t see any issues with it’ Participant 1‘I much preferred it when I could give oral answers to a simplified version in which she said, “These are just the main questions that we need to know”. That didn’t take nearly as long, and it was quite focused and straightforward’ Participant 2 |
| It was (not) easy to schedule completion and return of the questionnaires with other priorities in my life | ‘Well, as I say, I put it aside and then never got to it. So, clearly life got in the way for me. Yes, yes’ Participant 2‘No, not at all. It would never be a problem to sit down for ten minutes and answer the questions’ Participant 1 | |
| Completing the questionnaire was (not) enjoyable and/or straightforward | R: ‘And any struggles to do with any parts of the study at all so far?’‘No. Actually, in the questionnaire I asked about my health and this, that and the other, it was quite enlightening to thought about things’ Participant 7 | |
| ‘Tick box things. I think, probably … Sometimes, I ticked a box and I’m thinking, “Well, I’m ticking it, but maybe I need to explain a bit more”. Probably I’m ticking it because I was not on one side …. You know what I mean?’ Participant 5‘There was a lot of, “Hmm, how I do I answer this one? I’m not quite sure”’ Participant 9 | ||
| Social influences (9/9) | I did not feel pressurised to complete and return the questionnaires | R: ‘And did anyone support you to return the questionnaire?’P: ‘No, I mean, my partner’s here but he doesn’t really take any notice of my post anyway. No, I’ve got no pressures from anyone. Nobody encourages me either. At first, my sister half encouraged me to take part because I was about 90% sure I was going to do it, and then after talking to her and a couple of others, just things that were said I just said, Yeah, I’ll do it’ (also coded under belief statement, support from family members) |
| I (did not) receive support from family members to complete and return the questionnaires | ‘Once I’ve filled it in, my husband supports me in terms of he gets it to you, as in he posts it for me because I don’t go out to post it when I’ve got work or whatever. But that’s the only support, (inaudible) support, my husband is understanding. He understands the pain and he helps me when I’m suffering and stuff, but no, I don’t have any other support, not really’ Participant 6 | |
| R: ‘And does anyone support you to return the questionnaire?’P: ‘No, like I say, I just do it straightaway’R: ‘And it’s all off your own back, you’re able to post it yourself and such’P: ‘Yeah’ Participant 8 | ||
| Beliefs about consequences (9/9) | Completing and returning the questionnaires will help people with gallstones in the future | P: ‘Just to help other people further down the line, really. If the questionnaire helps the study, it can obviously benefit other people further down the line’ Participant 8 |
| The questions were not relevant to my circumstances | ‘I think filling it in would have been medical management, and I’ve not had any medical management. It was a bit like, yeah, I’ve not had anything to help with it so I was just filling it in in terms of the first time’ Participant 8.‘Well, there were a few. I mean, I started off well and then because nothing had happened … I hadn’t had any pain or anything and then I’d put them aside and then somebody would send another one and say, “You haven’t completed it”’ Participant 2 | |
| It’s important to fill in the questionnaires for the research team | ‘No, like I said, just the information for yourselves to go towards the study and get the information that you need, really. That’s the only thing that pushed me to do it if that makes sense’ Participant 8 | |
| Reinforcement (7/9) | (No) incentives or rewards would have encouraged me to complete and return the questionnaires | ‘I suppose so. Obviously if it’s going to benefit and you’re going to get something back, it would be nice …’ Participant 8‘Yeah, I just want to do it because I want to do it. I don’t really want anybody to try and bribe me to do it or encourage me to do it due to any kind of incentive, no. In some ways, actually that would put me off’ Participant 1 |
| Behavioural regulation (8/9) | It would (not) have been better to receive the questionnaire in a different format | ‘Yeah, I suppose that’s possible. Over the phone would be fine; face to face wouldn’t be ideal because I don’t live that close to [study centre]. Over the phone would be all right, I wouldn’t mind that, but in some ways that would be more difficult to fit into your day as well. Filling it out on a piece of paper, you can do that when you’ve got a few minutes to get it done and from my point of view, I’d probably find that easier to fit in’ Participant 1‘Initially. I mean, even if they were sort of three, six months apart, it would have been something. I find it easier to talk to a person probably than have to fill in these forms’ Participant 5 |
| I (do not) make a plan to help me complete and return the questionnaires | ‘I would actually think, “Well, I’ve got to get this done” and do it and then send it off. Saying that, yes, probably … You’re right, I would probably say, “Oh, I’m going to sit down this evening, fill this questionnaire in and then get it sent off.” So, yes, in that way, there was a plan’ Participant 5‘The questionnaire thing for me is a barrier because I’ll think, “Oh, I’ll put it to the side, I’ll do it when I’ve got time,” and I never get the time. I’ve got my second one to fill and it’s been there for months. Really bad!’ Participant 6 | |
| The time point of receiving the questionnaires influences questionnaire completion | ‘I suppose if the questions had been really complicated or difficult to answer, maybe. Or if you just can’t remember what exactly has happened over the time. I suppose the time between questionnaires, if it’s every three months then that’s not too bad, but if it was longer than that then I’d definitely have trouble remembering exactly what symptoms I’d had and how many times and all the fine details’ Participant 1‘We’re all busy and I think I’d have been more engaged with it, had I had a lot of symptoms and I was really desperately wanting to get sorted out, which I actually feel a little bit more now’ Participant 2 | |
| Beliefs about capabilities (9/9) | I was (not) confident that I could return the questionnaire(s) | R: ‘How confident are you that you could return a questionnaire when requested?’P: ‘100%, I know I can’ Participant 1‘As I say, it was the paperwork just kind of threw me and … I should have been more on top and organised with that, but’ Participant 2 |
| Intentions (9/9) | I am (not) committed to completing and returning the questionnaires | P: ‘Well, there’s not a lot … It’s commitment to answering the actual survey questions that took … not the survey, the questionnaires that come in. You’ve got to commit to that. I think you have to, because otherwise there’s no point you doing it’ Participant 5‘There’s something about these forms that you just look at them … like a tax form … You just look at it and think, “Oh, I’ll do it later.” You know’ Participant 2 |
| Knowledge (9/9) | The activities and tasks involved in the trial did (not) meet my expectations | ‘It’s the same as most questionnaires like that where you’ve got, I think, to give a level of how much something’s inconvenienced you. Yeah, just what you’d expect from a questionnaire. Sorry, I’m not very helpful on this one!’ Participant 1‘Well, actually the paperwork that comes through, some of the questions that we’re asked were sort of non-sp-…’ (inaudible) non-specific Participant 5 |
| I was (un)aware of when the trial/my participation ended | ‘I think at the appointment initially, they said that the trial had been extended. They didn’t give me a timeframe for how long it would go on for’ Participant 1‘Yeah, I know it’s every six months that they wanted me to do a questionnaire … two is coming to mind, two plus’ Participant 6 | |
| Skills (9/9) | Literacy and communication are key skills required to complete the questionnaires | ‘But actually, the actual skill involved is trying to be … I mean, hopefully, you’re literate enough to be able to do it …’ Participant 5 |
| You need a good memory to complete the questionnaire accurately | ‘It’s hard, it’s very hard. “During the past four weeks, to what extent has your physical health and emotional … how have they interfered with your personal life or your home life?” Things like that. You forget, do you know what I mean? You kind of forget in that four weeks what’s happened. They should ask me about it there and then, so this questionnaire, “How are you feeling today?” or whatever, or last week, or “When was your pain bad?” But the last four weeks, I can’t remember how it interfered with my neighbours, with my family. It is a valid question, it’s just me, I’m a busy person, I don’t have …’ Participant 6 | |
| Emotion (9/9) | Completing and returning the questionnaire(s) (does not) trigger unpleasant emotions such as boredom | R: ‘Yes, I understand. This may be a bit of an odd question but bear with me. What sorts of feelings come to mind when you think of returning a questionnaire?’P: ‘Just slightly bored’ Participant 2R: ‘What sort of feelings come to your mind when you think of returning a questionnaire? Any sort of feelings you experience?’P: ‘Not really, I think I’m quite matter-of-fact about it all’ Participant 7 |
| Completing the questionnaire brings me satisfaction and a sense of responsibility | ‘Well, you’ve completed a job, you know that it’s important, it helps in clinical trials, it’s helping finding out stuff. That’s the upside of doing it, and you feel satisfaction, that you’re making a difference’ Participant 4 | |
| Social professional role and identity (9/9) | My personality and social role influence my impression of the questionnaire in terms of content and format | ‘My (inaudible) thing is, honestly, I’m a social person. I like to speak to somebody, have a consultation and you will get your answers’ Participant 6‘I’m not one for paperwork, so … it’s not something I look forward to’ Participant 3 |
- Unclear expectations of trial participation: included beliefs about activities and tasks in the trial not meeting expectations, highlighting participants were (un)aware of when their participation in the trial ended, the relevance of the questionnaire to their circumstances (i.e. when not experiencing pain), and confusion over receiving the same questionnaire on multiple occasions.
- Personal attributes for questionnaire completion: the need to remember levels of pain experienced was mentioned as a skill required to complete the questionnaire, as was the need to concentrate which was sometimes reported as a barrier to completion and return.
- Significance of questionnaire non-return: participants reported feeling guilty for not returning questionnaires largely linked to them feeling they were letting the trial team down. Others felt satisfaction by completing and returning the questionnaires and saw the activity as their duty as a trial participant and that their contribution would help research.
- Commitment to returning questionnaires given other priorities: some participants reported they did not feel committed to completing and returning the questionnaire as it was not a priority for them, reporting they were too busy. In contrast, some participants recognised the negative consequences of not returning the questionnaire (such as wasting the researchers’ time) and so felt committed to completing.
- Individual preferences for presentation mode and timing of the questionnaires: there was variability among participants regarding preferences for questionnaire format with some preferring postal and others stating they would have preferred to complete it electronically or to speak with someone.
- Internal and external strategies to encourage questionnaire return: a range of strategies were reported that participants had used to help them complete questionnaire such as making a plan to complete and return, keeping a note of symptoms and completing the questionnaire as soon as they received it. There were mixed views about whether the use of incentives to encourage response would have been appropriate, but most participants suggested that receiving a prompt or reminder would have been/were helpful.
Sources of support in returning the questionnaires: participants reported that in some cases they received support from family members to return the questionnaires, but for the majority this was done independently. The trial office was noted as providing some level of support for all, which included reminder or completion of the questionnaire over the phone. Participants did not feel pressurised to complete and return the questionnaire, with some stating this was a barrier to completion.
Theoretical domains framework domains, associated belief statement and associated example quotes are presented in Table 27.
Phase 2: development and delivery of interventions to target recruitment and retention challenges
Strategies to improve trial recruitment and retention were developed, through discussion with Co-Chief Investigators and the PMG, in response to the data collected during the Phase 1 investigation. Feedback to trial sites was provided on a rolling basis, targeting sites in response to problems identified through the participants’ flow diagnostic data.
Site investigator meeting
A site investigators meeting was held in Birmingham in November 2018. This meeting was attended by 17 site representatives (which included consultant surgeons and RNs) from across 10 sites. Findings from the analysis of retention discussion in the audio consultations and participant interviews were shared and discussed. Meeting attendees were encouraged to reflect on the findings and asked to consider what they could do to help improve retention, and specifically, what changes they could make during trial recruitment to raise the profile of what was expected in terms of trial retention.
E-mail feedback on recruitment activity: site specific and generic
Guided by the diagnostic participant flow data, sites were provided with e-mail feedback on their current recruitment activity in relation to other centres (SEAR). The e-mail started with a thank you and general reminder of the aim of the trial and then a positive message about an aspect of their recruitment activity, for example, approaching all eligible patients. The content of the e-mail then moved on to highlight the target behaviour that we wanted them to change (e.g. randomising more of those approached by exploring preference), and data were shared on their performance in relation to other sites (see Figure 15, for example text). In addition to the broad areas of content, we also included BCTs within the e-mail text. A list of potential BCTs was proposed and discussed among the immediate process evaluation team regarding relevance and feasibility of inclusion in e-mail feedback to sites to target trial recruitment. Twelve BCTs were deemed relevant and feasible for delivery and incorporated into the site specific and generic feedback (Table 28).
FIGURE 15 Example e-mail content for site feedback (e-mail sent on behalf of the Clinical Co-Chief Investigator). (Figure not reproduced here; see the original article.)
| BCT | Example text |
|---|---|
| 1.2 Problem-solving | There are a number of generic tips on ways to discuss recruitment with potential participants in the attached materials. You could set aside some time to look at these and reflect back on the previous few recruitment conversations you’ve had – do you think there is anything you could improve upon? |
| 1.4 Action planning | During the consultation, offering a balanced explanation of both treatments and exploring patient preferences may ensure both treatments are given equal consideration. Specific suggestions on how to discuss these elements can be found in the attached materials. |
| 2.2 Feedback on behaviour | Site X is discussing the trial with upwards of XX% of eligible patients. |
| 2.7 Feedback on outcomes of behaviour | Although this is extremely positive, I also need to share with you that Site X is also the site that has one of the lowest percentages of participants that go on to be randomised. As you can see below, only XX% of trial discussions lead to randomised participants. |
| 3.2 Social support (practical) | Additionally, the trials team would like to offer your site the opportunity to receive further individualised feedback and training on recruitment. Please contact [research assistant] if you would like to arrange a date. |
| 5.1 Information about health consequences | As you will know, achieving our recruitment targets is crucial to being able to answer the research question posed by the C-GALL trial – what is the best treatment approach for patients with uncomplicated symptomatic gallstones? |
| 5.3 Info about social and environmental consequences | Crossovers, especially those in one direction, are problematic for a few reasons. Firstly, they suggest that participants have a strong preference for surgery and are unlikely to have been in equipoise when they agreed to participate. Secondly, a high number of crossovers will make the trial results unreliable and make it impossible for C-GALL to answer the research questions. |
| 5.5 Anticipated regret | I need to minimise the numbers of crossovers so that all the hard work we are doing to recruit participants is not wasted. |
| 6.2 Social comparison | The average across sites is XX% with some sites achieving upwards of XX%. |
| 9.1 Credible source | Letter sent on behalf of Co-Chief Investigators. |
| 10.4 Social reward | Thank you very much for your efforts on the C-GALL trial. I am really grateful for your ongoing support to achieve our recruitment targets. |
| 12.2 Restructuring the social environment | Why not get together with the rest of the research team in Site X and agree a plan about how to discuss C-GALL in the future. |
Thank you very much for your efforts on the C-GALL trial. I am really grateful for your ongoing support to achieve our recruitment targets. As you will know, achieving our recruitment targets is crucial to being able to answer the research question posed by the C-GALL trial – what is the best treatment approach for patients with uncomplicated symptomatic gallstones?
I would like to acknowledge the fantastic recruitment figures at Site X which reflects all your hard work. Site X is discussing the trial with upwards of XX% of eligible patients.
Although this is extremely positive, I also need to share with you that Site X is also the site that has one of the lowest percentages of participants that go on to be randomised. As you can see below, only XX% of trial discussions lead to randomised participants, the average across sites is XX% with some sites achieving upwards of XX%. So, although you are doing really well speaking to so many patients about the trial, this, unfortunately is not translating into a high proportion being randomised.
I see that around XX% of those who decline to take part had a preference for surgery. This is particularly high compared to the average of XX% across sites. During the consultation, offering a balanced explanation of both treatments and exploring patient preferences may ensure both treatments are given equal consideration. Specific suggestions of how to discuss these elements can be found in the attached materials.
The BCT-enhanced content information was included in the body of the e-mail. In addition to this main text, two attachments were provided. The first contained feedback on the content of the audio recordings with site-specific examples (where available) alongside examples of what had worked well at other sites (Table 29). The second attachment included a one-page ‘top tips’ sheet which aimed to address the core challenges identified in the consultations (Box 1). The sheet focused on tips regarding how to structure the conversation and explain trial-specific processes (e.g. randomisation), building on the examples of good practice provided in the first document they received. It also provided questions for them to ask participants to help explore treatment preferences. All sites were provided with this ‘top tips’ sheet along with either site specific or generic feedback on consultations.
| Examples from your site | Examples of what works well at other sites | |
|---|---|---|
| Balanced presentation about both treatments | ‘So, some people get infection of the gallbladder, which you may have already had, which can become more problematic. The risk of that is around about 10%/15% per year, okay. Other risks is something called pancreatitis, which the risk of that is about 1% to 2% a year, and that can be quite, potentially quite serious, and that is why historically if you like we’ve taken out a lot of gallbladders because people are worried that they don’t want that to happen to individuals. But that said, the surgery itself poses risks, and some people still find they get pain and symptoms and things after the operation as well … If you don’t have an operation, you are at risk of having an infection related to your gallbladder, which is probably what you had before’ | ‘There is one in three chance you may not get better after surgery, and there is a 10% risk of complication, so these are the risks of surgery. There are risks of medical management that there is a 0.7% chance per year for you to pick up a complication which might end up you having surgery at some point, so the arguments are quite balanced’ |
| P: ‘Yeah, okay’ R: ‘You can get another … there is other risks in terms of what we call pancreatitis. The risk of that is lower than the risk of getting an infection, but it is potentially quite a serious problem … it can need quite long hospital admission, and it’s even potentially life threatening, all right … The risk of that is relatively low, about 1% to 2% of people with gallstones will have that problem at some point in a year you know, but whether that will happen to you or not is almost impossible to predict, all right’. | ||
| Discussing preferencesUse direct and indirect questions – e.g. why do you prefer this treatment?, acknowledge their reasons and balance their views – e.g. highlight advantages of less preferred treatment and disadvantages of preferred | R: ‘Are you interested in the study or what do you think?’ P: ‘Just happy to get it gone’ R: ‘You just want an operation?’ P: ‘Yeah, just get it gone’ R: ‘You understand that there obviously are risks and’ (overspeaking) P: ‘Yeah, yeah, yeah’ R: ‘Fair enough’. | Patient: ‘I would rather have the surgery’‘Okay, no, that’s entirely your own choice …By having surgery it won’t necessarily get rid of the pain. It may not be, the pain may not be coming from the gallbladder and the gallstones, this is another reason why they suggest the trial … But you do get a priority; if you went on the trial and you were randomised for the management side of it but then all of a sudden the pain was too unbearable, you have got an open appointment to come back whenever you want to. It’s entirely – I’m not trying to persuade you one way or the other’ |
| Discuss uncertaintyRemind that worldwide there is no evidence that one treatment is better than the other and link this to whyC-GALL being done | I don’t think there’s a right and wrong answer, and I think that’s the thing, it’s difficult, almost impossible to predict in terms of whether you’re going to have an issue, when and where, and that’s why I’m talking to you about this study because I genuinely believe that I’m not … it’s not clear exactly what we should do. But like I say, some patients come to me with a very clear thing, ‘I want this doing, I don’t want this happening again’, and that’s it … and some don’t. | Now at the moment there is no good evidence to suggest which one is the better treatment and … because you fall in that criteria in which we are not sure about whether surgery is the right thing, or carrying on medical management is the right thing. We don’t know what the right option is … So the outcomes are balanced between two treatments and we don’t know which one is better than the other, and that’s the reason of running this trial. |
| Discussing crossovers | Well one thing I would add about the trial is that it’s not the kind of way we should … it’s meant to be set up. But if things change it doesn’t mean that you exclusively have to still stick with that allocation if that makes sense. So some patients for example I have involved the trial, they’ve been put into the medical management arm and watched for 18 months but then their symptoms have got quite a bit worse and they now want an operation, and that means that they just change, but they’re still part of the study but they’re part of the study that say, ‘Well this approach didn’t work for me’. | Explore preferences with patients, stress the need for participant to consider all treatments and equipoise, and don’t give impression to be biased to one treatment or the other. Don’t volunteer information about cross overs but give opportunity for participant to ask questions and address these. |
E-mail feedback was provided to all 21 sites, with generic and site-specific feedback dependent on presence of audio recordings. Sixteen of the sites were responsible for recruiting 400 of the 421 total patients (95%) recruited to the C-GALL trial. This suggests that the hypothesised incremental improvements in recruitment from receiving feedback will have had the opportunity to be realised across these high recruiting sites.
- This document provides suggestions for the clinical researcher team to help explain the C-GALL trial to patients
- This document is not a script – it provides suggestions that should help to improve informed decision-making and the informed consent discussion
- Please consider using some of these suggestions to complement your own consultation style/approach.
Amendments to the trial patient information leaflet
The PMG discussed the findings from the participant interviews and audio recordings and considered these in the context of the existing content of the trial PIL. It was agreed that the existing PIL could provide additional information regarding expectations about trial retention. Specifically, more information on when trial participants would receive questionnaires and how long they would receive them for that is frequency and duration. The importance of completion and return of the questionnaire was also reinforced as important to assess the impact of the different treatments, even if participants did not experience any symptoms. The importance of questionnaire data to produce reliable results was also emphasised. These amendments were made to the trial PIL and approved by the REC with an updated version implemented in June 2019 (see Appendix 2).
Updates to cover letters and newsletters to target questionnaire response
Recommendations to improve questionnaire return were made to the PMG based on the findings from the behavioural diagnosis findings in the Phase 1 interviews with trial participants. A list of relevant BCTs was generated from existing templates and C-GALL trial paperwork and compared to those identified as relevant from the patient interview findings. A total of 12 BCTs were identified as being relevant for return of participant-completed questionnaires in the C-GALL trial. Specific content was developed or enhanced within the cover letters and newsletter for trial participants to ensure these techniques were covered.
Table 30 presents the specific BCTs, the suggested text for inclusion in the letters, and the justification for inclusion based on interview findings. Changes to the questionnaire cover letters and to trial newsletters were implemented in June 2019.
| BCT | Suggested text | Justification for inclusion from participant interview findings |
|---|---|---|
| 1.4 Action planning11.3 Conserving mental resources | ‘The questionnaire requires you to answer questions about your general health over the past 3 months. It can help to keep a note of your symptoms in a diary throughout your participation in this trial. This will help you to complete the questionnaires quickly and as accurately as possible’. | Participants suggested that they struggled to remember their symptoms over the time period specified on the questionnaire. Many participants also said that it helped/would have helped to keep a note of their symptoms to facilitate accurate reporting. |
| 1.9 Commitment | ‘The C-GALL trial requires a strong commitment from you and [local hospital] to stay in the study until the end. This means your [local research team/consultant] has placed considerable trust in the patients they asked to join them in this research. Your [consultant] will not be able to fulfil their part in this study without the continued co-operation and participation of their patients’. | The majority of participants indicated the importance of committing to the study, which influenced questionnaire completion and return. Indicating that study participation requires commitment from participants to complete questionnaires at different time points may be effective. |
| 5.1 Information about health consequences5.2 Salience of consequences | ‘As you are aware, living with gallstones can be debilitating. Your participation in this study will help us gain insight into how different treatment options influence well-being. This will help us to identify ways we can support individuals with gallstones to improve their quality of life’. | Participants reported that they had hoped their participation in the study will help people with gallstones in the future. This was also cited as encouraging trial participation. |
| 2.7 Feedback on outcome of behaviour | ‘The information you provide in these questionnaires will contribute to the integrity of the results’. | Some participants mentioned that they felt guilty (or would have felt guilty) for failing to return the questionnaire. Many acknowledged that this was because they knew the questionnaires were important for the study. Emphasising this could serve a potent reminder. |
| 9.1 Credible source6.3 Information about others’ approval10.4 Social reward3.2 Social support (practical) | Newsletter:[Picture of Co-Chief Investigators]‘Co-Chief Investigators, Professor Craig Ramsay and Professor Irfan Ahmed would like to thank participants for their contribution to the trial to date. Without your support, we couldn’t do our research’[Picture of Trial Manager and Data Coordinator]‘The trial management team would like to express sincere thanks to those who have responded to the questionnaires we have administered so far. This is a vital component of the trial and we appreciate that it takes time and effort to complete. If you have any queries about the questionnaires, please feel free to get in touch via e-mail [indicate e-mail] or telephone [insert number]’ | Thanking participants within clinical trials could have an impact on their motivation to participate.A couple of participants in the qualitative study also indicated their appreciation for the support from the trial office. This was identified as a facilitator to questionnaire return: including pictures/quotes from the trial team and trial contact information will highlight the practical support available to them. |
| 2.2 Feedback on behaviour | If possible, include information about how many questionnaires have been completed, how many are left to complete and an indication of when their participation in the trial ends, tailored to the individual. Could draw attention to it by including this information within a table. | Participants often reported that they could not remember how many questionnaires they had completed/returned. Many participants also reported being unaware of when their participation in the trial ends – including this information will provide them with knowledge of their participation status and could increase their motivation to complete the trial. |
| 4.1 Instruction on how to perform the behaviour | ‘As part of the C-GALL study you are asked to complete five questionnaires at different time points. These questionnaires are identical but are designed to measure your well-being over a certain amount of time. It is important to complete the questionnaires even if you have not had surgery or are not experiencing any symptoms around the time that you receive the questionnaire’. | Many participants reported that they felt the questionnaire contained items that were not relevant to their circumstances, either because they had not had surgery or they were not experiencing symptoms around the time of questionnaire administration. Some were also confused about receiving the same questionnaire on multiple occasions. |
Discussion
The purpose of this process evaluation was to inform the delivery of the trial, and in particular to focus on the critical components of recruitment and retention of trial participants. We applied existing methods used to improve recruitment to surgical trials and enhanced their practicality by also considering challenges to trial retention. In addition, we applied a behavioural science lens through which to view the trial process problems – a novel contribution to process evaluations in ongoing trials to directly inform and improve recruitment and retention.
Audio recordings of trial consultations provide a rich data set to understand the complexities of decision-making around trial participation. Several studies using this approach have now demonstrated the value of content review and feedback on recruitment discussions.82 Similar to these existing embedded efforts to improve recruitment in trials we also experienced challenges associated with encouraging healthcare professionals at all sites to audio-record the consultations in which the trial was discussed. Of the 22 recruiting centres in C-GALL, 16 provided at least one consultation audio recording. Exploring ways to best encourage and support sites to routinise the audio recording of the consultations in which the trial is discussed needs to be revisited for future trials.
Findings from the audio recordings and participant interviews led to the PMG reviewing and revising the PIL to enhance the information on trial retention. Existing research has demonstrated that information on the consequences of not completing a trial is not routinely provided in trial PILs.87 When considered alongside our interview findings that indicate participants were not clear on the expectations of them during trial follow-up indicate the need for more directive information on trial retention is required from trial outset. This is further supported by research that highlighted potential trial participants deem understanding what they will have to do as a trial participant when making the initial decision to participate as highly important.88 We amended the C-GALL PIL based on the findings of our embedded process evaluation, but the learning generated is transferable for application to future trials.
Rather than being firmly prescriptive in its design, the embedded process evaluation allowed a targeted adaptive approach in order to be responsive to the needs of the trial. This flexibility was a core strength of the activity and allowed additional value to be gleaned from the investigations of trial process, for example, through exploring discussion of trial retention during recruitment consultations. This also allowed the process evaluation to contribute to the wider methodological literature through its investigation of trial retention in this way.68 The other strength of the activity was the use of behavioural theory to inform data collection, analysis and solution development. The application of behavioural science to problems of trial process is gaining attention.73 Our process evaluation built on these existing developments but elevated the approach by conducting a series of linked activities from multiple data sources to inform the development of participant-centred strategies.
There were a number of weaknesses. Notably, the sample of interviews with trial participants who were not able to complete the trial till the end constituted a small proportion of those who were initially invited. As such, the views represented by those interviewed may not represent the majority of participants who did not complete and return trial questionnaires. The other potential weakness of the study is that we did not formally evaluate any of the strategies which were implemented based on the findings of the process evaluation. Preliminary evaluation of the impact of the e-mail feedback (using before and after analysis) suggested it was effective but more robust evaluation (e.g. interrupted time series or randomised evaluation) is required and will be a focus of future activities. As mode of follow-up changed (from postal questionnaire to telephone follow-up) due to the pandemic, it was not possible to evaluate the participant-facing strategies focusing on trial retention (PIL, cover letter, newsletter).
Conclusion
The embedded process evaluation in C-GALL allowed exploration and targeted solution-focused strategies that aimed at improving recruitment and retention across the trial. The adaptive nature of the design of the evaluation allowed issues to be investigated in both a proactive and reactive response, improving efficiencies and outputs for the trial. There are some generalisable findings and recommendations for good practice for ongoing and future trials. These include ensuring that details of follow-up processes are shared with participants in both written and verbal information that is shared during informed consent, and highlighting the importance of completing questionnaires (including at multiple time points). The innovative adoption of a behavioural science approach to much of the process evaluation contributed novel insights into trial conduct challenges. Future efforts should focus on the formal evaluation of the approach and solutions developed.