Parts of this chapter have been reproduced from Tindall et al.92 © 2022 Tindall, Scott, Biggs, Hayward, Wilson, Cooper, Hargate, Wright and Gega. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms. The text below includes minor additions and formatting changes to the original text.
This section of the report provides a narrative summary of the challenges faced in implementing and delivering ASPECT. We describe the key challenges the study team faced (along with our approach to mitigation), including problems with (1) site set-up, (2) participant identification and recruitment, (3) data collection, (4) intervention training and delivery, and (5) COVID-19.
Setting up recruiting sites
Commissioning and service delivery restructuring
Recruitment to ASPECT coincided with the reorganisation of CAMHS by NHS clinical commissioning groups in response to the government papers Future in Mind93 and the Child Mental Health Green Paper.94 In parallel, there were also large reductions in local authority and NHS child mental health service funding, which occurred against a backdrop of large increases in child mental health referral rates across England. Consequently, many CAMHS underwent significant restructuring alongside a reduction in staff. Moreover, in some cases CAMHS that the trial team had set up to recruit and deliver treatment disbanded completely, or merged with other NHS trusts, further complicating trial implementation.
This restructuring lead to widespread service reprioritisation of the types of mental health difficulties experienced by CYP so that already limited) resources could be available to CYP who needed it the most. This often meant that specific phobias were deemed lower priority, with many CAMHS ceasing to provide usual phobia support (i.e. CBT). In some areas specific phobia cases were passed to school-based services that were still under development within the care pathway, and unable to offer multisession CBT as required by the ASPECT protocol. Furthermore, in some CAMHS where treatments for specific phobias were still delivered, mental health problem severity thresholds for service acceptance were often set at high levels, with many accepting CYP with a specific phobia only if they had a mental health comorbidity severe enough to reach this threshold. These commissioning and service delivery changes led to challenges, both in terms of trial set up and intervention delivery; for example, given that specific phobias were given lower priority, and often referred to other school-based services, gaps in service provision emerged and there were far fewer participants with phobias available to be recruited than was initially planned.
Solutions: adding new recruiting sites
The ASPECT team implemented a number of strategies to mitigate the impact of commissioning and service delivery changes on recruitment and intervention delivery. The most impactful strategy was to substantially increase the number of recruiting sites from the five that had been originally planned to 12 NHS trusts. Over the course of the trial, study recruitment was conducted in 14 geographical regions across England encompassing 12 NHS trusts comprising 26 CAMHS, three third-sector/voluntary services, one university-based CYP’s well-being service and a research clinic.
From 2017 to mid-2019, the ASPECT team approached approximately 77 NHS trusts to expand the number of recruiting sites, managing to open an additional nine geographical locations (12% of the total approached). This meant that 68 potential sites (88%) that we approached were unable to open to recruitment (Figure 1 illustrates a breakdown of reasons). Initially, our approach to identifying viable ASPECT sites involved preliminary discussions with NHS research and development (R&D) teams, senior clinicians and service managers, who determined whether or not services had sufficient specific phobia referrals and the resources to be part of the study. We found that any of these people could, individually or collectively, act as facilitators of or barriers to the establishment of a site. We learnt that involving all of these individuals at an early stage allowed us to work with these complex systems more effectively. We also learnt that adapting our approach to directly include the therapists involved in delivering low-intensity therapies allowed us to learn additional information about service referrals, capacity and resources.
FIGURE 1 A flow chart describing attempts to add additional recruiting sites to ASPECT. PI, principal investigator. (Figure not reproduced here; see the original article.)
Solutions: facilitating local commissioner–provider discussions about care pathway provision
As discussed in one locality, there was a clear gap in provision of phobia treatment in the care pathway, as commissioners had not specified who would treat CYP with phobia. The research team engaged with local commissioners and providers and over time this issue was resolved.
Training clinic
In the context of these discussions with commissioners and providers in this locality, it was agreed that complicated phobias (with comorbid conditions) would be treated by CAMHS and uncomplicated phobias would be treated by school-based workers funded by commissioners through the local authority. The latter clinical team indicated that they had not received recent CBT training for phobia. We, therefore, facilitated this and for a brief time two local CAMHS clinicians (with CAMHS trainees) worked with school-based workers who came through a training clinic that was termed the ‘research clinic’. This was a temporary arrangement to re-establish healthy care pathways and was successful. At the end of the study the care pathways in this area were working well.
Variation in the structure of services and phobia care pathways
As part of the Future in Mind93 paper, emphasis was placed on the notion of embedding a ‘single point of access’ (SPA) within local areas to allow all CAMHS mental health referrals to be processed via one system. This integrated approach aims to streamline referral criteria across a variety of services but was not always implemented successfully in some areas. Some localities had a SPA for each separate service and so were interpreting national guidance differently. The main challenge that arose in this context was that the care pathways for CYP with specific phobias were not always clearly defined across localities and varied widely between the study sites with multiple services (e.g. NHS CAMHS, voluntary agencies, school-based services and local authority-funded services). In some localities, CYP with specific phobias were routinely seen in CAMHS, whereas in others these individual cases were treated by new school-based services. Where CAMHS sites did routinely see phobia cases, variations were evident in the thresholds for service entry, the pathways to treatment and the length of time waiting for an initial assessment appointment.
Solutions to care pathway variation challenges
Owing to the variation in care pathways and service delivery, a ‘one size fits all’ approach to trial implementation could not be taken. A key strategy employed by the ASPECT team was to tailor the implementation of the trial to the specific structure of the clinical service. Over time, we learnt that the most effective way to engage with services was to involve all levels of the service hierarchy early on, including service and SPA managers, therapists, administration teams and the local R&D teams. This approach allowed us to identify the possible barriers to trial implementation from all perspectives within the service, and to work collaboratively with services to implement solutions that would facilitate the trial’s success. For example, one common barrier for clinical services was waiting list times to see a therapist, with these often exceeding the trial period of 6 months (a wait that was much longer than was expected when the trial was initially planned). This meant that if the trial team were to consent a participant, it was unlikely that the participant would have received any therapy before their final follow-up session. This is because the follow-up period was triggered at the point of randomisation rather than the start of therapy, an approach adopted to reduce bias created by variable rates. To address this, we worked with the clinical services to take a flexible recruitment approach, whereby CYP with phobias referred to the SPAs were recruited to the trial at the point when they were at the top of the waiting list, rather than at the point of referral. This process meant that clinical services could take part in the trial without long waiting times blocking recruitment; however, this was a resource-heavy process in that participants needed to be monitored on waiting lists by the clinical service with liaison with the trial team. To avoid placing additional pressure on services, this approach was flexible with services able to decide whether or not they adopted.
A second strategy that facilitated recruitment at clinical services was to ensure that clinicians and managers working in the SPAs were knowledgeable on the trial, and how it worked in each service. Those with greater ASPECT knowledge and ‘buy-in’ to the trial generally knew which pathways to place CYP with phobias onto, as well as the therapists who were ASPECT trained, and who to talk to in the trial team to ensure that waiting times were managed. In areas where this was not the case, the research team needed to take a more active role in reminding clinicians about the study and eligibility criteria to ensure that the study was routinely being offered. At times, this meant that specific phobias were not identified until an initial assessment appointment took place following a child or young person entering the service with a comorbidity.
Excess treatment costs
Excess treatment costs were applied for from Clinical Research Network funding streams and secured for many of the participating NHS trusts, enabling them to deliver OST as part of ASPECT. Overall, 11 Clinical Commissioning Groups committed to providing funding for ASPECT across Yorkshire and the Humber. However, mid-way through the recruitment period, in summer 2018, the National Institute for Health and Care Research amended the rules under which excess treatment costs could be claimed. As a result, NHS trusts needed to exceed a threshold based on their total expenditure before they could claim excess treatment costs. For many of the participating trusts, this threshold could not be reached. Under the new rules, trusts were informed that they had to use their own funds up to that threshold but many, because of financial difficulty, indicated that they were unable to do this. This had a direct impact on ASPECT, as it resulted in reduced access to additional funding during a period when clinical time was under considerable pressure.
Participant identification and recruitment
Originally, ASPECT was to adopt two methods of participant recruitment, the first from health-care and social care pathways in England (e.g. CAMHS or third-sector/voluntary organisations). The second involved identifying CYP through schools, checking that they met the eligibility criteria (including diagnosis of a phobia) and signposting them to the local treatment service. With regard to health-care and social care pathways, we have already detailed the clinical commissioning changes, variation in care pathways and difficulties with clinical service eligibility thresholds that substantially affected participant identification and recruitment earlier in the chapter. However, our intention to identify participants through schools (by posting invitation packs to families) also experienced significant challenges. First, owing to the structural changes to CAMHS described in Commissioning and service delivery restructuring, many services that were trained to deliver the trial treatments were overwhelmed with long waiting lists and limited resources. Therefore, many of our clinical services were reluctant to agree to speculative participant identification in schools, as it could further stretch already limited resources. Second, in those regions where CAMHS were happy for the trial team to contact schools, we had difficulties engaging both primary and secondary schools to take part. In total, after agreement with local clinical teams, the ASPECT team wrote to 121 schools across North East and North Cumbria, Hull and York; however, only a small proportion of schools replied despite follow-up, and fewer still were able to send out recruitment packs to families. Consequently, this strategy did not appear to be an efficient use of time to support recruitment to ASPECT. Finally, given that in many instances CAMHS were unable to accept speculative school referrals, we attempted to understand whether or not school-based mental health services themselves could recruit and deliver the trial treatments; however, only one school-based mental health service was able to commit the time and experience to deliver ASPECT treatments.
Collecting participant data
There were a number of logistical difficulties identified when embedding ASPECT in several sites specifically relating to the exposure included in both the baseline assessments and treatment deliveries. During completion of the BAT at baseline and the 6-month follow-up, and as part of exposure during therapy delivery, CYP had to come into close proximity to their phobic stimulus. Given that the majority of these procedures occurred in clinical services, careful consideration had to be given to ensuring that these could be conducted safely and that suitable spaces were available. Planning for animal and BII-type exposures was particularly resource heavy; for example, with BII phobias additional staff were required, including those trained in venepuncture, who demonstrated procedures as well as first-aiders who needed to be on call in the event that any child or young person experienced a vasovagal response. Animal-type exposures also required an additional member of staff to meet and greet animal handlers while the ASPECT researcher was completing data collection with families. It was also imperative for ASPECT researchers to brief and debrief staff who assisted with BAT assessments, and increased logistical planning was required to ensure that CYP did not come into contact with the stimulus until the BAT assessment. Additional staff were required for assessments with younger CYP to stay with the child during parent and child ADIS interviews, as responses to this were collected separately. Data collection required two separate rooms: one for the ASPECT researcher to collect baseline/follow-up measures and another to complete the BAT assessment. It was important for rooms to be close together and in a quiet location to ensure that the BAT assessment could be completed as sensitively as possible in the event that CYP became distressed. This presented some challenges in some CAMHS settings, as therapy rooms were already in high demand.
The research team worked flexibly, and alongside clinicians, to ensure that the requirements of the study could be completed, for example in the procurement of appropriate stimuli. Where a phobic stimulus was something that had the potential to cause harm (e.g. an animal), precautions were taken to ensure that the activities were completed safely. The research team also liaised closely with sites to ensure that any phobic stimulus could be brought to a site safely and within insurance remits. Where possible, additional suitable spaces to deliver baseline measures and therapy were identified.
Intervention training and delivery
Intervention training challenges
Training in the delivery of OST and the study procedures was provided to clinicians across all sites involved in ASPECT. The original intention had been to provide basic OST training to all designated trial therapists as well as providing advanced training to a separate more experienced group at each site, so that they could train other therapists within their respective localities. This approach aimed to mitigate the effects of staff turnover, while reducing the training burden on the central team; however, this proved unsuccessful, with those trained often leaving services, not having the capacity to train others or not feeling confident to deliver training. Consequently, clinical members of the central team provided training throughout the study, creating a need to direct resources towards this. This problem was compounded further, as the ASPECT team expanded the number of study sites almost threefold to meet recruitment demands. For example, originally, approximately 50 clinicians were trained to deliver OST; however, this figure increased to 170 by the end of the study. Many of these clinicians did not then go on to provide therapies for the study because of changes with contracts, capacity and availability. Furthermore, although some clinicians were enthusiastic to deliver therapies but unable to deliver them within the trial, others regarded the training as an opportunity for skill enhancement or curriculum vitae building, and not for future treatment delivery for ASPECT.
At times, limited planning of patient throughput resulted in significant delays between clinicians receiving training and them delivering therapy for the first time. As a result, some clinicians did not feel sufficiently confident to deliver the therapies and refresher training sessions were delivered in these instances. To address this issue, later sessions asked attendees to identify a potential phobia case ahead of the training. This not only ensured that clinicians had cases ready soon after completing their training but also assisted with training delivery, with real case examples used to guide the techniques learnt.
Intervention delivery challenges
ASPECT also faced several challenges with intervention delivery throughout the study. As described earlier in this chapter, in many CAMHS teams CYP experiencing mental health difficulties with more severe implications than specific phobias were often deemed higher priority in the context of limited resources. Therefore, in some situations, treatment priority had to be given to those with more severe comorbidities rather than focusing on phobia in the first instance. This resulted in some CYP not receiving phobia treatment until the conclusion of an additional therapy for a more severe comorbidity. This problem was compounded further in that the majority of CYP reporting specific phobias within CAMHS also report a primary comorbidity (e.g. anxiety, depression or eating disorders). Where this was prioritised for therapy, it meant that some CYP did not complete their phobia therapy before their 6-month follow-up measures were due for completion. It was left to clinicians to make appropriate decisions about the order of treatment. In other situations, the delivery of the study interventions needed to be carried out with care (e.g. where a child had a generalised anxiety as well) to accommodate the comorbidity (although the fundamental components of phobia treatment remained as described in training), with many clinicians being considerably experienced in doing this.
In addition, at times, individuals tasked with seeing these cases had limited experience of delivering phobia treatments; therefore, additional training, supervision and guidance were required in the delivery of both treatment arms. At times, the structure of services also impeded the delivery of treatment. Some clinicians, in particular those in school-based services, reported having maximum appointment times of 1 hour to see CYP; therefore, delivering therapy to those randomised to receive OST and requiring a session lasting up to 3 hours was difficult. In these instances, therapists often had to arrange OST sessions alongside holidays to be able to allocate enough time to the treatment sessions. In several instances a problem arose where a SPA clinician identified a child or young person as having a phobia and placed them on the waiting list only for the treating clinician to withdraw them from the study having decided either that a different course of treatment was more appropriate or that the child or young person did not have a phobia.
Staffing
Throughout ASPECT, there was a high turnover of staff in many of the services involved. Many sites identified their trainee psychologists or well-being practitioners as being best placed to deliver the study interventions. It was often felt that these individuals had the capacity to see trial participants, and accepting and treating phobia cases was deemed to provide good experience for professional development. However, many of these individuals were based in services for only a limited period and could, therefore, provide treatment to only a small number of individuals before their placements ended. This was particularly problematic in instances where CYP reached the top of the waiting list and were allocated to a clinician whose placement was nearly complete.
Dealing with COVID-19
In 2020, the COVID-19 global pandemic was announced, with a national lockdown commencing in England on 23 March 2020. Although recruitment to ASPECT had been completed in January 2020, a significant number of CYP were awaiting therapy, receiving therapy or due to complete their 6-month follow-up assessments when government restrictions were put in place. In response to the pandemic, immediate changes were implemented to ASPECT, with impacts on both treatment delivery and the completion of research-related activities.
All follow-up appointments after 23 March 2020 were conducted remotely, either by video conferencing or by telephone depending on participant resources and preference. Although the majority of follow-up measures could be completed this way, completion of the primary outcome measure (the BAT), which could not be administered remotely, had to be postponed until face-to-face contact could be resumed. However, we were unable to resume face-to-face contact by the end of the 6-month follow-up period in August 2020 and, therefore, a proportion of BATs (n = 48) could not be collected at the end of the trial. This equated to 24.4% of those completing a 6-month follow-up (21.6% randomised to CBT and 27% randomised to OST).
As ASPECT was a pragmatic trial, services were informed that therapy delivery during the time of the pandemic was to be based on individual policies and procedures. All services included in ASPECT reported that they had paused therapy delivery while government restrictions were in place. This significantly affected study timelines with many CYP reaching the time for their 6-month follow-up prior to receiving or completing therapy. When treatment deliveries recommenced, adaptations had to be made and therapies delivered remotely. While several clinicians were able to make suitable adaptations to the delivery of CBT, this was not the case for OST; many CYP allocated to OST had their remaining treatment paused.