Summary and conclusion

Introduction

This final chapter provides a brief summary of the key findings in the SOLID trial in relation to the aims and objectives of the study as outlined in Chapter 1, Research objectives. Each element of the study process has been discussed in detail previously. This chapter presents the overall conclusion to the study and suggests recommendations regarding a future trial. A key aim of the SOLID trial has been achieved in terms of adapting and testing the feasibility of conducting a trial of two behaviour change interventions (MET and SBNT) delivered to children in care, aged 12–20 years, within standard care pathways. However, the findings from our pilot trial are less optimistic regarding future evaluative work. Nevertheless, as this is the first UK-based RCT that has attempted to assess the feasibility of delivering behaviour change interventions to reduce current substance use in children in care, the findings from this study should be used to inform future work with children in care in the UK context.

Formative study and key stakeholder survey

As part of this research phase we succeeded in recruiting 65 key stakeholders, including children in care, non-care young people, drug and alcohol practitioners (front-line staff and service managers), local authority staff (social workers, managers and PAs), and carers (foster and residential), and completing a process of intervention adaptation. Key thematic findings informed the adaptation, manual development and integration of the interventions into children in care health and social work pathways to enhance acceptability. Children participating in risky substance use were seen by a specialist drug and alcohol practitioner. In standard practice, the first one or two sessions would have been used to conduct an initial assessment and establish the young person’s level of need. The study enhanced the offer of usual care once an individual engaged with services, as the intervention started within the first session. In addition, the adapted intervention allowed a maximum of six sessions to be delivered (compared with usual care, which was unstructured and young people could attend as many or few sessions deemed necessary to meet their identified needs). Participants engaged in the formative research phase fully and showed enthusiasm when contributing to the adaptation of the two interventions.

An electronically administered survey was completed with the service managers of young people’s drug and alcohol services. Eighty-two per cent of services in England completed the survey which illustrated high levels of variation with regard to screening and treatment pathways for children in care with risky substance use. There was no standardisation regarding which screening tools were used: 52 services (46%) reported developing a tool ‘to meet their local need’. The modalities of treatment differed across providers, the majority (n = 67, 55%) offered a bespoke service and 35 (29%) services reported only conducting structured work; however, even the latter could be delivered flexibly across sessions. None of the services reported delivering and adhering to a manualised evidence-based intervention.

Pilot randomised controlled trial

The pilot RCT was conducted to assess if rates of eligibility, recruitment and retention of children in care and acceptability of the interventions were sufficient to recommend a definitive multicentre RCT.

Predetermined STOP/GO criteria (green/red) were developed for progression to a definitive trial (see Table 28 for green for progress and red for stop).

Regarding the column reporting achievement (see Table 28, column 5), we assessed the key outcomes from the SOLID pilot trial and recommendations for a definitive trial are as follows.

Proportion of eligible participants consenting to pilot feasibility trial

The number of children in care screened was 860, of whom 211 were eligible.

Percentage of eligible participants consenting to feasibility trial: 112/211 × 100 = 53%.

Proportion recruited: 112/211 = 0.53 (amber).

Although the study did not recruit the target figure, lessons from this study could be used to maximise recruitment in a future study. The CRAFFT screening was completed by 860 (59%) of the 1450 children in care (aged 12–20 years) across the region, and 112 (53%) of the 211 eligible children in care were recruited and randomised into the study.

Proportion of children in care attending at least 60% of sessions as planned in combined intervention arms

MET

Four attended 100% of sessions offered, one attended 66% of sessions offered, one attended 50% of sessions offered, one attended 29% of sessions offered and 31 attended 0% of sessions offered. Therefore, 5 out of 38 (13%) attended > 60% of sessions offered.

SBNT

Two attended 100% of sessions offered and four attended 50% sessions offered. Therefore, 2 out of 38 (5%) attended > 60% of sessions offered.

Overall, 7 out of 76 (9%) attended > 60% of sessions offered (red).

Proportion of consented participants retained for measurement of key outcome data at 12 months

Sixty out of the 112 (54%) randomised provided follow-up data at 12 months (amber).

In total, 54% of recruited children were followed up to 12 months. Follow-up was particularly difficult due to frequent moves in this population and their changing contact details. However, the SOLID trial has shown that once participants were located and seen by the researcher, they were prepared to complete questionnaire data at both baseline and at follow-up. None of the participants asked to end the data collection process early and only one individual elected to not complete the TLFB at the 12-month follow-up.

Were interventions delivered with fidelity?

In total, only 26 intervention sessions were delivered to 15 young people out of the 76 children in care randomised to the two intervention arms. It was not possible to evaluate the fidelity of intervention delivery due to the low number of audio-recordings available, only nine audio-recordings were received.

Were interventions perceived acceptable by children in care and workers?

It was not possible to assess the acceptability of the MET and SBNT interventions due to the low number of participants who received the intervention sessions. Owing to the involvement of 24 children in care participants in the formative phase of the study, we believe the content should be acceptable. However, we can conclude that significant changes would need to be made to the location of these interventions within current service pathways, in order to increase participant engagement.

Does the value of information analysis show future research is worthwhile?

It was originally proposed that a value of information analysis would be conducted. Unfortunately, no useable data were available from the feasibility study and insufficient data were available from the literature to conduct the economic evaluation modelling exercise that underpins a value of information analysis. Therefore, an exploratory return on investment was conducted in order to identify the range of values for benefit and total cost consistent with a return on investment. The preliminary return on investment analysis suggests that a medium to large health effect would need to be demonstrated before the intervention would be considered cost-effective.

It is our judgement that criteria for progression to a definitive trial have not been met in this study.

As discussed in Chapter 5, major challenges were found in both screening and recruitment of children in care into the study. Screening of children by the six local authority social work teams took over a year to complete, with significant time and resource investment by the researcher and social work teams. The increased amount of time needed resulted from the impact of staff turnover, long-term sickness, restructuring of local authority sites and Ofsted inspections. Social workers acted as gatekeepers when deciding which children in care to screen, influenced, for example, by their preconceived beliefs of whether or not the young person used substances or were potentially too young to participate. Contact details for children within the care system were not always updated on the centralised systems, therefore obtaining the relevant contact details to make contact for the study follow-up was often problematic. These difficulties were fed back to team managers, leading to escalation to senior management level. It is hoped that future studies will benefit from this initial work to ensure that system data are accurate and up to date.

An embedded researcher or senior local authority member of staff seconded in a research role could provide an increasingly stable point of contact within the complex local authority setting, to facilitate more successful screening and recruitment.67 The availability of a visible individual to be present throughout the research period would act as a facilitator within what is, primarily, a less research-mature environment.

The process evaluation highlighted that, no matter how effective an intervention is perceived to be by practitioners during development, if participants do not engage with it, the potential benefits cannot be achieved. It is essential that the service considers different methods to increase engagement. The participant interviews identified valuable insights into the facilitators of and barriers to engaging with interventions, and highlighted the role of self-perception regarding ‘problematic’ substance use. The children in care recruited in to this study did not feel that they have a problem, often due to the normative view of their drug and alcohol use. Participants did not feel that their use warranted an intervention from a specialist drug and alcohol practitioner, therefore a referral to a specialist service does not feel salient. This suggests that a different delivery agent is required if intervention delivery is to be successful.

Recommendations for a future evaluation

We have learnt several lessons that could inform a future trial or other form of evaluative work with children in care.

The delivery of future interventions

The typical screen, refer and treat model used in the SOLID trial was problematic. Children in care were recruited into this study based on a screening outcome, suggesting risk due to substance use behaviour. The study was designed in this way to mirror the current system and was based on public health principles related to prevention to address avoidable future problems. However, it could be argued that being in the care system was itself a risk factor for increased drug and alcohol use. Moreover, the interventions were delivered by specialist drug and alcohol workers in a referral context. Any future trial needs to think about how we deliver harm reduction and preventative care for children in the social care system. This may well require additional resource. Accepting the view that being in care can generate increased risk for children and young people, a broader ‘care pathway’ approach may be needed in keeping with the principles of the thrive model currently used by the CAMHS for children and young people.119 The thrive model promotes five categories: thriving, getting advice and signposting, getting help, getting more help and getting risk support. Thus, substance use counselling would fit with ‘getting risk support’, but would be part of a process that understands wider determinants of risk and risk behaviour. By delivering the adapted intervention according to the thrive categories, it would allow a person-centred and needs-led approach to be delivered to children in care regarding their substance use.67 In the SOLID trial, we tried to use the existing drug and alcohol referral system to deliver novel interventions. This working between different sectors of the care system did not work. A new way of working, in which drug and alcohol workers become embedded to social care services and residential units, could be a better way of delivering these interventions. This fits with principles of place-based approaches,120 which argue that providers of services (in this case social workers and drug and alcohol practitioners) work together to improve health and care for the population they serve. Within the process evaluation, social workers stated that they would not be able to deliver these interventions unless they were given specific training and ring fenced time.

Evaluation design and support

Prior to beginning the SOLID pilot trial, we had secured senior ‘buy-in’ across six local authorities. We had a team of experienced researchers, including an academic social worker as a co-applicant, and we saw real excitement and enthusiasm from social work staff about research work being conducted in social care. However, social services departments are often less research mature than clinical contexts that have benefited from specific resources through the Clinical Research Networks (CRNs) to support research delivery over many years. Local authorities have experienced reduced funding due to economic austerity and many social work settings are understaffed and overstretched in terms of workload. Consequently, without additional, dedicated research support, potentially from an embedded researcher/social worker, then applied research in social care and, particularly, with children in care will prove difficult. This researcher would add much needed academic leadership within departments. They could be jointly funded and managed across academia and the local authority, which would give them clearance to engage clients and could significantly change the research culture within the units. Within the lifetime of the SOLID trial, the NIHR CRN has implemented changes to the eligibility criteria for studies. The NIHR CRN has now extended support into research taking place in non-NHS settings, such as health and social care and public health, these changes could facilitate the necessary change as it has within NHS research trials.121 In addition, future studies within the social care context would benefit from performing an organisational readiness study with social work teams and drug and alcohol services, to be involved in research and deliver interventions prior to implementation of a RCT or evaluation study. Pragmatic evaluation design coupled with additional research resource for children’s services are needed to evaluate these novel models of care at scale.

Screening and risk perception

Mass screening of children in care as part of ‘standard practice’ within the social work teams was not feasible. The findings of the SOLID trial have shown that attention needs to be paid to the identification, screening and intervention delivery for this high-risk group of young people. The screening results highlight that for this population, older teenagers are more likely to be reporting alcohol and/or drug use. Within this study, drug and/or alcohol use was most prevalent in young people aged between 16 and 19 years, with reported use peaking at 17 years of age [72 (8.4%) of respondents reporting use within the last 12 months]. However, the existing current models and pathways of care have been shown to be ineffective, resulting in a significant amount of need being unmet by services. Screening would need to be completed using an alternative method in a future study. The current screen and refer model used in the SOLID trial has been shown to be problematic. The completed screening tools could have detected problematic drug and/or alcohol use and provided an opportunity to offer an immediate therapeutic intervention. Having an embedded researcher with capacity to promptly engage a young person may have provided scope to tentatively explore the impact of their drug and/or alcohol use on their mental health, physical health, relationships and behaviour. This opportunistic interventions could be used to sensitively introduce a problem feedback component regarding their substance use and may have initiated a stronger recognition of risk by the young person. An embedded researcher within social care departments would have the potential to increase the uptake of screening with children in care, improve the engagement of social workers in very stretched services and provide an increasingly tailored response depending on the information provided at the screening stage. An embedded researcher could be an individual who is ‘either university based or employed with the purpose of implementing a collaborative, jointly owned research agenda in a host organisation’.122 This model is often found within a health-care setting, such as medicine and nursing; however, is less established within social care settings. Cheetham et al.122 provide an example of how having a researcher embedded within the local authority environment can increase the situated understanding of the cultures and norms of an organisation, and also recognise the realities of conducting public health interventions.122 An embedded researcher would have the capacity to take into consideration the context of the environment and stakeholders’ interests.123 In addition, the perceptions of ‘problem substance use’ differed between the children in care and practitioners working within the service systems currently in place. If the requirement is for children in care to be engaged with a substance misuse intervention and that intervention can be delivered opportunistically and quickly around the needs of children in care, then a place-based approach to intervention delivery needs to be taken.120 In addition, the child in care needs to perceive their substance use as an area they want to address. In our sample, many young people were more concerned about other problems in their lives and that substance use was a symptom not a cause of problems. Consequently, work on any substance use needs to be set into a wider context of adverse childhood experiences30 and difficulties experienced in care.89

Interventions

The MET and SBNT interventions modalities were chosen as they have both shown to be effective in decreasing substance use in participants, including adolescents.69–71 There is strong evidence for the use of brief interventions. A Cochrane review by Kaner et al.124 reported that at 1 year follow-up the amount of alcohol people drank each week had reduced and that longer counselling provided little additional benefit over brief interventions. However, we were unable to test the effectiveness of delivering brief interventions to this population of young people due to low attendance at the available sessions. Future studies would need to consider the best way to engage children in care in interventions that promote a harm reduction and prevention component, alongside offering a therapeutic approach.

Screening tool

The CRAFFT screening tool should be adapted to enable young people reporting substance use within the last 30 days, as opposed to 12 months as it is currently, to ensure that the information provided is current and relevant. The age range of young people being recruited into the study should be modified to 12–17 years. The rationale for suggesting this age range, is that data demonstrates that substance use is starting to emerge in children as young as 12 years. Among the young people screened, 66 forms (8%) were completed by children in care aged 12 years, with two (0.2%) children in care reporting substance use (alcohol, cannabis and NPS). By comparison, 93 forms (11%) were completed by children in care aged 13 years, with 12 (1.4%) positive for substance use, and 90 forms (10.5) completed by children in care aged 14 years, with 18 (2%) children in care reporting substance use. This suggests that if early intervention and preventative work is the aim of the study, 12 years is the correct age to commence screening, as it correlates with initiating substance use. This of course requires individuals to be approached sensitively due to their vulnerability, and issues of capacity and competency to engage with interventions would command further exploration.

Screening children in care aged ≥ 18 years also posed a dilemma. Young people within this age category are legally old enough to drink and by default did not identify their reported alcohol use as a problem as ‘alcohol is legal’. In addition, children in care were increasingly starting to transition into independent living accommodation, to enter the college environment in which exposure to alcohol in a social context was increasingly likely. Altering the inclusion age to children in care aged < 18 years would ensure that individuals were still ‘open’ to social services and would provide the necessary mechanism to follow up children in care. In addition, such young people are under the legal age to drink; therefore, any alcohol use could be deemed as risky and could warrant an intervention being delivered.

Conclusion

Current evidence shows that children in care are significantly more likely than their peers to use substances and have severe mental health morbidity. This study has shown that this population is amenable to be involved in research, with 65 participants engaging in the formative phase of the study and 109 participants engaging in the process evaluation. The enthusiastic ‘buy-in’ from children in care and professional participants within the more qualitative elements of the study, demonstrate that it may not be research, per se, that is the barrier to successful engagement, but it may be the rigid structure of the RCT protocol that is challenging for the children in care to participate in. Although the RCT is classed as the gold standard regarding making clinical decision, the social care context would benefit from alternative evaluations that can accommodate the increasingly complex causal pathways that occur between intervention delivery and behaviour change outcomes.125,126 Data have suggested that the intervention components of the adapted MET and SBNT approaches seem broadly acceptable to them, and that drug and alcohol practitioners perceived these approaches as helpful. However, the way that our behaviour change interventions were delivered and integrated into existing services and pathways was problematic. The interventions were being introduced into an already complex system that identified the difficulty of delivering standardised interventions, this can be seen by the lack of manualised interventions used within drug and alcohol services. Pettigrew et al.127 identify that interventions within public health need to be able to have sensitivity to features of the local context and our current design did not allow for this.

Moreover, our work showed that many children in care do not identify themselves as needing a drug and alcohol intervention, despite reporting use of licit and illicit substances and linked risky behaviours. This mismatch between professional’s and children in care’s views justifies further attention. The lessons learnt from the SOLID trial have implications for future evaluative work and we have suggested that a new model of embedded research in social care may be needed in the future. This additional capacity would enable research studies to be delivered in resource-constrained local authorities and also act as catalyst role models to help build a more research-conscious workforce in social care. In addition, future studies within the social care context would benefit from performing an organisational readiness study with social work teams and drug and alcohol services, to be involved in research and deliver interventions prior to implementation of a RCT or evaluation study. Finally, we do need to develop and evaluate models of prevention to help reduce the adverse outcomes of being a child in care. However, given the wider risks inherent in being placed in the care system, we need to embed substance use work within in a context of understanding wider determinants of risk behaviour. We also need to ensure that we address the specific concerns felt by the children whose outcomes we are trying to improve.