Process evaluation

Introduction

In keeping with the MRC guidelines,108 we used quantitative and qualitative methods to address the aims of the process evaluation. The qualitative work included semistructured interviews and focus groups conducted with five key groups of participants: (1) members of the local authority social work teams (social workers, PAs, social work team managers), (2) drug and alcohol practitioners, (3) children in care, (4) carers (foster and residential) and (5) the researchers conducting the study. The quantitative methods included collating data from drug and alcohol practitioner intervention logs and fidelity rating of audio-recordings of treatment sessions. This chapter provides a description of the methods used in the conduct and analysis of the interviews, followed by a discussion of the key findings and themes, and concludes with a critical discussion of the strengths and limitations of the work.

Aim

We aimed to understand and document the key lessons learned from implementing the SOLID trial in relation to the delivery of the adapted interventions the trial processes and to evaluate factors that would impact on delivering the intervention at scale. The process evaluation had three specific aims: (1) to examine the feasibility and acceptability of the interventions and the trial process; (2) to explore and aim to understand the mechanisms through which change occurred; and (3) to consider the role of context in shaping this change.109

The components of the process evaluation are shown in Table 22.

Method

Qualitative interviews

In-depth one-to-one interviews and focus groups were selected as the primary method of qualitative data collection. The aim of all sets of interview was to explore the acceptability of using the adapted MET and SBNT interventions and to elicit participants’ views on the study processes employed throughout the delivery of the project.

Drug and alcohol practitioners

The interviews and focus groups with drug and alcohol practitioners allowed us to explore the extent to which staff understood the principles and core components of the adapted SBNT and MET interventions, whether or not they valued and believed in them, if modification to their usual work plan had to be made to enable integration of the interventions, and to capture reflections on ways which the study process could be modified in future trials. Interviews with drug and alcohol service managers allowed us to explore how they perceived practitioners had engaged with the delivery of the interventions and the potential barriers they felt would have to be considered if thinking of a future definitive trial.

Social workers

Interviews with members of the social work team explored the social workers’ understanding of the purpose of screening all children in care, their opinions of the CRAFFT form as a screening tool and how it could be improved for future trials, alongside perceived barriers to and facilitators of completing the CRAFFT form with children in care.

In addition, three focus groups were carried out in November 2018, to further explore some of the main themes that had arisen within the process evaluation interviews with children in care. The interviews with social workers covered the same topic areas as the carer interviews and discussed whether or not social workers felt that they would be able to deliver the interventions, the potential facilitators of and barriers to them delivering the interventions and the learning for future trials to increase the number of young people participating in the interventions.

Interviews with all practitioners (social workers and drug and alcohol practitioners) took place at a time, date and location convenient to them. Informed consent was obtained, as previously outlined in Chapter 3, Consent, and interviews were audio-recorded.

The topic guide can be seen in the Report Supplementary Material 7.

Children in care

Interviews with children in care aimed to understand if participants had found the process of involvement in the study acceptable, including being contacted by the research team and completing the baseline and follow-up data collection questionnaires, as well as receipt of their allocated interventions. Interviews also explored how the research process and the interventions may be improved to make them more user-friendly and acceptable.

Carers (foster carers and residential workers)

The original focus of interviews and focus groups with carers aimed to explore their understanding of the study process and whether or not they perceived an intervention effect on the drug and alcohol use of young people in their care. However, placement instability was common in the participating children in care. As a result, many participants were not in the same placements as they had been when baseline data were collected. Others were living independently or were in residential placements. When a placement had sustained, some foster carers were not available for interview. The topic guide was therefore altered to explore what skills foster carers thought were necessary to engage and care for children with care experiences, the role they perceived social workers played in children in care’s support package and who they thought was best placed to provide support to children in care.

Interview topic guides were developed iteratively throughout the research process. Within the focus groups conducted at the final stage of the study, researchers discussed emergent findings from the interviews and explored whether or not carers felt that they would be an appropriate group to deliver the interventions, and any potential facilitators of and barriers to them delivering the interventions were discussed.

Researchers

Interviews were conducted with research associates who had conducted the study. Interviews explored the researcher role within the study, the barriers to and facilitators of working with each participant group (social workers, drug and alcohol practitioners, foster and residential carers and children in care) at both an individual level and an organisational level. Interviews also considered how research involving the different participant groups could be improved in future trials and the overall lessons learnt.

Sample strategy

One hundred and sixteen sets of data were collected from across five participant groups, including children in care (n = 37), carers (foster and residential) (n = 30), social workers (n = 27), drug and alcohol practitioners (n = 19) and researchers working on the project (n = 3). Social worker participants were sampled purposively to ensure maximum variation in terms of participants recruited from each local authority site, sex and level of experience. A snowball sampling method was used with foster carers and residential workers. Carers were recruited at the same time as children in care or through carers identifying further participants who could take part. All of the drug and alcohol practitioners involved with the study were approached to take part in an interview and all children in care approached to complete the 12-month follow-up were provided with an opportunity to take part in an interview. In addition, all the researchers working on the study took part in an interview. All participants were approached directly by the researchers. At each stage it was stressed that participation was entirely voluntary. Informed consent was taken at the beginning of every interview, ensuring that the participants had read and understood the participant information sheets and had been given an opportunity to ask any questions or raise concerns with the interviewer. Interviews lasted between 20 and 40 minutes and focus groups lasted between 60 and 90 minutes. They were all digitally audio-recorded and transcribed verbatim by professional transcribers.

Drug and alcohol practitioners

Each drug and alcohol practitioner who had been involved in the study (n = 14) was asked to participate in an interview for the process evaluation. Only one practitioner declined, citing previous involvement in a focus group in the development phase as the reason (see Chapter 2). Service managers were also approached to undertake an interview and interviews were completed with three out of six service managers. One service had recently been retendered and the manager felt unable to comment on the study. In a second study area, the practitioner who had been delivering the MET intervention was ‘acting up’ into the manager’s role. Last, in the third service, interviews were scheduled to take place on two occasions, but unfortunately were later cancelled owing to unforeseen circumstances.

Children in care

The interviews with children in care (n = 37) were completed at the same time as the 12-month follow-up data collection (as described above in Chapter 5, The 12-month follow-up data).

Carers

Interviews and focus groups with carers (n = 30) included a mixture of residential workers (n = 23) and foster carers (n = 7). To reach our target audience, the research team organised interviews and focus groups with workers employed within residential homes in three local authority sites where the participating children in care resided at, as well as with foster carers who had experience of caring for children in care within the 12–20 years age range of the study and experience of managing substance misuse issues.

Social workers

Interviews with members of the social work teams (n = 27) took place at two separate time points. In both instances, participants within the study sites who had been involved with the initial CRAFFT screening were approached by a member of the research team and asked to take part in an interview. Interviews took place at the social worker’s place of work at a time and date convenient to them.

The first interviews and focus groups took place from July to October 2017, with 14 participants of varied experience and knowledge. Participants included front-line social workers, senior social workers and team leaders, a strategic manager, a social work student and PAs. PAs work closely with social workers to advise, assist and support young people in local authority care.

Researchers

Research associates (n = 3) carrying out the day-to-day work of the project were also interviewed by a researcher who had no prior involvement in this study.

Table 23 shows the qualitative methods that participants engaged in and the sex of participants recruited into the process evaluation.

Qualitative analysis

Interview schedules for the drug and alcohol practitioners delivering the interventions were designed to highlight the four core concepts of normalisation process theory:112

1. coherence – a shared understanding of the work (do the workers understand the aims and the logic behind the intervention?)
2. cognitive participation – a shared agreement and engagement with the techniques of the work (do the workers ‘buy into’ and ‘own’ the aims and logic behind the intervention and the ways in which it is being implemented?)
3. collective action – agreement with the organisation of the work (what do the workers and managers do in practice to make the intervention work in their setting for themselves and the client group?)
4. reflexive monitoring – assessment and monitoring of the work (are the workers engaged enough to be able to suggest improvements to the logic or practice of the intervention?).

Normalisation process theory was chosen as it provided an evidence-based framework to consider factors that either promoted or inhibited the successful implementation of the adapted MET and SBNT interventions into routine practice.113 We analysed the data using the core constructs to examine the key barriers to successful delivery and integration of the interventions at the level of the system. Possible residual contamination of co-workers residing in the same premises but delivering different interventions was also explored within the process evaluation. The thematic analysis80 of all other data sources within the process evaluation was conducted as described in Chapter 3 of the report.

Qualitative findings

The qualitative findings from the thematic analysis are broadly categorised under four headings, namely exploring the acceptability of the screening tool, feasibility of the recruitment process, acceptability of the interventions and feasibility of retaining young people in the trial.

Was the screening process acceptable?

The CRAFFT screening tool was not wholly appropriate in its current format, areas of discussion arose regarding concerns about the screening tool. The CRAFFT form asked young people to identify their substance use within the last 12 months. Professionals, both social workers and drug and alcohol practitioners, thought that this time frame was too long and that a shorter time frame would produce more accurate results, and suggested modifying the CRAFFT form to capture data within a shorter time frame. Suggestions varied between the 28-day time frame used for the NDTMS up to a maximum of 3 months:

I would say 28 days because that way staff members can actually quantify it alongside the work that they’re doing anyway because that’s what we deal with on a day to day basis, that’s what we use as a guide . . . So I think the drug use needs to be more prevalent and more relevant within a shorter time frame upon referral.

In addition, the current method of scoring was not felt to be appropriate for distinguishing between use of a substance on a single occasion and problematic use:

The scoring of it, it fails to recognise when someone, by just saying, ‘Do you drink? Have you had a drink?’ It doesn’t distinguish different and potential levels of potential problems. So, it basically pulls in everybody so it produces a lot of negatives or potentially ends up producing a lot of work for no result. Many people then don’t need further information.

Lessons learnt regarding the screening process

The logistics of the screening process were also explored. A number of practical measures that could be taken to make the process more acceptable were discussed, such as having an embedded researcher. Areas for consideration in a future trial included who should screen and the age of participants to include in the study.

Methods of completing the screening tool

There was full agreement that screening should not be completed by posting CRAFFT forms out to children in care. None of the sites reported any forms being returned if they had been sent out for young people to complete independently:

I think we had a couple where we posted them out, and they didn’t do them. Whenever we send a letter out to a young person – I know this sounds really stupid, whenever we send the letter out to a young person, we put it in a white envelope. Whenever they see a brown envelope they think it’s a bill. It just adds to the pile.

Completing the CRAFFT forms face to face was the preferred method. In this study screening took place using paper forms, and it was suggested by social workers and children in care that completing the forms electronically would be preferable:

I mean there’s something you could send them by link to fill in, or as an app [application] or a link they could do. Sometimes they’ve got access to online. Then yes, I think they’d probably get more engaged if they’re being able to sit down without somebody asking them the direct questions, you fill in this link.

Electronically completing the form would allow the completed data to be uploaded/transferred to the research team immediately and would minimise risk of data being lost.

Reconsider the inclusion age range

Based on discussion with the children in care PPI group at the time of study design, the study aimed to screen and recruit young people between the ages of 12 and 20 years. Social workers raised concerns that 12 years was ‘too young’ to screen for drug use:

I thought 12 was a little bit young. I think from the responses that I got in terms of talking to workers, they were a little bit edgy around the 12 remit.

This was also voiced by one of the PAs, who felt that screening younger children in care may have caused participants to ‘panic’:

I think, personally, the age limit was a little bit low, just because some of the younger ones were like, ‘I’ve never done drugs, I’ve never done drugs, so what are you asking me this for?’ They were panicking, . . .

It was also much harder to follow up children in care within the 18–20 years age range, as in a number of cases they had transitioned to independent living and had elected to end contact with children’s services. Once an individual was closed to social service input, research staff could not obtain up-to-date contact details:

Eighteen for me would have been a time where, you know, young people are leaving care effectively, the care orders are lapsing and that might have been a natural point to have actually have ended.

Social workers could be a barrier to screening

It was also questioned whether or not children in care would be honest with their social worker when completing the CRAFFT form. Social workers had concerns that children in care may not be truthful:

There was a potential for some of the children to not want to share that information with the social worker. The social worker’s going out to complete a document or to ask those questions. I don’t know, in some cases, it’s not going to have a significant impact on the responses but there would be some children who may not feel that they can be entirely honest with the social worker.

This concern appeared to be unfounded. Although a small number of children in care reported that they would usually not choose to speak to their social worker about their drug and alcohol use, the majority stated that they did not have any concerns regarding completing the CRAFFT form in the presence of their social worker. The relationship that the a child in care had with their social worker was highlighted to be an important factor:

So, because I had a relationship with her, I was like, ‘Yes, that’s fine.’ There have been a few social workers in the past, where they have recommended things and because I haven’t got on with them, I’m like, ‘No, don’t worry. I don’t want anything off you’, sort of thing. Again, it just depends if you’ve got a relationship with that professional, in order for you wanting to do something.

Participants questioned whether social workers were the correct person to screen, due to other elements of their work taking priority over screening:

I mean I think the problem is, whenever we’ve got something extra to do on top of our day-to-day work, it adds an extra stress, which I think people find difficult and I think the problem is, like when you’ve got a kid whose running away or there’s something around child protection, filling in a questionnaire, isn’t top of your priority.

When social workers attempted to screen children in care as part of their standard visits, it was largely unsuccessful. It often took a single named point of contact (social work students, PAs, a visible researcher within the local authority site) to co-ordinate the site for screening to be successful.

All of the social work managers agreed that, in retrospect, having a researcher embedded in the local authority site would facilitate CRAFFT screening completion:

At least they’d have somebody who they can have that conversation with if they weren’t quite sure or if there was somebody on the phone that wanted a bit more information that maybe they didn’t have, you’ve obviously got a direct link.

Realistic time frames need to be agreed with study sites

A number of barriers were discussed within the process evaluation interviews regarding the study process. One of the main areas of concern was having a realistic time frame to complete the screening. The first issue raised was regarding the social workers’ ability to physically see the young people on their caseload and complete any necessary screening forms for the research study:

The visiting frequency has now changed for some young people. It was always a minimum of 4-weekly for children in care. That’s now been changed to 6-weekly. We, in this team, have young people, we’ve now got one young person who is only visited twice a year.

Despite the research team extending time frames due to poor response rates, and the extended time frames ultimately resulting in more forms being completed and returned, the social work managers deemed the ‘moving goalposts’ as ineffectual:

Initially we were going for one set time frame and it was all right then. I think we got an extra 28 days or something initially. Everyone was all right about that but then when you extended it again, I think the staff lost a bit of interest.

The local authority teams reported becoming apathetic to completing the screening:

There was lots of momentum. I remember feeling under loads of pressure. I’ve got to get this done. It should have started already. It hasn’t started. Then we moved, and we needed it done by the New Year . . . As soon as that message come through, certainly in my mind – probably to anyone else who heard it – it felt there wasn’t the urgency that we thought there was initially.

The complexities of conducting research in the real world need to be factored in to study design

The process evaluation interviews identified the real-world organisational barriers that created challenges when trying to conduct the SOLID trial. In many cases, this was the only research study being carried out in the social work departments and therefore there was a limited research culture. The following quote makes reference to the fact that in this study, social workers were expected to recruit children in care into the trial:

They are very important learning points about your study group, about the difficulties of multicentre trials and recruiting people from different occupations, recruiting people into research studies who have no research orientation, no research knowledge and no research motivation, quite frankly.

From the perspective of one local authority, significant transformation took place regarding the structure and physical location of the social work teams, with negative impact on engagement with the SOLID trial:

Transformation took place, the social workers came together from across the city and there were a broad range of priorities and cases moving across different teams, complicated matters. So, the timing of that certainly had an impact.

Teams were functioning, with strategic manager ‘off sick’ or open vacancies, this was reported in three out of the six local authority sites within the lifetime of the study:

I’m not making excuses for the service, but we have had different managers off sick. So, somebody who would have been key and pivotal to doing that, wasn’t here for a number of months, there’s another assistant manager been off, nearly 6 months would have carried that vacancy. So, for them, this has just been, I guess not as high priority as it should have been.

The pressure of preparing for, and enduring, Ofsted inspections was also experienced within two of the sites, this resulted in ‘core business’ being prioritised and all other commitments being suspended. There was an acknowledgement that the research is just another task to be undertaken ‘on top’ of and alongside the other commitments that teams have:

It was another ask on top of all the other asks . . . we will do our best to comply with it and as I said, it did come in at a difficult time for the service, both in terms of Ofsted and numbers of staff we have in post.

The pressures were also present for staff within the drug and alcohol services who experienced their services going through the retendering process. Four out of the six services were retendered or were preparing for the retendering process within the lifetime of this study. The potential disruption of these service tendering processes to the conduct and completion of research is an important consideration. Retendering brought additional pressures to realign with the expectations of a new service:

I came back to the workplace and, specifically for me, with the change of two buildings, that’s come in due to being TUPED [Transfer of Undertakings (Protection of Employment) Regulations] over to a new service, it was kind of just forgotten about, if I’m completely honest.

The pressure to complete all of their core business while being understaffed (two services had staff vacancies while taking part in this study) made it harder for practitioners to fully buy into and support the SOLID trial than they had first envisaged:

As I said before, time constraints, travel issues and our standard case load was impacting upon all of this so to us from a practitioner’s point of view, it became more of an onerous task.

In addition to the professional social workers’ and drug and alcohol workers’ views reported above, the two clinical trainers/supervisors acknowledged the pressures that had an impact on the study:

Services being changed dramatically and then being on a trial and people’s workloads being changed . . . the whole thing is just a bit precarious, I suppose, and you’re trying to get something going and doing everything you can. But there’s a fragility in the system . . .

Young people will over- and under-report their substance use

Respondents reported the potential for young people to either under- or over-report their substance use:

We know we’ve got some young people that will embellish potential use. We’ve got others that will under report.

I think what we tend to find with a lot of our young people . . . sometimes they overexaggerate, and I think where adults will reduce the amount that they’re taking and try and make it look a bit better. I think some of the young people might say that they’re using more than what they actually have been.

In reality, the question of whether a young person over- or under-reports within the initial screening is in many cases unimportant. What is important is whether or not the young people identify themselves as a ‘substance user’ at any level. The screening data still provide an opportunity to start a conversation with the young person, which could potentially lead on to the provision of further advice, information or therapeutic support.

Was it feasible to retain children in care within the intervention?

Children in care require a faster response time

Professional participants (drug and alcohol practitioners and residential workers) clearly articulated that they felt a barrier to retaining children in care within the interventions was the number of steps in the study process and the potential for a time lapse to occur between the social worker completing the screening with a young person to the researcher visiting to complete baseline, and the eventual step of a referral onto an external drug and alcohol service:

Particularly with young people and again the looked-after children that momentum has to be jumped on quite quickly. I think there has been a bit of a time lapse from where the referral has been completed and the researchers have been able to visit the young person to when it comes to us. Sometimes that has been 3 or 4 weeks. I know that has improved slightly but you have to be shit hot with any young person because a day can be a lifetime. Four weeks could be like a previous life.

There was consensus between the professionals that the whole process of screening, recruitment into the trial and first session of an intervention needs to happen within a week if it is to be successful:

I think it would have to be really trying to at least be within that same week, if at all possible. I know that is quite a high and very quick turnover but I do think it is important to get in there very quick.

A multitude of factors influenced how quickly a young person transitioned from completing the CRAFFT screening form to completing the baseline data collection with a researcher. The completed CRAFFT form had to be returned to the research office and then the researcher had to successfully establish contact with the young person. Following this, the randomisation process had to be undertaken, and the letters had to be generated and sent to the drug and alcohol service before the drug and alcohol practitioner could contact the young person to offer them an initial appointment.

The young person’s perception of need and readiness to change

The participants’ perception of need and their readiness to change were highlighted as significant factors affecting retention in the interventions offered. From the child in care’s perspective, despite consenting to involvement in the trial and agreeing to attend sessions with a drug and alcohol worker in discussions about the research at two separate time points with researchers, when the young people were actually contacted by the drug and alcohol worker they declined appointments. When discussing this with the research team within the process evaluation interviews, young people stated:

. . . there was a lady who called, but I didn’t take her up on it. Just wasn’t necessary.

I didn’t feel as though there was anything wrong with me drinking or anything, because I only had a couple if I was with friends or something.

Many children in care did not identify themselves as requiring any support, feeling that their substance use was not problematic and, therefore, they did not feel they wanted to attend intervention appointments:

I got offered [sessions], if I wanted the drug support or anything. I don’t need any support because I don’t have a problem. I smoke marijuana [cannabis] for pain, I don’t really see how people get addicted to it, to be honest. Maybe I have it, like, just when the pain’s unbearable I have a joint or something like that, but ye’.

Alongside young people reporting that they thought that interventions were not ‘necessary’, a multitude of other factors played a part in young people not attending sessions. Poor mental health was cited as a reason for failing to attend:

Well, as it happened I think I had went into hospital. I went into a mental health hospital just after you had thingy’d because I went a bit **** up. So, I couldn’t go.

Individuals remembered being offered sessions but did not manage to attend:

When you got that phone call, did you actually then take them up on any sessions?

I did, but then I had a bad period of my mental health, and I didn’t end up turning up.

Both drug and alcohol practitioners and children in care highlighted the importance of the individual identifying a need to change prior to attending interventions sessions:

We as a service require consent on the referral, for our generic referrals. That means the young person is consenting to want to have some kind of drug and alcohol intervention. That’s the top and bottom of it. They must identify themselves that there is a problem. What we were getting the [SOLID] referrals is that they didn’t identify that there was a problem. So, that was why the engagement wasn’t very good.

Young people had a good insight into the fact that they had to have a desire to engage with support, rather than attending due to feeling pressurised by someone else:

I know me especially and I know through my friends and that, if people make you do something that you don’t want to do, it’s not going to help.

Tom (young person, 16 years) articulated his thoughts about choice and control regarding decisions being made:

My social worker kept saying, ‘Do you want to be recommended for this service’ and I said, ‘No, I need to start living my life and the only way I’m going to do that is if you leave me alone’.

A number of young people acknowledge that making changes is hard and individuals need to be motivated. One individual captures this succinctly:

I think anyone who goes to therapy is quite a strong person. Just for the simple fact that’s what I’m going through now. It’s quite difficult. It’s a difficult thing to do. So, if you’re willing to go through therapy then you’re willing to change yourself.

From the research team perspective, this created an anomaly, as in reality young people had consented twice to attend intervention sessions prior to a referral being made to drug and alcohol services.

The process evaluation interviews explored whether or not opportunistic interventions would have been more effective. This may have helped to be more responsive to the young person’s needs.

The importance of contextual information: why children in care use substances

The majority of professionals displayed a pragmatic rationale as to why young people used substances:

They use them [substances], for lots of different reasons. It could be to block out what’s happened to them so actually they don’t really want to stop it, thank you very much, because it’s the only thing that they can manage with right now. To them it isn’t a problem. It is a strategy I guess to manage their day. Yes, there might be all those risky behaviours, all that stuff going on but it’s the only crutch they’ve got right now. You take that away, they’ve got nothing.

It was identified that if the young person had been exposed to substance use and it is recognised as ‘normalised’ behaviour, then they do not identify that their own use is problematic:

If they’ve certainly come from families where using substances isn’t necessarily an automatic negative. It’s part of their self-identity and that’s what they do with their friends. That’s how they enjoy their free time. It’s not always something that they consider anything that needs to be worked on.

Up until now, they don’t see alcohol and drugs problematic. They think it’s a part of life. They’ve seen it with their own families, and that’s part of their living.

There was the recognition from the professionals working in supported accommodation that until young people perceived their substance use as problematic, then it was unlikely they would do anything to change their behaviour:

At work you see a lot of people that, they think they don’t have a problem, and until they admit that they have there’s not really any helping them is there? You can’t do a lot until they admit it, in my eyes, my opinion.

Are the interventions acceptable to drug and alcohol practitioners?

One of the main aims of the process evaluation was to assess whether or not the principles and concepts of the interventions were acceptable to practitioners. The feasibility of intervention delivery also took into consideration the responses to the MET and SBNT training and supervision of the practitioners. One of the trainers stated:

By and large, these people came along with positive attitudes to training . . . as a trainer I want to motivate these people to practice in a particular way, because it’s good for them. It’s good for their practice and it’s good for their CVs [curricula vitae], and it’s good professional experience for them. I think, in the main, they were very open to it.

However, the consistency with which practitioners engaged with the clinical supervision was variable:

It was very mixed. There were a few highlights of people really doing hands-on work, and that was good. Then that went to another level where people aren’t doing it. But there were a handful of therapists that really ran with the idea and did really decent work.

Commitment to clinical supervision was difficult to gauge, as some drug and alcohol practitioners elected not to attend due to not delivering interventions to children in care as part of the study trial. They therefore deemed the clinical supervision sessions to be unnecessary.

How could future interventions be delivered differently?

Children in care emphasised the importance of flexibility when providing interventions, with traditional ‘talking therapy’ considered overly rigid and therefore hard to engage with. Multiple young people reinforced this belief:

I don’t like talking, it’s very easy to write it down on a piece of paper, than it is to say it, do you know what I mean?

I don’t know, it’s just hard to talk to people about saying, ‘Look, this is the problem now’, it’s hard to say isn’t it?

Instead, children in care and practitioners reported that methods which sought to engage participants in creative methods of working may be useful. It was equally acknowledged that writing and other creative methods of engagement might not suit everyone:

I think it’s each to their own, isn’t it? Some people would rather just sit down, talk one-to-one, get it done and that, and some people would rather show themselves in a different way.

In addition, alongside considering the content of the sessions, professionals thought that the low uptake of young people attending interventions could be attributed to the fact that it meant ‘another’ professional being involved in the care package for children in care. Discussion took place regarding amendments to the intervention delivery if the study was replicated. The main suggestion was that interventions should be delivered by a professional already involved in the young person’s care. However, the implication for this is that social workers, foster carers or residential care home workers may be expected to be involved in the delivery of interventions as key professionals involved in supporting children in care:

These children who have been taken into care have seen so many different people. They’ve had social workers and they’ve had foster carers, so they’ve been in court and they’ve been in trouble and all those kinds of things. You’re adding another person into the mix. What you probably want to think about is, ‘Who are all the people who are already in the mix by virtue of the legal requirements, which of those can we train to do this work?’

This prompted the research team to speak to professionals involved in children’s care and to explore whether or not they felt that delivering MET and SBNT sessions with a drug and alcohol focus was part of their remit, and whether or not they felt in a position to deliver the interventions.

Who is the most appropriate person to deliver the interventions?

None of the residential children’s home staff involved in the focus groups (n = 17) felt that the intervention delivery should become part of their role. They perceived that doing so could damage the relationships they had with children in care:

I think it would be more damaging for the young people, yes, definitely.

. . . because, ‘Oh God, they’re going to put a block on his finances. His risk assessment is going to go up dead high. They’re not going to let him stay at his friend’s house’.

Workers, within and across the focus groups with residential workers, identified that there is an inherent care–control dichotomy within their role with children in care. This creates a conflict wherein knowledge of the child in care’s substance use might mean that they would have to implement sanctions and this could, in turn, result in previous supportive relationships being dissolved:

I think you could alienate and might potentially damage our relationships if we went at it in the wrong way. It’s got to be an offer, not a requirement. It can’t be sanction-able.

It’s opt in rather than opt out.

They’re not going to be in any trouble. As a carer, I think if we could do that because otherwise it’s going to damage your relationship.

The desire to uphold supportive relationships meant that residential workers did not feel that they should be the professional to deliver specialist drug and alcohol sessions:

You’re also risking jeopardising your relationship if you’re coming in at a naggy level. They might pull away from you and then you’ve lost that link.

Furthermore, from the residential workers’ perspective, social workers did not have capacity to deliver specialist interventions, making it unrealistic to incorporate them into their current workload:

We get a statutory visit once a month and it’s about making that as much quality time as they can. They’ve got a number of placement issues to talk about and family issues. I think, actually, the expectation that they would be able to do that bit of work is, firstly, unrealistic and slightly unhelpful, if we’re talking about the need, the quality time, because their caseloads are ridiculous. They’re too diluted.

Social workers themselves agreed that they did not have the time, capacity or specialist knowledge to deliver the interventions. Rather, they believed that intervention delivery should be by a specialist drug and alcohol worker (as it was in this study) to give the work the credibility it deserved:

You need to be more specialised.

I think there’s that much more on the street now, and it’s changing.

You’d have to do training all the time . . . would you ask your social workers to do the mental health input? Not in a million years.

Exactly, its very specialist, isn’t it?

It’s very specialist.

It’s more intense than what we can do.

You’re not giving it the importance it has if you just make it up.

Carers (residential and foster) and social workers unanimously agreed that they lacked the level of specialist knowledge required to deliver an intervention regarding substance use. There was consensus within and across the focus groups conducted with social workers, foster carers and residential children’s home staff that it was not within their remit to deliver interventions regarding alcohol and drug use. A potential solution that was suggested by carers was for drug and alcohol practitioners to have regular sessions within care homes and/or within the social work teams, so that they become familiar to the young people. This, it was felt, would then enable the drug and alcohol worker to screen children in care and deliver interventions when necessary. One local authority area stated that this had happened previously and had been successful:

There have been services in the past which came here. They didn’t come here because any young person had a particular issue, they came here to be another friendly face, someone that isn’t on the staff team, so isn’t sanctioning. They’re not taking part of your money off you, they’re not challenging your behaviours. They’re just someone that comes from outside once a week, has tea, has a laugh, and then gradually gets into the C-Card, the sexual health, the drug and alcohol. Obviously that service isn’t available now, but when it was available we found it absolutely brilliant because the kids could go off one-to-one and they could open up about why they’re doing what they do.

It was believed that the current system was not as effective as it should be. The benefit of having specialist drug and alcohol workers was acknowledged; however, it was felt that the worker needed to be more of a familiar face to children in care:

Personally, I think it should be a more familiar, I even think as far as a specific person within the home setting more trained up. Or, for example, we’ve got three homes, even that one person between the three maybe. I mean, I don’t know what the answers are, but I do think that it definitely doesn’t work as it is now, as it proved.

Normalisation process theory analysis

Alongside the thematic analysis undertaken to identify the overarching themes (identified above) within the data across all participant groups, we applied normalisation process theory113 analysis to the data collected from drug and alcohol practitioners regarding the interventions. The findings identify how practitioners responded to the four core components of coherence, cognitive participation, collective action and reflexive monitoring.

Coherence

The drug and alcohol practitioners displayed coherence regarding the main aims and logic behind the interventions. Practitioners could clearly articulate the approaches that they were using when referring to SBNT. Examples practitioners provided were as follows:

The SBNT approach is about that young person finding a wider network of support for them. So, it doesn’t necessarily need to be one person, it could be various different people but it’s about using them to their advantage, to support their recovery.

My understanding was that it was getting the young person to identify a network of support that they could draw upon when they were feeling times of temptation or when they were feeling pressured into potentially making decisions and to hopefully get positive feedback and allowing them to make a rational decision based on the information that they were getting off that person or that group of people to change what they were potentially going to get drawn into.

Similarly, for MET, practitioners could identify the aim of the intervention:

In a nutshell, you are, obviously, trying to motivate the young people to change, but you are trying to bring out what they already want to change. So, it is about them . . . you know, they want the best for themselves, and it’s about bringing that out, I suppose, through motivational interviewing.

Cognitive participation

The practitioners expressed a shared agreement of what the interventions were trying to achieve and ‘bought into’ the aims. Drug and alcohol practitioners reported using the interventions with other young people already on their caseloads who were not participating in the study. The SBNT approach especially created positive feedback:

The concept of it all, again, is really good. You can’t deny, on paper, it’s good, it really is . . . Because everything’s in place for it to be really good, a successful thing.

The element of using the SBNT approach to help to explore support networks was welcomed. The approach provided a tool to instigate discussions about other people surrounding young people who could provide support when attempting behaviour changes:

I think it’s so important and I still use it now but I think that it’s so important that it is very time-limited, the capacity that professionals are able to work with somebody for and especially those within the ‘looked-after’ remit is that sometimes they aren’t able to identify anybody really, that they’re able to go towards. So, I think that by going in the early days and getting them to start thinking about who else they’ve got around them, who’s a positive influence, who they’re able to really communicate and talk to, more than anything really, is such a powerful and positive factor for someone.

In addition, creating the support network map prompted practitioners to have conversations regarding support networks and to explore the rationale for young people identifying sources of support, including those, who, on the face of it, would not be perceived as ‘positive’:

It’s a useful tool to be able to open up that dialogue and exploration.

Finally, the network map provided an opportunity for practitioners to challenge young people regarding their sources of support and to try to determine whether or not identified support networks could indeed help them to reach the end goals:

I thought there was some really good parts of it, in terms of picking out that supportive person. That is, as part of my assessment, I would ask the person to map out who was a good in their live. Bad influences, good influences, things like that. That part was really good. The practicalities, I think, of doing it on the ground, I didn’t think were going to be good. I didn’t think it was going to be practical, basically. Getting the supportive person in. I think we had discussions around the start, that supportive person could well be the dealer. It could well be the . . . they could identify anyway, that could be their dealer, or friends that they use with. That could be their only support network that they can identify. That was my first concerns around it. That, kind of went on, and in the end, I did think, actually, what’s good about this is, we can draw upon, what they are saying about their supportive person. Challenge that as well. Are they supportive, in what your goals are and where you are wanting to be?

Alongside the above positive views and experiences of SBNT, practitioners did also raise concerns regarding using the approach with children in care, who may struggle to identify sources of support:

I mean looking at SBNT with young people in care, personally, I don’t think it’s particularly effective in so far as those young people who are in care, in my experience and looking at the ones who we’ve got through, didn’t necessarily have that network to be able to draw upon. So that network of support was very, very minimal. One of the girls, she used to speak a lot to her 8-year-old nephew and it was 8-year-old nephew or mum, that was literally it. Others, it was key worker in the home. So from my point of view, we had our basic premise of what the programme should look like and the amount of sessions and stuff like that and really, you wouldn’t need all of those sessions for them to establish a network because their network didn’t necessarily exist, other than within that unit.

In addition, concerns were raised regarding the longevity of any support networks that were identified:

How do you identify the most positive support network? Original SBNT is very clearly about equality of relationships. When you’re with young people and then they start saying, ‘Oh, my school teacher, my YOT [youth offending team] worker’. You’re talking about . . . relationships of lasting length because your school teacher is your school teacher for this year.

Collective action

Practitioners reported feeling supported by their managers when trying to integrate the interventions into standard practice, they could access supervision regarding young people referred in through the study:

He [drug and alcohol manager] was really supportive and sometimes when I came back and I’d say, oh I’ve got a referral here I’m going to go to see this young person and when I came back I could discuss it with him definitely and he was very supportive of it.

Equally, practitioners felt that managers supported them to use the interventions with young people accessing the services outside of the SOLID trial, due to the intervention’s perceived effectiveness:

I mean now when I’m using it now, my manager is very, very supportive of it and is very interested in it as well and so we discussed that and she says to continue using it because it is very, very effective.

Managers also expressed being supportive of their staff and encouraging them to use the interventions within their standard work to benefit the young people accessing specialist services:

As a group, I suppose Adam [drug and alcohol worker] has experience of SBNT so I don’t think that there is any argument from us that, actually, as an approach, it’s really, really helpful.

Reflexive monitoring

The majority of the drug and alcohol practitioners who delivered the intervention to young people outside the study, reported that they did not think that six sessions of MET or SBNT would be enough to meet the needs of children in care, if a young person actually chose to engage with the intervention. The main belief behind this was the experience that practitioners had of initially engaging a young person in treatment services:

I know, myself, that I’ve sometimes had two or three sessions, before I’ve even been able to do an assessment. Because of that fire fighting, because of that building the relationship with kids who are just not interested. Trying to motivate them to get involved. By that point with SBNT, you’d have done half your intervention.

This was confirmed by another practitioner, who stated that the principles of the intervention and the resources available were helpful to motivate a young person, but that they did not think six sessions of MET as a standalone treatment option were sufficient:

All the techniques and the workbook, the worksheets . . . it could help move people on towards their treatment goals. I just think in all the children’s and young people’s services, to just do this or to have a certain client group that only receive this, it’s too restrictive for what young people need. Because a lot of them could end up coming out of it and going into more of the usual care. The techniques and the aim of it and the feel of it, is really positive. It is a good refresher. But it couldn’t be just exclusively MET.

The qualitative data identified that a number of elements in the study process were not deemed acceptable; however, potential solutions were also offered to make the work possible in a future study.

Quantitative data collection

Audio-recording methods

The drug and alcohol practitioners delivering the SBNT and MET intervention were requested to audio-record every session that they delivered, with consent from the participant. Following each audio-recorded session, the practitioners were requested to clearly label the session as outlined in the treatment protocol (session X, date 01.01.2019, young person’s initials) and upload it on to a secure server so that it was accessible to the research team and the clinical supervisors. The purpose of the audio-recordings was to assess the quality of the intervention delivery (treatment fidelity).

The research team (HA and RB) applied the validated UKATT Process Rating Scale (PRS), which was developed in the UKATT.114 The UKATT PRS is applicable to MET and SBNT and consists of 26 items, rated as follows:

1. maintaining structure
2. agenda setting
3. explanation of philosophy of treatment/treatment session
4. review intersession change
5. consistency of problem focus
6. end of session summary
7. homework
8. drinking feedback/negative consequences
9. alternative activities to drinking
10. eliciting client concern about drinking
11. social support for change
12. eliciting self-efficacy for change
13. involvement of others in behaviour change
14. commitment to drinking goal
15. identifying sources of support for change
16. ambivalence
17. creating conflict
18. eliciting commitment to change drinking
19. eliciting optimism for change
20. therapist as task orientated
21. therapist as active agent for change
22. reflective listening
23. collaboration
24. interpersonal focus
25. exploration of feeling
26. empathy.

Each item was rated on the extent to which the practitioners carried out each specific item and the quality of the therapist’s behaviour, using the following scales:

To what extent did the therapist perform the behaviour within each item?

How well did the therapist perform the behaviour within each item?

The research team assessed the extent to which the intervention delivery was true to the therapeutic principles on which it was based.

Audio-recording findings

In total, 26 sessions were delivered (MET n = 17, SBNT n = 9) and of these, practitioners only completed nine recordings in total (MET n = 3, SBNT n = 6). The small number of recorded interventions significantly limited our ability to assess the internal validity. Out of the 14 practitioners involved in the trial, only two practitioners submitted more than one recording for the same young person, two did not submit more than one recording for the same young person and 10 practitioners did not submit any recordings at all. (One practitioner did successfully manage to deliver all six sessions of MET; however, the young person declined to be audio-recorded.) Owing to the small number of findings, no conclusions can be drawn from these data. Further details are provided in Appendix 5 following the UKATT PRS.

Conclusion

Many participants in the study did not identify themselves as having a problem with drugs and alcohol, despite reporting using multiple substances. This could be due to their previous exposure to substance use and its normalisation within this population group. The qualitative process evaluation clearly showed that in a future trial the screening tool used needs to capture a young person’s substance use within the last month, to more accurately reflect their current use of alcohol and/or drugs, and the tool needs to be able to differentiate between recreational and problematic/excessive substance use. If the screening tool can establish that use is current and problematic, it would provide a seamless intervention window, linked with feeding back the screening results, even if the content of discussions is based on harm reduction rather than striving for an abstinence-based goal. An intervention accurately offered to young people who are current users may help to alleviate the problem experienced within this study of individuals having a diminished perception of need or readiness to change.

The trial implementation process, using existing referral systems, did not facilitate or encourage engagement of children in care throughout the study process. The difficulties of screening children in care, combined with the multiple steps and time lapses within the study process, resulted in few young people reaching services. However, multiple problems that were identified, such as how problematic the screening was perceived to be by social workers, the common problem of balancing real-world barriers (such as Ofsted) and competing demands of the social work role and the consistently reported need for faster response times throughout the study process, could all be addressed by having an embedded researcher or named practitioner to oversee the entire study process. A single point of contact to act as the ‘face’ of the research project, to screen, recruit, complete data collection and support each individual to attend the first intervention session would create an integrated service model and the necessary consistency to make the process feasible within a stretched system.

In conclusion, although the different professional groups understood the principles of the SBNT and MET approaches, were enthusiastic about the approaches within the formative work and had a shared agreement of what the interventions were trying to achieve, the study has also shown that, unless the interventions can be delivered in such a way that they dovetail into the often complex lives of children in care and the drug and alcohol practitioners’ workload, they are unlikely to become part of standard practice.