Quantitative results

Parts of this chapter have been reproduced from Gooding et al.69 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) licence, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by/4.0/. The text below includes minor additions and formatting changes to the original text.

The most important findings pertain to the efficacy, mechanisms and clinical outcomes of the CARMS RCT. However, a prelude to those findings is necessary with descriptions of (1) the flow of participants through the two arms of the CARMS RCT (therapy plus TAU vs. TAU alone) at the three assessment time points of baseline, 6 and 12 months, (2) breaches of protocol, including breaches where the RCT group became known to CARMS staff, (3) SAEs, (4) modes of conducting consent procedures and assessments, (5) therapy process measures, (6) participant characteristics and (7) baseline levels of primary and secondary outcome measures, mechanism measures and clinically relevant measures.

Participant flow

Randomisation into the baseline phase of the CARMS RCT was from 21 June 2017 to 25 November 2020, with the final 6-month assessment occurring in July 2022 and the final 12-month assessment completed on 10 January 2022. See Figure 1 for the CARMS CONSORT diagram depicting participant flow through the trial. No participants were lost from the trial due to consent withdrawal, loss of capacity or change of diagnosis.

FIGURE 1 Consolidated Standards of Reporting Trials diagram showing flow of participants through the CARMS RCT from referrals to 6- and 12-month follow-ups. Reproduced with permission from Gooding et al.69 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) licence, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by/4.0/. The figure includes minor additions and formatting changes to the original text. (Figure not reproduced here; see the original article.)

Flow of participants

Referrals and screening phases

Over 1000 potential participants were referred into the CARMS trial (see Figure 1). Just under half of referrals were lost prior to the care co-ordinator being able to establish consent-to-contact. A high proportion (over 75%) of potential participants who did provide consent-to-be-contacted progressed to the CARMS screening procedures, with over 80% of those individuals being confirmed as eligible to participate.

Baseline, 6- and 12-month follow-up phases

Two hundred and ninety-two participants were randomised into the baseline phase of the RCT overall, with 123 coming from GMMH, the largest participating NHS Trust; and 71, 84 and 14 coming from LSCFT and PCFT, respectively; and 5 from NWBHT. Retention was estimated at 75% in order to achieve a target sample size of 250, with the primary suicide outcome measure (ASIQ) being assessed at 6 months. Retention rates were high. The overall percentage of participants who had completed the ASIQ at 6 and 12 months was 86.3% (252/292) and 75.3% (220/292), respectively. The most frequent reason for being lost to follow-up was that the CARMS team was unable to progress with the participant due to a range of logistical reasons. For example, the participant did not attend appointments multiple times despite reminders; contact details for participants were, or became, invalid; participants moved out of area or left the UK; mental health teams could not provide risk information for participants (see Figure 1). If it was not possible to complete assessments 4 months past the follow-up deadline, then that participant was considered lost.

Breaches of protocol

There were seven breaches of protocol, all of which were discussed with the sponsor, the TSC Chair and the relevant REC Chair. The first three breaches involved three participants being given the wrong consent form to sign. Two were given consent forms for a qualitative workstream to sign rather than the RCT, and one participant was given a consent form for a qualitative workstream, but it was not the correct workstream. The fourth and fifth protocol breaches also involved consent forms, with two participants being given old versions of a consent form to sign. These breaches were resolved in a straightforward manner by informing the participants’ care co-ordinators and repeating the consent procedures with the correct forms. The sixth breach involved a participant who disclosed that they had been discharged from mental health services after signing the consent form and prior to starting assessments. Having established that relevant risk information could be provided by, and passed back to, this participant’s general practitioner, the decision was made to continue the assessments with this participant. The seventh breach comprised an issue of mistaken identity for one participant who was contacted in error. However, there were no consequent concerns raised by the participant once the situation had been explained.

Masking of allocated RCT group: one of the co-PIs, the trial statistician, RAs conducting the assessments for the RCT, the therapists delivering the intervention and those responsible for data handling procedures in the MCTU were masked with respect to participant allocation. The only breaches involved the CARMS RAs. All but two of these incidents were successfully handled by bespoke CARMS SOPs.1 Specifically, if a RA learnt of the trial allocation of a participant, then a different RA was immediately assigned to take over, completing assessments with that participant (see Protecting against detection bias). This meant that the assessments for just two participants, one in the 6-month phase and one in the 12-month phase, were conducted unmasked.

Serious adverse events over the course of Cognitive AppRoaches to coMbatting Suicidality

Serious adverse events can be problematic to evaluate in psychological intervention trials for numerous reasons.130 Two specific problems are that a small proportion of participants can have a very high frequency of such events, making the data difficult to interpret, and there is a higher probability that participants in the treatment group disclose SAEs during therapy sessions, meaning that there is a potential bias towards a higher frequency of reporting in the treatment group.130 Table 2 shows the different categories for SAEs for the treatment and control groups, both in terms of number of events recorded and number of different participants reporting those events.

As might be predicted, the overall SAE incidents which were due to self-harm and suicide attempts were detected more frequently in the treatment group compared to the control group. Similarly, SAEs incidents which were life-threatening, which were considered mild or moderate and which were expected were detected more frequently in the treatment, compared to the control, group. However, the number of actual participants experiencing SAEs in these categories was approximately equivalent in each group (with the exception of self-harm). No SAE was considered to be related to the assessments, nor to have occurred as a direct consequence of therapy sessions.

There were four participants who died over the course of the trial. Three participants had been allocated to the TAU group, two of whom died prior to their 6-month follow-up assessment time point, and one of whom died during their 12-month assessment period. One participant had been allocated to the treatment group but had not completed assessments within the allocated 4-month cut-off time window prior to their death, so was, technically, not part of the trial. Inquest records were available for three of the four deaths and indicated that they were considered to be drug related, with none of the deaths being recorded as death by suicide. The post-mortem report for the fourth participant who died concluded that it was a ‘non-suspicious death’, was a sudden unexpected death in epilepsy.

Modes of conducting patient information sheet and consent procedures and carrying out the assessments

Prior to the UK COVID-19 lockdown restrictions (March 2020–December 2021), the PIS and consent procedures, and completion of the assessments were carried out face to face, in person. However, after the lockdown restrictions ensued, these procedures were amended so that they could be carried out remotely. Appropriate governance and ethical approvals were obtained. There were five modes of taking consent used in CARMS, which were (1) in person (most frequent), (2) hard copy mail, (3) audio-recording (4) e-mail and (5) text message (most infrequent). The assessments were carried out either in person or by telephone (see CARMS CONSORT diagram, Figure 1, for details). Completing assessments remotely by video link was offered but not chosen by any participant for the PIS and consent procedures, nor for the assessments.

Therapy process measures

Basic therapy process measures, where possible, included number of sessions attended; therapy time across sessions in minutes; the highest level of CBT formulation attained as rated by the therapists (possible range 1–5); and client and therapist ratings of the therapeutic alliance using the WAI.

Number of therapy sessions

The median session number was 16 (mean = 14.52). Five participants assigned to the treatment group attended no sessions at all.

Time in therapy

The mean total time spent in therapy overall equated to around 12 hours (median = 13 hours). For those who attended at least one therapy session, the average time per session was around 50 minutes (mean = 49.97; SD = 10.05; median = 50.0; minimum = 12 minutes, maximum = 100 minutes).

Level of formulation

Formulation levels of 1 (very low), 2 (low), 3 (medium), 4 (high) and 5 (very high) were assigned by the therapists to their clients/participants, where possible.104 The mean formulation level was 3.74 (SD = 0.95; N = 138), and the median was 4.

Therapeutic alliance

Table 3 shows the therapeutic alliance ratings for the participants (clients) and the therapists. Mean alliance ratings were higher, and significantly so, when made by the participants compared to the therapists at both session 4 [N = 62, mean difference = 8.39, t(61) = 7.23, bootstrapped 95% CI 6.16 to 10.73; p < 0.001] and at the end of therapy [N = 64, mean difference = 8.88, t(63) = 10.86, bootstrapped 95% CI 7.23 to 10.39; p < 0.001]. The highest mean alliance rating was for the participants at the end of therapy.

Participant characteristics and baseline measures overall and across the two groups of the randomised controlled trial

Participant characteristics

Table 4 summarises baseline characteristics of the participants overall and split by treatment (therapy plus TAU) versus control (TAU alone) groups. Participants were largely of White/Caucasian ethnicity, single, with no dependants and not in paid employment. Around a third lived alone. There was an almost equal number of males and females. The most frequent (67.8%) diagnostic classification was for schizophrenia (i.e. F20 of ICD-10). The vast majority of the sample did not indicate receiving any form of psychological therapy at the time of recruitment into the trial. Those who were having therapy had it in a one-to-one format, as opposed to in a group. It was common for this therapy to cease while receiving CARMS therapy. These characteristics appeared similar across the two trial groups.

Outcome measures at baseline

Suicide outcome measures

For all suicide outcome measures, a higher score indicated more severe levels of suicidal thoughts and/or behaviours. As shown in Table 5, the primary suicidal ideation outcome measure of the ASIQ was similar across both trial groups at baseline. Secondary suicide outcome questionnaire measures of suicidal ideation (BSSI) and SPS were also comparable across the two trial groups at baseline. The diary self-report measure of suicidal thoughts and behaviours (TLFB) in the last 6 months appeared equivalent across treatment and control groups for frequency of suicide attempts (TLFB A), frequency of plans (TLFB C) and frequency of thoughts, which resulted in speaking with a mental health professional and/or mental health charity (TLFB D).

Mechanism outcome measures

A higher score on the mechanistic outcome measures indicates more negative appraisals or perceptions. Similar to the suicide outcome measures, mean scores for the mechanistic outcome measures of appraisals of emotional difficulties (DERS), problem-solving (SPSI), social support, defeat, entrapment and hopelessness were comparable across the two trial groups at baseline (see Table 5).

Other mental health symptom outcome measures

As with the suicide and mechanistic outcome measures, mean levels of depression, positive psychotic symptoms, negative psychotic symptoms, general symptoms of mental health problems, and distress associated with hallucinations and delusions were comparable across both trial groups at baseline (see Table 5).

Efficacy for primary and secondary suicide outcomes

Primary suicide outcome measure

As can be seen from Table 5 and Figure 2, there was a decrease in mean severity of suicidal ideation ASIQ scores at the critical 6-month follow-up time point compared to baseline overall, with that difference being greater for the treatment group than for the control group. However, the difference in suicidal ideation severity between the two trial groups at 6 months was not statistically significant [adjusted difference –7.36, (SE 4.10), p = 0.07; see Table 5]. At the 12-month follow-up time point, mean ASIQ scores appeared similar for the two trial groups, and also similar to ideation levels in the treatment group at 6 months. Indeed, there was no significant difference in ASIQ scores for the RCT groups at 12 months.

FIGURE 2 Mean primary suicidal ideation outcome scores (ASIQ) at baseline and 6- and 12-month time points for participants randomly allocated to the treatment (therapy plus TAU) and control (TAU alone) groups. Reproduced from Gooding et al.69 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by/4.0/. The figure includes minor additions and formatting changes to the original text. (Figure not reproduced here; see the original article.)

Secondary suicide outcome measures

Suicidal ideation measured by the BSSI, and suicide probability measured by the SPS, decreased across both 6 and 12 months relative to baseline, but differential decreases for those in the treatment compared to the control groups did not approach significance. Similarly, there were no significant differences between trial groups for reductions in self-harm, suicide attempts, plans or ideation assessed by the four TLFB diary measures across either the 6- or 12-month follow-up time points (see Table 5).

Compliance-adjusted analyses of the primary suicide outcome measure

The effect of compliance was modelled in two ways. The first modelling approach systematically excluded participants who completed no therapy sessions by 6 months. When this was done, the difference between the treatment and control groups was not significant [adjusted difference −7.13 (SE 3.68), 95% CI −0.39 to 0.00; p = 0.052]. The second modelling approach estimated the extent to which ‘dose’ of therapy, defined as session number, was related to reductions in suicidal ideation severity. An instrumental modelling approach was used, with randomisation as an instrument. Participants who were assigned to the treatment group but had zero sessions were included in the analysis. At 6 months, each additional session of therapy reduced the ASIQ score by 0.46 by 0.46 points, but this effect was not significant (95% CI −0.94 to 0.01, p = 0.056).

Mechanistic outcomes: differences between baseline and 6- and 12-month follow-up time points across the treatment and control groups

While negative appraisals of emotional processing and interpersonal problem-solving (total scores and subscale scores for the five different components of problem-solving) did appear to improve (i.e. become less negative) from baseline to 6 months overall, this improvement was not significantly greater for the treatment compared to the control groups. The same pattern of findings was evident for perceptions of being defeated, trapped and hopeless. However, when negative appraisals of social support were examined, there was a drop from baseline to 6 months, indicative of less negative perceptions, which was significantly greater (p = 0.048) in the treatment group compared to the control group. Across 12 months, there were no significant differential improvements in any of these six negative appraisals for the treatment compared to the control groups (see Table 5).

Mechanistic outcomes: mediation analyses

Mediation regression models (Table 6) were tested, with RCT group allocation (treatment vs. control) as the predictor variable, suicidal ideation severity (ASIQ) at 6 months as the outcome variable, and the six measures of negative appraisals (emotional difficulties, problem-solving, social support, defeat, entrapment, hopelessness) at 6 months as mediators.

There was a significant mediation effect for appraisals of social support (see Table 6). The direct pathway (c) between allocated group (treatment vs. control) and suicidal ideation at 6 months was not significant. The pathway (path a) between allocated group and social support was significant because SS-A at 6 months were more positive and less negative in the treatment compared to the control group. Path b was also significant because perceptions of social support at 6 months were significantly associated with suicidal ideation severity at 6 months. Of more importance, the indirect effect was significant, meaning that those in the treatment group had less negative SS-A at 6 months, which, in turn, was associated with reductions in the severity of suicidal ideation at 6 months (Figure 3). The proportion mediated was 38%.

FIGURE 3 A significant indirect effect of random allocation to treatment or control group on suicidal ideation (ASIQ) via SS-A at 6 months. The treatment effect operated on suicidal ideation severity by improving appraisals of poor social support.69 Reproduced from Gooding et al.69 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) licence, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by/4.0/. The figure includes minor additions and formatting changes to the original text. (Figure not reproduced here; see the original article.)

Other mental health outcomes

Although scores on measures of depression, psychotic symptoms (PANSS positive and negative), general symptoms of mental health (PANSS general) and distress from psychotic symptoms dropped from baseline to follow-up time points, there was no differential improvement observed for the treatment group compared to the control group at either 6 or 12 months. However, a self-report measure of harm due to drug use (DAST) showed a decrease across 6 months in the CARMS treatment group but no reduction in the TAU group, a difference which was significant (see Table 6).

Health economics outcomes: exploratory

Resource costs and QALYs are summarised by treatment group in Table 7. There were no significant differences for either costs or QALYs between the two RCT groups (therapy plus TAU vs. TAU alone). That has to be interpreted in the context that health economics analyses were exploratory in the CARMS project, and were also not sufficiently powered to detect QALY differences. Furthermore, the wide CIs attest to the wide range of health and mental health resources accessed by participants.