Results

Trial results

Participants were recruited to the study from NHS services in Northern England and Scotland. The first participant for the URB trial was randomised on 13 November 2018. The final participant was randomised on 15 September 2020. An internal pilot phase was conducted for 9 months after the commencement of recruitment (halfway through the planned 18-month recruitment phase). See figure 1 for recruitment progress. Due to the COVID-19 pandemic, several changes had to be made to the trial to ensure that recruitment and delivery of the interventions could continue. Recruitment was extended from 30 January 2020 to 31 May 2020 to allow as many participants to be consented as possible within the time frame. A further costed extension of 8 months was granted in November 2021 to allow for as much follow-up data to be collected as possible. The amended trial end date was 31 March 2022.

Potential participants were identified through the three clinical research sites, as well as through PICs. Clinicians were provided with information on the URB study and asked to introduce the study to families. This may have been in person at a clinic appointment, in a community setting/school, via telephone or by letter/e-mail. Clinicians gave potential families copies of the information sheets and an EoI form to be returned by the clinician or the family to the research team using the stamped addressed envelope provided. Families were also invited to take part through research databases such as ASD-UK. The local CRN at each site assisted with recruitment.

In total, 1829 potential participants were invited to participate into the trial. Please see figure 2 for CONSORT flow chart. Of these, 1563 declined and of these 422 potential participants reported their reasons for declining; 105 were not interested after further discussion, 41 did not consent and 14 were deemed not eligible. The main reasons for potential participants not being interested in the trial were due to other commitments such as child care, lack of time, financial issues and travel arrangements. These in combination accounted for 69 of the potential participants who declined. A further 11 potential participants did not consent following further information due to these factors. Other reasons recorded included concerns with a child participating in the study, no defined RRBs, child not living with the parent and a translator needed. There were several other factors that only affected one or two potential participants. Regarding the four participants deemed ineligible, this was due to the age range falling outside of the prespecified window.

Of those who were eligible, 227 were randomised. Of these, 113 were randomised to LAA and 114 were randomised to URB. As a consequence of missing data, 155 families have usable data at 24 weeks for the primary end point (see Appendix 2, Table 24).

FIGURE 1 Recruitment showing actual consented, randomised and projected participants for the overall trial alongside the pilot trial which was given the green light in August 2019 to progress through to the main trial. Due to COVID-19 restrictions, Lothian stopped recruitment in March 2020. (Figure not reproduced here; see the original article.)

FIGURE 2 The CONSORT diagram and the recruitment and retention to the trial. Reproduced from Grahame et al.1 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by/4.0/. The figure includes minor additions and formatting changes to the original text. (Figure not reproduced here; see the original article.)

Baseline data

Child characteristics

The summary of demographic variables used in the minimisation scheme for the randomised families (total = 227) are as follows:

The majority of children were from white ethnic backgrounds (85.9% vs. 14.1%). A breakdown of ethnicity by treatment arms showed that there were 85.84% in the LAA arm and 85.96% in the URB arm from white ethnic backgrounds. The largest age category was 3–5 years old (47.58%) whereas the smallest age category was 8–9 years (15.42%). Similarly, in both arms the majority were from the 3–5 years old age category (LAA, 46.9%; URB, 48.25%), whereas there were 16.81% in the LAA arm and 14.04% in the URB arm from the 8–9 years old category. In terms of gender, there were more males (79.74%) than females (20.26%) in the data set (males; LAA, 80.53%, URB, 78.95%). The families in the two trial arms were balanced on the variables (child’s ethnicity, age and gender) that were used in the minimisation scheme.

A summary of all the baseline measures at child level are provided in Table 4, including descriptive statistics on categorical demographic variables and baseline child scores.

Table 4 shows that children from a white ethnic background are in the majority. Six children had missing information on ethnicity (three in each trial arm) and four children (two in each arm) were unclassified. The mean baseline age in the LAA arm was 6.23 years while the mean age in the URB arm was 6.16 with variability slightly higher in the former than the latter. The values of the mean total scores for SRS-2 and VABS-3 were similar in both trial arms.

From March 2020 onwards, ADOS-2 assessments were not able to be undertaken with children once NHS COVID-19 restrictions were implemented. ADOS-2 scores were only obtained for two-thirds of the participants (155/227) and may therefore not be clinically representative of the full sample. The ADOS-2 scores are presented in the appendix (see Appendix 1, Table 23). The values of SRS-2 scores, 83.9 and 82.83 in the LAA and URB, arms respectively, are interpreted as severe and strongly associated with clinical diagnosis of autism (see Baseline characterisation and outcome measures).

The values of VABS-3 ABC scores were 67.4 and 67.85 in the LAA and URB arms, respectively, and fall within the low range of adaptive functioning, indicating difficulties with adaptive behaviour/daily living skills that are consistent with a clinical sample of autistic children. For comparison to a wider sample, based on standardisation data, a mean score of 63.3 was reported for autistic children aged 3–8 with a concurrent intellectual disability and a mean score of 76.8 was reported for autistic children aged 3–8 without a concurrent intellectual disability. The range of scores reported for the SRS-2 (57–90) and the VABS-3 ABC (42–111) suggests a sample of autistic children with mixed abilities, including children with and without a concurrent intellectual disability.

Furthermore, Table 4 presents the summary of the RBQ-2 baseline. In all baseline measures both trial arms were balanced in terms of available scores and the number of missing scores was similar in both trial arms. All comparisons here are exploratory with no formal statistical test. In general, the missing data pattern is similar in both trial arms.

Household and baseline characteristics of parent/carer

Table 5 shows the summary of child’s household relations and the distribution of the baseline characteristics of the parent/carer. Data related to the child’s other parent’s baseline characteristics (where applicable) can be found in Appendix 1, Table 22. Consistent with the child-level data, the majority of parents/carers were white. Most of the parents/carers were married. Summary of baseline parent/carer scores and details about educational qualifications and jobs of the parents/carers are also reported. Trial arms were almost balanced across all measures.

Sample size available for analysis at primary end point

As a consequence of missing data required to calculate CGI-I scores (primary outcome) (see Missing data), usable data for the primary outcome analysis at the primary end point (24 weeks) was available from 155 families. Given that the target sample size was for 250 families providing data for analysis (125 per arm), the study collected less than the expected data at the primary end point. The implications of this are noted in the analysis of the primary outcome (see Power consideration). All outcomes are summarised and analysed according to the ITT principle on the basis of the number of randomised families to trial arms. All families included in the analyses received the treatment that they were randomised to.

Attendance and fidelity of delivery of both intervention arms (Learning About Autism and Understanding Repetitive Behaviours)

Attendance at parent group sessions

A summary of attendance at parent group sessions showed that the average number of sessions attended in LAA and URB arms was five out of eight, according to the data provided by the sites.

Clinical Global Impression – Improvement scale inter-rater reliability

For 10% of the sample, the CGI-I were independently rated by a second trained researcher. Fleiss’ Kappa and Cohen’s Kappa were used to assess inter-rater reliability.

Fleiss’ Kappa as an index of inter-rater agreement between two raters was calculated and presented in Table 6. The agreement between raters was moderate by this measure (0.495) and significantly different from zero (p < 0.001). In addition, Cohen’s Kappa showed the agreement between raters was substantial (0.692) and significantly different from zero (p = 0.01). The conclusion from the two measures indicated that there was agreement between the two raters. This gives us confidence in the CGI-I ratings used in the primary analysis.

Fidelity of therapists’ delivery of Learning About Autism and Understanding Repetitive Behaviours intervention sessions

Analysis of fidelity of therapists’ delivery of the intervention to the manual was undertaken for both arms on 10% of randomly selected session recordings assessed by four independent raters. Sessions were selected for rating from face-to-face delivery (pre-COVID-19 restrictions) and online delivery (LAA: 8 face-to-face and 7 remote vs. URB: 9 face-to-face and 6 remote). For both arms the fidelity of delivery of the intervention was excellent: the LAA group overall percentage agreement between the independent raters was 93.6% (κ = 0.846), 92.7% for delivery (κ = 0.897) and 94.5% for content. For the URB group overall percentage agreement was 94.9% (κ = 0.91), 97.2% for delivery (κ = 0.963) and 90.1% for content.

Outcomes of serious adverse events and events of special interests

Among a total of five SAEs, two were hospital admissions due to appendix pain and mouth pain for the child (both in LAA) and one was hospital admission due to COVID-19 for parent/carer (in URB). The two other SAEs were a reported discharge from the hospital after having a heart attack for parent/carer (in URB) and one was a reported meningitis for the child (in URB). More details are reported in Appendix 2, Table 25.

Table 7 presents 24 ESI (see Chapter 2). Of these, 9 were reported within the LAA arm and 15 within the URB arm. The largest number of ESI were reported during the intervention phase of the URB arm (9/24) while six ESI were reported in each treatment arm during the follow-up phase of the study. Parental stress and school difficulties were the most frequently reported ESI across trial arms and three ESI were linked to COVID-19.

Among 24 ESI, at the end of the study 14 were reported as still ongoing, 1 was resolved which was in the URB arm and the status for 9 was unknown.

Among a total of 14 ongoing ESI across 3 sites, 10 were reported in CNTW and 4 in TEWV. The only one resolved ESI was located in CNTW and ESIs with unknown status were equally distributed across sites.

Primary analysis

Power consideration

There were a total of 72/227 (31.72%) data missing at the primary end point. A blinded sample size re-estimation indicated that a minimum of 179 families were needed to retain 80% power and by 224 families, 90% power is achievable. Therefore, in terms of the number of families that were randomised (227), the study has a minimum of 90% power. In terms of actual data available at 24 weeks for the primary end point, a post hoc power calculation based on 155 (227–72 = 155) families contributing to the primary end point at 24 weeks resulted in 70–75% power. This means that with the available data at 24 weeks, the trial collected fewer than the expected data and so is unable to answer, with any certainty, whether or not the URB intervention is statistically more effective compared with the LAA intervention.

Comparing improvement between trial arms according to Clinical Global Impression – Improvement scale at 24 weeks

In this section, the primary analysis is presented – Table 8 shows the estimate of OR, which is the ratio between the odds of improvement in the two arms (URB/LAA). Primary analysis was adjusted for the effects of gender, ethnicity and age.

The result shows that the odds of improvement on average was 39% higher in the URB arm compared to the LAA arm [OR 1.39, 95% confidence interval (CI) 0.52 to 3.69; p = 0.51]. However, the result is not statistically significant and the CI is sufficiently wide enough to contain differences favouring either URB or LAA. The primary analysis represented by risk difference is provided in Appendix 3.

As expected from the analysis, Figure 3 shows the mean probability of improvement in URB and the LAA arms. As noted in Table 8, there is no evidence of any difference between trial arms.

FIGURE 3 Predicted probabilities of improvement in trial arms. Reproduced from Grahame et al.1 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by/4.0/. The figure includes minor additions and formatting changes to the original text. (Figure not reproduced here; see the original article.)

The percentages of (non-)improvement according to CGI-I within the levels of gender, age and ethnicity per trial arms and in the entire sample are presented in Appendix 2, Figures 14–16.

Prespecified secondary analysis

Analyses of target behaviour vignette, improvement and impact on families

Each family provided two separate TBVs at baseline, which were then captured again at follow-up. An expert panel rated the follow-up vignettes in relation to change compared to baseline on a nine-point scale from ‘very much improved’ to ‘disastrously worse’ in relation to the behaviour reported and the impact on the family unit. Ratings of three or less (very much improved, markedly improved, definitely improved) were used to determine a positive response and this was used to define a ‘responder’, and binary variables (responder vs. non-responder) were calculated for the analysis (see Chapter 2, Secondary child outcome measure).

TVB change scores (behaviour and impact, responder vs. non-responder) were compared across the arms at each time point in vignette change scores overall.

Table 9 shows the results of the analysis of change in the TBV at 10, 24 and 52 weeks. Under ‘Positive response in behaviour improvement’, the odds of improvement on average were 92% higher in the URB arm compared to the LAA arm (OR 1.92, 95% CI 1.10 to 3.39; p = 0.02) at 24 weeks (primary end point). Unlike the primary analysis shown in Table 8, this result is statistically significant. Similarly, for ‘Positive response in impact on family’, the URB arm included significantly more responders than the LAA arm at 24 weeks (OR 2.10, 95% CI 1.14 to 3.92; p = 0.02).

Comparison of the change scores on the TBV between trial arms at 10 weeks shows that in the URB arm the participants were more likely to have a ‘Positive response in behaviour improvement’ and ‘Positive response in impact on family’ compared with the LAA arm. It is noteworthy that there was no evidence of any differences between the trial arms at 52 weeks for all the TBV measures.

Secondary analysis on continuous outcomes

Differences between and within treatment arms in secondary outcomes

From Table 10, it can be seen that there are few statistically significant differences between the URB and LAA arms in relation to changes in the child secondary outcomes. A significant difference is found between the URB arm and the LAA on the RBQ-2 Motor Sensory Behaviour subscale score at 10 weeks compared to baseline. However, it is noteworthy that in the LAA arm, the RBQ-2 Motor Sensory Behaviour score on average increased significantly at 10 weeks compared to baseline (+0.09, 95% CI 0.03 to 0.16), which may account for this difference between the arms.

For RBQ Teacher Mean Total score, a significant difference is also noted between the arms at 52 weeks compared to baseline. However, this is most likely accounted for by a decrease in the LAA arm at 52 weeks compared baseline.

Turning to the parent-reported measures, the parent/carer self-efficacy (PSE) score changed significantly at 10 weeks compared to baseline (+0.34, 95% CI 0.09 to 0.6) comparing URB with LAA with evidence of more parental self-efficacy in the URB arm compared with the LAA arm.

There were no other between arm differences at any time point on any of the secondary outcomes.

Changes over time within both arms can be seen in relation to the VABS-3 subscales at 24 weeks indicating some improvement in adaptive functions for children in both arms. It is important to note that, although these changes are statistically significant, they are small and do not represent a clinically meaningful change in adaptive function.

Changes are also apparent in a number of parent measures (PSE, APSI and WEMWBS) and family functioning measured by the AFEQ. The results indicate that parents in both trial arms may have benefited from the interventions they received. These changes are largely maintained to 52 weeks.

Missing data

Summary of missing data

Table 11 indicated that there was a total of 72 (31%) missing data sets in primary outcome with higher rate in the URB arm (14% vs. 18%). The highest rate of missing data in primary outcome was from the Lothian site (43%). Thirty-seven per cent of data were missing pre lockdown versus 19% post lockdown. There was no evidence that these were different (p = 0.21) using a logistic regression model.

Weighted generalised estimating equation for Clinical Global Impression – Improvement scale

As noted in the methodology section, we used WGEE to explore the impact of missing data on the primary outcome. Table 12 contains the result of WGEE on the primary outcome. There was no evidence of a difference between arms but the CI surrounding the odds of improvement was sufficiently wide enough to include important differences favouring either arm.

Completion, lost to follow-up and withdrawal

Eleven participants did not attend any parent group and therefore did not contribute to the primary outcome. In addition, two participants (0 in LAA and two in URB) withdrew but it was possible to use their data for CGI-I ratings. It did not appear that there was any difference between groups in this respect, but there did appear to be more withdrawals in the URB arm compared with the LAA arm (Table 13).