Quantitative results

This chapter examines and reports data on the feasibility of the recruitment of CAPRI and the delivery of the Young SMILES intervention. This entails reporting the number of families approached, the number of eligible children within the approached group, the number that were recruited from that eligible population and the number finally randomised and who took part. This chapter will also report on the number of children in each group (randomised to Young SMILES or to TAU) who completed data collection at each follow-up time point. Thereafter, as evidence of feasibility of delivery, we report on how many children and adolescents randomised to Young SMILES (as opposed to TAU) adhered to the intervention and how many sessions they completed.

In addition, we provide evidence on the feasibility of collecting data from children of all age groups (6–16 years) and their parents, using the chosen measures within the protocol. We present data that help to assess whether or not the chosen measures can capture change in the population over the time periods of use. The Young SMILES protocol and study design is neither tasked nor designed to estimate the effect of our intervention. As a result, we shall not make between-group comparisons in the chosen measures over the time periods of data collection. Such an examination will be undertaken in an appropriately powered and designed efficacy study.

Participant flow

The first pathway of recruitment into the study relied on NHS staff identifying eligible families through screening their patient caseloads and patient records within adult community mental health NHS teams, as well as contacting relevant care co-ordinators of patients with a diagnosis of severe mental illness who had children aged 6–16 years. Care co-ordinators were asked to contact their patients to pass on information about the study. Once their patient has confirmed their interest in the study, the care co-ordinator would either complete a verbal-consent-to-contact form and send it to the address of the research team, or encourage the patient to complete and send a consent-to-contact form to the same address. Adult community mental health services in the NHS routinely focus on adults. They are therefore not concerned with their children, unless there is a risk of harm to the child or a mental illness diagnosed for the child.

The second recruitment pathway followed an opportunistic approach of identifying families who came into contact with children’s charities during the period of the study’s recruitment. We relied on charity workers who assessed the child to identify whether or not their parent had severe mental illness, thereby gauging the potential eligibility of the child for Young SMILES. Charities, such as the NSPCC, focus on children and involve parents/carers who are well; they do not customarily assess or work with the mentally ill parent.

At the NSPCC sites, we do not have information about how many families NSPCC staff approached. Overall, 18 families reported an interest in participating and were screened for eligibility by phone, 15 of these were recruited via an NSPCC professional who had first seen each family and three via self-referral in response to a public advert or via signposting by a non-NSPCC professional. Three families had to be withdrawn by the research team because their children were eligible for the 12- to 16-year age group only, which did not get the chance to assemble because of recruitment difficulties for this cohort. In the NHS, out of the 1963 patient records that were screened by NHS staff, 162 (8%) families were considered potentially eligible. The research team asked the care co-ordinators to approach these 162 families. Care co-ordinators reported that a total of 33 families had expressed an interest, a response we took at face value, assuming that they had indeed approached all 162 potentially eligible families. One hypothesis is that front-line clinical staff could have acted as barriers to participant recruitment. This happens either by staff making decisions about the suitability of research on their client’s behalf or by gatekeeping access to research studies in other ways. This is likely to be a significant problem and to skew apparent recruitment uptake. Therefore, we cannot be sure how many of the 162 families were approached, or if care co-ordinators endorsed the study to the families they did approach.

However, we can reliably report that, of the 33 families whose details were provided to us by NHS care co-ordinators, 31 were screened by telephone but two were uncontactable. From the NSPCC, 18 families were screened by telephone. Out of a total of 49 families screened, 35 (71%) met inclusion criteria and received a face-to-face assessment and subsequent randomisation; nearly one-third of those (n = 12) were recruited in the NSPCC sites and two-thirds (n = 23) in the NHS. Out of the 35 randomised families, 33 families (18 in the Young SMILES group and 15 in the TAU group) provided baseline data. Of those families, 28 gave follow-up data at the primary end point (4 months post baseline): 15 out of 20 (75%) in the intervention group and 13 out of 15 (87%) in the control group.

When multiple children were recruited from the same family, only the index child’s data were included in analyses. Siblings were still offered the opportunity to attend Young SMILES (if their family was randomised to the intervention group). The index child was determined by the parent only. The parent was asked by the research team to choose which child they wanted to have their measures collected. The research team discovered that in all instances the parent’s decision was based on their perceptions of which sibling experiences the most difficulties in responding to parental mental illness. Only 6 out of the 20 families who were randomised to the intervention had children (seven in total) other than the index child of an age eligible to participate (aged 6–16 years). Baseline data are presented both for index children and for all children. Figure 1 presents participant flow from identification to follow-up.

FIGURE 1 The CONSORT participant flow diagram. (Figure not reproduced here; see the original article.)

Participant characteristics

The group of children that received Young SMILES (n = 18) was split into two age groups: 6- to 11-year-olds (n = 14, mean 9.2, SD 1.2) and 12–16 year olds (n = 4, mean 13.8, SD 1.7). The mean age of the children (n = 15) in the control group was 11 years (SD 2.7 years). The female-to-male ratio was 3 : 2 in both groups. All children were living with their parents. Most children (13/18, 72%, in the intervention group and 12/15, 80%, in the control group) were aware that their parent had severe mental illness. In total, 31 out of 33 parents checked the box as ‘living with a partner’ at baseline, which would suggest that the majority (81.8%) were unmarried parents. Table 7 details demographic characteristics for participating children in each group.

The majority of the parents in the participating families across both groups were mothers (30/33), white British (30/33), not in active employment (32/33) and aged 25–44 years (27/33). Families lived either in their own homes (10/33), in rented accommodation (8/33) or in social housing (12/33). Self-reported types of mental illness experienced by the parents included bipolar affective disorder, alcoholism, depression, anxiety, personality disorder, schizophrenia, panic disorder, substance abuse and phobia. In most cases, parents experienced a combination of these. Table 8 provides a breakdown of characteristics for the participating parents in each group (control and intervention).

The number of attended sessions by all children (n = 23) and their parents (n = 19) was not normally distributed (Table 9), so the median and interquartile range (IQR) were calculated in place of the mean and SD. The majority of children attended seven out of the eight sessions offered (IQR 6–8 sessions) and typically parents attended three out of the five sessions offered (IQR 2–5 sessions).

Outcomes

Table 10 provides an overview of the outcome measures we used in the study, along with their interpretation of scores and established cut-off points. See Appendix 3 for a more detailed description of every measure used.

We present QoL outcomes as self-reported by children and proxy-reported by parents using the Paediatric Quality of Life Inventory™ (PedsQL™)108 and KIDSCREEN™ (a health-related quality of life questionnaire for children and young people and their parents).126 We present outcome data from the indexed children only. As there were only six families who had more than one child, we did not estimate a family cluster intraclass correlation coefficient, because this would have been large and would have had very low precision. Little power would be gained from including multiple children per family, so our results are presented, as per protocol, using the index children as our sample, with no design effect of siblings who may have participated in the intervention. Parents’ outcomes (both proxy for their children and self-reported for themselves) are treated as a separate independent sample.

Paediatric Quality of Life Inventory

For all 23 items across both groups at all time points, total mean scores for child report ranged from 73.3 to 87.3 and for parent report ranged from 62.6 to 74.9. SDs ranged from 9.6 to 19.6 for child report and 7.1 to 23.4 for parent report. For the 15 items of psychosocial health, mean values ranged from 65.4 to 81.2 (SD 15.6 to 21.6) for self-report and from 56.1 to 66.8 (SD 14.3 to 21.1) for parent report. Furthermore, for the eight items of physical health, mean values ranged from 76.1 to 94.0 (SD 8.3 to 26.4) for self-report and from 63.5 to 83.0 (SD 12.4 to 27.6) for parent report (Table 11).

A meaningful cut-off point to identify those at risk of poor QoL has been proposed as one SD below the population mean.127 Most literature on child QoL is from medical settings (e.g. diabetes, cancer, asthma). Estimates of the minimum clinically important difference (MCID) for PedsQL total scores are available for children with diabetes: these estimates lie between 6.27 points (parent proxy-report) and 4.72 points (child self-report).128

KIDSCREEN-52

The 52-measure KIDSCREEN126 was completed by children aged 8–16 years in our sample as a self-report measure of their QoL and by their parents as a proxy-report measure of their children’s QoL. The KIDSCREEN-52 is not validated for 7-year-olds, so data for children of that age in our sample are available only from their parent’s proxy-report.

Child self-report KIDSCREEN mean scores across all 10 domains at all time points ranged from 42.7 to 64.5 (SD 8.8 to 17.5) in the TAU group and from 45.1 to 55.4 (SD 8.4 to 14.3) in the Young SMILES group. Parent proxy-report KIDSCREEN mean scores across all 10 domains ranged from 34.9 to 54.7 (SD 6.8 to 16.8) for the TAU group and 40.2 to 50.9 (SD 8.0 to 15.7) for the Young SMILES group.

The two randomised groups (Young SMILES vs. TAU) were comparable (< 1 point apart) at baseline scores in only three KIDSCREEN domains: self-perception (50.6 vs. 51.0) and autonomy (50.4 vs. 49.7) in the child self-report version, and physical health (43.0 vs. 42.8) in the parent proxy-report version. The widest gap in baseline scores between Young SMILES and TAU were in the child self-report for school (51.4 vs. 58.9) and in the parent proxy-report for bullying (44.6 vs. 36.4).

Child self-report scores across all KIDSCREEN domains were consistently higher than their corresponding parent proxy-report scores at all time points. We visually inspected the gaps between child and parent scores to identify KIDSCREEN domains with high discordance. In the Young SMILES group, we looked for scores that were at least 7.15 points apart [i.e. half the maximum SD (14.3) noted across all domains in the Young SMILES group]. In the TAU group, we looked for scores that were at least 8.75 points apart [i.e. half the maximum SD (17.5) noted across all domains in the TAU group].

In the Young SMILES group, we observed high discordance (at least 7.15 points) between child and parent scores in four domains: mood and emotion at 4 months and 6 months, autonomy at baseline and 4 months, parent relations at 4 months, and bullying at 4 months. In the TAU group, there was high discordance (at least 8.75 points) between child and parent scores in six domains: physical health at 6 months, psychological health at baseline and 6 months, mood and emotion at all time points, self-perception at 6 months, school at 4 and 6 months, and bullying at 4 and 6 months. One domain, mood and emotion, had high discordance between child and parent scores across all time points and in both Young SMILES and TAU groups (Table 12).

Child Health Utility 9D

To inform a future economic evaluation, we used the Child Health Utility 9D (CHU-9D).116 This is validated for children aged 7–17 years to estimate incremental health gain in quality-adjusted life-years (QALYs). By visually inspecting the results, means ranged from 0.7 to 0.9 and SDs from 0.1 to 0.2. With the caveat of small sample sizes, children in the TAU sample had ‘excellent’ QoL that remained so at all measurement time points. By contrast, QoL fluctuated from ‘good’ (baseline) to ‘excellent’ (4-month follow-up) and ‘very good’ (6-month follow-up) for the Young SMILES group. Notably, some children in the Young SMILES group had CHU-9D scores as low as 0.2, whereas the lowest score for the TAU group was 0.6 (Table 13).

Revised Child Anxiety and Depression Scale-25

Revised Child Anxiety and Depression Scale-25 (RCADS-25) scores across all groups and time points were in the non-clinical range. There were imbalances in baseline scores for Young SMILES versus TAU from both child-reported data (total, 47.5 vs. 42.2; anxiety, 47.6 vs. 42.6; depression, 47.8 vs. 43.2) and parent-reported data (total, 61.1 vs. 56.4; anxiety, 59.0 vs. 54.6; depression, 61.7 vs. 57.4), with scores consistently lower in the TAU group. Follow-up scores across all groups and all scales ranged from 39.3 to 62.4. SDs ranged from 10.0 to 20.2 at baseline and from 8.2 to 17.5 at follow-up.

Parent proxy-reported scores for both Young SMILES and TAU groups were much higher than the corresponding child self-reported scores across all time points. Using the established cut-off score of 65, the parents’ scores are approaching a proxy-reported clinical need for their children, whereas the children’s self-reported scores are much below the clinical threshold (Table 14).

Strengths and Difficulties Questionnaire

There were imbalances at baseline scores between Young SMILES and TAU for the Strengths and Difficulties Questionnaire (SDQ), which were > 1 point in the SDQ total score and in the emotional domain score. This is perhaps not surprising, given the small sample of children who provided self-report data. Despite these imbalances, the mean scores (rounded to the nearest integer) in total and across all domains denoted the same level of difficulties between the two groups (i.e. average, slightly raised, high or very high). Parent proxy-report mean scores at baseline were similar between Young SMILES and TAU in total and across all SDQ domains.

The discordance between child and parent scores within each randomised group at all time points was large, consistently indicating that parents perceived their children’s difficulties to be greater than the children’s assessment of their own difficulties. The only two domains where children and parents agreed on the level of difficulties were hyperactivity and prosocial behaviours. For these domains, all scores across all time points in both groups were close to ‘average’. These observations come with the caveat that we had three times more data from parents than from children, because a lot of the children in our sample were younger than 11 years and did not complete the self-report questionnaire.

At baseline, in both Young SMILES and TAU groups, children assessed their total difficulties to be ‘slightly raised’, whereas parents assessed the same to be ‘high’. At follow-up, children’s self-reported total difficulties were close to ‘average’ in both Young SMILES and TAU groups, whereas parents’ scores remained in the ‘high difficulties’ range for the TAU group and fluctuated between ‘slightly raised’ (6-month follow-up) and ‘very high’ (6-month follow-up) in the Young SMILES group. For the emotional and conduct domains, across all time points in both groups the children’s scores were close to ‘average’, whereas parents’ scores ranged between ‘slightly raised’ and ‘high’. For peer relationships, the children rated their difficulties as ‘high’, whereas their parents rated them as ‘very high’ (Table 15).

Mental Health Literacy questionnaire

Total Mental Health Literacy questionnaire (MHLq) mean scores across all time points ranged from 128.0 to 131.7 (SD 8.5 to 13.7) in the TAU group and from 130.4 to 134.5 (SD 12.6 to 16.6) in the Young SMILES group. For the three MHLq subdomains and across both randomisation groups, means ranged from 41.3 to 44.5 for help-seeking and first-aid skills, from 52.7 to 56.5 for knowledge and stereotypes, and from 26.6 to 32.6 for self-help strategies. The SDs across both the Young SMILES and TAU groups ranged from 3.2 to 6.4 for help-seeking and first-aid skills, from 4.3 to 7.7 for knowledge and stereotypes, and from 2.7 to 5.5 for self-help strategies.

On total MHLq scores, our sample’s baseline means (130.5 for Young SMILES and 128.0 for TAU) and follow-up means at 6 months (130.4 for Young SMILES and 131.7 for TAU) were notably lower than the mean baseline score (133.3) in Campos et al.’s113 group of children who were close to someone with mental illness. On the domain of the MHLq called ‘help-seeking and first-aid skills’, our sample’s baseline means (41.3 for Young SMILES and 41.7 for TAU) were marginally smaller than Campos et al.’s113 mean score (41.9), but higher at all follow-up time points.

We note that the instructions of summing up relevant items under each subscale came from communication with the authors of the MHLq. These instructions were different from the Campos et al.113 paper for two domains of the MHLq: knowledge and stereotypes (which included 18 items in the Campos et al. paper,113 as opposed to 15 items as we were instructed) and self-help strategies (which included five items in the Campos et al. paper,113 instead of eight as we were instructed). Therefore, we can compare Campos et al.’s113 sample with our sample only on the total MHLq score and the subscore of ‘help-seeking and first-aid skills’ (Table 16).

Parenting outcomes

Arnold–O’Leary parenting scale

Baseline mean scores in total and across all domains for both Young SMILES and TAU groups ranged from 4.5 (for verbosity) to 3.0 (for laxness); and SDs ranged from 0.6 to 1.2. At follow-up, mean scores in the Young SMILES group ranged from 4.1 (for verbosity) to 2.5 (for over-reactivity), while SDs ranged from 0.5 to 1.1. For the TAU group, mean scores ranged from 4.2 (for verbosity) to 2.7 (for laxness), while SDs ranged from 0.3 to 1.2.

The recommended clinical cut-off scores (for mothers, who predominated in our sample) are total score 3.2, laxness 3.6, over-reactivity 4.0 and verbosity 2.4.114,129 Verbosity was the only domain for which parents’ scores were above the clinical cut-off point at all time points and in both groups, indicating a possible area of need in our sample’s parenting style (Table 17).

We observed imbalances in baseline total scores for Parent Stress Index – Short Form (PSI-SF) between Young SMILES and TAU (107.4 ± 29.1 vs. 102.4 ± 24.7). At follow-up, the lowest mean total score was 84.9 (at 6 months for Young SMILES) and the highest was 98.2 (at 4 months for Young SMILES). SDs for the total scores ranged from 19.4 to 30.6. Mean scores and SDs for the subscales at baseline for Young SMILES and TAU, respectively, were for parental distress, 38.3 ± 12.0 and 33.3 ± 8.6; for parent–child dysfunctional interaction 32.7 ± 11.8 and 34.0 ± 12.6; and for difficult child 36.4 ± 11.8 and 37.9 ± 6.3. At follow-up, mean scores across all subscales ranged from 26.1 to 38.3 and SDs ranged from 6.3 to 12.6.

The cut-off score above which a family may need support and further services is 114 for the total score, 40 for parental distress and difficult child and 36 for parent–child dysfunctional interactions. Mean scores in total and for all subscales at all time points were lower than the cut-off point of needing support across both groups; however, the maximum total scores of 148 for the Young SMILES group and 159 for the TAU group at baseline suggest that some parents in the group experienced high parental distress (Table 18).

A construct called ‘Defensive Responding’ is also calculated from PSI-SF. This is calculated as the sum of seven items (1, 2, 3, 7, 8, 9 and 11) and helps with quality checking and interpreting the PSI-SF scores. If a parent scores < 10 in the total sum of the defensive responding items, they may be answering defensively and not being completely forthcoming in their responses. In our sample, parents’ mean scores ranged from 18.2 to 22.4 and SDs ranged from 5.0 to 7.8. Notably, some parents scored < 10, as indicated by minimum values of defensive responding scores in the range of 7 to 9 (Table 19).

Resource use over time

The collection of child resource utilisation data was piloted using the CA-SUS.130,131 We used the tool to identify the most important aspects of resource utilisation, as well as to assess the feasibility of collecting information on receipt of care and services from the NHS, social care, education, voluntary and third-sector organisations. We did not calculate costs for this resource use as part of this feasibility study.

The CA-SUS was adapted for our study in consultation with Professor Sarah Byford, the designer of the CA-SUS questionnaire. This adaptation involved removing the sections on out-of-pocket expenses and employment, removing the question on education type, removing the follow-on questions asking name of hospital for the hospital service use questions, removing complementary therapist (e.g. homeopath) from the list of community services, adding NHS walk-in services and NHS Direct to the list of community services, and simplifying the questions in the criminal justice services section. A copy of the CA-SUS version used in our study can be found in the additional files of this report [see the project web page: www.journalslibrary.nihr.ac.uk/programmes/hta/142901/#/documentation (accessed April 2020)].

Resource utilisation was collected retrospectively using parent recall, responding on behalf of the child. Parents completed the questionnaire on their own, but could ask the researcher for clarifications on any questions that they did not understand or found difficult to answer. The adapted version of the CA-SUS elicited information on resource use in the following categories: accommodation, education, hospital services, community services, medication and criminal justice services. Resource utilisation data were collected at three time points: baseline (recalling utilisation over the previous 6 months), 4-month follow-up (recalling utilisation since baseline) and 6-month follow-up (recalling utilisation since the last interview).

We obtained resource use data from 28 parents at baseline, 25 parents at the 4-month follow-up and 26 parents at the 6-month follow-up. This corresponds to the same number of children at each time point. Owing to the small sample size and infrequency of utilisation of many services, results are not reported separately by treatment arm.

Accommodation, education and hospital service use

Descriptive statistics for accommodation, education and the utilisation of hospital services are presented in Table 20. Results are summarised as the proportion of the sample who reported using a service during the recall period (for yes/no questions) and among those answering ‘yes’ the mean number of days/times used. The number of participants who completed the question is also reported to illustrate the completeness of data collection.

Response rates were high for the questions about accommodation, education and use of hospital services, with all respondents completing all of the initial questions. The only exception was the question about attendance at an accident and emergency (A&E) department, which was answered by only 23 out of 28 respondents at baseline. Although we cannot comment on the accuracy of the reported use, participants were willing and able to respond to questions regarding accommodation, education and use of hospital services using the CA-SUS.

All respondents reported that their child was living in domestic accommodation at all three time points. The average number of days absent from school or studies during the recall period was 4.3 at baseline, 6.5 at 4-month follow-up and 2.7 at 6-month follow-up. Hospital admission was rare, with no respondents reporting that their child had been admitted to hospital in the 6 months preceding baseline. In total, 0.04% of the sample reported being admitted to hospital at both the 4- and the 6-month follow-ups. Outpatient appointments were more common, with nearly one-third of the sample reporting their child having attended an outpatient appointment in the 6 months preceding baseline data collection. One-quarter (25%) of all respondents reported an ambulance having been called for their child in the 6 months preceding baseline, whereas 17% reported that their child attended A&E during the same period (Table 20).

Community services use

Descriptive statistics for community services use are presented in Table 21. Results are summarised as the mean number of contacts per individual. The number of participants who completed the question is also reported to illustrate the completeness of data collection.

Of the community services listed, GP surgery appointments were the most frequently used during all three periods. The mean number of GP surgery appointments per participant was 0.39 in the 6 months preceding baseline.

Participants reported no utilisation of the following services at any of the three data collection time points: GP home visits, psychiatrist, clinical psychologist, community psychiatric nurse, accommodation key worker, educational psychologist, drug/alcohol support worker, helpline (e.g. Samaritans) or NHS Direct.

The results from our feasibility study would suggest that these services are not used; therefore, to minimise participant burden, it may be advisable to remove some or all of these services from the list of community services questions. Alternatively, the lack of reporting could indicate problems with participant/parent recall. For a future trial, we will consult with stakeholders whether we should include or omit these community services, as well as how we can facilitate the completion of relevant items on the CA-SUS (see Table 21).

Medication and criminal justice services

Questions about medication prescriptions and interactions with criminal justice services were also included in the CA-SUS, but response rates in these sections were very low. No response data were recorded in the medical prescriptions section at any of the three time points. The criminal justice services section was also left blank by all respondents at both baseline and 4-month follow-up. At 6-month follow-up, three respondents gave answers to two of the questions asked in the criminal justice services section, covering nights spent in youth custody and being a victim of crime. As a result of the low response rate and the sensitivity of this information, these responses have not been reported in any more detail here.

Discussion

Participants

In our original protocol, we intended to recruit 60 families, equally split across two NSPCC sites (intended n = 40) and one NHS site (intended n = 20). We recruited to time and target in the NHS (actual n = 23 families), but we did not reach our intended recruitment target at the NSPCC sites (actual n = 12 families).

We have discussed in detail the reasons why recruitment was delayed by the transitioning into the new HRA process during the early months and first year of our award. This created an unforeseen and critical 11-month delay in our ability to start recruitment. During this period, the NSPCC sites that we had originally identified for recruitment were no longer available for the study or were closed. The delay also meant that, critically, recruitment start dates coincided with school holidays. Following extensive contact with NIHR, it was decided to foreshorten the recruitment period. This led to a number of protocol changes: to include 2 : 1 randomisation into the Young SMILES intervention and a new target to recruit 40 families in total. We requested an extension to the grant, but this was not feasible given the lack of costs available to support staff.

It is our view that the opportunistic recruitment method within the NSPCC was less effective at identifying suitable families than the caseload review and record screening approach of recruitment within NHS services. By the same token, relying on gatekeepers who work primarily with children was less effective than going directly to gatekeepers who work with mentally ill parents. This is important for future recruitment of families into similar studies in the NHS. Intuitively, researchers may be tempted to follow a less fruitful recruitment pathway via CAMHS to recruit children into a study that delivers an intervention for children. However, our experience taught us that we could reach more eligible families within a limited time frame by engaging with care co-ordinators from adult mental health teams, especially when risk factors for a child are associated with their family. Furthermore, the children whom we targeted did not necessarily have mental illness and, therefore, may have gone unidentified by services. Indeed, in our sample, few endorsed symptoms likely to fulfil criteria for a clinical diagnosis.

In our feasibility study, we found that only a small pool of families (8%) in NHS secondary care adult mental health services were eligible for the Young SMILES trial (i.e. had a parent with SMI and children aged 6–16 years). The NHS recruitment pathway drew from a large population (all adults who were in receipt of secondary care mental health services), but only a fraction of this population (8%) was potentially eligible. We found that adults in secondary care services either were too young to have children aged ≥ 6 years (especially in early intervention for psychosis services) or had adult children (e.g. parents with lifelong severe mental illness). Parents with severe mental illness who have school-age children are not necessarily in receipt of secondary care services, but they are looked after in primary care. Future research studies that evaluate interventions for children and families experiencing severe mental illness can boost recruitment by screening GP and pharmacy records.

The research team was able to contact just one-fifth of potentially eligible families. We cannot tell whether the remaining potentially eligible families were approached by care co-ordinators but declined participation or if the care co-ordinators did not approach these families in the first place. Once families were identified as eligible, agreed to participate and were randomised, on average, across both intervention and control groups, there was a high rate (> 80%) of completion of follow-up measures. Children typically had high adherence to the intervention by completing seven out of the eight offered sessions. The overwhelming majority of recruited parents were mothers, although there was a balance of sexes among participating CAPRI.

Outcomes

Differences in baseline scores are a useful indicator of the comparability of the intervention and the control groups, and the success of randomisation. There were visually noticeable differences in mean scores at baseline between the two randomised groups across all outcome measures. Such differences are likely to be the result of chance rather than bias, given that the randomisation was done remotely with a computer program. This is not surprising given our small sample size. The observed baseline imbalances could be the result of factors other than compromised randomisation (e.g. the children’s age). In our sample, the proportion of 6- to 11-year-olds was slightly higher in the intervention group than in the control group (78% vs. 67%). Expected imbalances at baseline are an important consideration for a future RCT’s design and outcome selection. Such imbalances can be minimised at the RCT’s design phase through stratification of participants, and further adjusted at the point of analysis by including baseline scores as covariates.

Overall, the children’s self-reported QoL and functioning were high, but their emotional and mental health needs were below the clinical threshold within each outcome measure and across all time points. By visually inspecting the outcome data at both follow-up points across all outcome measures, there are neither alarming results nor an obvious and consistent direction of scores in relation to time or outcomes, even when baseline scores were similar between the intervention and the control group.

We refrain from making statistical between-group comparisons on QoL and mental health outcomes at follow-up, because our RCT was not intended to estimate between-group differences in outcomes. Given that our sample for analysis included only 33 participants (Young SMILES, n = 18; TAU, n = 15), unsurprisingly, it was not powered to do so. Furthermore, there was an imbalance in baseline scores between the randomised groups, so any differences at follow-up would not necessarily indicate a treatment effect. For example, for the PedsQ, the means for Young SMILES versus TAU at 6-month follow-up were 73.3 and 87.2, indicating that the TAU group had a higher QoL. However, this imbalance was also reported at baseline (Young SMILES vs. TAU was 74.0 vs. 83.2, respectively), so we would be likely to see no intervention effect after controlling for baseline scores, even if we had sufficient power to justify a between-group comparison.

Parents’ proxy-reported scores consistently tended to overestimate areas of need and difficulty for their children, as well as underestimating their children’s QoL compared with the children’s corresponding self-reported scores. Discordance between child- and parent-reported scores is common in the literature for outcomes based on the PedsQL132 and KIDSCREEN.133 Parental mental health can also influence their reports of the QoL of their children, particularly in the areas of physical well-being and social support, with elevated levels of parental depression and anxiety being associated with lower QoL scores among their children.134 Although self-report is preferable for measuring QoL from the child’s perspective, the parent’s view of their child’s QoL is equally important as it can influence help-seeking and service utilisation.135 It will be useful for future research studies that evaluate interventions for children and families experiencing severe mental illness to include both child self-report and parent-proxy versions of a QoL questionnaire.

Parent self-reported outcomes across all time points indicated that parents in our sample were, on average, below the cut-off scores for distress and difficulties with their children and that their parenting style was positive.

Mental health literacy

We refrain from making statistical comparisons of MHLq scores between Young SMILES and TAU because of our small sample size and the observed baseline imbalances. By visually inspecting the data, there is no sense of direction of MHLq scores in favour of either the intervention or the control. There is no information in the literature that would help us to interpret the observed differences in mental health literacy between different samples or between the intervention and control groups. The same applies to differences within the same group at different time points (e.g. by knowing what the minimal difference in MHLq scores that is important in a clinical context, similar to a MCID). We also do not know what the normative values are for MHLq, so that we can gauge whether baseline mental health literacy in our sample was ‘better’ or ‘poorer’ than in other samples or in the general population or ‘average’. Future studies need to set benchmark criteria against which we can interpret MHLq scores and a MCID for MHLq. This is needed to assess whether or not an intervention such as Young SMILES can lead to a meaningful change in mental health literacy.

Campos et al.113 identified three predictors of better mental health literacy based on the MHLq: being female, being a teenager (as opposed to being aged < 13 years) and being close to someone with mental illness. We would expect that, on account of having a mentally ill parent, our participating children have better mental health literacy than their peers in the general population. Our sample had more girls than boys, which would act in favour of better mental health literacy. However, the average age in both Young SMILES and TAU groups was < 13 years, and this may have contributed to lower MHLq scores than in Campos et al.’s sample.113 Future studies of interventions that aim to improve mental health literacy in children of mentally ill parents need to consider stratification according to the children’s sex and age.

It is worth noting that the MHLq includes items that are relevant to severe mental illness, such as schizophrenia. However, it also refers to other mental illnesses, such as depression and anxiety. A scoping review of all mental health literacy measures136 identified 69 knowledge measures (e.g. assessing ability to recognise mental illness and factual knowledge of terminology, aetiology, diagnosis, prognosis and consequences of mental illness) and 35 help-seeking measures (e.g. assessing help-seeking attitudes, intentions and actual help-seeking behaviours). Despite the plethora of mental health literacy measures, it is not immediately clear which ones are more appropriate for children with severely mentally ill parents, with respect to their content.

A recent literature review of mental health literacy content for children with severely mentally ill parents137 highlighted the need for adding family-contextual content (e.g. dealing with stigma experiences, managing stress and communicating about parental mental illness) to the standard content of mental health literacy that is designed for children in the general population. This suggests that generic mental health literacy measures may not be fit for assessing the utility of an intervention for children with mentally ill parents. Future studies that aim to improve mental health literacy in children with severely mentally ill parents need to use measures that allow the addition of family-contextual content.

In the context of interventions like Young SMILES, whose ultimate aim is to improve QoL, mental health literacy is a mediating variable (i.e. a mechanism by which we expect the intervention to work to produce the desirable outcome). Therefore, not only does the intervention need to bring about meaningful change in MHLq scores, but this change should be positively correlated with a subsequent change in QoL outcomes. Our feasibility study was too small to enable us to explore associations between changes in mental health literacy and changes in QoL. Future studies that seek to explore the mediating effects of mental health literacy on children’s QoL need to estimate the sample size that will enable such a mediational analysis.

Resource utilisation

We noted high completion rates of the resource utilisation questionnaire across all time points (> 90% of all participants). Participants were willing and able to respond to questions from the CA-SUS regarding accommodation, education, hospital service utilisation and community services utilisation. However, of the 27 community services listed in the CA-SUS questionnaire, participants reported no utilisation of nine of these services at all three time points. To minimise participant burden, it may therefore be advisable to remove some or all of these services from the list of community services questions in a future trial.

Outpatient appointments were the most common type of health-care use, with nearly one-third of the parents reporting that their child had attended an outpatient appointment within the previous 6 months. Ambulance use and A&E attendance were also relatively high in our sample. Other than GP appointments and input from a social worker and a family support worker, children were scarcely reported to use any other community or social care services.

Response rates were zero for questions regarding medication prescriptions. They were also very low for questions regarding interactions with the criminal justice system. Given that these categories are important, especially if a future economic evaluation takes a societal rather than just a health and personal social services perspective, for an intervention that is likely to have an impact on resource use in these areas, we will consider alternative methods of data collection in addition to a self-report questionnaire.

Data missingness

Table 22 gives an overview of all missing data for each outcome measure within each group from baseline to the primary end point (4 months).