Overview and aims
The health economic component of this feasibility study had two main aims. The first involved the modification and testing of an appropriate version of the CA-SUS, for the collection of service use data in this population. The second aim was to explore recent literature to assess how realistic it might be to include a measure of health-related quality of life (HRQoL) capable of generating quality-adjusted life-years (QALYs), for use in a cost–utility analysis, in this very young age group.
Part 1: modification and testing of the Child and Adolescent Service Use Schedule
Methods
The CA-SUS is a service use schedule that was developed originally for use in economic evaluations of mental health services for children and adolescents. It is generally completed in interview with parents, although it can be completed in interview with older children (i.e. adolescent populations). The CA-SUS is a retrospective measure that asks respondents to report use of various services over a specific period, for example over the previous 3 months. The CA-SUS contains a number of sections that can be included or excluded dependent on the economic perspective of the study, for example hospital services, community health and social care services, medication, accommodation and education.
The CA-SUS has been successfully applied in a number of populations similar to the Video-feedback Intervention to promote Positive Parenting and Sensitive Discipline, Foster Care (VIPP-FC) population, including looked-after populations and pre-school populations.50,74,75 Because study participants, economic perspective, and duration and timing of follow-up vary by study, the CA-SUS must be adapted for the purpose of each new study. These earlier versions of the CA-SUS formed the basis of a modified version suitable for the current population,50,74,75 with modifications being determined, initially, by expertise in the research group and, subsequently, by testing the acceptability and comprehensiveness of the VIPP-FC CA-SUS in the feasibility study.
The VIPP-FC CA-SUS was administered in interview with foster carers during the baseline interview, covering service use over the previous 4 weeks, and at the 4-month follow-up, covering service use in the period between the baseline assessment and the follow-up interview date. The economic perspective was the NHS and social services perspective, preferred by NICE, plus education sector resources (support services provided in schools and education facilities attended), given the age of the population.
The initial draft version of the VIPP-FC CA-SUS was tested and modified in two ways. First, using feedback from research assessors who supported participants to complete the CA-SUS in interviews, we assessed research assessors’ and respondents’ understanding of the questions and modified the wording or added an explanation when anything was unclear. Second, we assessed the measure’s comprehensiveness in capturing all relevant services by identifying services used by young participants but not included in the draft CA-SUS (missing items), and any items included in the draft CA-SUS but not used by the young participants (redundant items). To assess missing items, research assessors probed for and recorded any services used by young participants that had not been covered during the course of the CA-SUS interview.
Amendments to the CA-SUS were made during the data collection period to provide an opportunity to test an improved version during the feasibility study and at the end of data collection. Initial amendments were made after 12 interviews had been completed (n = 9 at baseline and n = 3 at the 4-month follow-up) and involved discussions between the research assessors and CA-SUS designers to explore the findings so far. Research assessors highlighted any difficulties with clarity and any missing items that were being reported by a number of participants, and necessary adjustments were made. The new version of the CA-SUS was then used for all subsequent baseline and follow-up interviews. At the end of the study, all hard-copies of completed CA-SUS questionnaires (n = 58) and the CA-SUS data contained in the study database were checked for errors, redundant items (no recorded use), missing items (recorded under ‘any other service use’) and any additional feedback from research assessor notes on the hard copies of the CA-SUS.
Results
Feedback from research assessors suggested that the CA-SUS was easy to use, taking between 5 and 10 minutes to complete, and was acceptable to both interviewers and interviewees. Interviewers found that foster carers were able to answer all of the questions and there was no need to interview social workers to fill any gaps.
A summary of CA-SUS item completeness, including percentage of missing data (no response provided), redundant items (no use recorded) and missing items (additional service use recorded under ‘any other service use’) is provided in Table 6; for reference, the adjusted version of the CA-SUS is presented in Appendix 2. Some items contained in the CA-SUS are excluded from Table 6; these were not intended to be costed and included in the economic evaluation, but were of interest to the research group (e.g. statement of special education needs, days of school missed).
| CA-SUS section | Service use item | Missing data (%) | Redundant items (no use recorded) | Number of missing items (‘other’ items reported) |
|---|---|---|---|---|
| A: accommodation | A1 Type of accommodation | 0 | Items 01 and 05–13a | 0 |
| B: education | B1 Type of education | 0 | Items 03 and 04a | 8 |
| C: hospital services | C1 Hospital admission | 0 | N/A | N/A |
| C2 Inpatient specialty | 0 | N/A | 3 | |
| C3 Outpatient/day-patient appointment | 0 | N/A | N/A | |
| C4 Outpatient/day-patient specialty | 0 | N/A | 8 | |
| C5 Accident and emergency attendance | 0 | N/A | N/A | |
| D: community services | D1 Service type | 0 | Items 01, 03, 12, 18, 19 and 20a | 12 |
| D1 Number of contacts | 0 | N/A | N/A | |
| D1 Average duration of contacts | 16 | N/A | N/A | |
| E: medication | E1 Any medication? | 0 | N/A | N/A |
| E2 Name and dose | 0 | N/A | N/A |
Overall, almost all of the questions in each section of the CA-SUS were completed appropriately and almost all of the questions contained a response; therefore, the issue of missing data was not identified as a problem, which suggests that participants were able to understand the questions and found them acceptable.
The accommodation section (A1) was 100% completed, with no implausible values. The only accommodation types reported were formal foster care, provided by local authorities or private agencies, and formal kinship care. All other accommodation types were reported as zero use (redundant items), including domestic accommodation with parents/adoptive parents, informal foster care including informal kinship care, residential care provided by a local authority or private agency, residential mother and baby facility, bed and breakfast, boarding house or hotel, refuge, supported housing and short break/respite care. No ‘other’ accommodation was reported; thus, there were no missing items.
Type of education (B1) was 100% completed, with no implausible values. No children received home tuition or were in residential schools (redundant items). Additional schooling types recorded under ‘other’ included portage (n = 2), special needs school (n = 2), speech and language therapy at school (n = 1), early risers (n = 1), breakfast club (n = 1), small support group (n = 1) and booster classes (n = 1). A number of responses appear to relate to school-based services, rather than education types, and thus can be excluded from this section (e.g. breakfast club, small support group, booster classes). However, it may be appropriate to consider them for inclusion in the community service section of the VIPP-FC CA-SUS, with some rewording. For example, ‘speech and language therapy at school’ can be classified alongside other community speech and language contacts. Most of the remaining responses lacked enough detail to determine what exactly was being reported and thus were ignored (e.g. early risers, small support group and booster classes). One service was reported that the team felt could potentially be relevant to the population (portage: a home-visiting educational service for pre-school children with special educational needs and disabilities), and thus should be added to the community services section of the VIPP-FC CA-SUS. The only responses focused on education types (e.g. education facilities) were for special needs schools; again, the team felt that this could potentially be relevant for this population and so should be added to a the VIPP-FC CA-SUS.
In terms of hospital services, use of inpatient, outpatient/day-patient and accident and emergency services (C1, C3 and C5; yes/no response required) was 100% completed. Number of inpatient nights, number of outpatient/day-patient appointments and details of any accident and emergency attendances were all 100% completed (C2, C4 and C6). However, a number of additional responses were recorded under ‘other’, where specialty was unknown (C2 and C4). These responses are used to help identify the appropriate specialty. Responses for inpatient specialty included bronchoscopy, magnetic resonance imaging and sleep study. Responses for outpatient/day-patient specialty included bronchoscopy, eye appointment, immunology, blood test, heart screen, sleep test, physiotherapy and genetic test.
The community services section (D1) was 100% completed, with zeros recorded to indicate no service use. No service use was recorded for GP home visits, GP telephone contact, family therapist, education welfare officer, advice service or helpline. Some feedback was received from research assessors. First, they suggest a need to distinguish between a child’s social worker and a foster carer’s supervising social worker. Second, they reported that item 19, ‘advice service’, did not seem to apply to this population, perhaps because foster carers instead turn to their supervising social worker. In terms of number of contacts with each service accessed, these sections were 100% complete. Average duration of contacts with community services, however, was missing for a number of services, including community psychiatric nurses (missing for one out of one contact reported), counsellors (missing for two out of three contacts reported), social worker (missing for 1/48 contacts reported), educational psychologists (missing for one out of one contact reported), speech and language therapist (missing for 4/15 contacts reported).
Additional community services reported under ‘other’ included physiotherapist (n = 2), community paediatrician (n = 1), personal education plan (n = 1), family partnership nurse (n = 1), adoption social worker (n = 1), play therapist (n = 1), community nurse (n = 1), foster support worker (n = 1), dietitian (n = 1), occupational therapist (n = 1) and audiologist (n = 1). Some of these services can be reclassified within the existing list of services. For example, ‘adoption social worker’ can be included under ‘social worker’. ‘Community nurse’ and ‘family partnership nurse’ are community nurses, and can be classified alongside ‘school/college nurse, health visitor, midwife or district nurse’, suggesting the need to reword this item to make clear that this should include all nurses working in the community. ‘Foster support worker’ could be classified alongside ‘family support worker’, again suggesting the need to reword this item to make clear that this should include any support workers. All but one of the remaining services were reported by only one respondent. However, categories emerged that could be useful to add, for example there were a range of different ‘therapists’ (physiotherapist, occupational therapist, play therapist) that may usefully be added to the VIPP-FC CA-SUS.
The medication section was 100% completed, with 15 out of 58 respondents reporting use of prescribed medication.
Conclusions and recommendations
Items to be removed
Items for which no use was reported and for which the research team did not feel strongly that the item may be important for a larger sample include the following:
- accommodation – ‘residential mother/baby facility’; ‘bed and breakfast’, ‘boarding house’ or ‘hotel’; ‘refuge’; ‘supported housing’
- education – ‘residential school’, ‘home tuition’
- community services – ‘family therapist’, ‘advice services’, ‘helplines’.
Items to be added
Items reported under ‘other’ sections in the CA-SUS that were reported by a number of respondents that may thus be important to the population, and so should be added to the CA-SUS, include the following:
- education – ‘specialist school [e.g. specialist schools for children with emotional and behavioural disorders (EBD) school, special needs school]’
- community services – ‘portage’, ‘physiotherapist/occupational therapist/play therapist’.
Items requiring rewording
Feedback from research assessors and responses given under ‘other’ in the community services section of the CA-SUS suggest that the following should be reworded:
- replace ‘general practitioner – home’, ‘general practitioner – surgery’ and ‘general practitioner – telephone’ with ‘general practitioner (in surgery, by telephone or at home)’
- change ‘school/college nurse, health visitor, midwife or district’ to ‘nurse in school/college or in the community (e.g. health visitor, midwife, district nurse)’
- change ‘social worker’ to ‘social worker focused on the child (including adoption social worker)’
- change ‘family support worker’ to ‘support worker (e.g. family support worker, foster support worker)’
- change ‘speech and language therapist’ to ‘speech and language therapist in the community or in school/college’.
Part 2: measurement of health-related quality of life in young children (aged ≤ 6 years)
Methods
The second aim involved exploration of recent literature to assess how realistic it might be to include a measure of HRQoL capable of generating QALYs in this young age group. QALYs are a generic, preference-based measure of HRQoL that are used in cost–utility analysis and allow comparison across different disorders for resource allocation purposes. Without such a measure, it is difficult to make clear resource allocation decisions through the application of the NICE threshold of £20,000–30,000 per QALY. However, when the VIPP-FC feasibility study was proposed, no validated measure existed for children aged ≤ 6 years and methodological guidance for alternative approaches was limited. We therefore proposed exploring any developments in the literature that occured during the course of the feasibility study that might provide better guidance.
We searched various databases and websites for guidance in very young children, including the NIHR website, the NICE website, clinical trial registries (ClinicalTrials.gov, the EU Clinical Trials Register) and websites for instruments known to be used to derive child health utilities in the UK. We contacted experts in child health utilities asking for their insights, any new evidence and if they were aware of any ongoing work developing or testing child health utility measures in children aged ≤ 6 years.
Results
Current literature
A review of current literature identified few publications focused on the measurement and valuation of utilities in children. One 2020 review explored valuation methods of preference-based measures of health for economic evaluation in child and adolescent populations,76 one explored agreement between self-reported versus proxy-reported utility values77 and a third reported a systematic review and meta-analysis of childhood health utilities.78 None of these reviews considered methods for children aged ≤ 6 years.
The consensus from the literature seems to be that it is still too difficult to measure self-reported health in very young children, even with a well-designed patient outcome measure.79 Very young children and children with developmental or cognitive impairments are likely to be less able than older children and adolescents to perceive, understand and value their subjective experience of health.76,77
In these situations, proxy reports could be completed by parents, caregivers or health-care professionals. This does, however, rely on the proxies accurately perceiving and understanding a child’s subjective experience of their health. For older children, studies that have compared child report with proxy report have generally found that agreement was poorer on subjective attributes (e.g. cognition, emotion, pain) than on physical attributes (e.g. mobility, self-care, speech, vision) of HRQoL.77 The situation for very young and infant children is often complicated by the severity and intensity of the conditions they are suffering from and the treatments they are receiving in response,77 in this population, proxy-reporting of HRQoL, which focuses on symptoms and physical aspects of the condition, may have more validity. For example, the Infant Health-Related Quality of Life instrument, a mobile-based generic measure for infants aged < 12 months, covers seven items including sleeping, feeding, stooling, mood, skin and interaction.80
Developments in progress
Current NICE guidance states that:
When necessary, consideration should be given to alternative standardised and validated preference-based measures of health-related quality of life that have been designed specifically for use in children. The standard version of the EQ-5D [EuroQol-5 Dimensions] has not been designed for use in children. An alternative version for children aged 7–12 years is available, but a validated UK valuation set is not yet available.
A recent report by the Office of Health Economics notes that NICE is reviewing its recommendations on child utilities and QALYs and will make recommendations as part of the current NICE review of its methods for technology appraisals, which will publish a final programme manual in December 2020 after a public consultation in the summer of 2020.79,82 A search of the NICE website on 2 March 2021 did not produce any further information on this review.
Review of known child health utility measures
There are two main instruments used in the UK to measure child health utilities, the Child Health Utility Index-9 Dimensions (CHU-9D) and the EuroQol-5 Dimensions-Youth version (EQ-5D-Y).
The CHU-9D website, hosted by the University of Sheffield, states that studies are under way trialling a proxy version with children aged 5–7 years and a proxy version with guidance notes for children aged < 5 years [www.sheffield.ac.uk/scharr/sections/heds/mvh/paediatric/about-chu9d (accessed 25 March 2020)]. To our knowledge, these studies are, at the time of writing, still ‘work in progress’ (Oliver Rivero Arias, University of Oxford, February 2020, personal communication; and Donna Rowen, University of Oxford, February 2020, personal communication). Interestingly, a 2020 review of valuation methods of child health utilities reported the appropriate age range for the CHU-9D as aged 4–17 years, rather than the aged 7–17 years age range stated on the CHU-9D website.76 The aged < 7 years age range refers to the proxy version, although the authors acknowledge that there is no published evidence to support the validity of this version (Donna Rowen, personal communication). The CHU-9D is currently being revalued in the UK using discrete choice experiments with adult responders (Donna Rowen, personal communication).
The EQ-5D-Y is a ‘child-friendly’ version of the EuroQol-5 Dimensions, three-level version (EQ-5D-3L); it contains the same questions and levels, but with more comprehensible wording for young people. The EQ-5D-Y covers young people aged 8–15 years, with a proxy version available for children aged 4–7 years.83 The EuroQol Group is currently developing an international valuation protocol for the EQ-5D-Y. In a forthcoming review of the psychometric performance of the EQ-5D-Y, CHU-9D, Health Utilities Index Mark 2 (HUI2) and Health Utilities Index Mark 3 (HUI3), the EQ-5D-Y performed best in terms of psychometric properties, followed by the CHU-9D, with the HUI2 and HUI3 being used in more studies, but with more mixed evidence (Donna Rowen, personal communication).
Conclusions and recommendations
In summary, although proxy versions for children aged ≤ 6 years are available for both the CHU-9D and EQ-5D-Y, there is no evidence currently to support the validity or reliability of these proxy versions. In addition, the EQ-5D-Y proxy measure is for use with children aged 4–7 years and, although no minimum is stated for the CHU-9D, it is possible that neither measure is suitable for children aged 0–3 years. Given these findings, it is not currently possible to recommend any specific measures for inclusion in a future trial of VIPP-FC; thus, the primary economic evaluation should be a cost-effectiveness analysis using the primary clinical outcome measure or using an assessment of meaningful change on the primary clinical outcome measure. However, neither option is ideal, given the limitations of cost-effectiveness analysis using clinical measures, compared with cost–utility analysis using QALYs, for which the results can be assessed against a national threshold, such as the cost per QALY threshold preferred by NICE. We therefore additionally recommend that a future trial of the VIPP-FC may be a good opportunity to compare the construct validity and sensitivity to change of the proxy versions of the CHU-9D and/or the EQ-5D-Y by exploring the relationship of these measures with clinical outcomes such as the SDQ and CBCL.