Assessment of cost-effectiveness

Review of existing economic model evaluations

Methods for systematic review of economic evaluations

The initial protocol sought to conduct a systematic search to identify full economic evaluations of digitally enabled therapies for people with tic disorders. Given the lack of anticipated evidence, searches were expanded to include full economic evaluations of any intervention for people with tic disorders. The following databases were searched, with no time, language, or publication type restriction:

• Ovid MEDLINE
• Ovid EMBASE
• Ovid PsycInfo® (American Psychological Association, Washington, DC, USA)
• NHS Economic Evaluations Database
• International HTA Database (INAHTA)
• Proquest EconLit Cost-Effectiveness Analysis (CEA) Registry.

Detailed search strategies are provided in Appendix 1. The websites of relevant professional organisations [e.g. International Society for Pharmacoeconomics and Outcomes Research (ISPOR) Scientific Presentations Database] and health technology agencies [e.g. the National Institute for Health and Care Excellence (NICE), the Canadian Agency for Drugs and Technologies in Health (CADTH), the Pharmaceutical Benefits Advisory Committee, the Institute for Clinical and Economic Review (ICER)] were screened for supplementary reports. Reference lists of all incorporated studies were manually reviewed to identify additional relevant studies. Additional data and information provided by the companies was assessed for relevance to the decision problem and included in results summaries where appropriate.24

The review included full economic evaluations of any intervention for people with tic disorders. For the purposes of the review, full economic evaluations are defined as comparative analyses of costs and outcomes within the framework of cost-utility, cost-effectiveness, cost-benefit, or cost-minimisation analyses. Economic evaluations conducted alongside single effectiveness studies or decision-analytic models were included.24

The key findings from included economic evaluations were summarised in tabular format and narratively synthesised. All included studies were appraised with respect to the NICE reference case checklist for economic evaluations.44 Reporting quality of studies was assessed using the Consolidated Health Economic Evaluation Reporting Standards45 checklist, and any decision models were quality assessed using the Philips et al. (2004) checklist.

Results for systematic review of economic evaluations

The search identified 1242 potentially relevant articles. One further study was provided by one of the companies post search date, leading to a total number of articles of 1243.41 An initial screen by the study information specialist excluded 1171 irrelevant titles and abstracts and duplicates. The remaining 72 abstracts were reviewed by a health economist reviewer Dwayne Boyers (DB), of which 10 were retained for full text screening. The final list of studies included four publications describing three studies.35,41,46,47 Further details of the screening results are provided in the PRISMA flow chart in Figure 8.48

FIGURE 8 Preferred Reporting Items for Systematic Reviews and Meta-Analyses flow chart for economic evaluation studies. (Figure not reproduced here; see the original article.)

Summary of existing systematic review of economic evaluations

Tables 11 and 12 summarise the characteristics and results of the included studies.

The quality assessment of the included studies is summarised in Table 13. Full details of quality assessment scores for decision modelling studies are presented in Table 28, Appendix 3.

Economic model overview

A Markov cohort model was constructed in TreeAge Pro 2023 software (TreeAge Software, Inc., Williamstown, MA, USA) to assess the expected costs and quality-adjusted life-years (QALYs) of different treatments for managing tics and TS in children and young adults. Model outputs are used to calculate the expected net monetary benefit (NMB) for each strategy at a range of cost-effectiveness thresholds. As demonstrated in the review of economic evaluation studies, there were few examples of cost-effectiveness models for tic disorders and the model structure was therefore based on the model developed alongside the recently published RCT by Hollis et al., which compared online ERP with online psychoeducation for tics in children.47 Two separate comparisons of cost-effectiveness are presented in this chapter:

1. Online ERP therapy compared to online psychoeducation in children and adolescents. Despite some uncertainties in modelled parameters, the External Assessment Group (EAG) is satisfied that the evidence base on cost-effectiveness for ORBIT is sufficient to enable decision-making. It should however be noted that the online psychoeducation comparator does not align directly with the NICE scope, where face-to-face psychoeducation was stated as the preferred comparator. Comparisons against this standard of care remain an area of future research need.
2. Neupulse stimulation compared to a waitlist control (i.e. no stimulation) in young adults. The EAG notes that there is no evidence regarding the cost-effectiveness of Neupulse. The only existing clinical evidence (transition probabilities) to populate the economic model comes from short-term (4-week), non-published data that were provided to the EAG through personal communication with the company. The EAG are, therefore, of the view that there is currently insufficient evidence to decide on the cost-effectiveness case for Neupulse at the moment, and longer-term follow-up data were required. Further evidence generation is required to identify medium and long-term trajectory of clinical outcomes, to provide transition probabilities for extrapolation beyond 4 weeks, and to understand the full costs of intervention delivery and support. The EAG therefore consider the assessment of Neupulse to remain highly uncertain and more appropriate for an early value assessment.

For comparison 1, data are available from the ORBIT study, and this study has been used to replicate the model structure for the current assessment. We use a 5-state Markov model describing different severity of tics and TS classified according to quintiles of the YGTSS-TTSS score. An option for an absorbing state of ‘no tics’ is included to explore the impact on results of an assumption that some people may gain complete control of tics and TS over time, into adulthood, without treatment. The cohort can die from any state during the model according to all-cause age and sex adjusted mortality probabilities. There is no excess mortality associated with tics and TS included in the model.

The cohort can progress through different severity levels of tics in either 6-monthly cycles (ORBIT) or 4-weekly cycles (Neupulse), according to a set of transition probabilities obtained from the respective RCT data and personal communication with the manufacturer of Neupulse.36,47 A range of alternative assumptions are explored to illustrate the impact of different assumptions on tics progression over the longer term, post treatment, including extrapolation in the longer term based on last observed health state transitions, fixing the cohort in their last observed health state longer term, and switching ERP transitions to the online psychoeducation arm of ORBIT longer term. Intervention cost data are obtained from the literature, though the current price of Neupulse, if it were to be implemented in NHS practice, is not publicly available. Mean Child Health Utility-9 Dimensions (CHU-9D) based utilities and NHS perspective costs associated with each YGTSS-TTSS defined health state were determined using data reported in the ORBIT study.

In line with the NICE reference case, we take a UK NHS perspective, over a modelled lifetime time horizon. Costs and outcomes accruing beyond the first year were discounted at the recommended rate of 3.5% per year.49 A summary of the model characteristics is provided in Table 14.

Model structure

After conducting a rapid literature review of existing economic evaluations of any intervention, in any population for CTDs or TS, we identified one published Markov decision-analytic model that extended the analysis of cost-effectiveness from the ORBIT trial for treating tics in children aged 9–17 years.35 Full details of the ORBIT trial are provided in Table 3, with results of the clinical effectiveness outcomes detailed in Tables 4, 6 and 8. As the ORBIT study model was the only decision modelling study that closely meets the scope for this project, we have built the current model structure around the health states defined in that study. The EAG’s clinical expert advisor confirmed that the model structure from the ORBIT was appropriate for decision-making as it captured disease severity using the most used outcome measure for patients with tics and TS, the YGTSS score. The Markov cohort model structure is illustrated in Figure 9.

FIGURE 9 Schematic diagram of the Markov model structure of the economic model. In each model cycle, the cohort are also exposed to a risk of all-cause mortality where they enter the ‘death’ state from any other model state. Death is an absorbing state in the model, based on age-specific mortality rates reported for males and females in the UK life tables.51 (Figure not reproduced here; see the original article.)

The model comprises five mutually exclusive health states that describe the severity of tics and TS, defined as quintiles of the YGTSS-TGSS score as depicted in the schematic above. The cohort can move through the model states, according to a set of transition probabilities that allow for progression, stability or regression of symptoms in each model cycle. Two different cycle lengths are considered depending on the comparison being evaluated: a 6-month cycle length for ORBIT and a 4-week cycle length for Neupulse. The 4-week cycle length was required for Neupulse due to it being the only available data. Cycle lengths were chosen given the limited availability of data, especially for Neupulse.

Health states are defined according to the level of tic severity as measured using the YGTSS, TTSS.50 The classification was based on the severity structure proposed by Bloch and Leckman, which was also applied in the ORBIT modelling, where tics are classified according to quintiles of the TTSS on the YGTSS: very mild (0–9), mild (10–19), moderate (20–29), severe (30–39), and very severe (40–50).9 An additional, semi-absorbing, tics-free health state was considered for scenario analysis. The purpose of the scenario analysis was to allow for a proportion of patients to become permanently tics-free over time, into adulthood. However, based on clinical expert advice, this was not considered appropriate for the base-case analysis. Definitions of each health state are summarised in Table 15.

An important evidence gap relating to the model structure is that the long-term potential for people with CTDs or TS to become tic-free over time, into adulthood is unclear. Scenario analyses explore the potential for this based on one small study, but caution is required regarding heterogeneity in the underlying severity of that study compared to the studies included in our review.

Population

The modelled populations were different for comparison 1 (ORBIT) and comparison 2 (Neupulse). This reflects the likely different populations in which these treatments would be used in UK clinical practice (e.g. Neupulse in an older population and with more severe tics and TS). The modelled starting cohort’s characteristics are therefore obtained from the key clinical trials of both interventions as described below.

Online Remote Behavioural Treatment for Tics

The population entering the model for ORBIT and online psychoeducation consists of children and young people, diagnosed with CTDs who continue to report bothersome tics after initial psychoeducation, which is in line with the NICE scope.52 The specific participant characteristics were obtained as the mean age and gender proportion across the full study population in the ORBIT trial. The detailed proportion of participants starting in each severity health state undergoing online ERP and psychoeducation treatment was based on data from the ORBIT trial. Most participants entered the model in the moderate, severe or very severe states, with comparatively fewer starting with mild or very mild tics according to the YGTSS.

Neupulse

For Neupulse, the population is restricted to individuals aged 12 and older with confirmed or suspected TS and CTD, with moderate to severe tics. The starting population in the model for the Neupulse and waiting list control arms of the model was therefore based on the mean characteristics across the Neupulse and waitlist control groups from the Maiquez, 2023 trial.36 The Neupulse trial included participants with comparatively more severe tics and TS and significantly higher YGTSS-TGSS score compared to the ORBIT trial population. This necessitated separate comparisons of different populations for CEA. Data for the distribution of the cohort starting in each modelled state at baseline were not available from the published literature. Starting proportions according to our modelled definitions were provided through personal communication with a coauthor from the Maiquez study (Personal communication, Neupulse, May 2024). Modelled population characteristics for both comparisons are summarised in Table 16.

A key evidence gap for the Neupulse comparison relates to a lack of available data in the public domain regarding the starting distribution of the cohort across Markov model states. However, information was helpfully provided by the company. It was not possible to conduct a fully incremental analysis, including a comparison or Neupulse with ORBIT as the clinical trials were conducted in different population groups. Importantly, there was no evidence to enable a cost-effectiveness assessment of any of the interventions compared to current UK standard of care, as defined in the NICE scope (online/face-to-face psychoeducation). The online psychoeducation arm of ORBIT uses the BiP platform and may therefore be a more active comparator than current UK clinical practice. Due to the lack of available data, the assessment therefore relies on the comparisons considered in the key clinical trials.

Perspective, time horizon and discounting

The base-case analysis reports incremental cost per QALY gained over a lifetime time horizon (up until age 100) from a UK NHS and Personal Social Services perspective. All costs and QALYs beyond 12 months were discounted at a rate of 3.5% per year in line with the NICE reference case.49 Shorter time horizons (2, 5 and 10 years) are explored in scenario analyses to reduce the uncertainty associated with longer-term tics and TS progression. For the ORBIT comparison, the model starting point is 18 months, as the model includes the costs and QALY payoffs reported in the clinical trial up to 18 months’ follow-up. Discounting formulae are adjusted accordingly. A 2-year time horizon therefore refers to 18 months plus 2 years of extrapolation for the ORBIT comparison. No cost-effectiveness trial data are available for the Neupulse; therefore, the model is started at time zero.

A key evidence gap relates to long-term extrapolations beyond 18 months for the ORBIT comparison and beyond 4 weeks for the Neupulse comparison. This is a significant source of uncertainty, and it may therefore be appropriate to consider shorter time horizons given the lack of robust long-term data on tic/TS progression with and without treatment.

Model parameters: transition probabilities

Transition probabilities based on observed data

For the ORBIT study, costs and QALYs from the trial are used directly within the economic model as payoffs up to 18 months (first three model cycles). The starting proportions for the Markov cohort were taken as the distribution of health state occupancy at the end of the ORBIT study follow-up at 18 months. Beyond that time point, baseline transition probabilities were derived from the online psychoeducation arm of the ORBIT trial follow-up, calculated from observed transitions between 6 and 18 months, aiming to assess the natural course of tic progression post intervention.34,47 Specific transitions between 6–12 and 12–18 months were not reported. The calculation of transition probabilities in the ORBIT study excluded transitions between baseline and 6 months to remove the initial steeper improvement in tics in both arms of the trial from the calculation of long-term extrapolation transition probabilities. The 6-monthly transitions calculated from the observed data between 6 and 18 months were applied in the model from cycle 3 onwards in the base-case analysis. Transition probabilities for ERP were also obtained directly from the ORBIT study, using a similar approach for data from the intervention arm of the trial.

For Neupulse and waiting list control, a similar approach was taken to deriving transition probabilities. There was no trial-specific cost nor QALY data available to inform the model; therefore, the starting point for the cohort was time zero. Transition probabilities were not available from the published literature. Instead, 4-week transitions were obtained through personal communication with the company. The EAG provided a pre-specified transition matrix form, in which count data from the study were entered and returned to the EAG for use in the model.

All transition probabilities are incorporated into the model using Dirichlet distributions using alpha values reflecting transition counts back calculated from transition probabilities reported in Table 17 of the ORBIT NIHR report, assumed to reflect 6-monthly transitions, though it is not explicitly stated in the report how these were derived. For Neupulse, the Dirichlet distributions are parameterised using 4-week alpha counts provided through personal communication with the company. Data underpinning these distributions are provided in Table 17.

Evidence gaps: it should be noted that Neupulse transition probabilities were not available from the published literature and were instead sourced through personal communication with the company. Data were provided up to 4 weeks, which the EAG considers a short time horizon. These data are insufficient to make any strong predictions of longer-term outcomes and clinical effectiveness data up to 6 months, anticipated to be published in future, will help reduce the uncertainty somewhat. Transition probabilities for all comparisons (ORBIT and Neupulse) are derived from very small counts. This leads to substantial uncertainty regarding treatment effect sizes in the longer term. For example, transition probabilities derived from < 10 observations should be interpreted particularly cautiously as they may lead to misleading deterministic base-case results. Larger studies of the interventions are required, with longer-term follow-up, particularly for Neupulse required to improve modelling projections.

Transition probabilities to the absorbing ‘tics free’ state in scenario analyses

Inclusion of a semi-absorbing state of ‘tics free’ is explored in a scenario analysis. The intention of this state is to model a proportion of people who may ‘grow out’ of tics in the longer term, returning to a QoL assumed equal to that of the general population. The proportion of the cohort entering the absorbing ‘tics free’ state in the scenario analysis is derived from a study reporting that, over one-third (37%) of a combined sample of across two studies of N = 82 children with TS were completely tic-free. We use the median follow-up from Bloch et al., of 7.3 years to fit an exponential distribution to the median to determine a linear progression into the tics free state over time for this scenario analysis.9

Since cycle-specific transitions into the tics-free state were unavailable, we used an exponential formula based on the median time to an event to estimate the proportion of individuals who become tic-free in each 6-monthly cycle. Transitions to the tic-free state were applied from each modelled tic severity state at an equal rate due to a lack of evidence to support a tic severity adjustment.

The cycle-specific transition into the tics-free state was calculated as:

P(t) = exp (ln (0.5)/median number of cycles × current cycle)

Where:

• P(t) is the probability of being disease-free at time t.
• ln is the natural logarithm.
• 0.5 represents the 50th percentile (median).
• The median number of cycles is the time at which 50% of the cohort remains disease-free.
• The current cycle is the specific time point for which we are calculating the probability.

Evidence gap: the long-term trajectory towards a tic-free state, that is the potential for people to have completely resolved tics over time, into adulthood is not well understood. There appears to be limited long-term data, and it is unclear how the time to achieve tics-free would depend on the underlying severity of chronic tics of TS. Further evidence is required to better inform modelling of any potential possibility for transition to a tics-free state over time and whether different intervention types might have differential impacts on these outcomes. Our simplistic scenario analysis assumes that transitions occur at the same rate from any state of the model. This assumption is likely to lack face validity, given that people become tics gradually over time. However, in the absence of alternative data, this was deemed the most appropriate approach in which to conduct the scenario analysis.

Longer-term transition probabilities beyond the end of the observed period from trial data

The EAG note that there is significant uncertainty and a lack of data to describe the long-run transitions between model health states beyond the observed periods in the respective trials. The tentative base-case analysis is formulated on the assumptions which the EAG consider potentially plausible, and in line with the clinical expert advice we have received, but a range of different assumptions are applied in scenario analyses. For online psychoeducation and ORBIT, we assume that the transition probabilities between 6 and 18 months can be extrapolated over a lifetime horizon. These transitions exclude the initial improvement in both arms seen up to 6 months in the study. We assume that, to achieve these outcomes, ongoing use of the BiP platform would be required in both arms of the model. However, the costs of ongoing therapist support would not be incurred. This assumes that both ERP and online psychoeducation are successful in the longer term and that learnt behaviours can be implemented indefinitely.

For Neupulse, the mechanism of action is electrical stimulation. It is therefore assumed that the intervention is effective only so long as it is used. For the base-case analysis, we assume, similarly, to ORBIT that transition probabilities from 4 weeks can be extrapolated longer term. Again, this would require that Neupulse treatment is continued indefinitely. However, it may be more appropriate to assume that full effectiveness for Neupulse is achieved at 4 weeks and that ongoing treatment would not lead to further improvement in symptoms, with the cohort instead being held in their observed state at 4 weeks.

All these assumptions are surrounded by considerable uncertainty, and it is not possible to derive a clear base-case set of assumptions. For that reason, we have reported several scenario analyses as follows:

• Assuming that longer-term transition probabilities switch from the active intervention to comparator transitions beyond the observed time points. This would be a pessimistic assumption for Neupulse. However, it would be optimistic for ORBIT given that there is some evidence that transition probabilities may be favourable to psychoeducation longer term with initial gains potentially being reversed.
• Assuming that each cohort is held in their last observed state for the remainder of the economic model.
• Given that none of these scenarios for long-term modelling are supported by robust evidence and are, at best, speculative, the model time horizon has been reduced to 10, 5 and 2 years in scenario analyses to minimise the impact of long-term extrapolation assumptions on results.

Evidence gaps: long-term transition probabilities are unknown for all the interventions. It is unclear how transition probabilities for Neupulse would change beyond the study intervention phase (4 weeks). There is some suggestion that transition probabilities for online psychoeducation may be slightly favourable to psychoeducation longer term suggesting that gains in the ERP arm might not be sustained longer term. However, all these data are highly uncertain, and it is more appropriate to consider the balance of uncertainty around model outcomes using probabilistic analyses, with the uncertainty captured through small counts informing the Dirichlet distributions of transitions. Future, longer-term follow-up studies are required to better understand the long-term impact of interventions on chronic tics and TS.

Model parameters: costs and resource use

Intervention costs: Online Remote Behavioural Treatment for Tics and online psychoeducation

All cost items for ERP and online psychoeducation were sourced directly from the ORBIT trial. It was not possible to identify each item of resource use in sufficient detail to enable a recalculation of costs using current Personal Social Services Research Unit costs of staff time. Therefore, costs reported in the ORBIT study are inflated to 2023 values for application in the model. The Neupulse device cost was obtained from the company upon request. A summary of intervention costs, inflated from those reported in the ORBIT study is provided in Table 18. The costs for consideration include:

• The BiP platform costs (fixed cost of setting up platform and variable cost per text sent from the platform, based on usage). The BiP platform was used in both arms of the study in the ORBIT trial, with an assumed throughput for calculation of per person costs of N = 222 (combined sample across both arms of the ORBIT study). However, the ORBIT study only included costs in the ERP arm for their economic modelling. The EAG consider it appropriate to include the costs of the platform in both arms of the study to align the resource use incurred with the benefits derived. The throughput is varied between 100 and 1000 for sensitivity analyses.
• Costs of therapist support time, including a fixed component for training and a variable component of service delivery and patient support.

For the EAG analyses, costs were split into an ‘upfront’ initial cost, as described above, plus and ongoing cycle-specific cost based on variable cost of platform usage. The ongoing cycle-specific costs are applied to both arms in the base-case analysis, on the assumption that long-term transition probabilities can only be achieved if use of the platform is continued indefinitely over time. These assumptions are tested in scenario analyses. Full details of the intervention costing are provided in Table 18.

Evidence gaps: The ORBIT study used the BiP platform in both study arms, but costs in the ORBIT economic evaluation only included costs in the ERP arm. The EAG preferred to include the costs in both arms because this reflects the potential resource use incurred in the trial. However, it is unclear how the availability of the BiP platform would have impacted on the effectiveness of the online psychoeducation arm of the study, but it is plausible that it would have overestimated effectiveness in the comparator arm relative to standard care where the platform was not available. Similarly, the exact throughput for the BiP platform is unclear, and this impacts on costs. For the longer-term extrapolation, the EAG includes some ongoing costs of platform access for the model time horizon because we also extrapolate short-term transition probabilities over the longer-term time horizon. However, it is unclear which combination of intervention costs and long-term extrapolations is most appropriate.

Intervention costs: Neupulse

The cost for Neupulse consists of two components: an initial device purchase cost and a monthly subscription cost (Table 19). A definitive purchase and subscription cost to the NHS was not available, but the company confidentially provided tentative prices that could be used to inform the committee’s decision-making. Given the uncertainty of the definitive price that would be applicable in UK clinical practice, if rolled out on the NHS, the economic model explored the impact of a range of different prices on cost-effectiveness results. The model assumed that the intervention cost would need be incurred indefinitely to maintain the intervention’s effectiveness. An ongoing monthly subscription cost is also applied in each model cycle. The monthly subscription includes access to an app for controlling the device, daily disposable hydrogel pads, storage of medical data associated with the therapy, and access to both digital and human product support resources. It was assumed that there were no intervention costs assigned to the waiting list control arm of the model and that no additional costs would be incurred associated with training or therapist support as Neupulse is intended as a self-administered device, with all required patient training provided within the company provided materials.

Evidence gap: the costs of Neupulse are not currently publicly available and have been provided to the EAG through personal communication with the company. It is unclear how the costs provided would translate to costs in UK NHS practice if the device was made available to the NHS. However, it is feasible that a combination of factors, including additional costs of sales to the NHS, or reductions due to economies of scale, would be relevant, meaning that the cost to the NHS may not be accurately represented by the costs used in the economic modelling. While the company suggest that there are no training or staff costs associated with the use of the device, there are, as yet, no data to support or refute this assumption.

Health state costs

The EAG’s understanding of UK clinical practice is that management of children with tics and TS is most likely to take place in paediatrics, or CAMHs services, with management of adults typically occurring in community mental health teams of neurology clinics. Within the ORBIT study, the authors compared resource use and costs between ERP and online psychoeducation across several different categories of costs (specialist tic services, community services, hospital services, accident and emergency and medication costs). They found no clear differences between groups for any of the cost categories, despite potential clinical benefits. The authors also explored the impact of tic severity on these cost categories. The only cost category identified as being significantly determined by tic severity was the use of specialist tic services. Therefore, the model used health state costs that were informed by a regression analysis to determine health state-specific costs (defined as use of specialist tic services), with adjustment for age, comorbidities using the development and wellbeing assessment, and gender.

The EAG’s base-case analysis uses the cost data from the trial on the grounds that it is a conservative estimate of the potential cost-savings of moving between different health states but acknowledging that it might not capture the full range of services utilised by patients with chronic tics and TS. The EAG’s clinical expert and one response to a survey of specialist committee members for this topic suggested that improvements in tic severity might also reduce the need for medication. The EAG therefore explored a scenario analysis where an additional medication cost was added to the moderate and severe YGTSS health states, but not to the mild/very mild states. Medication usage also likely varies across UK clinical practice, both in terms of the proportion of patients receiving medication and the distribution of medications among those who do. For the scenario analysis, the EAG applied 6-month costs of Clonidine (assumed 50%) and Haloperidol (assumed 50%) in the moderate state with a cost of Clonidine (45%), Risperidone (10%) and Haloperidol (45%) applied in the severe and very severe tics states. Costs of medications for the scenario analysis were sourced from eMIT 2023 prices as:

• Clonidine 25 mcg, dose up to 150 mcg per day, unit cost £2.95 per pack size of 112 tablets, 10 packs required per 6-month model cycle, cost per cycle: £29.50.
• Haloperidol 5 mg, dose of up to 5 mg per day, unit cost £0.84 per pack size of 28 tablets, 6 packs required per 6-month model cycle, cost per cycle: £5.04.
• Risperidone 1 mg, average dose of approximately 5 mg per day, unit cost £0.77 per pack size of 60 tablets, 16 packs required per 6-month cycle, cost per cycle: £12.32.

Based on the weightings above, an additional cost of £17.27 and £16.78 is added to the severe and very severe tic states respectively.

All cost parameters were incorporated into the model probabilistically using gamma distributions. Where a measure of spread was not available for any of the cost parameters, the standard error (SE) was assumed to be 10% of the mean. All cost parameters are summarised in Table 20.

Evidence gap: There is limited evidence external to the clinical trial with regards to the most appropriate health state costs to apply in the economic model. A survey of clinical experts for this assessment failed to provide any additional information to reflect variation in UK clinical practice. The EAG are also aware tha4t many people may not have access to services specifically to treat their tics/TS and that in some cases, these services may be embedded within other services for other comorbid conditions. Care is required to avoid double-counting of health state costs, and further research is needed to better understand the care pathways followed by patients with tics and TS in UK clinical practice. Similarly, there are no data from the ORBIT study regarding the health state costs to apply for adults. This is an added uncertainty, particularly for the Neupulse assessment.

Model parameters: health state utility values

Health state utility values were obtained from the ORBIT study. In the ORBIT study, utilities were calculated from responses to the CHU-9D.53,54 Responses within the trial were provided by a mix of patient (young person) and proxy (assumed parent) reports at baseline and at 3, 6, 12 and 18 months’ follow-up. It is unclear whether child self-reports, parent proxy-report or a combination of both were used to derive health state utility values (HSUVs) for application in the model. Responses to the CHU-9D were converted to utilities using a preference-based algorithm.55 As with costs above, mild and very mild states incurred the same utility as did severe/very severe states. The estimation approach for HSUVs was not described in the study, and, therefore, it was unclear whether these were based on regression estimates or descriptive data. For the scenario analysis that includes the semi-absorbing tics free state, utility is assumed to be equal to the very mild/mild health state, acknowledging that people with tics and TS often have multiple comorbidities and it would likely be inappropriate to assign UK general population norms, despite the absence of tics.

Table 21 summarises the HSUVs obtained from the ORBIT trial data and applied in the model. All utilities are incorporated using beta distributions. When applied to health states, QALY payoffs were halved to accommodate the 6-monthly cycle duration. The utility of the death state is set to zero. All utility inputs to the model are age and sex adjusted, allowing for reduced HSUVs in all health states as the cohort ages through the model. Adjustment multipliers were calculated as general population EuroQol-5 dimensions (EQ-5D) based utility at each stage/general population utility at the start age of the model cohort. At any given time, the UK general population norm is calculated using the method described by Ara and Brazier.56

Evidence gaps: As demonstrated in the clinical-effectiveness review, the relationship between tic severity and HRQoL is unclear. Changes in the clinical outcome do not necessarily lead to changes in QoL or HSUVs. This is an important area for future research. While some evidence is available from the ORBIT study for children, there are no studies reporting the relationship between tic severity and EQ-5D in adult patients. The EAG understands the rationale for pooling very mild/mild and very severe/severe states for calculating HSUVs. However, if a true relationship between tic severity and generic HRQoL exists, the approach may underestimate the QALY benefits of effective treatments. It is unclear if the lack of statistical significance of health state in the ORBIT study is due to small sample sizes, or a true lack of effect. Further studies are required to better understand the relationship between tic severity and QALY benefits. The lack of evidence is particularly acute in an adult population.

Model analysis

Results are presented as incremental cost per QALY gained from a UK NHS perspective. ICERs were determined for pairwise comparisons of ERP versus psychoeducation and active Neupulse stimulation versus waitlist control.

The cost-effectiveness of each intervention versus comparators was evaluated at an indicative threshold value of £20,000–30,000 per QALY gained.57

The model is constructed to be fully probabilistic. Probabilistic analyses were performed for each modelling scenario, using Monte Carlo simulation with 50,000 iteration runs (the minimum number of runs required to achieve convergence of probabilistic ICERs), to construct the cost-effectiveness acceptability curves (CEACs). This involved varying the inputs by randomly assigning parameter values from predefined uncertainty distributions as described in the model input parameter tables. A probability distribution was assigned to each model input parameter, reflecting the degree of uncertainty due to sampling variation. Gamma distributions were applied to represent uncertainty surrounding cost inputs, beta distributions were used for utility parameters, and Dirichlet distributions58 were applied for transition probabilities. When SEs for parameters were unknown, they were assumed to be 10% of the parameter value to define the PSA distributions. The CEACs show the probability of an intervention being cost-effective compared to comparators for a range of cost-effectiveness threshold values. Results are also presented as iterations of simulations for intervention versus comparator on the cost-effectiveness plane to illustrate the impact of parameter uncertainty on results.

Scenario analysis

There are multiple points of residual uncertainty surrounding many of the key assumptions and model inputs, both for the comparison of ORBIT versus online psychoeducation and Neupulse versus waitlist control. This makes it difficult for the EAG to define a clear set of base-case assumptions. However, we have selected a set of assumptions that, on balance, appear plausible as a starting point and conduct extensive scenario analyses around these. Table 22 summarises some of the key assumptions, residual uncertainties and EAG scenarios conducted to illustrate that uncertainty.

Model validation and face validity checks

One health economist built and parameterised the economic model. Model formulae and parameter inputs were verified against source by a second health economist, who also conducted a range of face validity checks, changing one model parameter at a time and re-running results to assess consistency. Adaptations were made where issues were identified. The EAG were unable to identify any longer-term data against which the model outputs could be externally validated. The model outputs were instead presented and discussed among the team and clinical expert did not identify any issues of concern regarding the plausibility of the outputs. However, they did flag that the highly uncertain evidence base meant that caution should be exercised when interpreting the results of a lifetime horizon model, given the lack of longer-term follow-up beyond 18 months for ORBIT and lack of published transition probability data for Neupulse.

The EAG has undertaken a range of further verification tests, based on an adaption of those proposed by Tappenden et al.59 The results of these verification checks are provided in Table 23, applied to the EAG’s preferred base-case analysis. No further issues were identified with the final version of the EAG’s model.

Results: Online Remote Behavioural Treatment for Tics

Markov cohort traces

Figures 10 and 11 illustrate the base-case Markov cohort traces for online psychoeducation and ORBIT respectively over a lifetime horizon. The cohort traces are very similar, particularly after the first few model cycles. This is due to differences up to 18 months being quite quickly offset by slightly favourable, but highly uncertain transition probabilities for online psychoeducation.

FIGURE 10 Markov cohort trace – online psychoeducation. (Figure not reproduced here; see the original article.)

FIGURE 11 Markov cohort trace – ORBIT. (Figure not reproduced here; see the original article.)

Base-case analysis

It was not possible to determine a definitive base-case set of assumptions. However, a set of assumptions, as outlined in Table 22 are used as a starting point for exploring uncertainty. It is important to consider the full range of scenario analyses carried out and note that some scenarios have a major impact on the reported ICERs and probability of cost-effectiveness. Probabilistic and deterministic ICERs are reported in Table 24. The deterministic analysis suggests a low ICER of £9289 for ORBIT compared to online psychoeducation, whereas the probabilistic ICER shows ORBIT to be, on average, more costly with minimal differences in effectiveness (i.e. dominated by online psychoeducation). While these results may appear significantly different to each other, they should be interpreted in the context of the uncertainty surrounding the results output. The magnitude of QALY gains and losses are small in the context of a lifetime horizon, driven by broadly similar transition probabilities for both arms of the model, slightly favouring online psychoeducation, offsetting early QALY gains observed in the trial.

The results should be interpreted considering the uncertainty surrounding the model outputs. Observation of the spread of iterations from the 50,000 iterations from the probabilistic analysis on the cost-effectiveness plane (Figure 12) indicates substantial uncertainty. An approximately equal proportion of iterations lie above and below the £20,000 per QALY threshold line, indicating that neither strategy is clearly optimal under base-case assumptions. Furthermore, many of the simulated cost and effect pairs lie in the northwest quadrant of the plane, indicating that we cannot rule out ERP being more costly and less effective over a lifetime horizon. CEACs, provided in Figure 13 also illustrate that the uncertainty persists across all threshold values of willingness to pay (WTP) for a QALY gain.

FIGURE 12 Base-case incremental scatter plot of simulations on the cost-effectiveness plane for ORBIT vs. online psychoeducation. (Figure not reproduced here; see the original article.)

FIGURE 13 Base-case CEACs for ORBIT and online psychoeducation. (Figure not reproduced here; see the original article.)

Scenario analysis

The results of a range of scenario analyses described throughout the report and summarised in Table 22 are provided to further explore uncertainty surrounding key modelling assumptions, including intervention costing, long-term extrapolation of transition probabilities and the potential for inclusion of a tics free semi-absorbing state in the model. Results of the scenario analyses are reported in Table 25. All analyses are probabilistic, based on 50,000 iterations, and probabilities of cost-effectiveness at a £20,000 and £30,000 threshold value of WTP for a QALY gain are reported for completeness, and to illustrate the residual decision uncertainty. Accompanying deterministic results are provided in Appendix 4, Table 29. In general, results were most sensitive to assumptions about the costs of psychoeducation, including whether the variable costs of the platform are included. When combined with uncertainty around the most appropriate extrapolation assumptions, uncertainty increased further. The results show an improving cost-effectiveness case for ORBIT over shorter-term time horizons. This is because long-term transition probabilities are extrapolated from the 18-month observed trial data. Transition probabilities between 12 and 18 months tended for favour the comparator, psychoeducation group (despite the overall profile of effectiveness at 18 months favouring ORBIT). Extrapolating these transitions over a lifetime horizon therefore favours the psychoeducation group. The maintenance of a treatment effect over the longer term is a key driver of cost-effectiveness results and different assumptions lead to substantial variation in the ICER. This also explains why re-setting transition probabilities to the psychoeducation arm might be anticipated to increase the ICER substantially for ERP but this is not observed in the model outputs. The probability of ORBIT being cost-effective at a WTP threshold of £20,000 per QALY gained varies between 45% up to 90% in a highly optimistic scenario where it is assumed that the cohort remain in their last observed health state at the end of the trial follow-up period.

In terms of overall findings, it should also be acknowledged that the psychoeducation comparator delivered as part of the ORBIT study was potentially more beneficial than the treatment options widely available to patients in UK NHS practice. In such scenarios, estimates of cost-effectiveness of ORBIT compared to usual care may lead to lower ICERs, and this is an area for future research.

Results: Neupulse

Markov cohort traces

Figures 14 and 15 show the modelled Markov cohort traces for waiting list control and Neupulse (assumed standard of care) respectively.

FIGURE 14 Base-case Markov cohort traces for wait list control. (Figure not reproduced here; see the original article.)

FIGURE 15 Base-case Markov cohort traces for Neupulse. (Figure not reproduced here; see the original article.)

The Markov cohort traces show a clear difference between the Neupulse arm, with most of the cohort in the mild health state over time, compared to the waiting list control arm, with the majority in the moderate health state. These cohort traces reflect the optimistic transition probabilities provided by the company to the EAG. However, it is crucial to note that these are based on one set of transitions, between baseline and 4 weeks only, and are based on very small samples. It is therefore likely that the relative effectiveness of the intervention is overestimated in the longer term as it is highly unlikely that the level of effectiveness seen up to 4 weeks would be maintained over a full lifetime horizon.

Base-case analysis

Given the lack of available data to populate the Neupulse model, it is not reasonable to derive a definitive base-case analysis. Instead, the potential uncertainty surrounding results should be interpreted across the range of scenario analyses presented. Neupulse was more costly than the waiting list control at the prices provided by the company. The additional costs are driven by the costs of device replacement and ongoing subscription which would be the minimum costs required to allow extrapolation of highly optimistic 4-weekly transition probabilities over a lifetime horizon. This scenario likely represents the maximum feasible QALY gain that could be achieved if an intervention could almost entirely resolve all tics to the ‘mild tics’ health state. Whether this is achievable in the longer term is unclear. Full details of total Neupulse costs are confidential in nature and so were provided only to the committee to aid in decision-making.

Scenario analysis

The results of scenario analyses conducted to explore the sensitivity of the base-case results to a range of modelling assumptions were provided confidentially to decision-makers. Results, in terms of the probability of cost-effectiveness under various scenarios are detailed in Table 26. All analyses are probabilistic, with the probability of cost-effectiveness indicated. Results were most sensitive to assumptions about the long-term extrapolation from 4 weeks onwards, clearly demonstrating the need for future research around long-term effectiveness. Different assumptions cause wide variation in the ICER between just over £10,000 per QALY gained in an optimistic scenario analysis that extrapolates 4-weekly transitions for a lifetime (probability of cost-effective = 87%) to over £300,000 per QALY in the less optimistic scenario analysis (probability cost-effective = 0%) where transitions are crossed over to the waitlist control group after the observed 4-week period. Deterministic analyses are provided in Table 30, Appendix 4.

Given the confidential nature of initial and ongoing Neupulse intervention costs, two-way scenario analyses are conducted to illustrate the impact of varying intervention and subscription costs on results. These scenarios are applied to two alternative assumptions about long-term transition probabilities (4-weekly transitions are carried forward, and the cohort fixed in state after 4 weeks, assuming maximum effectiveness achieved at 4 weeks). Analyses are based on NMBs with a WTP = £20,000 per QALY. Results are reported in Figures 16 and 17. Results of these analyses further emphasise the substantial residual uncertainty regarding intervention costs and long-term extrapolation assumptions.

FIGURE 16 Two-way scenario analysis of initial and subscription costs for Neupulse (assumes long-term transition probabilities extrapolated). (Figure not reproduced here; see the original article.)

FIGURE 17 Two-way scenario analysis of initial and subscription costs for Neupulse (assumes cohort held in last observed state). (Figure not reproduced here; see the original article.)