Abstract

OBJECTIVES: Poor mental wellbeing is one of the leading causes of long-term sickness absence from work despite conditions, such as stress and anxiety, being treatable and often preventable. The aim of this analysis was to conduct economic modelling and provide data on costs and benefits to employers who are considering implementing a workplace intervention to improve wellbeing. Additionally, the analysis was used to assess any changes in employee outcomes (e.g. productivity and staff turnover).

METHODS: A cost-consequence model with a one-year time horizon was developed to assess the impact of workplace mental wellbeing interventions. Because all workplaces are different, it is not useful to present one single base case. Instead, the model generates a series of hypothetical case studies, with varying levels of absenteeism, presenteeism and staff turnover, as well as different levels of productivity and staff replacement costs. Several mental wellbeing interventions were compared with 'no intervention' (current practice) to calculate the total incremental costs and incremental cost per employee.

RESULTS: In the hypothetical case study with 50 employees and an intervention cost of £100, the intervention had a net cost saving of £3,395 per employee. Savings were due to reductions in absenteeism and presenteeism. Sensitivity analysis assessed the impact of varying each input, to reflect that the inputs will vary substantially for each individual organization and setting. The intervention is more likely to be cost saving when the baseline level of absenteeism and staff turnover are high, and the level of presenteeism and intervention cost are low.

CONCLUSIONS: Mental wellbeing interventions may influence a range of outcomes but outcomes demonstrating a mental wellbeing benefit to employees may be challenging to translate into monetary value. The model can be used by decision makers and employers to understand the potential economic and wellbeing implications of implementing a mental wellbeing intervention in the workplace.

Abstract

OBJECTIVES: In the UK, the National Institute for Health and Care Excellence (NICE) requires that "relevant" cost-effectiveness studies and health-related quality-of-life (HRQoL) data are identified using "systematic" and "transparent" searches and cost and healthcare resource use (CHRU) data are "identified systematically". Systematic literature reviews (SLRs) can be resource intensive and time consuming and this study aims to help determine whether the SLR approach is an efficient use of limited resources.

METHODS: We reviewed what methods companies submitting to NICE claimed to use (systematic versus other), whether their methodologies reflected this, what effect this had on how Technology Appraisals (TAs) were appraised, and what this may mean for acceptability of future methods. A sample of company submission documents for NICE TAs (1 March to 18 June 2024) were retrieved.

RESULTS: 22 TAs were identified. 20 single TAs (STAs) described using SLRs to identify EEs and HRQoL. 19 of these STAs described using SLRs for CHRU data. 1 STA submission did not describe a CHRU SLR, instead using reference costs and a pragmatic review of similar models. The Evidence Assessment Group (EAG) did not comment on this approach in its critique. 2 TAs were rapid assessments, meaning SLRs were not feasible. Instead, a pragmatic approach used evidence from 2 living reviews.

CONCLUSIONS: Economics reviews in NICE TAs continue to mostly be SLRs despite no explicit requirement for this. We identified no explicit criticism from the EAG where a pragmatic approach was taken. Because SLRs expend large resources, this research suggests that a carefully considered pragmatic approach could be sufficient to identify relevant data using transparent and replicable methods. Advances in reviewing — for example the integration of artificial intelligence in record selection or data extraction — may help to bridge the gap and make companies more comfortable in using pragmatic reviews for highly valuable TA submissions.

Abstract

OBJECTIVES: Modifying the delivery of healthcare in order to reduce environmental will come at some cost, whether that cost is monetary (i.e. expenditure to reduce harmful effects) or via health (patients switching to less harmful, but less cost-effective treatments). This study aimed to estimate the public's views about such trade-offs.

METHODS: 508 members of the public completed a discrete choice experiment (DCE), which asked them to state their preferences between different health system outcomes, measured by (i) life expectancy, (ii) carbon emissions and (iii) impact on biodiversity. Two alternative sets of questions were presented. The first reported that the environmental effects would be felt within the respondent's own country of residence, and the second reported that the effects would occur elsewhere in the world.

RESULTS: People were willing to lose an average of 0.75 years of life expectancy in order to see a 5% reduction in carbon emissions. They were willing to lose 1.59 years of life expectancy to save 100 species from becoming extinct. Conversely, people would require an increase of 1.55 years of life expectancy to see carbon emissions increase by 5% and would require 2.88 additional years of life expectancy if 100 more species were to become extinct. When the environmental impact was described as occurring outside the respondent's own country, the strength of preference was reduced by approximately 50%

CONCLUSIONS: Whilst this study was exploratory and should not be used for policy making, it demonstrates that it is feasible to elicit opinions from the public around trade-offs between health outcomes and environmental outcomes.

Abstract

OBJECTIVES: Risk of bias (RoB) tools aim to identify systematic error or deviation from the truth in primary studies. While RoB assessment (RoBA) of randomised controlled trials (RCTs) is well established, assessment of other trial designs is less standardised. We investigated which tools are currently used for RoBA in systematic reviews (SRs) of non-randomised studies. To consider how RoBA has changed over the past 20 years, we compared our findings with those of Deeks et al (2003), who conducted an evaluation of RoBA tools in SRs including non-randomised intervention studies.

METHODS: A pragmatic search of Medline identified 66 2023 SRs, including 940 non-randomised primary studies. Due to lack of consistency in review authors' descriptions of primary studies, we conducted our own assessment, classifying them as non-randomised controlled trials (21), single arm trials (183), cohort studies (198), or case series (538).

RESULTS: Across 39 SRs including non-randomized comparative studies, ten different RoB tools were used. Six SRs (15%) conducted no RoBA of included studies. Across 65 SRs including non-comparative studies, 14 RoB tools were used, with 15 SRs (23%) conducting no RoBA of the included single group studies. For comparative and non-comparative studies, most SRs used a Newcastle-Ottawa, Joanna Briggs Institute (JBI), or MINORS tool for RoBA.

Compared with 2003, more authors are now conducting RoBA (86% v. 33% in 2003) and more authors are using an existing standardised tool (80% v. 14% in 2003). However, for up to 55% of the 2023 SRs, the choice of tool may not have been appropriate.

CONCLUSIONS: Specific tools used for RoBA of non-randomised studies have changed over the past two decades, with a positive trend towards a greater awareness of the importance of RoBA, and consistency in the tools used. However, matching each study design to the most appropriate tool remains challenging.

Abstract

OBJECTIVES: CAR-T is a type of personalised immunotherapy treatment that has may provide a proportion of people with haematological cancers that are considered difficult to treat, with a potential 'cure'. A mixture cure model (MCM) is a type of survival model that uses event data from clinical trials to account for cured and non-cured sub-populations within the overall trial population and is considered appropriate for modelling CAR-T outcomes. Immature, highly censored trial data makes it difficult for health technology assessment (HTA) bodies to trust predicted cured and non-cured proportions, however. The objective was to quantify the impact of increasingly mature clinical trial data on the predictive capabilities of MCM.

METHODS: Published 'ZUMA-1' trial (axicabtagene ciloleucel for treating refractory large B-Cell lymphoma) Kaplan Meier (KM) data were digitised to generate pseudo individual participant data (IPD). The 'full' IPD were truncated to replicate three early trial cuts with increased censoring, representing new datacuts, which were validated with trial publications where available. MCMs were fit to the full dataset and the early trial cuts, using standard parametric distributions for the 'uncured' population. Four analyses were compared using graphed long-term survival predictions, goodness-of-fit statistics, median survivals and cure fractions.

RESULTS: Application of MCMs to early trial cuts representing 70% and 60% censoring did not accurately reflect the long-term survival in the full dataset. When censoring is 60%, MCMs are not expected to accurately predict long-term survival. When censoring is <60%, the perceived risk that subsequent data collection may change the trial findings is anticipated to be relatively low. Further, with <60% censoring, more consideration should be given to the model capabilities and alternative modelling methods compared to the statistical fit of MCM.

Abstract

OBJECTIVES: Randomised controlled trials (RCTs) are the gold standard for assessing efficacy and safety. Increasingly, health technology assessment (HTA) considers evidence from non-randomised studies. Guidance recommends synthesising different designs separately due to their different inherent biases/limitations. If reviewers misclassify studies, it can affect which studies are included, potentially impacting review findings and the robustness of evidence available to decision-makers and patients. This research aims to develop a clear study design classification system based on ROBINS-I terminology, for use by reviewers of any experience, to use when performing HTA of pharmaceutical interventions.

METHODS: We performed a pragmatic web-based search for existing tools and appraised them, from which to develop a clear algorithm. Tool utility, consistency and user experience was first assessed by web-based survey in a small internal sample of reviewers, each independently using the system to categorise 18 published studies. Following improvements, the updated version was tested in a larger group of reviewers from multiple commercial and public organisations.

RESULTS: We present a graphic tool to identify study designs when performing HTA of pharmaceuticals. In piloting, a median of 7 reviewers (range 4-8) categorised each study. Rater agreement varied widely, with 100% agreement on the designs of 3/18 studies (17%), and =75% agreement on one design for 9/18 studies (50%). The most common sources of disagreement were between different types of cohort studies, and between case series and controlled cohort studies, largely due to inconsistent reporting. Results from testing the revised tool in the larger sample will be available shortly.

CONCLUSIONS: The pilot tool led to too much variation in study design categorisation to be useful. Consequently we present a revised version evaluated across a larger sample of reviewers. Further research will also investigate whether using the tool would change the results of systematic reviews, using a sample of published reviews.

Abstract

OBJECTIVES: Over the past decade, digital health technologies (DHTs) have become increasingly common healthcare interventions and their prevalence is growing further. DHTs can be used across a range of pathways, rather than for the treatment of specific health conditions. This may lead to many issues, including not having a clearly defined comparator in health economic evaluation. Furthermore, pathways differ locally and regionally which makes identifying a relevant comparator especially challenging. This research describes the approaches to this issue when evaluating the health economic impact of DHTs and recommends suggestions for future considerations.

METHODS: A pragmatic literature review was undertaken to identify research that had sought to provide clarity or had outlined a framework for the evaluation of DHTs. This was conducted using unstructured searches on PubMed and Google Scholar. Following this, a series of expert panel discussions and interviews were undertaken whereby the approaches to evaluating DHTs were discussed.

RESULTS: Regardless of the purpose of the DHT, the choice of comparator will be a function of how the intervention interacts with non-digital health care. The DHT may complement or substitute other types of health care delivery or administration systems. The relevant comparator may be easier to identify in settings where the intervention being implemented is in an area where a DHT is already used. The exception is where the new DHT has a wider aspect than the current DHT.

CONCLUSIONS: Identifying the relevant comparator can be difficult when evaluating DHTs. Comparators may differ at local, regional and national levels, particularly where a DHT is replacing part, or all, of a face-to-face care pathway. Decision makers should be supported to develop a framework for the evaluation of DHTs. Each DHT should have an adapted scoping approach, depending on the elements involved, to ensure a suitable approach to evaluation is used.