Our latest research, all in one place. Browse our collection of journal articles, reports and conference proceedings to see how we’re contributing to HEOR research. Remember to:
OBJECTIVES: Risk of bias (RoB) tools aim to identify systematic error or deviation from the truth in primary studies. While RoB assessment (RoBA) of randomised controlled trials (RCTs) is well established, assessment of other trial designs is less standardised. We investigated which tools are currently used for RoBA in systematic reviews (SRs) of non-randomised studies. To consider how RoBA has changed over the past 20 years, we compared our findings with those of Deeks et al (2003), who conducted an evaluation of RoBA tools in SRs including non-randomised intervention studies.
METHODS: A pragmatic search of Medline identified 66 2023 SRs, including 940 non-randomised primary studies. Due to lack of consistency in review authors' descriptions of primary studies, we conducted our own assessment, classifying them as non-randomised controlled trials (21), single arm trials (183), cohort studies (198), or case series (538).
RESULTS: Across 39 SRs including non-randomized comparative studies, ten different RoB tools were used. Six SRs (15%) conducted no RoBA of included studies. Across 65 SRs including non-comparative studies, 14 RoB tools were used, with 15 SRs (23%) conducting no RoBA of the included single group studies. For comparative and non-comparative studies, most SRs used a Newcastle-Ottawa, Joanna Briggs Institute (JBI), or MINORS tool for RoBA.
Compared with 2003, more authors are now conducting RoBA (86% v. 33% in 2003) and more authors are using an existing standardised tool (80% v. 14% in 2003). However, for up to 55% of the 2023 SRs, the choice of tool may not have been appropriate.
CONCLUSIONS: Specific tools used for RoBA of non-randomised studies have changed over the past two decades, with a positive trend towards a greater awareness of the importance of RoBA, and consistency in the tools used. However, matching each study design to the most appropriate tool remains challenging.
OBJECTIVES: CAR-T is a type of personalised immunotherapy treatment that has may provide a proportion of people with haematological cancers that are considered difficult to treat, with a potential 'cure'. A mixture cure model (MCM) is a type of survival model that uses event data from clinical trials to account for cured and non-cured sub-populations within the overall trial population and is considered appropriate for modelling CAR-T outcomes. Immature, highly censored trial data makes it difficult for health technology assessment (HTA) bodies to trust predicted cured and non-cured proportions, however. The objective was to quantify the impact of increasingly mature clinical trial data on the predictive capabilities of MCM.
METHODS: Published 'ZUMA-1' trial (axicabtagene ciloleucel for treating refractory large B-Cell lymphoma) Kaplan Meier (KM) data were digitised to generate pseudo individual participant data (IPD). The 'full' IPD were truncated to replicate three early trial cuts with increased censoring, representing new datacuts, which were validated with trial publications where available. MCMs were fit to the full dataset and the early trial cuts, using standard parametric distributions for the 'uncured' population. Four analyses were compared using graphed long-term survival predictions, goodness-of-fit statistics, median survivals and cure fractions.
RESULTS: Application of MCMs to early trial cuts representing 70% and 60% censoring did not accurately reflect the long-term survival in the full dataset. When censoring is 60%, MCMs are not expected to accurately predict long-term survival. When censoring is <60%, the perceived risk that subsequent data collection may change the trial findings is anticipated to be relatively low. Further, with <60% censoring, more consideration should be given to the model capabilities and alternative modelling methods compared to the statistical fit of MCM.
OBJECTIVES: Randomised controlled trials (RCTs) are the gold standard for assessing efficacy and safety. Increasingly, health technology assessment (HTA) considers evidence from non-randomised studies. Guidance recommends synthesising different designs separately due to their different inherent biases/limitations. If reviewers misclassify studies, it can affect which studies are included, potentially impacting review findings and the robustness of evidence available to decision-makers and patients. This research aims to develop a clear study design classification system based on ROBINS-I terminology, for use by reviewers of any experience, to use when performing HTA of pharmaceutical interventions.
METHODS: We performed a pragmatic web-based search for existing tools and appraised them, from which to develop a clear algorithm. Tool utility, consistency and user experience was first assessed by web-based survey in a small internal sample of reviewers, each independently using the system to categorise 18 published studies. Following improvements, the updated version was tested in a larger group of reviewers from multiple commercial and public organisations.
RESULTS: We present a graphic tool to identify study designs when performing HTA of pharmaceuticals. In piloting, a median of 7 reviewers (range 4-8) categorised each study. Rater agreement varied widely, with 100% agreement on the designs of 3/18 studies (17%), and =75% agreement on one design for 9/18 studies (50%). The most common sources of disagreement were between different types of cohort studies, and between case series and controlled cohort studies, largely due to inconsistent reporting. Results from testing the revised tool in the larger sample will be available shortly.
CONCLUSIONS: The pilot tool led to too much variation in study design categorisation to be useful. Consequently we present a revised version evaluated across a larger sample of reviewers. Further research will also investigate whether using the tool would change the results of systematic reviews, using a sample of published reviews.
OBJECTIVES: Over the past decade, digital health technologies (DHTs) have become increasingly common healthcare interventions and their prevalence is growing further. DHTs can be used across a range of pathways, rather than for the treatment of specific health conditions. This may lead to many issues, including not having a clearly defined comparator in health economic evaluation. Furthermore, pathways differ locally and regionally which makes identifying a relevant comparator especially challenging. This research describes the approaches to this issue when evaluating the health economic impact of DHTs and recommends suggestions for future considerations.
METHODS: A pragmatic literature review was undertaken to identify research that had sought to provide clarity or had outlined a framework for the evaluation of DHTs. This was conducted using unstructured searches on PubMed and Google Scholar. Following this, a series of expert panel discussions and interviews were undertaken whereby the approaches to evaluating DHTs were discussed.
RESULTS: Regardless of the purpose of the DHT, the choice of comparator will be a function of how the intervention interacts with non-digital health care. The DHT may complement or substitute other types of health care delivery or administration systems. The relevant comparator may be easier to identify in settings where the intervention being implemented is in an area where a DHT is already used. The exception is where the new DHT has a wider aspect than the current DHT.
CONCLUSIONS: Identifying the relevant comparator can be difficult when evaluating DHTs. Comparators may differ at local, regional and national levels, particularly where a DHT is replacing part, or all, of a face-to-face care pathway. Decision makers should be supported to develop a framework for the evaluation of DHTs. Each DHT should have an adapted scoping approach, depending on the elements involved, to ensure a suitable approach to evaluation is used.
OBJECTIVES: When modelling the impact of an intervention and comparator, they usually have the same population. However, interventions may alter the clinical pathway, which can impact the target population. This can be a challenge to model, for example, the underlying prevalence of the condition explored might be altered in the new population. We present possible approaches to address this using a case study involving the introduction of a home testing kit for sexually transmitted infections (STI).
METHODS: The home testing kit would reach a much larger population with differing risks. An economic model was developed to investigate the health and cost impact of implementing home testing kits compared to standard in-clinic testing for STIs. The key model outcomes were total incremental cost and the number of complications averted from excess cases being detected through home testing.
Data on prevalence was used to inform comparator population prevalence of STIs, test uptake, and positive results. A meta-analysis of 7 studies estimated that home testing increased positive tests by 71%. For the intervention, evidence on increase in positive tests was leveraged to explore scenarios where prevalence estimates were non-equivalent to the comparator population.
RESULTS: The use of the home test is estimated to be cost-effective with an ICER of £3,865 when compared with standard STI testing assuming equivalent prevalence. The inclusion of a differential increase in completed tests and positive diagnoses resulted in a larger ICER (£3,865 and £13,877 for the optimistic scenario and the lower risk scenario, respectively).
CONCLUSIONS: Modelling different populations between the intervention and comparator can be challenging. This case study outlines an approach to addressing this by leveraging evidence to draw assumptions and guide scenario analyses.
BACKGROUND: A virtual ward (VW) supports patients who would otherwise need hospitalization by providing acute care, remote monitoring, investigations, and treatment at home. By March 2024, the VW programme had treated 10 950 patients across six speciality VWs, including heart failure (HF). This evaluation presents the economic assessment of the Liverpool HF VW.
METHODS AND RESULTS: A comprehensive economic cost comparison model was developed by the York Health Economics Consortium (University of York) to compare the costs of the VW to standard hospital inpatient care [standard care (SC)]. The model included direct VW costs and additional costs across the care pathway. Costs and resource use for 648 patients admitted to the HF VW were calculated for 30 days post-discharge and total cohort costs were extrapolated to a full year. Primary outcomes included costs related to length of stay, readmissions, and NHS 111 contact. The total cost for the HF VW pathway, including set-up costs, was £467 524. This results in an incremental net cost benefit of £735 512 compared with the total SC cost of £1 203 036, indicating a substantial net cost benefit of £1135 per patient per episode (PPPE). This advantage remains despite initial setup expenses and ongoing costs such as home visits, virtual consultations, point-of-care testing, and home monitoring equipment.
CONCLUSION: Our HF VW model offers a substantial net cost benefit, driven by reduced hospital stays, fewer emergency department visits, and lower readmission rates. The study highlights the importance of considering system-wide impacts and continuous monitoring of VWs as they develop.
BACKGROUND: AcuPebble SA100 ('AcuPebble') is a novel wearable medical device to diagnose obstructive sleep apnoea (OSA). This paper investigates the potential economic impact of the technology in the UK through cost savings analysis, and the redirection of savings into further diagnoses.
METHODS: A cost comparison study was conducted, comparing AcuPebble to the standard diagnostic approach of home respiratory polygraphy (HRP) and in-clinic polysomnography (PSG), estimating the net benefit value (NBV) and return on investment (ROI). Cohort size was varied to model the effects of volume discounted pricing and staff training costs. To demonstrate the potential for cost savings, data on the healthcare costs of undiagnosed OSA patients were used to quantify the benefit of increased OSA diagnosis rates, as facilitated by AcuPebble.
RESULTS: For 500 uses of AcuPebble, the NBV in the diagnostic pathway over one year would be in excess of £101,169, increasing to £341,665 for 1,500 uses, £1,263,993 for 5,000 uses, and to £2,628,198 for 10,000 uses, with ROIs of 2.02, 3.03, 5.05, and 6.56, respectively. Given an initial cohort of 1,500 patients, 4,555 extra AcuPebble studies could be completed by redirecting resources from HRP/PSG. Direct cost savings to the NHS from resultant lower undiagnosed rates could be between £24,147 and £4,707,810, based on the cost per use and the percentage of tests that result in a positive diagnosis (varied from 25 to 75% positives).
CONCLUSIONS: AcuPebble presents an opportunity for substantial healthcare savings, enabling an increase in the number of people tested, diagnosed and treated for OSA.
