Our latest research, all in one place. Browse our collection of journal articles, reports and conference proceedings to see how we’re contributing to HEOR research. Remember to:
OBJECTIVES: The UK NHS is committed to achieving net zero direct carbon emissions by 2040. Nitrous oxide (N2O) is a greenhouse gas used for pain relief during contractions in labor. Destruction technology (DT) can help the NHS achieve its target by breaking N2O down into nitrogen and oxygen. This study evaluated the impact of introducing N2O DT in labou=r wards on NHS emissions and costs.
METHODS: A cost-consequence model was developed to estimate the per delivery incremental cost, and NHS cost impact, of implementing DT. A pragmatic literature search was conducted to identify all model inputs. N2O emissions were monetized by converting into CO2 equivalents (CO2e) and using the 2022 marginal abatement CO2e costs. Scenario analysis explored the impact with varying rates of N2O use and one-way deterministic sensitivity analysis was used to identify the main drivers of results.
RESULTS: N2O DT is estimated to be cost saving to the NHS when valuing emissions monetarily. Per-delivery savings vary from £14.09 to £135.20 for emergency caesarean section labors (which use less N2O) and labors using N2O for every contraction, respectively. The per-delivery cost of mobile DT units is £5.57, and £10.10 for central units. It would cost the NHS £22,512,210 and £40,765,953 to purchase and maintain mobile and central DT units, respectively, for all UK consultant-led labor wards. Sensitivity analysis shows the amount of N2O used, the carbon value, and the effectiveness of DT impact results most, though DT remains cost saving in all scenarios.
CONCLUSIONS: The reduction in emissions offsets the cost increase of DT when these emissions are monetized. This raises the wider question of whether the environmental impact of technologies should be formally quantified in cost-effectiveness analyses because it may be a key driver of the conclusions. This study also adds to evidence from European studies to support DT implementation.
OBJECTIVES: Single-arm trials (SATs) are becoming more common in an era of precision medicines. SATs typically lack trial randomization due to small sample sizes and limited statistical power, but health technology assessment bodies like The National Institute for Health and Care Excellence (NICE) require comparative effectiveness estimates for reimbursement decision making. Matching-adjusted indirect comparisons (MAICs) are one statistical method used to indirectly compare single-arm evidence to relevant comparators. This research aims to quantify the impact of MAIC parameter inclusion and sample size on propagated economic model uncertainty.
METHODS: We used a NICE technology appraisal (TA781) and supporting publications to create a simulated individual participant data (SIPD) set, conduct MAICs using the published comparator data, calculate parametric survival analysis and populate a partitioned-survival model. Two cuts of the SIPD were selected, the 'full' sample size (n=174) and an arbitrarily selected smaller subset ('small', n=30). For both cuts, MAICs were conducted using all variables (ALL) and high priority variables (HP) only.
RESULTS: For the full [small] samples, the effective sample size (ESS) reduced by 40% [46%] (ALL) and 25% [18%] (HP). The full sample MAICs displayed similar levels of uncertainty in the probabilistic economic model. The HP small sample MAIC displayed more uncertainty than both full sample MAICs. The small sample ALL MAIC displayed the most uncertainty, with probabilistic iterations spread across all four quadrants of the cost-effectiveness plane.
CONCLUSIONS: Where uncertainty is driven by low ESS, we recommend consideration of whether a MAIC is truly informative. If a MAIC is deemed necessary, we recommend preserving ESS by only including HP variables whilst transparently outlining the impact this could have on the MAIC quality and supplementing with analyses, such as naïve comparison and qualitative discussion. We suggest these analyses are presented to decision makers so the original shape of iterations is known.
OBJECTIVES: Generalised gamma (GG) is one of six probability distributions recommended by the National Institute for Health and Care Excellence (NICE) Decision Support Unit (DSU) for survival analysis. A persistent issue was investigated, where some survival models fitted using the GG distribution would produce deterministic results that functioned appropriately, however, the mean survival would be overestimated in the probabilistic sensitivity analysis (PSA). The objective of this analysis was to identify the cause of this overestimation.
METHODS: The reconstructed individual participant data (IPD) were used to provide example survival analysis inputs. The GG parametric survival models were fitted using the R package 'flexsurv'. Extrapolation and PSA were performed in both Microsoft Excel and R to control for any differences between the two.
RESULTS: The formulae used in Excel and R for extrapolation were found to be identical. The cause of the overestimation was identified as PSA samples where extrapolated survival was constant at 100% due to high parameter variance and covariance (particularly of the shape parameter 'Q') in the GG model. This is statistically and clinically implausible. The potential for survival overestimation in the PSA is not observable unless the survival analysis coefficients are applied probabilistically. The erroneous PSA samples only occurred when the GG distribution was used. This issue was observed in both Excel and R, and in raw and reconstructed IPD.
CONCLUSIONS: When reporting a GG model, it is recommended to check for: high variance and covariance in the survival model parameters; PSA samples with 100% survival; incongruence between mean survival in the PSA and deterministic estimates. If these elements are present, the GG model is inappropriate due to the inadvertent inclusion of statistically and clinically implausible survival probabilities in the PSA and the high levels of overall uncertainty this represents.
OBJECTIVES: Healthcare systems face significant strain due to growing demand, with urgent and emergency care centers particularly affected. 'Hospital at Home' (HaH) has been identified as a potential solution, allowing patients to receive acute care at home or in community settings. HaH facilitates early hospital discharge (step-down care) or prevents hospital admission (step-up care). This research examines the challenges in generating evidence for technology-enabled HaH initiatives and highlights factors for future evaluation.
METHODS: A pragmatic literature review was conducted to assess safety, clinical effectiveness, and cost effectiveness of HaH initiatives. Gap analysis identified priority areas for future research and issues in evidence generation. The authors leveraged their experience from an early value assessment for NICE on virtual wards for acute respiratory infection to inform the review and gap analysis.
RESULTS: Evidence, though limited, suggests HaH is potentially safe and effective. Clinical effectiveness varies by patient cohort and virtual ward model (step-up, step-down, mixed). Most studies were non-comparative or underpowered. Case studies of HaH initiatives in the NHS lacked peer review, involved small samples, and were not transparent about costs. Key issues in evaluation include variability in features between technology-enabled HaH initiatives, population and subgroup differences, and potential distortions in comparison with standard care. Future research should focus on prospective cohort studies to understand clinical and resource outcomes, impacts of different technological features, and true resource use.
CONCLUSIONS: Technology-enabled HaH initiatives are complex interventions. Preliminary evidence suggests that they may benefit healthcare system resources, but comprehensive evaluations are crucial to understand their clinical efficacy, safety, risks and costs. Comprehensive evaluations are vital as HaH initiatives are rapidly implemented across global healthcare systems. Future studies should determine the effectiveness of HaH initiatives across different clinical areas, identify effective features, and be used to determine the optimal implementation and management of HaH in different settings.
OBJECTIVES: Quick and equitable vaccine access is a global priority. However, the vaccine assessment is complex, meaning there is variation in vaccine schedules between countries and limited opportunities to develop a centralized framework. This heterogeneity further contributes to inequity of vaccine access. This work aims to increase awareness of the key vaccine assessment elements in EU and non-EU countries.
METHODS: Pragmatic desk-based research was conducted in June 2024 to explore the key stages involved in vaccine market access and how these differ across England, Italy, Germany, France and the United States (US). Where available, data were extracted about the stakeholders involved in vaccine appraisal, the key assessment factors and value framework considered, and the number and type of vaccines included in the national vaccination schedule.
RESULTS: National Immunization Technical Advisory Groups (NITAGs) are key stakeholders in all five countries but play different roles. In some countries the NITAG is solely responsible for vaccine appraisal (England, Germany, US), but in other countries the vaccine appraisal is conducted by the health technology assessment (HTA) body (Italy) or both the NITAG and HTA body in parallel (France). There are key differences in the vaccine assessment by NITAGs and HTA bodies. For example, NITAGs consider public health impact, which is not considered by HTA bodies. There are also differences in the value framework between NITAGs in different countries. For example, only England's NITAG formally considers the disease impact on quality of life of carers. Consequently, the key vaccine assessment factors differ between countries, resulting in a different number of vaccines in each country's vaccination schedule.
CONCLUSIONS: Several between-country differences in vaccine market access are identified; for example, the role of NITAGs, vaccine assessment factors, and value frameworks. Vaccine developers should consider these results when planning market access strategies to ensure rapid and equitable vaccine access across countries.
OBJECTIVES: Non-inferiority and clinical equivalence clinical trials can be used to determine whether a health technology is, at least, no worse than an existing treatment. There is a large body of literature and guidance on this topic, with substantial variation in definitions and practice, which can make it challenging to robustly demonstrate or assess claims of non-inferiority or clinical equivalence. This study aimed to provide actionable recommendations in the appraisal of claims of non-inferiority and clinical equivalence.
METHODS: International guidelines and published literature were reviewed to identify approaches for the conduct and reporting of non-inferiority or clinical equivalence studies. Guidelines from health technology assessment (HTA) and regulatory bodies were considered, and literature reviews from 2010 to 2023 were identified. The results of the reviews were supplemented with findings from an expert panel and synthesized to form a series of recommendations, using case studies from the National Institute for Health and Care Excellence (NICE).
RESULTS: The majority of guidelines (13/14) discussed, to varying extents, methods to determine the non-inferiority margin and how the analysis should be conducted. Despite this, the rationale for the margin was not reported in over 50% of 273 blinded randomized controlled trials between 1966 and 2015. Evidence of non-inferiority or clinical equivalence presented in NICE Medical Technology Evaluation Program appraisals (for health technologies) is often of lower quality than in Technology Appraisal appraisals (for pharmaceuticals), increasingly concluding that further evidence generation is required.
CONCLUSIONS: Despite clear guidance, the quality of reporting in non-inferiority and clinical equivalence trials is consistently poor. Prior to presentation of trial evidence, HTA submissions that claim non-inferiority or equivalence should present the technical, biological and/or pharmacokinetic reasonings that support the claim. HTA bodies should introduce more precise definitions of non-inferiority and clinical equivalence so that evidence standards are more likely to be met.
OBJECTIVES: The recent development and rise in accessibility of large language models (LLMs) has generated excitement around their possibilities for reducing the resource burden of conducting reviews. Following testing, we assessed the cost, accuracy, and accessibility of LLMs to reviewers, and consider what types of reviews LLMs are currently best suited to assist with.
METHODS: We conducted internal testing of a LLM, Claude 3 Opus, via the chat interface. We used the tool to conduct high level data extraction for a targeted review, highly granulated extraction for a systematic review, and risk of bias assessment of RCTs.
RESULTS: The LLM via a chat interface was highly accessible, inexpensive, and saved significant time in conducting high level qualitative data extraction for a pragmatic review. Outputs were standardized and easy to manipulate and integrate into our existing work process. Extracting accurate granular data for a systematic review proved more difficult, with the model failing to interpret complexities of patient flow, struggling to respond accurately to lengthy, detailed prompts, and the subsequent checking, correcting, and formatting outweighing any time saved. The model identified some relevant content for conducting risk of bias assessment with the Cochrane RoB 1 tool, although lacked context, and human judgement was needed for final decision making.
CONCLUSIONS: LLM chat interfaces offer significant time savings for pragmatic reviews, although copyright issues exist in uploading published papers for synthesis. Optimal performance for systematic reviews is unlikely to be achieved without fine tuning a version of the model with archive data. This process is currently costly, commercial confidentiality must be considered, and the skill set required is outside the scope of many review teams. Developers should ensure that any LLM based tools for reviewing can be integrated into clients' existing processes with the use of standardized import and export formats such as CSV or RIS.
