Institut National d'Excellence en Santé et en Services Sociaux

BACKGROUND: Interest in value-based healthcare, generally defined as providing better care at lower cost, has grown worldwide, and learning health systems (LHSs) have been proposed as a key strategy for improving value in healthcare. LHSs are emerging around the world and aim to leverage advancements in science, technology and practice to improve health system performance at lower cost. However, there remains much uncertainty around the implementation of LHSs and the distinctive features of these systems. This paper presents a conceptual framework that has been developed in Canada to support the implementation of value-creating LHSs. METHODS: The framework was developed by an interdisciplinary team at the Institut national d'excellence en santé et en services sociaux (INESSS). It was informed by a scoping review of the scientific and grey literature on LHSs, regular team discussions over a 14-month period, and consultations with Canadian and international experts. RESULTS: The framework describes four elements that characterise LHSs, namely (1) core values, (2) pillars and accelerators, (3) processes and (4) outcomes. LHSs embody certain core values, including an emphasis on participatory leadership, inclusiveness, scientific rigour and person-centredness. In addition, values such as equity and solidarity should also guide LHSs and are particularly relevant in countries like Canada. LHS pillars are the infrastructure and resources supporting the LHS, whereas accelerators are those specific structures that enable more rapid learning and improvement. For LHSs to create value, such infrastructures must not only exist within the ecosystem but also be connected and aligned with the LHSs' strategic goals. These pillars support the execution, routinisation and acceleration of learning cycles, which are the fundamental processes of LHSs. The main outcome sought by executing learning cycles is the creation of value, which we define as the striking of a more optimal balance of impacts on patient and provider experience, population health and health system costs. CONCLUSIONS: Our framework illustrates how the distinctive structures, processes and outcomes of LHSs tie together with the aim of optimising health system performance and delivering greater value in health systems.

BACKGROUND: According to Donabedian's health care quality model, improvements in the structure of care should lead to improvements in clinical processes that should in turn improve patient outcome. This model has been widely adopted by the trauma community but has not yet been validated in a trauma system. The objective of this study was to assess the performance of an integrated trauma system in terms of structure, process, and outcome and evaluate the correlation between quality domains. METHODS: Quality of care was evaluated for patients treated in a Canadian provincial trauma system (2005-2010; 57 centers, n = 63,971) using quality indicators (QIs) developed and validated previously. Structural performance was measured by transposing on-site accreditation visit reports onto an evaluation grid according to American College of Surgeons criteria. The composite process QI was calculated as the average sum of proportions of conformity to 15 process QIs derived from literature review and expert opinion. Outcome performance was measured using risk-adjusted rates of mortality, complications, and readmission as well as hospital length of stay (LOS). Correlation was assessed with Pearson's correlation coefficients. RESULTS: Statistically significant correlations were observed between structure and process QIs (r = 0.33), and process and outcome QIs (r = -0.33 for readmission, r = -0.27 for LOS). Significant positive correlations were also observed between outcome QIs (r = 0.37 for mortality-readmission; r = 0.39 for mortality-LOS and readmission-LOS; r = 0.45 for mortality-complications; r = 0.34 for readmission-complications; 0.63 for complications-LOS). CONCLUSION: Significant correlations between quality domains observed in this study suggest that Donabedian's structure-process-outcome model is a valid model for evaluating trauma care. Trauma centers that perform well in terms of structure also tend to perform well in terms of clinical processes, which in turn has a favorable influence on patient outcomes. LEVEL OF EVIDENCE: Prognostic study, level III.

OBJECTIVES: To validate and compare performance of an International Classification of Diseases, tenth revision (ICD-10) version of a combined comorbidity index merging conditions of Charlson and Elixhauser measures against individual measures in the prediction of 30-day mortality. To select a weight derivation method providing optimal performance across ICD-9 and ICD-10 coding systems. RESEARCH DESIGN: Using 2 adult population-based cohorts of patients with hospital admissions in ICD-9 (2005, n=337,367) and ICD-10 (2011, n=348,820), we validated a combined comorbidity index by predicting 30-day mortality with logistic regression. To appreciate performance of the Combined index and both individual measures, factors impacting indices performance such as population characteristics and weight derivation methods were accounted for. We applied 3 scoring methods (Van Walraven, Schneeweiss, and Charlson) and determined which provides best predictive values. RESULTS: Combined index [c-statistics: 0.853 (95% confidence interval: CI, 0.848-0.856)] performed better than original Charlson [0.841 (95% CI, 0.835-0.844)] or Elixhauser [0.841 (95% CI, 0.837-0.844)] measures on ICD-10 cohort. All weight derivation methods provided close high discrimination results for the Combined index (Van Walraven: 0.852, Schneeweiss: 0.851, Charlson: 0.849). Results were consistent across both coding systems. CONCLUSIONS: The Combined index remains valid with both ICD-9 and ICD-10 coding systems and the 3 weight derivation methods evaluated provided consistent high performance across those coding systems.

The concept of adaptive licensing (AL) has met with considerable interest. Yet some remain skeptical about its feasibility. Others argue that the focus and name of AL should be broadened. Against this background of ongoing debate, we examine the environmental changes that will likely make adaptive pathways the preferred approach in the future. The key drivers include: growing patient demand for timely access to promising therapies, emerging science leading to fragmentation of treatment populations, rising payer influence on product accessibility, and pressure on pharma/investors to ensure sustainability of drug development. We also discuss a number of environmental changes that will enable an adaptive paradigm. A life-span approach to bringing innovation to patients is expected to help address the perceived access vs. evidence trade-off, help de-risk drug development, and lead to better outcomes for patients.

BACKGROUND: Oral immunotherapy (OIT) is an emerging approach to the treatment of patients with IgE-mediated food allergy and is in the process of transitioning to clinical practice. OBJECTIVE: To develop patient-oriented clinical practice guidelines on oral immunotherapy based on evidence and ethical imperatives for the provision of safe and efficient food allergy management. MATERIALS AND METHODS: Recommendations were developed using a reflective patient-centered multicriteria approach including 22 criteria organized in five dimensions (clinical, populational, economic, organizational and sociopolitical). Data was obtained from: (1) a review of scientific and ethic literature; (2) consultations of allergists, other healthcare professionals (pediatricians, family physicians, nurses, registered dieticians, psychologists, peer supporters), patients and caregivers; and patient associations through structured consultative panels, interviews and on-line questionnaire; and (3) organizational and economic data from the milieu of care. All data was synthesized by criteria in a multicriteria deliberative guide that served as a platform for structured discussion and development of recommendations for each dimension, based on evidence, ethical imperatives and other considerations. RESULTS: The deliberative grid included 162 articles from the literature and media reviews and data from consultations involving 85 individuals. Thirty-eight (38) recommendations were made for the practice of oral immunotherapy for the treatment of IgE mediated food allergy, based on evidence and a diversity of ethical imperatives. All recommendations were aimed at fostering a context conducive to achieving objectives identified by patients and caregivers with food allergy. Notably, specific recommendations were developed to promote a culture of shared responsibility between patients and healthcare system, equity in access, patient empowerment, shared decision making and personalization of OIT protocols to reflect patients' needs. It also provides recommendations to optimize organization of care to generate capacity to meet demand according to patient choice, e.g. OIT or avoidance. These recommendations were made acknowledging the necessity of ensuring sustainability of the clinical offer in light of various economic considerations. CONCLUSIONS: This innovative CPG methodology was guided by patients' perspectives, clinical evidence as well as ethical and other rationales. This allowed for the creation of a broad set of recommendations that chart optimal clinical practice and define the conditions required to bring about changes to food allergy care that will be sustainable, equitable and conducive to the well-being of all patients in need.

BACKGROUND: Sodium-glucose cotransporter-2 (SGLT-2) inhibitors could increase the risk for diabetic ketoacidosis (DKA). OBJECTIVE: To assess whether SGLT-2 inhibitors, compared with dipeptidyl peptidase-4 (DPP-4) inhibitors, are associated with an increased risk for DKA in patients with type 2 diabetes. DESIGN: Population-based cohort study; prevalent new-user design between 2013 and 2018. (ClinicalTrials.gov: NCT04017221). SETTING: Electronic health care databases from 7 Canadian provinces and the United Kingdom. PATIENTS: 208 757 new users of SGLT-2 inhibitors were matched by using time-conditional propensity scores to 208 757 recipients of DPP-4 inhibitors. MEASUREMENTS: Cox proportional hazards models estimated site-specific hazard ratios (HRs) with 95% CIs of DKA comparing receipt of SGLT-2 inhibitors with receipt of DPP-4 inhibitors, which were pooled by using random-effects models. Secondary analyses were stratified by molecule, age, sex, and prior receipt of insulin. RESULTS: Overall, 521 patients were diagnosed with DKA during 370 454 person-years of follow-up (incidence rate per 1000 person-years, 1.40 [95% CI, 1.29 to 1.53]). Compared with DPP-4 inhibitors, SGLT-2 inhibitors were associated with an increased risk for DKA (incidence rate, 2.03 [CI, 1.83 to 2.25] versus 0.75 [CI, 0.63 to 0.89], respectively; HR, 2.85 [CI, 1.99 to 4.08]). Molecule-specific HRs were 1.86 (CI, 1.11 to 3.10) for dapagliflozin, 2.52 (CI, 1.23 to 5.14) for empagliflozin, and 3.58 (CI, 2.13 to 6.03) for canagliflozin. Age and sex did not modify the association; prior receipt of insulin appeared to decrease the risk. LIMITATIONS: There was unmeasured confounding and no laboratory data were available for the majority of patients, and molecule-specific analyses were conducted at a limited number of sites. CONCLUSION: SGLT-2 inhibitors were associated with an almost 3-fold increased risk for DKA, with molecule-specific analyses suggesting a class effect. PRIMARY FUNDING SOURCE: Canadian Institutes of Health Research.

Artificial intelligence (AI) is seen as a strategic lever to improve access, quality, and efficiency of care and services and to build learning and value-based health systems. Many studies have examined the technical performance of AI within an experimental context. These studies provide limited insights into the issues that its use in a real-world context of care and services raises. To help decision makers address these issues in a systemic and holistic manner, this viewpoint paper relies on the health technology assessment core model to contrast the expectations of the health sector toward the use of AI with the risks that should be mitigated for its responsible deployment. The analysis adopts the perspective of payers (ie, health system organizations and agencies) because of their central role in regulating, financing, and reimbursing novel technologies. This paper suggests that AI-based systems should be seen as a health system transformation lever, rather than a discrete set of technological devices. Their use could bring significant changes and impacts at several levels: technological, clinical, human and cognitive (patient and clinician), professional and organizational, economic, legal, and ethical. The assessment of AI's value proposition should thus go beyond technical performance and cost logic by performing a holistic analysis of its value in a real-world context of care and services. To guide AI development, generate knowledge, and draw lessons that can be translated into action, the right political, regulatory, organizational, clinical, and technological conditions for innovation should be created as a first step.

OBJECTIVE: To analyze relations among injury, demographic, and environmental factors on function, health-related quality of life (HRQoL), and life satisfaction in individuals with traumatic spinal cord injury (SCI). DESIGN: Prospective observational registry cohort study. SETTING: Specialized acute and rehabilitation SCI centers. PARTICIPANTS: Participants (N=340) from the Rick Hansen Spinal Cord Injury Registry (RHSCIR) who were prospectively recruited from 2004 to 2014 were included. The model cohort participants were 79.1% men, with a mean age of 41.6±17.3 years. Of the participants, 34.7% were motor/sensory complete (ASIA Impairment Scale [AIS] grade A). INTERVENTIONS: None. MAIN OUTCOME MEASURES: Path analysis was used to determine relations among SCI severity (AIS grade and anatomic level [cervical/thoracolumbar]), age at injury, education, number of health conditions, functional independence (FIM motor score), HRQoL (Medical Outcomes Study 36-Item Short-Form Health Survey [Version 2] Physical Component Score [PCS] and Mental Component Score [MCS]), and life satisfaction (Life Satisfaction-11 [LiSat-11]). Model fit was assessed using recommended published indices. RESULTS: Goodness of fit of the model was supported by all indices, indicating the model results closely matched the RHSCIR data. Higher age, higher severity injuries, cervical injuries, and more health conditions negatively affected FIM motor score, whereas employment had a positive effect. Higher age, less education, more severe injuries (AIS grades A-C), and more health conditions negatively correlated with PCS (worse physical health). More health conditions were negatively correlated with a lower MCS (worse mental health), however were positively associated with reduced function. Being married and having higher function positively affected Lisat-11, but more health conditions had a negative effect. CONCLUSIONS: Complex interactions and enduring effects of health conditions after SCI have a negative effect on function, HRQoL, and life satisfaction. Modeling relations among these types of concepts will inform clinicians how to positively effect outcomes after SCI (eg, development of screening tools and protocols for managing individuals with traumatic SCI who have multiple health conditions).

Abstract Objective To compare the risk of cardiovascular events between sodium glucose cotransporter 2 (SGLT2) inhibitors and dipeptidyl peptidase-4 (DPP-4) inhibitors among people with type 2 diabetes in a real world context of clinical practice. Design Multi-database retrospective cohort study using a prevalent new user design with subsequent meta-analysis. Setting Canadian Network for Observational Drug Effect Studies (CNODES), with administrative healthcare databases from seven Canadian provinces and the United Kingdom, 2013-18. Population 209 867 new users of a SGLT2 inhibitor matched to 209 867 users of a DPP-4 inhibitor on time conditional propensity score and followed for a mean of 0.9 years. Main outcome measures The primary outcome was major adverse cardiovascular events (MACE, a composite of myocardial infarction, ischaemic stroke, or cardiovascular death). Secondary outcomes were the individual components of MACE, heart failure, and all cause mortality. Cox proportional hazards models were used to estimate site specific adjusted hazards ratios and 95% confidence intervals, comparing use of SGLT2 inhibitors with use of DPP-4 inhibitors in an as treated approach. Site specific results were pooled using random effects meta-analysis. Results Compared with DPP-4 inhibitors, SGLT2 inhibitors were associated with decreased risks of MACE (incidence rate per 1000 person years: 11.4 v 16.5; hazard ratio 0.76, 95% confidence interval 0.69 to 0.84), myocardial infarction (5.1 v 6.4; 0.82, 0.70 to 0.96), cardiovascular death (3.9 v 7.7; 0.60, 0.54 to 0.67), heart failure (3.1 v 7.7; 0.43, 0.37 to 0.51), and all cause mortality (8.7 v 17.3; 0.60, 0.54 to 0.67). SGLT2 inhibitors had more modest benefits for ischaemic stroke (2.6 v 3.5; 0.85, 0.72 to 1.01). Similar benefits for MACE were observed with canagliflozin (0.79, 0.66 to 0.94), dapagliflozin (0.73, 0.63 to 0.85), and empagliflozin (0.77, 0.68 to 0.87). Conclusions In this large observational study conducted in a real world clinical practice context, the short term use of SGLT2 inhibitors was associated with a decreased risk of cardiovascular events compared with the use of DPP-4 inhibitors. Trial registration ClinicalTrials.gov NCT03939624 .

PURPOSE: Artificial intelligence (AI) raises many expectations regarding its ability to profoundly transform health care delivery. There is an abundant literature on the technical performance of AI applications in many clinical fields (e.g. radiology, ophthalmology). This article aims to bring forward the importance of studying organizational readiness to integrate AI into health care delivery. DESIGN/METHODOLOGY/APPROACH: The reflection is based on our experience in digital health technologies, diffusion of innovations and healthcare organizations and systems. It provides insights into why and how organizational readiness should be carefully considered. FINDINGS: As an important step to ensure successful integration of AI and avoid unnecessary investments and costly failures, better consideration should be given to: (1) Needs and added-value assessment; (2) Workplace readiness: stakeholder acceptance and engagement; (3) Technology-organization alignment assessment and (4) Business plan: financing and investments. In summary, decision-makers and technology promoters should better address the complexity of AI and understand the systemic challenges raised by its implementation in healthcare organizations and systems. ORIGINALITY/VALUE: Few studies have focused on the organizational issues raised by the integration of AI into clinical routine. The current context is marked by a perplexing gap between the willingness of decision-makers and technology promoters to capitalize on AI applications to improve health care delivery and the reality on the ground, where it is difficult to initiate the changes needed to realize their full benefits while avoiding their negative impacts.

STUDY DESIGN: Clinical practice guidelines. OBJECTIVES: To develop the first Canadian clinical practice guidelines for treatment of neuropathic pain in people with spinal cord injury (SCI). SETTING: The guidelines are relevant for inpatient and outpatient SCI rehabilitation settings in Canada. METHODS: The CanPainSCI Working Group reviewed the evidence for different treatment options and achieved consensus. The Working Group then developed clinical considerations for each recommendation. Recommendations for research are also included. RESULTS: Twelve recommendations were developed for the management of neuropathic pain after SCI. The recommendations address both pharmacologic and nonpharmacologic treatment modalities. CONCLUSIONS: An expert Working Group developed recommendations for the treatment of neuropathic pain after SCI that should be used to inform practice.

The ageing of the population and the increasing need for long-term care services are global issues. Some countries have adapted homecare programs by introducing an intervention called reablement, which is aimed at optimizing independence. The effectiveness of reablement, as well as its different service models, was examined. A systematic literature review was conducted using MEDLINE, CINAHL, PsycINFO and EBM Reviews to search from 2001 to 2014. Core characteristics and facilitators of reablement implementation were identified from international experiences. Ten studies comprising a total of 14,742 participants (including four randomized trials, most of excellent or good quality) showed a positive impact of reablement, especially on health-related quality of life and service utilization. The implementation of reablement was studied in three regions, and all observed a reduction in healthcare service utilization. Considering its effectiveness and positive impact observed in several countries, the implementation of reablement is a promising avenue to be pursued by policy makers.

BACKGROUND: Although antineoplastic agents are critical in the treatment of cancer, they can potentially cause hypersensitivity reactions that can have serious consequences. When such a reaction occurs, clinicians can either continue the treatment, at the risk of causing a severe or a potentially fatal anaphylactic reaction, or stop the treatment, although it might be the only one available. The objective of the present work was to evaluate the effectiveness of methods used to prevent and treat hypersensitivity reactions to platinum- or taxane-based chemotherapy and to develop evidence-based recommendations. METHODS: The scientific literature published to December 2013, inclusive, was reviewed. RESULTS: Premedication with antihistamines, H2 blockers, and corticosteroids is not effective in preventing hypersensitivity reactions to platinum salts. However, premedication significantly reduces the incidence of hypersensitivity to taxanes. A skin test can generally be performed to screen for patients at risk of developing a severe reaction to platinum salts in the presence of grade 1 or 2 reactions, but skin testing does not appear to be useful for taxanes. A desensitization protocol allows for re-administration of either platinum- or taxane-based chemotherapy to some patients without causing severe hypersensitivity reactions. CONCLUSIONS: Several strategies such as premedication, skin testing, and desensitization protocols are available to potentially allow for administration of platinum- or taxane-based chemotherapy to patients who have had a hypersensitivity reaction and for whom no other treatment options are available. Considering the available evidence, the Comité de l'évolution des pratiques en oncologie made recommendations for clinical practice in Quebec.

Marginal structural Cox Models (Cox MSMs) have been used to estimate the causal effect of a time-varying treatment on the hazard when there exist time-dependent confounders, which are themselves also affected by previous treatment. A Cox MSM can be estimated via the inverse-probability-of-treatment weighting (IPTW) estimator. However, IPTW estimators suffer from large variability if some observations are assigned extremely high weights. Weight truncation has been proposed as one simple solution to this problem, but truncation levels are typically chosen based on ad hoc criteria that have not been systematically evaluated. Bembom et al. proposed data-adaptive selection of the optimal truncation level using the estimated mean-squared error (MSE) of a truncated IPTW estimator for cross-sectional data. Based on a similar principle, we proposed data-adaptive approaches to select the truncation level that minimizes the expected MSE for time-to-event data with time-varying treatments. The expected MSE is approximated by using either observed statistics as a proxy for the true unknown parameter or using cross-validation. Simulations confirm that simple weight truncation at high percentiles such as the 99th or 99.5th of the distribution of weights improves the IPTW estimators in most scenarios we considered. Our newly proposed approaches exhibit similarly good performance and may be applied in a wide range of settings.

There is broad agreement among health-care stakeholders that more must be done to ensure that patients have timely access to new and innovative medicines. Assuming that industry will continue to develop such medicines at a sustainable rate, regulators and payers become the gatekeepers. Regulators, starting in the late 1980s/early 1990s, and, more recently, payers have implemented a variety of early-access pathways or initiatives, and this practice is continuing even today. This article describes the specific approaches that have been taken in four economically developed regions, reviews their success rates, and suggests possible new directions.

BACKGROUND: Although asthma morbidity can be prevented through long-term controller medication, most patients with persistent asthma do not take their daily inhaled corticosteroid. The objective of this study was to gather patients' insights into barriers and facilitators to taking long-term daily inhaled corticosteroids as basis for future knowledge translation interventions. METHODS: We conducted a collective qualitative case study. We interviewed 24 adults, adolescents, or parents of children, with asthma who had received a prescription of long-term inhaled corticosteroids in the previous year. The one-hour face-to-face interviews revolved around patients' perceptions of asthma, use of asthma medications, current self-management, prior changes in self-management, as well as patient-physician relationship. We sought barriers and facilitators to optimal asthma management. Interviews were transcribed verbatim and transcripts were analyzed using a thematic approach. RESULTS: Patients were aged 2-76 years old and 58% were female. Nine patients were followed by an asthma specialist (pulmonologist or allergist), 13 patients by family doctors or pediatricians, and two patients had no regular follow-up. Barriers and facilitators to long-term daily inhaled corticosteroids were classified into the following loci of responsibility and its corresponding domains: (1) patient (cognition; motivation, attitudes and preferences; practical implementation; and parental support); (2) patient-physician interaction (communication and patient-physician relationship); and (3) health care system (resources and services). Patients recognized that several barriers and facilitators fell within their own responsibility. They also underlined the crucial impact (positive or negative) on their adherence of the quality of patient-physician interaction and health care system accessibility. CONCLUSIONS: We identified a close relationship between reported barriers and facilitators to adherence to long-term daily controller medication for asthma within three loci of responsibility. As such, patients' adherence must be approached as a multi-level phenomenon; moreover, interventions targeting the patient, the patient-physician interaction, and the health care system are recommended. The present study offers a potential taxonomy of barriers and facilitators to adherence to long-term daily inhaled corticosteroids therapy that, once validated, may be used for planning a knowledge translation intervention and may be applicable to other chronic conditions.

: Many determinants are related to vaccine hesitancy. These determinants should be taken into account when health professionals engage with vaccine-hesitant individuals.

BACKGROUND: Injury is second only to cardiovascular disease in terms of acute care costs in North America. One key to improving injury care efficiency is to generate knowledge on the determinants of resource use. Socio-economic status (SES) is a documented risk factor for injury severity and mortality but its impact on length of stay (LOS) for injury admissions is unknown. This study aimed to examine the relationship between SES and LOS following injury. This multicenter retrospective cohort study was based on adults discharged alive from any trauma center (2007-2012; 57 hospitals; 65,486 patients) in a Canadian integrated provincial trauma system. SES was determined using ecological indices of material and social deprivation. Mean differences in LOS adjusted for age, gender, comorbidities, and injury severity were generated using multivariate linear regression. RESULTS: Mean LOS was 13.5 days. Patients in the highest quintile of material/social deprivation had a mean LOS 0.5 days (95 % CI 0.1-0.9)/1.4 days (1.1-1.8) longer than those in the lowest quintile. Patients in the highest quintiles of both social and material deprivation had a mean LOS 2.6 days (1.8-3.5) longer than those in the lowest quintiles. CONCLUSIONS: Results suggest that patients admitted for traumatic injury who suffer from high social and/or material deprivation have longer acute care LOS in a universal-access health care system. The reasons behind observed differences need to be further explored but may indicate that discharge planning should take patient SES into consideration.

BACKGROUND: The last decade has seen growing interest in scaling up of innovations to strengthen healthcare systems. However, the lack of appropriate methods for determining their potential for scale-up is an unfortunate global handicap. Thus, we aimed to review tools proposed for assessing the scalability of innovations in health. METHODS: We conducted a systematic review following the COSMIN methodology. We included any empirical research which aimed to investigate the creation, validation or interpretability of a scalability assessment tool in health. We searched Embase, MEDLINE, CINAHL, Web of Science, PsycINFO, Cochrane Library and ERIC from their inception to 20 March 2019. We also searched relevant websites, screened the reference lists of relevant reports and consulted experts in the field. Two reviewers independently selected and extracted eligible reports and assessed the methodological quality of tools. We summarized data using a narrative approach involving thematic syntheses and descriptive statistics. RESULTS: We identified 31 reports describing 21 tools. Types of tools included criteria (47.6%), scales (33.3%) and checklists (19.0%). Most tools were published from 2010 onwards (90.5%), in open-access sources (85.7%) and funded by governmental or nongovernmental organizations (76.2%). All tools were in English; four were translated into French or Spanish (19.0%). Tool creation involved single (23.8%) or multiple (19.0%) types of stakeholders, or stakeholder involvement was not reported (57.1%). No studies reported involving patients or the public, or reported the sex of tool creators. Tools were created for use in high-income countries (28.6%), low- or middle-income countries (19.0%), or both (9.5%), or for transferring innovations from low- or middle-income countries to high-income countries (4.8%). Healthcare levels included public or population health (47.6%), primary healthcare (33.3%) and home care (4.8%). Most tools provided limited information on content validity (85.7%), and none reported on other measurement properties. The methodological quality of tools was deemed inadequate (61.9%) or doubtful (38.1%). CONCLUSIONS: We inventoried tools for assessing the scalability of innovations in health. Existing tools are as yet of limited utility for assessing scalability in health. More work needs to be done to establish key psychometric properties of these tools. Trial registration We registered this review with PROSPERO (identifier: CRD42019107095).

The application of artificial intelligence (AI) may revolutionize the healthcare system, leading to enhance efficiency by automatizing routine tasks and decreasing health-related costs, broadening access to healthcare delivery, targeting more precisely patient needs, and assisting clinicians in their decision-making. For these benefits to materialize, governments and health authorities must regulate AI, and conduct appropriate health technology assessment (HTA). Many authors have highlighted that AI health technologies (AIHT) challenge traditional evaluation and regulatory processes. To inform and support HTA organizations and regulators in adapting their processes to AIHTs, we conducted a systematic review of the literature on the challenges posed by AIHTs in HTA and health regulation. Our research question was: What makes artificial intelligence exceptional in HTA? The current body of literature appears to portray AIHTs as being exceptional to HTA. This exceptionalism is expressed along 5 dimensions: 1) AIHT’s distinctive features; 2) their systemic impacts on health care and the health sector; 3) the increased expectations towards AI in health; 4) the new ethical, social and legal challenges that arise from deploying AI in the health sector; and 5) the new evaluative constraints that AI poses to HTA. Thus, AIHTs are perceived as exceptional because of their technological characteristics and potential impacts on society at large. As AI implementation by governments and health organizations carries risks of generating new, and amplifying existing, challenges, there are strong arguments for taking into consideration the exceptional aspects of AIHTs, especially as their impacts on the healthcare system will be far greater than that of drugs and medical devices. As AIHTs begin to be increasingly introduced into the health care sector, there is a window of opportunity for HTA agencies and scholars to consider AIHTs’ exceptionalism and to work towards only deploying clinically, economically, socially acceptable AIHTs in the health care system.