|

 

Initiative-elga.at

ORIGINAL INVESTIGATION HEALTH CARE REFORM An Empirical Model to Estimate the Potential Impact
of Medication Safety Alerts on Patient Safety, Health
Care Utilization, and Cost in Ambulatory Care
Saul N. Weingart, MD, PhD; Brett Simchowitz, BA; Harper Padolsky, MD; Thomas Isaac, MD, MBA, MPH;Andrew C. Seger, PharmD; Michael Massagli, PhD; Roger B. Davis, ScD; Joel S. Weissman, PhD Background: Because ambulatory care clinicians over-
potentially serious, 125 (34-307) significant, and 228 (85- ride as many as 91% of drug interaction alerts, the po- 409) minor ADEs. Accepted alerts may have prevented tential benefit of electronic prescribing (e-prescribing) a death in 3 (IQR, 2-13) cases, permanent disability in with decision support is uncertain.
14 (3-18), and temporary disability in 31 (10-97). Alertspotentially resulted in 39 (IQR, 14-100) fewer hospital- Methods: We studied 279 476 alerted prescriptions writ-
izations, 34 (6-74) fewer emergency department visits, ten by 2321 Massachusetts ambulatory care clinicians and 267 (105-541) fewer office visits, for a cost savings using a single commercial e-prescribing system from Janu- of $402 619 (IQR, $141 012-$1 012 386). Based on the ary 1 through June 30, 2006. An expert panel reviewed panel's estimates, 331 alerts were required to prevent 1 a sample of common drug interaction alerts, estimating ADE, and a few alerts (10%) likely accounted for 60% of the likelihood and severity of adverse drug events (ADEs) ADEs and 78% of cost savings.
associated with each alert, the likely injury to the pa-tient, and the health care utilization required to address Conclusions: Electronic prescribing alerts in ambula-
each ADE. We estimated the cost savings due to e- tory care may prevent a substantial number of injuries prescribing by using third-party–payer and publicly avail-able information.
and reduce health care costs in Massachusetts. Becausea few alerts account for most of the benefit, e-prescribing Results: Based on the expert panel's estimates, elec-
systems should suppress low-value alerts.
tronic drug alerts likely prevented 402 (interquartile range[IQR], 133-846) ADEs in 2006, including 49 (14-130) Arch Intern Med. 2009;169(16):1465-1473 electronic prescribing harm, decrease unnecessary utilization of (e-prescribing) is well es- health care services, and save money.
tablished in the acute care To understand the potential benefits of hospital, its safety ben- medication safety alerts in ambulatory care, efits in ambulatory care are less well un- we conducted a multifaceted study of a derstood. Early studies1,2 showed that ru- commercial e-prescribing system serving dimentary electronic order entry, without 2321 Massachusetts ambulatory care pro- advanced decision support such as drug viders in 2006. We hypothesized that the interaction and allergy alerts, resulted in alerts that clinicians accepted would, in ag- legible prescriptions but no difference in gregate, benefit patients, lower health care the rate of adverse drug events (ADEs) costs, and help to validate the continueduse of these systems.
See Invited Commentary
at end of article
compared with paper-based prescribing.
In subsequent studies,3-5 however, inves-tigators discovered that clinicians with ac- To examine the impact of medication safety cess to these features overrode as many as alerts, we modified a conceptual framework thatcharacterizes prescribers' responses to drug safety 91% of drug interaction and allergy alerts.
alerts and the resulting consequences Although overriding alerts may jeopar- (Figure 1).6,7 When an alert is triggered, a cli-
Author Affiliations are listed at
dize the potential impact of these sys- nician has the following 3 options: (1) override the end of this article.
tems, it is possible that even the small num- the alert and leave the prescription intact, (2) can- (REPRINTED) ARCH INTERN MED/ VOL 169 (NO. 16), SEP 14, 2009 2009 American Medical Association. All rights reserved.
Health care utilization Permanent disability Temporary disability Office visit with 1 – Intercept rate Prescription filled Office visit without DDI alerts triggered laboratory findings Figure 1. Conceptual model for estimating the patient safety impact, health care utilization, and cost savings attributable to accepted medication safety alerts.
Alerts are generated in an electronic prescribing system. ADE indicates adverse drug event; DDI, drug-drug interaction; and ED, emergency department.
cel the prescription, or (3) change the prescription to another medi- drug, we studied the last drug alert that the prescriber trig- cation. We combined canceled prescription orders and those gered sorted by physician, patient, prescribed generic drug name, changed to an alternate medication and termed these accepted date, and time.3 This approach eliminated 146 425 alerts, leav- alerts. Some accepted alerts prevent ADEs from occurring. The ing 133 051. Each DDI belongs to a broader class-class inter- number of prevented ADEs is a function of the number of alerts action (CCI): bupropion hydrochloride with citalopram hy- generated, the acceptance rate, and the probability that the alerted drobromide, for example, is a DDI within the smoking cessation interaction would have resulted in an ADE if allowed to reach agents and selective serotonin reuptake inhibitor antidepres- the patient. The number of ADEs prevented should be dis- sant CCI. We selected the 100 most frequently accepted CCIs; counted by the number of prescriptions that pharmacists do not these alerts represented 56.5% of all accepted alerts. Finally, fill or patients do not collect. The severity of the ADE may affect we selected the most commonly accepted DDI within each CCI.
the type of injury, health care utilization, and cost of care.
EXPERT PANEL ESTIMATES
Because there are no published data estimating the probability We examined medication alerts generated by the users of and severity of harm caused by most DDIs, we used a modified PocketScript, an e-prescribing application developed by Zix- Delphi technique (a type of consensus method) to characterize Corp (Dallas, Texas) that allows clinicians to transmit prescrip- the clinical utility, patient safety benefit, and cost savings asso- tions electronically to a pharmacy via a desktop computer or a ciated with accepted alerts.8-10 We recruited a 7-member expert handheld device. The system creates a profile of a patient's ac- panel of 4 Massachusetts physicians and 3 pharmacists (includ- tive medications based on previously written e-prescriptions. When ing investigators S.N.W., T.I., and A.C.S.), inviting panelists based a prescriber attempts to order a drug, the system checks whether on their experience in primary care medicine and patient safety the prescribed medication interacts with any medications on the patient's profile, drawing on a list of medication interactions main- Investigators oriented panelists to the project by means of a con- tained by Cerner Multum, Inc (Denver, Colorado). If an inter- ference call and practiced sample cases. Panelists then reviewed action is detected, a warning banner is displayed showing the se- the same 100 medication safety alerts and, informed by their ex- verity of the interaction (high, medium, or low), and a description perience, made a series of judgments about each alert. We provided of the interaction is available through a drug reference guide.
information about each DDI taken from pharmaceutical referencetexts, including Micromedex,11 Clinical Pharmacology,12 Lexi- SELECTION OF MEDICATION SAFETY ALERTS
Comp,13 Epocrates,14 and a research study by Malone et al.15 For each DDI, panelists estimated the probability that the in- We studied all e-prescriptions written and all medication safety teraction would result in an ADE—defined as an injury due to alerts generated by 2321 eligible Massachusetts clinicians medication use—and, if so, whether the most likely ADE would who used the PocketScript e-prescribing system for at least 1 be serious, significant, or minor.2,16,17 Serious ADEs could cause e-prescription from January 1 through June 30, 2006. ZixCorp organ system dysfunction, such as a seizure or major gastroin- provided information on all drug-drug interaction (DDI) alerts testinal tract bleeding. Significant ADEs could cause symptoms generated during the study period, along with the prescribers' such as a rash or fever and/or laboratory abnormalities such as action on receiving the alert. Prescribers included physicians thrombocytopenia or hyperkalemia. Minor ADEs could cause (79.2%) and nonphysicians (20.8%), including specialists in fam- minimal injury, such as flushing or dyspepsia. Because alerts were ily medicine (14.5%), internal medicine (13.1%), pediatrics generated for initial prescriptions rather than renewals, panel- (13.7%), psychiatry (2.5%), and other specialties (26.5%). Spe- ists assumed that the medications were being prescribed to- cialty information was unavailable for 29.7% of prescribers.
gether for the first time. Panelists judged the severity of potential These clinicians wrote 1 833 254 prescriptions for 60 352 ADEs on the basis of the most likely reaction (rather than the worst- patients and generated 279 476 drug interaction alerts during case scenario) of the typical patient who would receive the drug the study period. Multiple alerts may have appeared for the same combination (in terms of age, comorbidities, and dosage). We prescription attempt if a prescribed drug interacted with more asked panelists to classify each incident by likely frequency, in- than 1 medication on the patient's profile. To avoid double- jury, and health care utilization using the categories in Table 1.
counting alerts and multiple prescription attempts for the same Finally, panelists estimated the probability that a given drug in- (REPRINTED) ARCH INTERN MED/ VOL 169 (NO. 16), SEP 14, 2009 2009 American Medical Association. All rights reserved.
Table 1. Expert Panel Instructions for Rating Drug Interaction Alerts
Rating Task
Rate the probability that each interaction will result in a serious, significant, or minor ADE No evidenceTheoretical basis but not seen in practiceCase reports (incidence ⬍0.1%)Rarely seen in clinical practice (incidence ⬍1%)Sometimes seen in clinical practice (incidence 1%-5%)Seen often in clinical practice (incidence ⬎5%) Rate the most likely health consequence if there is a serious ADE (repeat for significant Permanent disabilityTemporary disabilitySymptoms lasting ⱖ30 dSymptoms lasting ⬍30 dAbnormal laboratory results only Choose the most likely resource used if there is a serious ADE (repeat for significant Emergency department visitOffice visit with new medicationOffice visit without new medicationTelephone call or e-mailNo additional services Estimate the likelihood that the prescription is intercepted and not dispensed Panelists entered a value from 0%-100% Abbreviation: ADE, adverse drug event.
teraction would have been intercepted in a community phar- number of injuries, extent of health care utilization, and costs.
macy and the prescription not dispensed.
We assumed that nonintercepted prescriptions were filled at a Panelists scored each interaction independently. We se- rate of 90% based on a previous study.6 lected 15 interactions with the greatest scoring discrepancies Because our data set included 6 months of prescription infor- and presented them in a second conference call for discus- mation, we annualized the results to provide a more intuitive pre- sion. Participants rescored these DDIs, and these final scores sentation of the data. We extrapolated the analysis of the 100 most were used in the analyses.
commonly accepted DDIs, each selected from 1 of the top 100CCIs, to the CCIs as a whole. In doing so, we assumed that each DDI presented to the panelists was representative of other DDIswithin the same CCI subset. To estimate the total cost savings We used published sources and payer data to estimate the costs and the impact of medication safety alerts on patient safety in Mas- to third-party payers associated with specified categories of health sachusetts attributable to PocketScript prescribers, we then ex- care utilization. The average cost of a medical hospitalization was trapolated from the 100 most common CCIs to the entire uni- estimated at $9000 (Adrienne Cyrulik, MPH, Blue Cross/Blue verse of alerts, assuming that weighted average rates would apply Shield of Massachusetts, written communication, June 17, 2008).
(Figure 2). A test of our assumptions supported this method:
Based on a study of the cost of emergency department visits,18 the average acceptance rate for the top 100 DDIs in the top 100 the average cost of an emergency department visit was calcu- classes was 12.2%, whereas the acceptance rate for all other DDIs lated as $427, adjusted to 2006 dollars using the Consumer Price in the top 100 classes was 12.9%. The average acceptance rate Index for medical services.19 The cost of a physician visit due to for the 100 most commonly accepted CCIs was similar to that of an ADE ($111) was derived from the average national charge in the remaining CCIs (10.6% vs 10.0%).
2006 dollars for a 25-minute office visit.20 For physician visits that We performed sensitivity analyses to test the stability of our re- generated additional prescriptions, we estimated the average cost sults over a range of assumptions. Given the limited number of re- of a filled prescription at $58, calculated using the sales of pre- viewers for each DDI and the possibility of a skewed distribution scription drugs in 2006 ($192 024 661635) divided by the num- of scores, we calculated upper- and lower-bound estimates of the ber of prescriptions dispensed (3 308 896262).21 The cost of tele- likelihood of the type of ADE, injury, and health care utilization phone calls with clinicians was assumed to be $0 because clinicians using the interquartile range (IQR) (25th-75th percentile) for each are not commonly reimbursed for this service.
DDI. We varied our assumptions about the pharmacy interceptionrate (using the panel's estimate in the base case and 10% in an al- ternate model) and the rate with which patients filled their pre-scriptions (90% in the base case and 70% in an alternate model).
We examined the interrater reliability of panelists' first-round judgments regarding the probability that each DDI would pro- the cost and likelihood of hospitalization. We used the average cost duce a serious, significant, or minor ADE; the type of injury to of a US hospitalization ($26 555) as a high-range estimate, calcu- the patient; and the resulting health care utilization. Overall, lated by dividing total hospital revenue from US inpatient hospi- agreement was satisfactory, with a range of 86% to 94% and ␬ tal admissions in 2006 ($939 459 919425) by the total number of scores of 0.49 to 0.69 (P ⬍.001).
admissions (35 377 659).22 We used the panelists' median scores to estimate the type This study was approved by the institutional review board of of ADE, injury, and health care utilization for each DDI. We the Dana-Farber Harvard Cancer Center. Statistical analyses were then used the cost estimates, the number of CCI alerts, and the performed using commercially available software (SAS [SAS In- mathematical models shown in the eAppendix (http://www stitute, Inc, Cary, North Carolina] and Stata 9.1 [StataCorp, Col- .archinternmed.com) to calculate the number of prevented ADEs, lege Station, Texas]).
(REPRINTED) ARCH INTERN MED/ VOL 169 (NO. 16), SEP 14, 2009 2009 American Medical Association. All rights reserved.
1 833 254 Prescriptions 227.2 ADEs prevented, 279 476 DDI alerts $227 480 saved for top 100 CCIs (January 1 to June 30, 2006) Extrapolate top 100CCIs to all CCIs Removed 146 425 duplicate alerts Annualize estimates Extrapolate representative DDI calculation Estimated 402 ADEs prevented, 133 051 Unique alerts to 100 top CCIs (representing 56.6% $402 619 saved, for all alerts of all accepted alerts): 113.6 ADEs prevented, $113 740 saved ˜8.9% Alert acceptance rate (January 1 to June 30, 2006) 11 883 Accepted alerts Modified Delphi process in which panelists assess probability of ADE, likely consequence of ADE, health care utilization, and likelihood of pharmacy interception for 100 representative Select 100 most commonly accepted DDIs; discrepancies reconciled for interactions with greatest score differentials Select the most frequent DDI alert within each Present panelists with 100 of the 100 most commonly accepted CCI alerts representative DDI alerts Figure 2. Flow diagram depicting selection of drug-drug interaction (DDI) alerts for analysis and extrapolation of the results of the analysis to all DDI alerts
generated by a single electronic prescribing system. Data are from Massachusetts in 2006. ADE indicates adverse drug event; CCI, class-class interaction.
Table 2. Number of Prevented ADEs and Injuries
in Massachusetts in 2006 Owing to Accepted Medication
Safety Alerts (Base Case) From a Single Electronic
No. of Alerts
to Prevent
Prevented Events Per Year
Estimated No. of Prevented ADEs Percentage of Drug Alerts Prevented injuries Permanent disability Figure 3. Cumulative number of serious, significant, and minor adverse drug
Temporary disability, ⬍1 y events (ADEs) prevented by safety alerts. Data were obtained from a cohort Symptoms lasting ⱖ30 d of electronic prescribers in Massachusetts in 2006. A small percentage ofalerts accounted for most of the estimated benefits.
Symptoms lasting ⬍30 d Abnormal laboratory results ening, 125 were significant, and 228 were minor (ap-proximately 0.17 ADEs per year for each prescriber). The Abbreviations: ADE, adverse drug event; IQR, interquartile range.
IQR (133-846 ADEs per year) shows the variability as-sociated with these judgments.
The panel's estimates showed that many alerts were required to prevent a single ADE because relatively few ADEs AND RELATED INJURIES
DDIs posed a risk of harm. Overall, clinicians encoun-tered 331 alerts to prevent a single ADE (of any sever- During the 6-month study period, Massachusetts clini- ity) and 2715 alerts to prevent a single serious ADE. As cians encountered DDI alerts in 7.3% of e-prescribing shown in Figure 3, a small percentage of alerts pre-
attempts and overrode 91.1% of the encountered DDI vented most ADEs. Ten percent of alerts were estimated alerts. The crude rate of alert encounters—calculated to prevent 60% of ADEs, and only 6 of the 100 most com- without removing potential duplicate alerts—was monly accepted DDI alerts were judged to have pre- vented at least 1 serious ADE per year (Table 3). Sev-
Table 2 shows the estimated number of ADEs that
enteen percent of alerts were judged unlikely to prevent were prevented in 2006 when 2321 Massachusetts ADEs of any severity.
PocketScript users accepted medication safety alerts, based Of the 402 ADEs potentially prevented by medica- on the expert panel's assessment of risk and harm. In this tion alerts, the expert panel judged that alerts most com- base-case analysis, electronic alerts prevented an esti- monly prevented symptoms that persisted less than 30 mated 402 ADEs, of which 49 were serious or life threat- days (n=272 [67.7%]) (Table 2). Of the 49 serious ADEs, (REPRINTED) ARCH INTERN MED/ VOL 169 (NO. 16), SEP 14, 2009 2009 American Medical Association. All rights reserved.
Table 3. Selected DDI Alerts and Estimates of Serious ADEs Prevented
Probability CCI Alerts
of Serious Accepted, Prescription
DDI Alert
Alert Text
CCI Alert
Fill Rate
High-Value Interaction Alerts
Warfarin with ciprofloxacin Some quinolone antibiotics have Coumarins and been reported to potentiate indanediones with the hypoprothrombinemic effect of warfarin and othercoumarin anticoagulants Diltiazem hydrochloride with Additive reductions in heart rate, Calcium channel cardiac conduction, and cardiac contractility may occur when calcium channel blockers, especially verapamil and diltiazem, are usedconcomitantly with␤-blockers Sedatives, tranquilizers, muscle Narcotic analgesic combinations with and other CNS depressants narcotic analgesic may have additive CNS- and/or respiratory-depressanteffects with propoxyphene Hydrochlorothiazide-triamterene Concomitant use of ACE ACE inhibitors with combination with lisinopril may increase the risk ofhyperkalemia Ibuprofen with prednisolone The combined use of oral corticosteroids and NSAIDs may increase the potential forserious toxic effects in the GItract, including inflammation,bleeding, ulceration, andperforation Sedatives, tranquilizers, muscle Narcotic analgesic combination with lorazepam combinations with and other CNS depressants may have additive CNS- and/or respiratory-depressanteffects with propoxyphene Low-Value Interaction Alerts
Amoxicillin with azithromycin Although some in vitro data Aminopenicillins with indicate synergism between macrolide antibiotics andpenicillins, other in vitro dataindicate antagonism Bupropion hydrochloride with The use of bupropion is Smoking cessation associated with a dose-related risk of seizures Bupropion with methylphenidate The use of bupropion is Smoking cessation associated with a dose-related risk of seizures Azole antifungals such as Topical steroids with clotrimazole combination itraconazole and ketoconazole anti-infectives with with atorvastatin calcium increase the plasma concentrations of some HMG CoA reductase inhibitors and may increase the risk ofrhabdomyolysis The use of bupropion is associated with a dose-related risk of seizures smoking cessationagents accepted alerts may have prevented a patient death in 3 COST SAVINGS
cases, permanent disability in 14, and temporary disabil-ity in 31. Clinicians encountered thousands of alerts Preventing ADEs with e-prescribing may have resulted (4292-44 350) to prevent a single potential death, dis- in 39 (IQR, 14-100) fewer hospitalizations, 34 (6-74) ability, or case of prolonged symptoms.
fewer emergency department visits, 267 (105-541) fewer (REPRINTED) ARCH INTERN MED/ VOL 169 (NO. 16), SEP 14, 2009 2009 American Medical Association. All rights reserved.
Table 3. Selected DDI Alerts and Estimates of Serious ADEs Prevented (continued)
Probability CCI Alerts
of Serious Accepted, Prescription
DDI Alert
Alert Text
CCI Alert
Fill Rate
Low-Value Interaction Alerts (continued)
Although some in vitro data potassium combination with indicate synergism between macrolide antibiotics and penicillins, other in vitro dataindicate antagonism Potassium chloride with Concomitant use of agents with Minerals and tolterodine tartrate electrolytes with (eg, antihistamines, phenothiazines, skeletalmuscle relaxants, tricyclicantidepressants, class IAantiarrhythmics [especiallydisopyramide]) maypotentiate the risk of upper GItract mucosal damageassociated with oral solidformulations of potassiumchloride Potassium chloride with Concomitant use of agents with Minerals and electrolytes with (eg, antihistamines, phenothiazines, skeletalmuscle relaxants, tricyclicantidepressants, class IAantiarrhythmics [especiallydisopyramide]) maypotentiate the risk of upper GItract mucosal damageassociated with oral solidformulations of potassiumchloride Ibuprofen with sertraline SRIs may potentiate the risk for NSAIDs with bleeding in patients treated with agents that affect hemostasis such asanticoagulants, plateletinhibitors, thrombininhibitors, thrombolyticagents, or agents thatcommonly causethrombocytopenia Calcium–vitamin D combination The bioavailability of orally Vitamin and mineral with multivitamin administered iron may be combinations with reduced by concomitant administration of an antacid or other agents with Abbreviations: ACE, angiotensin-converting enzyme; ADE, adverse drug event; CCI, class-class interaction; CNS, central nervous system; DDI, drug-drug interaction; GI, gastrointestinal; HMG CoA, 3-hydroxymethyl-3-glutaryl coenzyme A; NSAID, nonsteroidal anti-inflammatory drug; SRI, serotonin reuptakeinhibitor.
office visits, and 60 (8-109) fewer telephone calls to cli- 78% of cost savings, derived largely from 32 prevented nicians. As shown in Table 4, e-prescribing alerts were
hospitalizations. Thirty-one percent of alerts resulted in judged to yield annual savings of $402 619 (IQR, savings of less than $100 annually, and 19% were judged $141 012-$1 012 386). Divided by the total number of eli- to have no health care utilization impact from the per- gible Massachusetts e-prescribers using the system spective of third-party payers.
(n = 2321), the average savings per clinician in 2006 was$173 (IQR, $61-$436). The bulk of the savings (86.8% of the total) was attributed to the 39 prevented hospi-talizations. Although office visits were the most com- Given the inherent uncertainty in estimating the likeli- monly averted service (n = 267 [66.4% of services]), the hood of ADEs, associated injuries, and health care utiliza- cost per hospitalization dominated the calculation.
tion, we analyzed the impact of alternate assumptions on In a finding similar to the analysis of ADEs, a small our results. Compared with the base-case scenario, we found number of alerts likely accounted for most of the sav- moderate differences (⬍30%, or ⬍121 ADEs) in the num- ings. Ten percent of alerts were estimated to account for ber of ADEs when we varied the rate at which patients filled (REPRINTED) ARCH INTERN MED/ VOL 169 (NO. 16), SEP 14, 2009 2009 American Medical Association. All rights reserved.
Table 4. Prevented Health Care Costs in Massachusetts in 2006 Owing to Accepted Medication Safety Alerts (Base Case)
From a Single Electronic Prescribing System
No. of Alerts
Prevented Health Costs Per Year
to Save $1000
Emergency department visit Office visit with new medication Office visit without new medication Telephone call to clinician No additional services 402 (100.1)
141 012-1 012 386 Abbreviations: IQR, interquartile range; NA, not applicable.
a Percentages may not total 100 because of rounding.
MA admission cost, high ADE likelihood, median hospitalization likelihood MA admission cost, median ADE likelihood, high hospitalization likelihood US average admission cost, median ADE likelihood, median hospitalization likelihood MA admission cost, median ADE likelihood, median hospitalization likelihoodMA admission cost, median ADE likelihood, low hospitalization likelihood MA admission cost, low ADE likelihood, median hospitalization likelihood Estimated No. of Prevented ADEs 100 Estimated Savings, $ Percentage of Drug Alerts Percentage of Drug Alerts Figure 4. Sensitivity analysis of the cumulative number of prevented adverse
drug events (ADEs) owing to medication safety alerts. Varying assumptions
about the rate with which patients filled prescribed medications (70% in
Figure 5. Sensitivity analysis of the cumulative cost savings owing to
model 2 vs 90% in the base case), the rate of pharmacy interceptions (10% medication safety alerts. Varying assumptions about the cost of in model 3 vs panelists' median estimates in the base case), and both factors hospitalization ($9000 for a Massachusetts [MA] admission and $26 555 for (70% pharmacy fill rate and 10% pharmacy interception rate in model 4) did the average cost of a hospitalization in the United States), the likelihood of an not result in drastically different estimates of prevented ADEs. The 75th and adverse drug event (ADE) (panelists' median estimates in the base case vs 25th percentile curves demonstrate the range of panelists' estimates the 25th and 75th percentile estimates in alternate scenarios), and the regarding the frequency with which alerts prevented ADEs, using base-case likelihood of hospitalization given an ADE (panelists' median estimates in the base case vs 25th and 75th percentile estimates in alternate scenarios)affected the estimated cost savings owing to electronic prescribing.
their prescriptions (70% in model 2 vs 90% in the base case), sulted in projected cost savings of $146 000 and $487 000, the likelihood that the pharmacist would intercept a po- respectively (Figure 5).
tentially dangerous prescription (10% in model 3 vs the pan-elists' median estimate in the base case [median, 0.10%; IQR,0%-1%]), and the combination of these factors (70% fill rate and 10% interception rate in model 4 [Figure 4]).
These alternate assumptions resulted in estimates of 313,
Using a modified Delphi technique and data on 1.8 367, and 285 prevented ADEs, respectively, relative to the million prescriptions and 135 051 unique alerts gener- base case (402 ADEs).
ated by ambulatory care clinicians, we estimated that Modifying these factors also had a moderate effect e-prescribing alerts possibly averted 133 to 846 ADEs in (⬍30%, or ⬍$121 000) on cost savings ($313 000, Massachusetts in 2006, including 14 to 130 potentially se- $368 000, and $286 000 for models 2 through 4, respec- rious ADEs that could have caused 2 to 13 deaths and 13 tively, compared with the base-case savings of $403 000).
to 115 disabilities. Alerts may have prevented 125 to 715 Savings were highly dependent on assumptions about the hospitalizations, emergency department visits, and office cost of hospitalization in Massachusetts. When used as visits, for a total savings to the health care system of $141 012 an upper-bound estimate (rather than the base-case as- to $1 012 386. Extrapolating these results to all 38 847 pre- sumption of $9000 per hospitalization), the average cost scribers (both inpatient and outpatient) in Massachusetts of a US hospitalization ($26 555) yielded savings of $1.08 suggests that expanding e-prescribing statewide might pre- million (IQR, $383 000-$2.77 million). In addition to vent more than 6700 ADEs per year, including 50 deaths, varying the cost of hospitalization, we also conducted sen- and result in cost savings of approximately $6.7 million.
sitivity analyses using panelists' 25th and 75th percen- Our study provides perspective on earlier investiga- tile estimates of the likelihood of hospitalization for each tions of medication safety alerts. In previous studies, com- DDI. Lower and upper hospitalization estimates re- puterized prescribing in the hospital reduced noninter- (REPRINTED) ARCH INTERN MED/ VOL 169 (NO. 16), SEP 14, 2009 2009 American Medical Association. All rights reserved.
cepted serious medication errors by 55%, preventable ADEs cial drug interaction databases used throughout North by 17%, and hospital costs by $887 to $8958.16,23-28 Deci- sion analytic models similar to the approach used herein Second, the decisions of Massachusetts clinicians to have been used to estimate the cost of drug-related mor- accept or override electronic alerts may not reflect the bidity in the United States at more than $177 billion in behavior of clinicians in other regions. However, in a re- 2000.7 However, we are aware of no studies that have es- lated study of e-prescribing, we found that the rates and timated the safety benefits or cost savings from the use of types of drug interaction alerts accepted by Massachu- e-prescribing alerts in the ambulatory environment.
setts clinicians were similar to those of clinicians in New Given the recent passage of the Medicare Improve- Jersey and Pennsylvania.34 Nevertheless, these studies ex- ments for Patients and Providers Act of 2008,29 which pro- amined clinicians who have generally opted into the sys- vides federal incentives for the adoption of e-prescribing tem and therefore may reflect the experience of a rela- and imposes penalties on providers who fail to switch to tively committed population of e-prescribers rather than the electronic systems, our findings offer timely informa- what is expected from novice users.
tion to government and private industry. The benefits of Third, the structured implicit judgments of the ex- medication alerts in ambulatory care are likely derived from pert panel were dependent on the makeup of the panel a small fraction of alerts encountered by frontline clini- and the experience of its members. Although the use of cians. Indeed, only 10% of drug interaction alerts were es- a modified Delphi technique enhanced the reliability of timated to account for 60% of ADEs and 78% of cost sav- this process, another panel with different members might ings in our study. Seventeen percent of alerts had no have offered different estimates.
discernible patient safety benefit, and 31% made a small Finally, our analysis took a conservative approach to cal- (⬍$100/y) contribution to cost savings. Accordingly, cli- culating the impact of e-prescribing in ambulatory care. The nicians probably reviewed thousands of alerts to prevent models did not consider the effect of alerts on clinician be- a single serious ADE. In fact, our study may underesti- havior other than aborting a prescription, such as coun- mate the alert burden by examining only a subset of alerts; seling a patient or monitoring for drug interactions. We did we excluded more than 100 000 potentially redundant not address benefits from e-prescribing that might derive alerts and did not include formulary adherence or allergy from standardization of dosing, reduction in duplicate alerts in our analyses.
therapy, prevention of allergic reactions, and elimination The phenomenon of alert fatigue has been well de- of illegible prescriptions,35 nor did we attempt to assess the scribed,3,15,30-32 and the disproportionate relationship be- impact of alerts on patient satisfaction and drug adher- tween the number of alerts and the patient safety and fi- ence. We also focused on the estimated financial benefit nancial benefits of the system makes one wonder whether to third-party payers, rather than the costs of medical the juice is, in fact, worth the squeeze. Our findings sug- injuries borne by patients and their families. Given the lack gest that the savings attributable to prevented ADEs may of data on these issues, our study used methods that are be insufficient to cover the costs to third-party payers of in line with those of other studies that have attempted investing in e-prescribing systems. However, our esti- to estimate the cost of illness and the benefits of mates do not take into consideration savings that might ac- e-prescribing.6,7,10,15 Future research should attempt to quan- crue owing to improved formulary adherence, increased tify the return on investment to medication safety alerts, use of generic drugs, legible prescriptions, and the preven- including reduction in medical liability premiums and losses, tion of allergic reactions to prescribed drugs, and they do to determine when the cost of these systems and the bur- not account for the effect on the patients and families of den of excessive alerts detract from patient care. Studies lost wages and illness-related morbidity. Most important, could also examine differences between drug interaction we believe that the technology's ability to prevent ADEs databases and their effect on patient safety.
makes it worthwhile, and our findings suggest that signifi- Our study suggests that drug alerts have the poten- cant efficiencies could be gained by reducing overalert- tial to prevent harm and reduce health care costs. To do ing. Doing so would mitigate alert fatigue, thereby increas- so, however, clinicians need relief from alerts with little ing the percentage of clinically significant alerts accepted clinical value.
and the number of ADEs averted. Previous studies havedemonstrated that tiering alerts and interrupting prescrib- Accepted for Publication: May 15, 2009.
ers for only the most serious warnings are effective strat- Author Affiliations: Center for Patient Safety, Dana-
egies for increasing alert acceptance rates.4,33 Farber Cancer Institute (Drs Weingart, Padolsky, Isaac, This study has several limitations. First, its generalizabil- and Seger and Mr Simchowitz), Tufts University School ity may be restricted by the use of a single e-prescribing of Medicine (Dr Padolsky), Division of General Medi- system and drug interaction alert database. However, in cine and Primary Care, Beth Israel Deaconess Medical 2008, the PocketScript system was used by 8% of Massa- Center (Drs Weingart, Isaac, and Davis), Division of Gen- chusetts prescribers and approximately 4000 eligible eral Medicine, Brigham and Women's Hospital (Dr Seger), prescribers in 18 states (Christopher Yu, MPH, ZixCorp, Massachusetts College of Pharmacy and Health Sci- written communication, August 19, 2008). Its features, in- ences (Dr Seger), Institute for Health Policy, Massachu- cluding required fields, pick lists with available dose setts General Hospital (Dr Weissman), and Executive forms, and DDI alerts, are common to many commercial Office of Health and Human Services, Commonwealth and home-grown e-prescribing systems. In addition, Cerner of Massachusetts (Dr Weissman), Boston, Massa- Multum, Inc, which maintains the medication safety alerts chusetts; PatientsLikeMe, Cambridge, Massachusetts (Dr for the PocketScript system, is 1 of several major commer- Massagli); and Department of Community and Family (REPRINTED) ARCH INTERN MED/ VOL 169 (NO. 16), SEP 14, 2009 2009 American Medical Association. All rights reserved.
Medicine, University of Massachusetts Medical School, 9. Fink A, Kosecoff J, Chassin M, Brook RH. Consensus methods: characteristics Worcester (Dr Weissman).
and guidelines for use. Am J Public Health. 1984;74(9):979-983.
10. Park RE, Fink A, Brook RH, et al. Physician ratings of appropriate indications for six Correspondence: Saul N. Weingart, MD, PhD, Center for
medical and surgical procedures. Am J Public Health. 1986;76(7):766-772.
Patient Safety, Dana-Farber Cancer Institute, 44 Binney St, 11. Micromedex Healthcare Series (electronic version). Thomson Healthcare. http: Boston, MA 02115 ([email protected]).
//www.thomsonhc.com. Accessed February 29, 2008.
Author Contributions: Dr Weingart had full access to
12. Clinical Pharmacology Web site. Gold Standard Inc. http://www.clinicalpharmacology all the data in the study and takes responsibility for the .com/. Accessed February 29, 2008.
13. Lexi-Comp Inc Web site. http://www.lexi-comp.com/institutions/products integrity of the data and the accuracy of the data analy- /online/. Accessed February 29, 2008.
sis. Study concept and design: Weingart, Padolsky, and 14. Epocrates Web site. http://www.epocrates.com. Accessed February 29, 2008.
Weissman. Acquisition of data: Weingart and Seger. Analy- 15. Malone DC, Abarca J, Hansten PD, et al. Identification of serious drug-drug in- sis and interpretation of data: Weingart, Simchowitz, Pa- teractions: results of the partnership to prevent drug-drug interactions. J Am Pharm dolsky, Isaac, Massagli, Davis, and Weissman. Drafting Assoc (2003). 2004;44(2):142-151.
16. Bates DW, Leape LL, Cullen DJ, et al. Effect of computerized physician order en- of the manuscript: Weingart, Simchowitz, and Padolsky.
try and a team intervention on prevention of serious medication errors. JAMA.
Critical revision of the manuscript for important intellec- tual content: Weingart, Padolsky, Isaac, Seger, Massagli, 17. Morimoto T, Gandhi TK, Seger AC, Hsieh TC, Bates DW. Adverse drug events Davis, and Weissman. Statistical analysis: Weingart, Pa- and medication errors: detection and classification methods. Qual Saf Health Care.
dolsky, Isaac, and Davis. Obtained funding: Weingart and 18. Dennehy CE, Kishi DT, Louie C. Drug-related illness in emergency department Weissman. Administrative, technical, and material sup- patients. Am J Health Syst Pharm. 1996;53(12):1422-1426.
port: Weingart, Simchowitz, Padolsky, and Seger. Study 19. US Department of Labor, Bureau of Labor Statistics. Consumer Price Index for all urban consumers (CPI-U) for the US city average for medical care services, Financial Disclosure: None reported.
1982-1984. http://www.bls.gov/cpi/home.htm#tables. Accessed March 24, 2008.
20. Wasserman Y. Physicians' Fee Reference 2005. 22nd ed. Milwaukee, WI: Medi- Funding/Support: This study was supported by a grant
cal Publishers Ltd; 2005.
from Blue Cross Blue Shield of Massachusetts.
21. Top 200 brand-name drugs by retail sales in 2006. Drug Topics. February 19, Role of the Sponsor: Blue Cross Blue Shield of Massa-
chusetts was not involved in the design and conduct of /drugtopics/072007/405100/article.pdf. Accessed March 24, 2008.
the study; the collection, management, analysis, and in- 22. American Hospital Association. AHA Hospital Statistics, 2008 Edition. Chicago, IL: Jossey-Bass/AHA Press; 2008.
terpretation of data; or the preparation, review, or ap- 23. Bates DW, Spell N, Cullen DJ, et al; Adverse Drug Events Prevention Study Group.
proval of the final manuscript.
The costs of adverse drug events in hospitalized patients. JAMA. 1997;277 Additional Information: The eAppendix is available at
24. Tierney WM, Miller ME, Overhage JM, McDonald CJ. Physician inpatient order Additional Contributions: Christopher Yu, MPH, Angus
writing on microcomputer workstations: effects on resource utilization. JAMA.
1993;269(3):379-383.
MacDonald, and Geoff Bibby, BC, of ZixCorp, provided 25. Mekhjian HS, Kumar RR, Kuehn L, et al. Immediate benefits realized following the electronic prescription and alert data; Adrienne implementation of physician order entry at an academic medical center. J Am Cyrulik, MPH, and Jessica Fefferman, MPH, of Blue Med Inform Assoc. 2002;9(5):529-539.
Cross Blue Shield of Massachusetts, assisted in concep- 26. Evans RS, Pestotnik SL, Classen DC, et al. A computer-assisted management program for antibiotics and other antiinfective agents. N Engl J Med. 1998; tualizing the project and providing third-party-payer 27. Schneider PJ, Gift MG, Lee YP, Rothermich EA, Sill BE. Cost of medication- related problems at a university hospital. Am J Health Syst Pharm. 1995;52(21):2415-2418.
28. Bates DW, Teich JM, Lee J, et al. The impact of computerized physician order entry on medication error prevention. J Am Med Inform Assoc. 1999;6(4):313-321.
1. Gandhi TK, Weingart SN, Borus J, et al. Adverse drug events in ambulatory care.
29. Medicare Improvements for Patients and Providers Act of 2008, Pub L No. 110- N Engl J Med. 2003;348(16):1556-1564.
275 ( July 15, 2008).
2. Gandhi TK, Weingart SN, Seger AC, et al. Outpatient prescribing errors and the 30. Glassman PA, Simon B, Belperio P, Lanto A. Improving recognition of drug in- impact of computerized prescribing. J Gen Intern Med. 2005;20(9):837-841.
teractions: benefits and barriers to using automated drug alerts. Med Care. 2002; 3. Weingart SN, Toth M, Sands DZ, Aronson MD, Davis RB, Phillips RS. Physi- cians' decisions to override computerized drug alerts in primary care. Arch In- 31. Ko Y, Abarca J, Malone DC, et al. Practitioners' views on computerized drug- tern Med. 2003;163(21):2625-2631.
drug interaction alerts in the VA system. J Am Med Inform Assoc. 2007;14 4. Shah NR, Seger AC, Seger DL, et al. Improving acceptance of computerized pre- scribing alerts in ambulatory care. J Am Med Inform Assoc. 2006;13(1):5-11.
32. Magnus D, Rodgers S, Avery AJ. GPs' views on computerized drug interaction 5. Payne TH, Nichol WP, Hoey P, Savarino J. Characteristics and override rates of or- alerts: questionnaire survey. J Clin Pharm Ther. 2002;27(5):377-382.
der checks in a practitioner order entry system. Proc AMIA Symp. 2002:602-606.
33. Paterno MD, Maviglia SM, Gorman PN, et al. Tiering drug-drug interaction alerts 6. Johnson JA, Bootman JL. Drug-related morbidity and mortality: a cost-of- by severity increases compliance rates. J Am Med Inform Assoc. 2009;16(1): illness model. Arch Intern Med. 1995;155(18):1949-1956.
7. Ernst FR, Grizzle AJ. Drug-related morbidity and mortality: updating the cost- 34. Isaac T, Weissman JS, Davis RB, et al. Overrides of medication alerts in ambu- of-illness model. J Am Pharm Assoc (Wash). 2001;41(2):192-199.
latory care. Arch Intern Med. 2009;169(3):305-311.
8. Stewart J, O'Halloran C, Harrigan P, Spencer JA, Barton RJ, Singleton SJ. Iden- 35. Fischer MA, Vogeli C, Stedman M, Ferris T, Brookhart MA, Weissman JS. Effect tifying appropriate tasks for the preregistration year: modified Delphi technique.
of electronic prescribing with formulary decision support on medication use and cost. Arch Intern Med. 2008;168(22):2433-2439.
(REPRINTED) ARCH INTERN MED/ VOL 169 (NO. 16), SEP 14, 2009 2009 American Medical Association. All rights reserved.

Source: http://www.initiative-elga.at/ELGA/allgemein_infos/ArchIntMed_e_prescribing.pdf