|

 

How to use an article about harm

How to Use an Article about Harm

Mitchell Levine, Stephen Walter, Hui Lee, Ted Haines, Anne Holbrook, Virginia Moyer, for the Evidence Based Medicine Working
Based on the Users' Guides to Evidence-based Medicine and reproduced with permission from JAMA. (1994;271(20):1615-1619). Copyright 1995, American Medical Association. • Introduction • I. Are the results of the study valid • II. What are the results • III.What are the implications for my practice • Resolution of the Scenario
Clinical Scenario
You are having lunch in the hospital cafeteria when one of your colleagues raises the issue of the safety of beta-adrenergic
agonists in the treatment of asthma. Your colleague feels uncertain about how to respond to patients asking him about media reports of an increased risk of death associated with these medications. Another colleague mentions a key article on this topic that generated much of the publicity, but she cannot recall the details. You all agree that this is an issue which arises frequently enough in your practices that you should become familiar with the evidence contained in the article that your patients have heard about. You volunteer to search the literature for the key article and report back to your colleagues in the next few days.
The Search
The next day you do a Medline search using the following terms: asthma (mh) (mh stands for MeSH heading, indexing terms
used by National Library of Medicine personnel); adrenergic beta receptor agonists (mh); adverse effects (sh) (sh stands for Sub-heading). You limit the search to "Abridged Index Medicus journals" knowing that you will likely find the article your colleague recalled seeing within this list of major medical journals. Your Medline search (1990-93) identifies 38 citations. There were nine original studies, seven review articles, and 22 letters, editorials, and commentaries. Of the nine original articles, only one is an epidemiologic study assessing the risk of death associated with inhaled beta-adrenergic agonists, and you think this is the article to which your colleague referred. The study describes a 2.6 fold increased risk of death from asthma associated with the use of beta-adrenergic agonist metered dose inhalers [1].
Introduction
Clinicians often encounter patients who may be facing harmful exposures, either to medical interventions or environmental
agents. Are pregnant women at increased risk of miscarriage if they work in front of video display terminals? Do vasectomies increase the risk of prostate cancer? Do hypertension management programs at work lead to increased absenteeism? When examining these questions, physicians must evaluate the validity of the data, the strength of the association between the putative cause and the adverse outcome, and the relevance to patients in their practice (Table 1). The current article in our series of User's Guides to the medical literature will help you evaluate an individual article assessing an issue of harm. To fully assess the cause and effect relationship implied in any question of harm requires consideration of all the information available. Systematic overviews (e.g. meta-analysis) can provide an objective summary of all the available evidence, and we will deal with how to use an overview in a subsequent article in this series. Using such an overview requires an understanding of the rules of evidence for individual studies, and this article covers the basic rules for observational (non- randomized) studies. I. Are the results of the study valid?
A. Primary Guides
1. Were there clearly identified comparison groups that were similar with respect to important determinants of outcome,
other than the one of interest?
In a study which identifies a harmful exposure, the choice of comparison groups has an enormous influence on the credibility of
the results. Because the design of the study determines the comparison groups, we will review the basic study designs that clinicians encounter when assessing whether their patients have been or might be exposed to a potentially harmful factor 2).
Randomized Trials
A randomized trial is a true experiment in which patients are assigned, by a mechanism analogous to a coin flip, to either the
putative causal agent or some alternative experience (either another agent or no exposure at all). Investigators then follow the patients forward in time and assess whether they have experienced the outcome of interest. The great strength of the randomized trial is that we can be confident that the study groups were similar not only with respect to determinants of outcome that we know about, but also those we do not know about. In prior articles in this series, we have shown how readers of articles about therapy can use the results of randomized trials [2] [3]. Randomized trials are rarely done to study possible harmful exposures, but if a well-designed randomized trial demonstrates an important relationship between an agent and an adverse event, clinicians can be confident of the results. For instance, the Cardiac Arrhythmia Suppression Trial (CAST) is a randomized trial which demonstrated an association between the anti-arrhythmic agents encainide, flecainide and moricizine and excessive mortality [4] [5]. As a result, clinicians have curtailed their use of these drugs and become much more cautious in using other anti-arrhythmic agents in the treatment of non-sustained ventricular arrhythmias.
Cohort Studies
When it is either not feasible or not ethical to randomly assign patients to be exposed or not exposed to a putative causal agent,
investigators must find an alternative to a randomized trial. In a cohort study, the investigator identifies exposed and non-
exposed groups of patients and then follows them forward in time, monitoring the occurrence of the outcome. You can appreciate the practical need for cohort studies when subjects cannot be "assigned" to an exposure group, as occurs when one wants to evaluate the effects of an occupational exposure. For example, investigators assessed perinatal outcomes among children of men exposed to lead and organic solvents in the printing industry using a cohort of all males who had been members of printers unions in Oslo, and on the basis of job classification fathers were categorized as to their exposure to lead and solvents. In this study exposure was associated with an eight fold increase in preterm births, but no significant impact on birth defects [6]. Cohort studies may also be performed when harmful outcomes are infrequent. For example, clinically apparent upper gastrointestinal hemorrhage in non-steroidal anti-inflammatory drug (NSAID) users occurs approximately 1.5 times per 1,000 person-years of exposure, in comparison with 1.0 per 1,000 person-years in those not taking NSAIDs (assuming a stable risk over time) [7]. A randomized trial to study this effect would require approximately 6,000 patient-years of exposure to achieve a 95% probability of observing at least one additional serious gastrointestinal hemorrhage among treated patients, and a substantially larger sample size (approximately 75,000 patient-years per group) for adequate power to test the hypothesis that NSAIDs cause the additional bleeds [8]. Such a randomized trial would not be feasible, but a cohort study, particularly one in which the information comes from a large administrative data base, would be. Because subjects in a cohort study select themselves (or are selected by a physician) for exposure to the putative harmful agent, there is no particular reason they should be similar to non-exposed persons with respect to other important determinants of outcome. It therefore becomes crucial for investigators to document the characteristics of the exposed and non-exposed subjects and either demonstrate their comparability, or use statistical techniques to adjust for differences. In the association between NSAIDs and the increased risk of upper gastrointestinal bleeding, age is associated both with exposure to NSAIDs and with gastrointestinal bleeding, and is therefore called a possible "confounding variable". In other words, since patients taking NSAIDs will be older, it may be difficult to tell if their increased risk of bleeding is because of their age or because of their NSAID exposure. When such a confounding variable is unequally distributed in the exposed and non-exposed populations, investigators use statistical techniques which correct or "adjust for" the imbalances. Even if investigators document the comparability of potentially confounding variables in exposed and non-exposed cohorts, or use statistical techniques to adjust for differences, there may be an important imbalance in prognostic factors that the investigators don't know about or have not measured that may be responsible for differences in outcome. It may be, for instance, that illnesses that require NSAIDs, rather than the NSAIDs themselves, are responsible for the increased risk of bleeding. Thus, the strength of inference from a cohort study will always be less than that of a rigorously conducted randomized trial. Case-Control Studies
When the outcome of interest is either very rare or takes a long time to develop, cohort studies also may not be feasible.
Investigators may use an alternative design in which they identify cases, patients who have already developed the outcome of interest (e.g. a disease, hospitalization, death). The investigators then choose controls, persons who do not have the outcome of interest, but who are otherwise similar to the cases with respect to important determinants of outcome such as age, sex and concurrent medical conditions. Investigators can then assess retrospectively the relative frequency of exposure to the putative harmful agent among the cases and controls. This observational design is called a case-control study. Using a case-control design, investigators demonstrated the association between diethylstilbestrol (DES) ingestion by pregnant women and the development of vaginal adenocarcinoma in their daughters many years later . A prospective cohort study designed to test this cause and effect relationship would have required at least twenty years from the time when the association was first suspected until the completion of the study. Further, given the infrequency of the disease, a cohort study would have required hundreds of thousands of subjects. Using the case-control strategy, the investigators defined two groups of young women - those who had suffered the outcome of interest (vaginal adenocarcinoma) were designated as the cases (n=8), and those who did not have the outcome, as the controls (n=32). Then, working backwards in time, the exposure rates to DES were determined for the two groups. Analogous to the situation with a cohort study, investigators had to ensure balance, or adjust for imbalances, in important risk factors in cases and controls (e.g. intrauterine x-ray exposure). The investigators found a strong association between in utero DES exposure and vaginal adenocarcinoma which was extremely unlikely to be attributable to the play of chance (p<0.00001) without a delay of 20 years, and requiring only 40 women. As with cohort studies, case-control studies are susceptible to unmeasured confounders. Therefore, the strength of inference that can be drawn from the results may be limited.
Case series and case reports
Case series and case reports do not provide any comparison group, and are therefore unable to satisfy the requirements of the
first primary guide. Although descriptive studies occasionally demonstrate dramatic findings mandating an immediate change in physician behavior (e.g. thalidomide and birth defects), there are potentially undesirable consequences when actions are taken in response to weak evidence. Bendectin (a combination of doxylamine, pyridoxine and dicyclomine used as an antiemetic in pregnancy) was withdrawn as a result of case reports suggesting it was teratogenic [10]. Later, a number of comparative studies demonstrated the relative safety of the drug [11] but they could not eradicate a litigious atmosphere which prompted the manufacturer to withdraw the drug from the market. Thus, many pregnant women who could have benefited were denied the symptom relief the drug could have offered. In general, clinicians should not draw conclusions about cause and effect relationships from case series, but recognize that the results may generate questions for regulatory agencies and clinical investigators to address.
Design Issues -- Summary
It is apparent that, just as for questions of therapeutic effectiveness, clinicians should look for randomized trials to resolve issues
of harm. It is also apparent that they will often be disappointed in this search, and must be satisfied with studies of weaker
design. Whatever the design, however, they should look for an appropriate control population before making a strong inference about a putative harmful agent. For RCT and cohort studies the control group should be similar in baseline risk of outcome. For case-control studies the derived exposed and non exposed groups should also be similar with respect to determinants of
outcome.
2. Were the exposures and outcomes measured in the same way in the groups being compared?
In case-control studies, ascertainment of the exposure is a key issue. Patients with leukemia, when asked about prior exposure
to solvents, may be more likely to recall exposure than would a control group, either because of increased patient motivation (recall bias) or greater probing by an interviewer (interviewer bias). Clinicians should attend to whether investigators used strategies, such as blinding subjects and interviewers to the hypothesis of the study, to minimize bias. For example, in a case- control study describing the association between psychotropic drug use and hip fracture, investigators established drug exposure by examining computerized claims files of the Michigan Medicaid program, a strategy that avoided both recall and interviewer bias [12]. As a result, the clinician has more confidence in the study's findings of a two fold increase in risk of hip fracture. Exposure opportunity should also be similar amongst cases and controls. There is evidence suggesting a 2.7 fold increased risk of homicide amongst individuals keeping a gun in their home. It would be important to know that the control group had a similar opportunity for gun possession, otherwise the true risk could be different from the study results - increased if the controls had a greater opportunity, decreased if the controls had a lesser opportunity for gun possession [13]. In RCTs and cohort studies, ascertainment of outcome is the key issue. Investigators have reported a three-fold increase in risk of malignant melanoma in individuals working with radioactive materials. One possible explanation for some of the increased risk might be that physicians, aware of a possible risk, search more diligently and therefore detect disease that might otherwise go unnoticed (or detect disease at an earlier point in time). This could result in the exposed cohort having an apparent, but spurious, increase in risk, a situation we 3. Was follow-up sufficiently long and complete?
An additional point relating to the measurement of outcomes is the need for adequate follow-up in randomized trials and cohort
studies. As discussed in a previous article in this series [2] , losing patients to follow-up threatens the validity of the results
because the patients who are lost may have very different outcomes from those available for assessment. The longer the follow- up period required, the greater the possibility that the follow-up will be incomplete. In a well-executed study, investigators determined the vital status of 1,235 of 1,261 white males (98%) employed in chrysotile asbestos textile operation between 1940 and 1975. The relative risk for lung cancer death increased monotonically from 1.4 to 18.2 with cumulative exposure among asbestos workers with at least 15 years since first exposure [15]. Because the 2% missing data were unlikely to effect the results and the follow-up was sufficiently long, the study allows relatively strong inference about the increase in cancer risk with asbestos exposure.
B. Secondary Guides
4. Is the temporal relationship correct?
Does exposure to the harmful agent precede the adverse outcome? The reports of increased suicidal ideation associated with
the use of the anti-depressant fluoxetine illustrate the importance of this question [16]. Did the thoughts of suicide occur after the fluoxetine was administered or were the patients given this drug because they were already showing signs of clinical deterioration? A meta-analysis of controlled trials of treatment for depression did not confirm the apparent association [17].
5. Is there a dose-response gradient?
We are more confident attributing an adverse outcome to a particular exposure if, as the quantity or the duration of exposure to
the putative harmful agent increases, risk of the adverse outcome also increases. The risk of dying from lung cancer in male physician smokers is dose dependent; the risk increases by 50%, 132% and 220% for 1-14, 15-24 and 25 or more cigarettes smoked per day, respectively [18].
II. What are the results?
1. How strong is the association between exposure and outcome?
We have described the most common way of expressing an association between exposure and outcome, the relative risk, in
detail in an earlier paper in this series [3]. In brief, the relative risk is the risk (or incidence) of the adverse effect in the exposed group divided by the risk of the adverse effect in the unexposed group. Values greater than 1 represent an increase in risk associated with the exposure, while values less than 1 represent a reduction in risk. To illustrate, in a cohort study assessing in- hospital mortality following non-cardiac surgery in male veterans, 23/289 patients with a history of hypertension died, compared with 3/185 patients without. The relative risk for hypertension was 4.9 [19]. The relative risk tells us that death occurs almost 5 times more often in the hypertension patients than in normotensive patients. The estimate of relative risk depends on having samples of exposed and unexposed patients, where the proportion of the patients with the outcome of interest can be determined. The relative risk is therefore not applicable to case-control studies in which the number of cases and controls, and therefore the proportion of individuals with the outcome, is chosen by the investigator. For case-control studies, instead of using a ratio of risks, we use a ratio of odds: the odds of a case-patient being exposed, divided by the odds of a control-patient being exposed. Using a simple 2 X 2 table relative risks (RR) and odds ratios (OR) can be represented as depicted in Table 3. When the outcome of interest is rare in the population from which the sample of cases was drawn, which is often the reason for employing a case-control design in the first place, the odds ratio closely approximates the relative risk. When considering both study design and strength of association, we may be ready to interpret a small increase in risk as representing a true harmful effect when the study design is strong (such as a randomized trial). A much higher increase in risk might be required of weaker designs (such as cohort or case-control studies) as subtle findings is more likely to be due to subtle flaws in design. Very large values of RR or OR represent strong associations that are less likely to be due to confounding or bias. 2. How precise is the estimate of the risk?
In a previous paper in this series we have shown how the clinician can evaluate the precision of the estimate of treatment effect
by examining the confidence interval around that estimate can take the same approach with articles assessing
risk. In a study in which the investigators have shown an association between an exposure and an adverse outcome, the lower limit of the estimate of relative risk associated with the adverse exposure provides a minimal estimate of the strength of the association. In a study where the investigators fail to demonstrate an association (a "negative" study), the upper boundary of the confidence interval around the relative risk tells the clinician just how big an adverse effect may still be present, despite the failure to show a statistically significant association.
III. What are the implications for my practice?
1. Are the results applicable to my practice?
If you are convinced that the results of the study are valid for the population that was studied, you then have to decide whether
you can extrapolate the results to patients in your own practice. Are your patients similar to those described in the study with respect to morbidity, age, race or other potentially important factors? Are there clinically important differences in the treatments or exposures between your patients and the patients studied? For example, the risk of thrombophlebitis associated with oral contraceptives described in the 1970's may not be applicable to the patient of the 1990's because of the lower estrogen doses currently in use. Similarly, increases in uterine cancer secondary to postmenopausal estrogen probably don't apply to women
who are also taking concomitant progestins tailored to produce monthly withdrawal bleeding.
2. What is the magnitude of the risk?
The relative risk and the odds ratio do not tell us how frequently the problem occurs, only that the observed effect occurs more or
less often in the exposed group compared to the unexposed group. Thus, the reader needs a method for assessing clinical importance. In our discussion of therapy we described how the clinician can calculate the number of patients she must treat to prevent an adverse event [3]. When the issue is harm, the clinician can use data from a randomized trial or cohort study to make an analogous calculation to determine how many people must be exposed to the harmful agent to cause an adverse outcome. From the CAST study, over an average of 10 months of follow-up, mortality was 3.0% and 7.7% for placebo and encainide/flecainide patients respectively. The absolute risk increase was 4.7%, the reciprocal of which tells us that, on average, for every 21 patients we treat with encainide or flecainide for about a year, we will cause one excess death [4]. This contrasts with NSAIDs and upper gastrointestinal bleeding. Of 2,000 unexposed patients, two will suffer a bleed each year. Of 2,000 patients taking NSAIDs, three will suffer a bleed each year. Thus, if we treat 2,000 patients with NSAIDs we can expect a single additional bleeding event [6].
3. Should I attempt to stop the exposure?
After evaluating the evidence that an exposure is harmful, determining subsequent actions may not be simple. There are at least
three aspects the physician must consider in making a clinical decision [20].
• One is the strength of inference; how strong was the study or studies that demonstrated harm in the first place? • Second, what is the magnitude of the risk to patients if exposure to the harmful agent continues? • Third, what are the adverse consequences of reducing or eliminating exposure to the harmful agent? Clinical decision making is simple when both the likelihood of harm, and its magnitude, are great. Because the evidence of increased mortality from encainide and flecainide came from a randomized trial, we can be confident of the causal connection. Since treating only 21 people will result in an excess death, it is no wonder that clinicians quickly curtailed their use of these anti-arrhythmic agents when the study results became available. The clinical decision is also made easier when an acceptable alternative for avoiding the risk is available. For example, beta blockers prescribed for the treatment of hypertension can result in symptomatic increase in airways resistance in patients with asthma or chronic airflow limitation, mandating the use of an alternative drug, such as a thiazide diuretic, in susceptible patients [21]. Even if the evidence is relatively weak, the availability of an alternative can result in a clear decision. The early case-control studies demonstrating the association between aspirin use and Reye's syndrome were relatively weak and left considerable doubt about the causal relationship. Although the strength of inference was not great, the availability of a safe, inexpensive, and well-tolerated alternative, acetaminophen, justified use of this alternative agent in children at risk of Reye's syndrome [22]. In contrast to the early studies regarding ASA and Reye's syndrome, multiple well-designed cohort and case-control studies have consistently demonstrated an association between NSAIDs and upper gastrointestinal bleeding, and our inference about harm has been therefore relatively strong. However, the risk of an upper gastrointestinal bleed is quite low, and we don't have safer equally efficacious anti-inflammatory alternatives available. We are therefore probably right in continuing to prescribe NSAIDs for the appropriate clinical conditions.
Resolution of the Scenario
The study you retrieved on the risks of inhaled beta-adrenergic therapy used a case-control design relying on computer record
linkages between health insuranc. The data base for the study included 95% of the population of the province of Saskatchewan in western Canada. The investigators matched 129 cases of fatal or near fatal asthma with 655 controls who were also asthmatics. The investigators attempted to control for potential confounders, such as disease severity. Their measures of disease severity included number of hospitalizations in the previous 24 months and an index of the aggregate use of medications. They found an association between the routine use of large doses of beta adrenergic agonist metered dose inhalers and death from asthma (odds ratio, 2.6 per canister per month, 95% confidence interval, 1.7 to 3.9). The study satisfied the validity criteria in Table 1 quite well. The investigators chose an appropriate control population and corrected for measurable potential differences in important prognostic factors in the treatment and control groups; exposure and outcome were measured the same way in treatment and control groups; the temporal relationship is correct; and they found a dose-response gradient. However, the study used a case-control design rather than a randomized trial, and we remain uncertain whether differences in unmeasured prognostic variables between the treatment and control groups explain the results. In other words, it is still possible that the patients who used more beta-agonists were sicker, and this (rather than their increased use of the drug) explains the increased risk of death. The magnitude of the association is moderate, and although the baseline risk of death from asthma (44 deaths in 12,301 medicated asthmatics, 0.3%) is low enough that we would have to treat large numbers of patients before the drugs were responsible for a death, reducing preventable deaths is extremely important. The fact that the data came from a population- based study suggests the results are widely generalizable. Thus, as an individual study on the subject, you find the results of an "association" between inhaled beta-adrenergic agonist use and death both believable and relevant to your practice. Because it is not an RCT you are less certain about a true causal relationship underlying the observed association. This would require a systematic review of all the evidence in the literature to move further in deciding whether heavy use of beta-agonists cause additional asthma deaths. You tell your inquiring patients that there is an increased risk of death in heavy users of inhaled beta-adrenergic agonists, but that you cannot be certain whether it is due to the drug or possibly the consequence of having severe disease. Intermittent use of inhaled beta-agonist therapy in patients with reversible airflow obstruction provides an attractive alternative to more intensive administration, and many clinicians have responded to the results of this and other studies by choosing this alternative approach.
References

1. Spitzer WO, Suissa S, Ernst P et al. The use of beta-agonists and the risk of death and near death from asthma. N Engl J Med
326. 501-6 (1992).
2. Guyatt GH, Sackett DL, Cook DJ. Users' guides to the medical literature: II. How to use an article about therapy or prevention: A. Are the results of the study valid? Evidence-Based Medicine Working Group. J A M A 270. 2598-601 (1993). 3. Guyatt GH, Sackett DL, Cook DJ. Users' guides to the medical literature: II. How to use an article about therapy or prevention: B. What are the results and will they help me in caring for my patients? Evidence-Based Medicine Working Group. J A M A 271. 4. Echt DS, Liebson PR, Mitchell LB, Peters RW, Obias-Manno D, Barker AH, Arensberg D, Baker A, Friedman L, Greene HL. Mortality and morbidity in patients receiving encainide, flecainide, or placebo. The Cardiac Arrhythmia Suppression Trial Mortality and morbidity in patients receiving encainide, flecainide, or placebo. The Cardiac Arrhythmia Suppression Trial. N Engl J Med 324. 781-8 (1991). 5. The Cardiac Arrhythmia Suppression Trial II Investigators, Anonymous. Effect of the antiarrhythmic agent moricizine on survival after myocardial infarction. The Cardiac Arrhythmia Suppression Trial II Investigators. N Engl J Med 327. 227-33 (1992). 6. Irgens LM, Daltveit AK, Andersen A, Kristensen P. Perinatal outcome among children of men exposed to lead and organic solvents in the printing industry. Am J Epidemiol 137. 134-44 (1993). 7. Carson JL, Strom BL, Soper KA, et al, West SL, Morse ML. The association of nonsteroidal anti-inflammatory drugs with upper gastrointestinal tract bleeding The association of nonsteroidal anti-inflammatory drugs with upper gastrointestinal tract bleeding. Arch Intern Med 147. 85-8 (1987). 8. Walter SD. Determination of significant relative risks and optimal sampling procedures in prospective and retrospective comparative studies of various sizes. Am J Epidemiol 105. 387-97 (1977). Herbst AL, Ulfelder H, Poskanzer DC. Adenocarcinoma of the vagina. Association of maternal stilbestrol therapy with tumor appearance in young women. N Engl J Med 284. 878-81 (1971). Soverchia G, Perri PF. Two cases of malformation of a limb in infants of mothers treated with an antiemetic in a very early phase of pregnancy. Pediatria Medica e Chirurgica 3. 97-9 (1981). Holmes LB. Teratogen update: bendectin. Teratology 27. 277-81 (1983). Ray WA, Griffin MR, Schaffner W, et al, Baugh DK, Melton LJ, III. Psychotropic drug use and the risk of hip fracture. N Engl J Med 316. 363-9 (1987). Kellerman AL, Rivara RP, Rushforth NB et al. Gun ownership as a risk factor for homicide in the home. N Engl J Med 329. Hiatt RA, Fireman B. The possible effect of increased surveillance on the incidence of malignant melanoma. Prev Med 15. 652-60 (1986). Dement JM, Harris RL, Symons MJ, Shy CM, Harris RL, Jr. Exposures and mortality among chrysotile asbestos workers. Part II Mortality Exposures and mortality among chrysotile asbestos workers. Part II: mortality. Am J Med 4. 421-33 (1983). Teicher MH, Glod C, Cole JO. Emergence of intense suicidal preoccupation during fluoxetine treatment. Am J Psychiatry 147. 207-10 (1990). Beasley CM, Dornseif BE, Bosomworth JC, Sayler ME, Rampey AH, Heiligenstein JH, Thompson VL, Murphy DJ, Masica DN. Fluozetine and suicide: a meta-analysis of controlled trials of treatment for depression. Br Med J 303. 685-92 (1991). 18. Doll R, Hill AB. Mortality in relation to smoking: ten years' observation of British doctors. Br Med J 1. 1399-410 (1964). Browner WS, Mangano DT, Li J. In-hospital and long-term mortality in male veterans following noncardiac surgery. The Study of Perioperative Ischemia Research Group. J A M A 268. 228-32 (1992). 20. Sackett DL, Haynes RB, Guyatt GH, Tugwell P. Second Edition. Clinical epidemiology, a basic science for clinical medicine.(1991) Second Edition.Boston/Toronto: Little, Brown and Company. Ogilvie RI, Burgess ED, Cusson JR, Feldman RD, Leiter LA, Myers MG. Report of the Canadian hypertension society consensus conference: 3. pharmacologic treatment of essential hypertension. Can Med Assoc J 149. 575-84 (1993). Soumerai SB, Ross-Degnam D, Kahn JS, Ross-Degnan D. Effects of professional and media warnings about the association between aspirin use in children and Reye's syndrome. Millbank Quart 70. 155-82 (1992). Copyright 2001 Evidence-Based Medicine Informatics Project Table 1: Users' Guides for an Article About Harm
I. Are the results of the study valid?
• Primary Guides: outcome, other than the one of interest? • Were the outcomes and exposures measured in the same way in the groups being compared? • Was follow-up sufficiently long and complete? • Secondary Guides: • Is the temporal relationship correct? • Is there a dose response gradient? II. What are the results?
• How strong is the association between exposure and outcome? • How precise is the estimate of the risk? III. Will the results help me in caring for my patients?
• Are the results applicable to my practice? • What is the magnitude of the risk? • Should I attempt to stop the exposure? Table 2: Directions of inquiry and key methodologic strengths and weaknesses for different study designs.
Starting Point
Assessment
Strengths
Weaknesses
adverse event status internal validity feasibility, generalizability adverse event status feasible when randomization of susceptible to threats to exposure not possible internal validity overcomes temporal delays, susceptible to threats to may only require small sample internal validity Table 3. Estimates of Relative Risks and Odds Ratios for Exposed and Unexposed Patients
Adverse Event (Case)
No Adverse Event (Control)
Exposed a
Not Exposed c
Relative Risk = [a/a+b]/[c/c+d] Odds Ratio = [a/c]/[b/d]

Source: http://peds.arizona.edu/sites/default/files/curriculum-files/r-usersguide_harm.pdf