Search by:

Source

Donaldson L, Fitzgerald E, Flower O, Delaney A. Review article: Why is there still a debate regarding the safety and efficacy of intravenous thrombolysis in the management of presumed acute ischaemic stroke? A systematic review and meta-analysis. Emerg Med Australas 2016;28(5):496–510.

Study Population: 10,431 patients in 26 randomized trials comparing intravenous thrombolysis with placebo or standard care in acute ischemic stroke

Efficacy Endpoints

Good functional outcome, defined as a modified Rankin Score of 3 or less, i.e. some residual disability requiring assistance but able to walk and care for personal needs independently

Harm Endpoints

Symptomatic intracranial hemorrhage (as defined by individual trials) and overall mortality

Narrative

This is the third NNT summary of thrombolytics for acute ischemic stroke. The first gave thrombolytics, as a class, a "red color recommendation: no benefit." The second gave alteplase, a single agent, a "green color recommendation: benefit>harm." As no relevant trials were published between the two, both author groups examined essentially the same data and arrived at opposing conclusions. We believe it would be hubris to presume this third summary will arrive at the one true answer. We focus, therefore, on the uncertainty we believe leads to conflicting interpretations.

The systematic review we chose to summarize includes 26 randomized trials of more than 10,000 participants, assessing the benefits of thrombolysis for acute ischemic stroke.1 The authors report a 3.2% improvement in good neurologic outcome, a 5.4% increase in symptomatic intracranial hemorrhage, and a 2.5% increase in mortality. However, we question the certainty implied by these summary numbers.

There are multiple relevant systematic reviews with varying methods but similar findings. In one widely cited review, Emberson and colleagues reported only on alteplase (a problem we discuss below) and find a 5% improvement in neurologic outcomes, a 5.5% increase in intracranial hemorrhage, and a 1.4% increase in 90 day mortality that was not statistically significant.2 The 2014 Cochrane review by Wardlaw and colleagues arrives at similar conclusions with significant improvement in neurologic outcomes, increased intracranial hemorrhage, and increased mortality.3 Thus our conclusions and discussion are unchanged by choice of review, and reflect our belief that pooling data on this topic is overly simplistic and masks profound uncertainty.

Caveats

One of the great conceptual difficulties of summary statistics like the number-needed-to-treat (NNT) is the implication of certainty. A major strength is its simplicity, making complex research easier to understand. A weakness, however, is also simplicity, hiding the complexity of research, ignoring confidence intervals, and obscuring biases. For most topics, these details are more important than any individual number.

Sources of uncertainty:

Conflicting individual trial results: A major source of uncertainty is the differing results of trials. Among 26 trials in this systematic review, 24 research groups found no benefit in their selected primary outcome.1 Moreover, the two that claim a benefit (NINDS part 2 and ECASS III)4^, 5 had baseline imbalances that may explain the difference. In re-analyses adjusting for imbalances in both these trials the benefits disappear.7^, 8 However, in some re-analyses of NINDS-2 the benefit is maintained, adding to the uncertainty.9^, 10

Clinical heterogeneity of individual trials: The 26 trials are clinically heterogeneous, enrolling stroke patients of differing demographics, treatment times, stroke severities, anatomic territories, and thrombolytic agents. The author of the first NNT summary felt this was too much heterogeneity for appropriate pooling, a position supported by the major differences in conclusions drawn depending on which studies an author group chooses to include.

Selective emphasis on trials claiming benefit: It is circular and erroneous logic to claim efficacy for thrombolytics based on the trial characteristics of the two positive trials. First, there is legitimate debate about whether they were truly positive. Second, selectively highlighting positive results is a form of the "Texas sharpshooter fallacy".11 For instance, because both NINDS and ECASS III used alteplase, some have suggested alteplase is a superior agent.4^, 5 On close inspection this logic falters: few trials have compared thrombolytic agents head to head, nine additional trials of alteplase are negative, and systematic reviews consistently find no heterogeneity of effect between agents.3^, 5^, 6 Moreover, in evaluating drug efficacy, establishing a class effect is generally a prerequisite for debating or comparing individual agents.12 Therefore, while it may increase complexity, we believe it is a mistake to exclusively examine data from the agent used in the two trials that claimed benefit.

Likewise, while there are theoretical reasons to think early treatment is better, this has not been directly tested and is not strongly supported by data. Neither Donaldson et al. nor the Cochrane review find an interaction between time to treatment and effect.1^, 3 IST-3, the largest trial of thrombolytics for stroke, found better outcomes among those treated after 4.5 hours than in patients treated at 3-4.5 hours from onset. Again, we feel it is best to consider this literature as a whole rather than using time windows selected based on outlying (i.e. positive) results.

Individual trial bias: Bias is a major source of uncertainty in all scientific research. Importantly, using the GRADE tool,13 Donaldson et al. rate the risk of bias as “serious” for all outcomes.1 One notable source is the outcome scales used, for instance the modified Rankin score. This score is known to be subjective with poor inter-rater reliability and questionable validity. When trained neurologists examine the same patients there is substantial variability in Rankin score assignments.14^, 15 Compounding the problem, some trials assessed patients by phone or mail, a choice certain to increase variability and imprecision. For example in IST-3, which contributes nearly 40% of subjects in the Donaldson meta-analysis,16 results were obtained using telephone and mail follow-up, and non-blinded. This subjectivity is important, because removing this trial from the pooled analysis removes any finding of benefit.

Early stoppage and low power: Because larger trials are weighted more heavily in a meta-analysis, early termination (which reduces trial size) can significantly affect results. Five thrombolytic trials were stopped early for harm or futility.17^, 18^, 19^, 20^, 21 Together these would have enrolled more than 2,000 additional subjects who, had they been included, may have neutralized or even reversed findings from the two trials claiming benefit, NINDS2 and ECASS III (combined n=1,445).4^, 5 Furthermore, while over 10,000 subjects were enrolled in stroke trials, some individual trials for acute myocardial infarction enrolled far more, and in aggregate those trials included more than 60,000.22^, 23 The comparatively small number of participants in stroke trials means chance findings like baseline imbalances are both more likely and more influential, furthering uncertainty.

Harms and overlapping outcomes: In contrast to the heterogeneous data on benefits, the data on harms are more certain. Exact numbers vary based on definitions and whether one focuses on fatal, symptomatic, or any hemorrhage, but an increase in intracranial hemorrhage is certain. More importantly, there is also an increase in mortality with thrombolytics.1^, 3

Prior NNT summaries directly compared the number of patients with intracranial hemorrhage to the number with a good neurologic outcome, which can be misleading. Any harms due to intracranial hemorrhage are incorporated into a final neurologic outcome assessment, and therefore good functional outcome is far more important than intracranial hemorrhage.

There is, however, a difficult comparison to consider between the chance of neurologic benefit and the increased risk of death. Thrombolytics appear to increase mortality;1^, 3 however many patients may be willing to accept this risk for an increased chance of functional improvement. Shared decision making for this already very difficult decision is made even harder by the fact that a mortality harm is more certain than any benefit.

Financial conflicts: A final important source of uncertainty in this research arises from financial conflicts of interest, which are unfortunately common in medical research. Although they do not always invalidate findings, they complicate interpretation because studies with financial conflicts are more likely to report positive findings.24^, 25^, 26^, 27 Such conflicts are well documented in both the original thrombolysis research and the subsequent guidelines28 and are known to affect the quality and conclusions of systematic reviews. This may be an unconscious contributor to the divergent conclusions seen for this topic.29

Like all such work, our summary has inherent limitations. Not included in this review, for instance, are newer trials using advanced imaging to select patients for thrombolysis,30^, 31^, 32^, 33 trials we felt should not be combined with studies using thrombolytics based largely on clinical criteria. Also of note, the Donaldson review uses a different cut-off on the modified Rankin Score (≤3) than others. The authors argue being able to walk and care for one’s self independently is the threshold most would consider a good outcome. While this cut-off may seem unconventional, other reviews find similar harms and benefits using different cutoffs, suggesting to us that cutoff points are not an important differentiator of conclusions.2^, 3

The uncertainty in these data may help explain differing views of thrombolysis for stroke: trials are heterogenous with many sources of bias; rare positive findings are marred by baseline imbalances but continue to have outsize influence; outcomes were assessed using imprecise, subjective scales; early terminations tilt the data; outcomes overlap; and conflicts abound. There is no simple summary. We find the only appropriate conclusion is uncertainty.

Summaries of the literature cannot answer questions the literature cannot answer. Thrombolytics for stroke has been among the most debated topics in medicine for three decades, perhaps due to underlying data deficiencies. The Donaldson et al. meta-analysis suggests a potential functional outcome benefit which must be balanced against increased mortality. However, the authors warn us that the findings are at serious risk of bias, and pooling results introduces more bias. Three NNT summaries have now arrived at three conclusions. This question will not be answered by re-analyzing the same data. We need additional, carefully executed trials.

In the meantime, it is difficult to offer clear guidance. Clinicians should be aware of the weaknesses in these data, and patients deserve to know as well. We believe the data are not compelling enough for specific recommendations. Shared decision making is essential, and physicians are left with the difficult task of guiding patients through a decision without a clear answer.

See theNNT.com's previous reviews of this topic:
Thrombolytics for Acute Ischemic Stroke, March 25, 2013
Tissue Plasminogen Activator (tPA) For Acute Ischemic Stroke, January 11, 2019

Author

Justin Morgenstern, MD; W. Ken Milne, MD
Supervising Editors: Shahriar Zehtabchi, MD

Published/Updated

April 21, 2021