Abstract

Background:Clinical prediction tools assist in clinical outcome prediction. They quantify the relative contributions of certain variables and condense information that identifies important indicators or predictors to a targeted condition. This systematic review synthesizes and critically appraises the methodologic quality of studies that derive both clinical predictors and clinical predictor tools used to determine outcome prognosis in patients suffering from aneurysmal subarachnoid hemorrhage (SAH).

Methods:This systematic review included prospective and retrospective cohort studies, and randomized controlled trials (RCTs) investigating prognostic factors and clinical prediction tools associated with determining the neurologic outcome in adult patients with aneurysmal SAH.

Results:Twenty-two studies were included in this systemic review. Independent, confounding, and outcome variables were studied. Methodologic quality of individual studies was also analyzed. Included were 3 studies analyzing databases from RCTs, 8 prospective cohort studies, and 11 retrospective cohort studies. The most frequently retained significant clinical prognostic factors for long-term neurologic outcome prediction include age, neurological grade, blood clot thickness, and aneurysm size.

Conclusions:Systematic reviews for clinical prognostic factors and clinical prediction tools in aneurysmal SAH face a number of methodological challenges. These include within and between study patient heterogeneity, regional variations in treatment protocols, patient referral biases, and differences in treatment, and prognosis viewpoints across different cultures.

Keywords: Aneurysmal subarachnoid hemorrhage, aneurysms, clinical outcome prediction, health research methodology, prognosis, systematic review

INTRODUCTION

Clinical prediction tools assist in clinical outcome prediction. This systematic review synthesizes and critically appraises methodologic quality of studies that derive both clinical predictors and clinical predictor tools used to determine outcome prognosis in patients suffering from aneurysmal subarachnoid hemorrhage.

Clinical prediction tools

Clinical prediction tools assist in clinical outcome prediction, in establishing the likelihood of presence or absence of a condition, as well as in determining potential therapeutic courses of action. As such, they complement clinical opinion and judgment. Clinical prediction tools quantify the relative contributions of certain variables and condense information that identifies important indicators or predictors to a targeted condition.[ 1 2 3 4 5 6 12 19 31 35 36 ]

Methodologic assessment of clinical prediction tools pertains to their derivation and validation. In their development, the study from which the database is developed is critiqued for its study protocol (including inclusion and exclusion criteria, setting, patient recruitment, effective power with sample size of at least 10 patients cases for each predictor variable, description of patient characteristics and follow-up, report and handling of missing data, and subgroup analyses), relevance of predictor variables and outcomes studied (justification and definition of variables and outcomes used, with attention to their coding and reproducibility), description of mathematical models (whether these models are both statistically and clinically sensible). In terms of model performance and validation, clinical prediction tools should be presented with a discussion of the types of performance measures used, as well as the types of validation used (including internal validation techniques such as data splitting, boot-strapping, and external validation techniques, like adopting the derived rules in an external population).[ 1 2 3 4 5 6 12 19 31 35 36 ]

Aneurysmal subarachnoid hemorrhage

Intracranial aneurysmal subarachnoid hemorrhage (SAH) affects about 45,000 individuals in North America and 600,000 individuals worldwide annually. Aneurysmal SAH is associated with a mortality rate of at least 45% in the first 30 days following rupture.[ 22 ] Apart from the primary neurological injury from the aneurysmal rupture itself, other secondary injury processes can further worsen an individual's neurological condition and eventual clinical outcome. These processes include both neurological processes (such as delayed stroke, re-bleeding, brain swelling, vasospasm induced strokes, seizures, and hydrocephalus), and systemic medical complications (such as myocardial infarction, fever, and pulmonary edema).[ 13 21 22 ] Together, these processes can lead to long-term disability. Types of disability include physical, neurocognitive, and psychological impairment. Long-term reductions in health-related quality of life are common, even though the case fatality of aneurysmal SAH has slowly declined due to prompt diagnosis and repair, as well as improved critical care medical management.[ 13 21 22 ]

Objectives

The purpose of this systematic review is to synthesize and critically appraise methodologic quality of studies that derive both clinical predictor tools and clinical predictors used to determine outcome prognosis in patients suffering from aneurysmal SAH, with inclusion of studies with data generated from both prospective and retrospective cohort studies, and randomized controlled trials (RCTs).

METHODS

This systematic review was designed based on a predefined protocol.

Study eligibility criteria

We included prospective and retrospective cohort studies, and RCTs investigating clinical prediction tools and prognostic factors associated with determining neurologic outcome in adult patients with aneurysmal SAH. We excluded prognostic studies and grading schemes based on expert opinions, those for traumatic SAH and perimesencephalic SAH. Eligible studies were limited to those published from January 1, 1995 to March 31, 2014, due to differences in diagnostic modalities and treatment prior to this point.

Literature search

Two reviewers (Benjamin Lo [BL], Hitoshi Fukuda [HF]) independently searched a number of electronic databases. Relevant studies were identified from Ovid MEDLINE, Ovid EMBASE, Web of Science, the Cumulative Index to Nursing and Allied Health Literature, without language restrictions. To include gray literature, we also searched ProceedingsFirst and PapersFirst. We used the search terms aneurysmal SAH, clinical prognosis, and prediction rules.

Study selection and data collection process

Investigators (BL and HF) reviewed all titles and abstracts, and full reports of all potentially relevant trials. The initial literature search (January 1, 1995 to March 31, 2014) yielded 2,863 citations [ Figure 1 ]. Screening by title and abstract and citation yielded 121 items. Of these 121 items, reviewers BL and HF reached agreement on 70 items for inclusion, 42 items for exclusion, and were unsure on 9 items. Consensus conference was held with the assistance of a third reviewer, Yusuke Nishimura (YN). Inter-rater reliability was high (estimated kappa 0.85 (95% confidence interval [CI] 0.80–0.90) for citation and abstract screening). Seventy-nine full-text articles were identified as potentially relevant and were assessed with the further exclusion of articles due to an incomplete variable and outcome reporting, inappropriate patient inclusion and exclusion criteria, and inappropriate predictor models used.

Figure 1

Flow diagram of study selection

Investigators BL and HF then independently applied the inclusion criteria to the full reports. Each trial report was examined carefully for its methodologic quality. As outlined in the “methodologic quality assessment” section, each article was appraised in nine areas. Of the 198 items assessed in 22 articles, BL and HF reached agreement on 160 items, disagreed on 30 items, and were unsure on 8 items (kappa statistic = 0.85, 95% CI 0.80–0.90). Disagreements were resolved through consensus discussions and YN, the third reviewer.

For data collection, the reviewers (BL, HF) extracted relevant data using a data extraction form, piloted on a sample of included studies. Disagreements were resolved by consensus discussions and YN, the third reviewer.

Methodologic quality assessment

For this systematic review, we sampled the quality checklist using Delphi methods for prescriptive clinical prediction rules (QUADCPR),[ 9 ] and criteria proposed by Bouwmeester et al. 2012[ 5 ] for methodologic quality assessment of clinical prediction research. The following areas were used in methodologic quality assessment:

Study design – Including description of study protocol, inclusion and exclusion criteria, study setting, and recruitment

RESULTS

Study search and selection

The initial literature search (January 1, 1995–March 31, 2014) yielded 2863 citations [ Figure 1 ]. These were screened by title and abstract. Seventy-nine full-text articles were identified as potentially relevant and were assessed with the further exclusion of articles due to an incomplete variable and outcome reporting, inappropriate patient inclusion and exclusion criteria, and inappropriate predictor models used. Twenty-two studies were included in this systemic review, with Table 1 examining the independent, confounding, and outcome variables, and Table 2 examining their methodologic quality.

Table 1

Variables investigated in existing clinical prognostic models in aneurysmal SAH

Table 2

Methodological assessment of clinical prognostic models on aneurysmal SAH

Study results and synthesis of results

This systemic review of both clinical prediction tools and prognostic factors in patients with aneurysmal SAH comprised 3 studies analyzing databases from RCTs, 8 prospective cohort studies, and 11 retrospective cohort studies. The most frequently retained significant clinical prognostic factors for long-term neurologic outcome prediction include age (n = 7: Germanson et al. 1998,[ 10 ] McGirt et al. 2007,[ 23 ] Ogilvy et al. 2006,[ 28 ] Rabinstein et al. 2004,[ 30 ] Risselada et al. 2010,[ 32 ] Rosengart et al. 2007,[ 33 ] Karamanakos et al. 2012[ 16 ]), neurological grade (n = 6: Germanson et al. 1998,[ 10 ] Kahn et al. 2006,[ 15 ] McGirt et al. 2007,[ 23 ] Ogilvy et al. 2006,[ 28 ] Rabinstein et al. 2004,[ 30 ] Risselada et al. 2010,[ 32 ] Karamanakos et al. 2012[ 16 ]), blood clot thickness (n = 4: Ogilvy et al. 2006,[ 28 ] Rabinstein et al. 2004,[ 30 ] Risselada et al. 2010,[ 32 ] Rosengart et al. 2007[ 33 ]), and aneurysm size (n = 2: Rosengart et al. 2007,[ 33 ] Risselada et al. 2010[ 32 ]).

Methodological quality of included studies

The included 22 studies all had thorough descriptions of study protocols, including inclusion and exclusion criteria. Representative patient cohorts were included in these studies. With the exception of one study (Soehle et al. 2007[ 34 ]), all studies had adequate patient sample sizes to ensure effective study power. Predictor variables were adequately defined in all studies. Patients were followed from 1 to 12 months after aneurysmal rupture for assessment of neurological outcomes, with small proportions of patients lost to follow-up. Outcome assessments were performed in 15 of 22 studies. In addition, most studies (19 of 22 studies) accounted for potential confounding variables and stratification in their analyses. Univariate and multivariable logistic regression analyses were used for most studies (21 of 22 studies). However, only 6 of 22 studies checked for model performance including good calibration (agreement between predicted probabilities and observed outcome frequencies) and good discrimination (ability to distinguish between patients with and without the outcome).[ 2 3 4 ] Finally, studies of clinical predictors and prediction models in aneurysmal SAH are prone to patient selection and referral biases, as well as recall bias in outcome assessments.

DISCUSSION

This systematic review was conducted to synthesize current evidence on prognostic factors affecting the outcome in aneurysmal SAH and to appraise the methodologic quality of studies investigating these clinical outcome prediction tools.

Methodological issues

Systematic reviews for clinical prognostic factors and clinical prediction tools in aneurysmal SAH face a number of methodological challenges. These include within and between study patient heterogeneity, regional variations in treatment protocols, patient referral biases, and differences in treatment and prognosis viewpoints across different cultures.

Between-center differences in treatment and patient populations influence patient prognosis and clinical outcomes[ 7 8 10 11 14 15 16 17 18 20 22 23 24 25 26 27 28 29 30 32 33 34 37 38 ] These center cluster effects should be taken into account when determining the effect sizes of individual prognostic factors. In addition, prognostic variables may be co-dependent.[ 7 8 10 11 14 15 16 17 18 20 22 23 24 25 26 27 28 29 30 32 33 34 37 38 ] Exploration of interactions between variables is important as they reflect the interrelated pathophysiologic mechanisms of brain-body associations in aneurysmal SAH.

Unlike a recently performed systematic review on clinical prediction models in aneurysmal SAH,[ 13 ] this systematic review included:

Studies that provide clear definitions of predictor variables

This systematic review also attempted to overcome other methodological limitations by including high quality cohort studies and RCTs in prognosis fulfilling a number of quality assessment criteria, namely, those proposed by QUADCPR,[ 9 ] and criteria proposed by Bouwmeester et al. 2012.[ 5 ] In addition, all included studies had clearly defined predictor and outcome variables, effective study power, as well as clinically and statistically sensible prediction tools, and prognostic factors.

Across most studies, the core and most frequently retained clinical outcome predictors in aneurysmal SAH include age,[ 10 16 23 28 30 32 33 ] neurological grade,[ 10 15 16 23 28 30 32 ] aneurysm size,[ 32 33 ] and blood clot thickness.[ 28 30 32 33 ]. Yet, a number of other systemic, physiologic, and neurologic parameters may also turn out to be important clinical outcome predictors. These factors are usually not as frequently included in clinical outcome prognostic studies on aneurysmal SAH. For instance, even though the majority of studies (n = 20) used univariate and multivariable logistic regression analyses for determination of significant prognostic factors and clinical prediction tools, only 6 of the 22 studies checked for model performance. Lack of knowledge about these model performance parameters may perpetuate one's lack of awareness about other possible entities that may influence the clinical prediction model, like potential interactions that may exist between core predictors.

CONCLUSION

Studies attempting to elucidate prognostic factors in aneurysmal SAH are affected by a number of methodologic limitations. This systematic review attempted to overcome some of these methodologic limitations by synthesizing high-quality RCTs and cohort studies. Yet, these synthesized epidemiologic studies did not attempt to clarify underlying mechanisms of how ruptured brain aneurysms influence other body systems. Brain-body associations carry a significant impact on patients’ clinical outcomes. Together, existing methodologic limitations of epidemiologic studies on outcome prognosis in aneurysmal SAH readily influence the quality of clinical insight gained in this area.

References

1. Adams ST, Leveson SH. Clinical prediction rules. BMJ. 2012. 344: d8312-

2. Altman DG, Royston P. What do we mean by validating a prognostic model?. Stat Med. 2000. 19: 453-73

3. Bartfay E, Bartfay WJ. Accuracy assessment of prediction in patient outcomes. J Eval Clin Pract. 2008. 14: 1-10

4. Beattie P, Nelson R. Clinical prediction rules: What are they and what do they tell us?. Aust J Physiother. 2006. 52: 157-63

5. Bouwmeester W, Zuithoff NP, Mallett S, Geerlings MI, Vergouwe Y, Steyerberg EW. Reporting and methods in clinical prediction research: A systematic review. PLoS Med. 2012. 9: 1-12

6. Brismee J, Sizer P. Potential pitfalls of clinical prediction rules. J Man Manip Ther. 2008. 16: 182-

7. Chiang VL, Claus EB, Awad IA. Toward more rational prediction of outcome in patients with high-grade subarachnoid hemorrhage. Neurosurgery. 2000. 46: 28-35

8. Claassen J, Vu A, Kreiter KT, Kowalski RG, Du EY, Ostapkovich N. Effect of acute physiologic derangements on outcome after subarachnoid hemorrhage. Crit Care Med. 2004. 32: 832-8

9. Cook C, Brismée JM, Pietrobon R, Sizer P, Hegedus E, Riddle DL. Development of a quality checklist using delphi methods for prescriptive clinical prediction rules: The QUADCPR. J Manipulative Physiol Ther. 2010. 33: 29-41

10. Germanson TP, Lanzino G, Kongable GL, Torner JC, Kassell NF. Risk classification after aneurysmal subarachnoid hemorrhage. Surg Neurol. 1998. 49: 155-63

11. Heuer GG, Smith MJ, Elliott JP, Winn HR, LeRoux PD. Relationship between intracranial pressure and other clinical variables in patients with aneurysmal subarachnoid hemorrhage. J Neurosurg. 2004. 101: 408-16

12. Hier DB, Edelstein G. Deriving clinical prediction rules from stroke outcome research. Stroke. 1991. 22: 1431-6

13. Jaja BN, Cusimano MD, Etminan N, Hanggi D, Hasan D, Ilodigwe D. Clinical prediction models for aneurysmal subarachnoid hemorrhage: A systematic review. Neurocrit Care. 2013. 18: 143-53

14. Juvela S. Alcohol consumption as a risk factor for poor outcome after aneurysmal subarachnoid haemorrhage. BMJ. 1992. 304: 1663-7

15. Kahn JM, Caldwell EC, Deem S, Newell DW, Heckbert SR, Rubenfeld GD. Acute lung injury in patients with subarachnoid hemorrhage: Incidence, risk factors, and outcome. Crit Care Med. 2006. 34: 196-202

16. Karamanakos PN, von Und Zu, Fraunberg M, Bendel S, Huttunen T, Kurki M, Hernesniemi J. Risk factors for three phases of 12-month mortality in 1657 patients from a defined population after acute aneurysmal subarachnoid hemorrhage. World Neurosurg. 2012. 78: 631-9

17. Kramer AH, Bleck TP, Dumont AS, Kassell NF, Olson C, Nathan B. Implications of early versus late bilateral pulmonary infiltrates in patients with aneurysmal subarachnoid hemorrhage. Neurocrit Care. 2009. 10: 20-7

18. Krishnamurthy S, Kelleher JP, Lehman EB, Cockroft KM. Effects of tobacco dose and length of exposure on delayed neurological deterioration and overall clinical outcome after aneurysmal subarachnoid hemorrhage. Neurosurgery. 2007. 61: 475-80

19. Laupacis A, Sekar N, Stiell IG. Clinical prediction rules. A review and suggested modifications of methodological standards. JAMA. 1997. 277: 488-94

20. Lindvall P, Runnerstam M, Birgander R, Koskinen LO. The Fisher grading correlated to outcome in patients with subarachnoid haemorrhage. Br J Neurosurg. 2009. 23: 188-92

21. Lo B, Macdonald RL. Health-related quality of life after aneurysmal subarachnoid hemorrhage: New data from a large series in Germany. World Neurosurg. 2013. 79: 243-4

22. Macdonald RL, Cusimano MD, Etminan N, Hanggi D, Hasan D, Ilodigwe D. Subarachnoid Hemorrhage International Trialists data repository (SAHIT). World Neurosurg. 2013. 79: 418-22

23. McGirt MJ, Woodworth GF, Ali M, Than KD, Tamargo RJ, Clatterbuck RE. Persistent perioperative hyperglycemia as an independent predictor of poor outcome after aneurysmal subarachnoid hemorrhage. J Neurosurg. 2007. 107: 1080-5

24. Miss JC, Kopelnik A, Fisher LA, Tung PP, Banki NM, Lawton MT. Cardiac injury after subarachnoid hemorrhage is independent of the type of aneurysm therapy. Neurosurgery. 2004. 55: 1244-50

25. Mocco J, Ransom ER, Komotar RJ, Schmidt JM, Sciacca RR, Mayer SA. Preoperative prediction of long-term outcome in poor-grade aneurysmal subarachnoid hemorrhage. Neurosurgery. 2006. 59: 529-38

26. Naidech AM, Drescher J, Ault ML, Shaibani A, Batjer HH, Alberts MJ. Higher hemoglobin is associated with less cerebral infarction, poor outcome, and death after subarachnoid hemorrhage. Neurosurgery. 2006. 59: 775-9

27. Naidech AM, Janjua N, Kreiter KT, Ostapkovich ND, Fitzsimmons BF, Parra A. Predictors and impact of aneurysm rebleeding after subarachnoid hemorrhage. Arch Neurol. 2005. 62: 410-6

28. Ogilvy CS, Cheung AC, Mitha AP, Hoh BL, Carter BS. Outcomes for surgical and endovascular management of intracranial aneurysms using a comprehensive grading system. Neurosurgery. 2006. 59: 1037-42

29. Qureshi AI, Suri MF, Sung GY, Straw RN, Yahia AM, Saad M. Prognostic significance of hypernatremia and hyponatremia among patients with aneurysmal subarachnoid hemorrhage. Neurosurgery. 2002. 50: 749-55

30. Rabinstein AA, Friedman JA, Nichols DA, Pichelmann MA, McClelland RL, Manno EM. Predictors of outcome after endovascular treatment of cerebral vasospasm. AJNR Am J Neuroradiol. 2004. 25: 1778-82

31. Raymond J, Naggara O, Guilbert F, Altman DG. Assessing prognosis from nonrandomized studies: An example from brain arteriovenous malformations. AJNR Am J Neuroradiol. 2011. 32: 809-12

32. Risselada R, Lingsma HF, Bauer-Mehren A, Friedrich CM, Molyneux AJ, Kerr RS. Prediction of 60 day case-fatality after aneurysmal subarachnoid haemorrhage: Results from the International Subarachnoid Aneurysm Trial (ISAT). Eur J Epidemiol. 2010. 25: 261-6

33. Rosengart AJ, Schultheiss KE, Tolentino J, Macdonald RL. Prognostic factors for outcome in patients with aneurysmal subarachnoid hemorrhage. Stroke. 2007. 38: 2315-21

34. Soehle M, Chatfield DA, Czosnyka M, Kirkpatrick PJ. Predictive value of initial clinical status, intracranial pressure and transcranial Doppler pulsatility after subarachnoid haemorrhage. Acta Neurochir (Wien). 2007. 149: 575-83

35. Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N. Assessing the performance of prediction models: A framework for traditional and novel measures. Epidemiology. 2010. 21: 128-38

36. Toll DB, Janssen KJ, Vergouwe Y, Moons KG. Validation, updating and impact of clinical prediction rules: A review. J Clin Epidemiol. 2008. 61: 1085-94

37. van den Bergh WM, Algra A, van der Sprenkel JW, Tulleken CA, Rinkel GJ. Hypomagnesemia after aneurysmal subarachnoid hemorrhage. Neurosurgery. 2003. 52: 276-81

38. Yoshimoto Y, Tanaka Y, Hoya K. Acute systemic inflammatory response syndrome in subarachnoid hemorrhage. Stroke. 2001. 32: 1989-93