Abstract

Background: Health literacy profoundly impacts patient outcomes as patients with decreased health literacy are less likely to understand their illness and adhere to treatment regimens. Patient education materials supplement in-person patient education, especially in cerebrovascular diseases that may require a multidisciplinary care team. This study aims to assess the readability of online patient education materials related to cerebrovascular diseases and to contrast the readability of those materials produced by academic institutions with those of non-academic sources.

Methods: The readability of online patient education materials was analyzed using Flesch-Kincaid Grade Level (FKGL) and Flesch Reading Ease (FRE) assessments. Readability of academic-based online patient education materials was compared to nonacademic online patient education materials. Online patient education materials from 20 academic institutions and five sources from the web were included in the analysis.

Results: Overall median FKGL for neurovascular-related patient online education documents was 11.9 (95% CI: 10.8–13.1), reflecting that they are written at a 12^th grade level, while the median FRE was 40.6 (95% CI: 34.1–47.1), indicating a rating as “difficult” to read. When comparing academic-based online patient education materials to other internet sources, there was no significant difference in FRE and FKGL scores (P = 0.63 and P = 0.26 for FKGL and FRE, respectively).

Conclusion: This study demonstrates that online patient education materials pertaining to cerebrovascular diseases from major academic centers and other nonacademic internet sites are difficult to understand and written at levels significantly higher than that recommended by national agencies. Both academic and nonacademic sources reflect this finding equally. Further study and implementation are warranted to investigate how improvements can be made.

Keywords: Cerebrovascular disease, Cerebrovascular surgery, Health literacy, Patient education, Readability

INTRODUCTION

Cerebrovascular neurosurgery is a neurosurgical subspecialty in which many of its disease entities require complex decision-making. Such decision-making aims to balance potential neurological risks with the inherent risk of a surgical or endovascular intervention. This complex decision-making process also involves evaluating the use of multiple treatment modalities and collaboration with multiple providers from different biases and backgrounds. As a result, it can be challenging for patients as they try to navigate medical decisions for themselves, their families, and their loved ones. The increasing incidence of incidentally discovered cerebrovascular disorders only makes these decisions more challenging.[ 8 , 16 - 18 ]

In response to these challenges, academic neurosurgery departments have created online education materials for patients with the primary objectives of defining diagnoses, describing risk factors, explaining disease natural history, and outlining available treatment options of cerebrovascular conditions. In addition to the description of treatment options, online materials typically aim to discuss anticipated results and potential complications that pertain to cerebrovascular surgery. However, an assessment of the efficacy of how well cerebrovascular-related patient educational materials communicate essential information to patients is lacking.

The National Institute of Health (NIH) recommends that patient education materials are written at a 6^th grade level,[ 19 ] since the average American reads at the 6^th–8^th grade level.[ 28 ] Other national agencies such as the American Medical Association (AMA) and the United States Department of Health and Human Services also make similar recommendations.[ 21 ] Scales have been developed to evaluate how easily a piece of written text can be read. The Flesch-Kincaid Grade Level (FKGL) and Flesch Reading Ease (FRE) are two of the most commonly utilized and accessible readability tools. FKGL provides an estimate of the academic grade level needed to read and understand a particular piece of written material.[ 10 , 13 ] FRE focuses on how difficult a passage is to read and leads to a rating from 0 to 100.[ 10 ] Higher scores denote increased reading ease with 100–91 being rated “very easy,” 90–81 “easy,” 80–71 “fairly easy,” 70– 61 “standard,” 60–51 “fairly difficult,” 50–31 “difficult,” and 30–0 “very difficult.”[ 10 ]

The aim of this study is to analyze the readability of patient education materials pertaining to cerebrovascular conditions. We sought to examine the readability of materials from academic centers and compared these to other more easily available online sources.

MATERIALS AND METHODS

To identify institutions with readily available patient education materials related to cerebrovascular conditions, the listing of academic centers in the US News and World Report was utilized. Table 1 is the list of the institutions included in the survey. Centers were further divided by regions: Northeast, South, Midwest, and West. A search of neurosurgery and neurology departmental websites was performed by the primary authors (CSE and SA). Patient materials arising from Google searches related to particular cerebrovascular neurosurgery conditions were also evaluated.

Table 1:

List of academic institutions and nonacademic websites included in the study.

Documents were chosen for study evaluation if they contained content that discussed five common cerebrovascular conditions: Dural arteriovenous fistulas, brain arteriovenous malformations, carotid stenosis, cavernomas, and brain aneurysms. Each academic institution had up to five documents included per topic, while the first five articles from nonacademic sites were analyzed for each topic. A “Diagnosis” subgroup was created using documents relating to epidemiology, symptoms, and diagnosis. A “treatment” subgroup was made with articles connected to treatment, management, and prognosis. Only online content written in English was included in the analysis.

All documents were assessed using plain text with Microsoft Word for Mac version 16.20. The FKGL and FRE for each article were generated. This technique has been previously described by Sabharwal et al.[ 24 ] The formula for FKGL is as follows: (0.39 * average number words per sentence) + (11.8 * average number of syllables per word) − 15.59. The formula for FRE is: 206.835−1.015 × (words/sentences) − 84.6 × (syllables/words). Intraclass correlation coefficients were calculated for both examiners (CSE, SA). Intra-rater reliability, for both FKGL and FRE, was ascertained using 10 randomly selected articles. An analogous method was used for inter-rater reliability. Twenty documents were independently analyzed and randomly selected for readability score (CSE, SA).

Median values and 95% confidence intervals were calculated for FKGL and FRE for each article included. A oneway analysis of variation was calculated across regions and conditions. Two-tailed t-tests were used to compare documents from academic centers to alternate internet information. Statistical significance was set at P < 0.05. All statistical analyses were performed using GraphPad 7 (GraphPad), VassarStats (http://www.vassarstats.net ), and Microsoft Excel (Microsoft Excel).

RESULTS

Eighteen of 20 included institutions (90%) [ Table 1 ] had readily available online patient education materials, and a total of 75 documents were included and reviewed. In total, there were 72 documents with information for the diagnostic subgroup and 71 documents with information for the treatment subgroup. Intra-rater and inter-rater reliability for both FKGL and FRE was 1.0, indicating concordance in the methodology between the authors performing the primary analysis.

The median FKGL for all documents across all institutions was 11.7 (95% CI: 10.7–12.4), and the median FRE was 42.9 (95% CI: 41.3–46.1). Median FKGL and FRE scores across geographical regions are shown in Table 2 . There were no significant differences between the FKGL and FRE scores among geographical regions within the U.S. [ Table 2 ]. The median FKGL of documents in the diagnosis subgroup was 11.3 (95% CI: 10.3–11.9), while the median FRE was 46.2 (95% CI: 43.1–48.4). The difference in medians between all regions for FKGL and FRE in the diagnosis subgroup was not statistically significant [ Table 2 ]. The median FKGL of documents in the treatment subgroup was 12.1 (95% CI: 11.2–13.3), while the median FRE was 39.7 (95% CI: 37.5– 43.6). These differences were not statistically significant [ Table 2 ]. There was a significant difference between median diagnostic and treatment groups in terms of the FRE, indicating documents dealing with diagnostic information were significantly more difficult to read than those of treatment and outcome (P = 0.01). In terms of FKGL, there was a trend for articles dealing with treatment and outcome to contain more challenging reading material, compared with the diagnostic subgroup (P = 0.09).

Table 2:

Median FKGL and the FRE overall across the United States geographical regions.

Table 3 shows that the FKGL and FRE scores stratified by disease type. There were significant differences in overall FRE scores among disease conditions; however, there was no difference in overall FKGL (P = 0.19 and P = 0.02 for overall FKGL and FRE, respectively). There was no statistical difference in diagnostic FKGL or FRE score (P = 0.06 and P = 0.22 for FKGL and FRE, respectively). There was a significant difference in FRE terms for the treatment subgroup, but no differences in FKGL scores (P = 0.29 and P = 0.03 for FKGL and FRE, respectively).

Table 3:

Median FKGL and the FRE overall across the specific conditions.

The overall median FKGL for all documents from the internet [ Table 1 ] was 11.9 (95% CI: 10.8–13.1), and the median FRE was 40.6 (95% CI: 34.1–47.1). When comparing the information from academic institutions with information from the internet, there was no difference in overall FRE and FKGL scores (P = 0.63 and P = 0.26 for FKGL and FRE, respectively). The median FKGL of documents in the diagnosis subgroup was 11.1 (95% CI: 10.2–11.8), while the median FRE was 46.3 (95% CI: 43.5–49.0). There was no difference in the diagnostic information when compared to academic institutions (P = 0.47 and P = 0.20 for FKGL and FRE, respectively). The median FKGL of documents in the treatment subgroup was 13.1 (95% CI: 11.2–13.9), while the median FRE was 34.9 (95% CI: 28.7–43.5). There was no difference in the diagnostic information when compared to academic institutions (P = 0.73 and P = 0.18 for FKGL and FRE, respectively) [ Table 4 ].

Table 4:

Median FKGL and the FRE overall across the specific conditions for alternate internet information.

DISCUSSION

Cerebrovascular neurosurgery encompasses a number of complex conditions that are often difficult for patients to understand. To further complicate matters, these conditions are frequently managed by multiple providers from numerous specialties, some of which may potentially be located at separate institutions. Often, these providers will have conflicting ideas and offer contrasting advice to patients.[ 7 , 30 ] This is especially important for cerebrovascular neurosurgery, in which making informed treatment decisions require a sophisticated understanding of the future risk compared with up-front procedure-related risk. After provider consultations, patients may leave the office with an exaggerated estimate of risk and lack a clear understanding of treatment plans, emphasizing the difficulty in communicating this type of information.[ 14 ] Not to mention, patients may not remember all important details following an appointment, especially if already predisposed to lapses in consciousness due to age or the anxiety-inducing nature of much of this content.[ 20 ]

These common patient experiences do not mean that patient education efforts are doomed to fail. For example, an in-depth educational and interactive informed consent process involving booklets, cartoons, videos, questionnaires, and interviews was found to significantly increase comprehension in patients with unruptured intracranial aneurysms as compared to controls.[ 21 ] Similarly, the use of 3D-printed models in displaying unruptured intracranial aneurysms was shown to greatly increase patient understanding following preoperative consultation.[ 16 ] Many of these interventions are costly or impractical to implement on a large scale. As a result, clinic-based patient education initiatives still fail to address the gap of improving the accessibility and readability of internet-based patient materials in a world where patients are increasingly reliant on the internet for health-care information.[ 3 ]

In our study, we found that online patient education materials pertaining to cerebrovascular disease provided by academic medical centers were often challenging to read and properly understand. Difficulties in readability and comprehension were also observed when comparing the readability of cerebrovascular neurosurgical information provided by medical centers to those found on other sites. These challenges are suggestive that, in aggregate, medical information is written at too high of a reading level. To add, our data suggest that this is not a problem limited to tertiary and academic medical institutions. A similar, though more limited, investigation into the readability of online reading materials for brain aneurysms also came to the conclusion that patient education materials are written at levels too high for patients to understand.[ 11 ] More general online neurosurgical information has been shown to suffer from a similar lack of readability.[ 2 , 15 , 22 ]

The American Medical Association and National Institutes of Health have recommended that patient education materials be written at the 6^th-grade level or below.[ 19 ] This is largely due to the fact that health literacy has been shown to be a significant factor with regards to health-care outcomes.[ 29 ] Low health literacy is associated with increased mortality, length of stay, and increased costs to the health-care system, reaching as high as 236 billion dollars annually.[ 5 , 6 ] Conversely, patient education interventions have been shown to make a positive economic impact and improve patient and provider satisfaction.[ 1 , 26 ]

It is clear that there is a need for more readable patient education materials to improve health literacy, which has already resulted in improvements in literacy within other medical specialties.[ 25 ] A significant reduction in FKGL scores from 10.4 to 9.3 was observed when looking at patient education materials from the Association of American Orthopedic Surgeons website from 2008 to 2014.[ 23 ] Over a similar period of time, Eltorai et al. noted improvements in FKGL from 11.5 to 10.7 in spine-related patient education materials.[ 9 ] Some of the documents analyzed were from the American Association of Neurological Surgeons, where there was only a slight improvement in FKGL from 12.1 to 11.8.[ 9 ] These positive results suggest that improvements are attainable within other neurosurgical subspecialties, such as cerebrovascular neurosurgery.

This study is inherently limited in scope. The FKGL and FRE tests are unable to account for accompanying images and videos found online. Additional differences in presentation and layout that may affect a patient’s reading experience and understanding could not be considered. Furthermore, we are unable to comment on how the readability of these documents is affected by translations into other languages or for websites written in other languages. Language discordance is an important factor in promoting health literacy and has been associated with increased risk of readmission, longer length of stay, and even infection rates.[ 12 , 15 , 27 ] Moreover, while readability assessments have been performed using other languages, much of the evidence within neurosurgery is not recent and thus provides an avenue for the future study.[ 4 ]

CONCLUSION

Cerebrovascular conditions can be challenging for patients to understand and often require input by physicians from multiple subspecialties, presenting patients with different perspectives on risk and conflicting advice for treatment. Patient education materials provide an essential adjunct to in-person patient education during a consultation and can make all the information they are presented with more easily comprehended. However, our study indicates that currently available information from major U.S. academic centers and other nonacademic websites pertaining to cerebrovascular disease are written at a level significantly higher than recommended by national agencies. The authors of these articles should factor the NIH and AMA guidelines of patient education materials to allow patients to make informed health-care decisions. The future studies should focus on improving the readability of the articles to understand the effects of patient outcomes related to these diseases.

Declaration of patient consent

Patient’s consent not required as there are no patients in this study.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.

References

1. Abbott SA. The benefits of patient education. Gastroenterol Nurs. 1998. 21: 207-9

2. Ammanuel SG, Edwards CS, Alhadi R, Hervey-Jumper SL. Readability of online neuro-oncology-related patient education materials from tertiary-care academic centers. World Neurosurg. 2020. 134: e1108-14

3. Bailey SC, O’Conor R, Bojarski EA, Mullen R, Patzer RE, Vicencio D. Literacy disparities in patient access and health-related use of Internet and mobile technologies. Health Expect. 2015. 18: 3079-87

4. Bartels U, Hargrave D, Lau L, Esquembre C, Humpl T, Bouffet E. Analysis of paediatric neuro-oncological information on the Internet in German language. Klin Padiatr. 2003. 215: 352-7

5. Berkman ND, Sheridan SL, Donahue KE, Halpern DJ, Crotty K. Low health literacy and health outcomes: An updated systematic review. Ann Intern Med. 2011. 155: 97-107

6. Dewalt DA, Berkman ND, Sheridan S, Lohr KN, Pignone MP. Literacy and health outcomes: A systematic review of the literature. J Gen Intern Med. 2004. 19: 1228-39

7. Ducroux C, Fahed R, Khoury NN, Gevry G, Kalsoum E, Labeyrie MA. Intravenous thrombolysis and thrombectomy decisions in acute ischemic stroke: An interrater and intrarater agreement study. Rev Neurol (Paris). 2019. 175: 380-9

8. Ellis JA, Barrow DL. Supratentorial cavernous malformations. Handb Clin Neurol. 2017. 143: 283-9

9. Eltorai AE, Cheatham M, Naqvi SS, Marthi S, Dang V, Palumbo MA. Is the readability of spine-related patient education material improving? An assessment of subspecialty websites. Spine (Phila Pa 1976). 2016. 41: 1041-8

10. Flesch R. A new readability yardstick. J Appl Psychol. 1948. 32: 221-33

11. Gupta R, Adeeb N, Griessenauer CJ, Moore JM, Patel AS, Kim C. Evaluating the complexity of online patient education materials about brain aneurysms published by major academic institutions. J Neurosurg. 2017. 127: 278-83

12. John-Baptiste A, Naglie G, Tomlinson G, Alibhai SM, Etchells E, Cheung A. The effect of English language proficiency on length of stay and in-hospital mortality. J Gen Intern Med. 2004. 19: 221-8

13. Karliner LS, Kim SE, Meltzer DO, Auerbach AD. Influence of language barriers on outcomes of hospital care for general medicine inpatients. J Hosp Med. 2010. 5: 276-82

14. Kim PS, Choi CH, Han IH, Lee JH, Choi HJ, Lee JI. Obtaining informed consent using patient specific 3D printing cerebral aneurysm model. J Korean Neurosurg Soc. 2019. 62: 398-404

15. Kincaid JP, Fishburne RP, Rogers RL, Chissom BS.editors. Derivation of New Readability Formulas Personnel Flesch Reading Ease Formula) For Navy Enlisted Personnel. Research Branch Report. 1975. p. 8-75

16. King JT, Yonas H, Horowitz MB, Kassam AB, Roberts MS. A failure to communicate: Patients with cerebral aneurysms and vascular neurosurgeons. J Neurol Neurosurg Psychiatry. 2005. 76: 550-4

17. Lunsford LD, Niranjan A, Kano H, Monaco EA, Flickinger JC. Leksell stereotactic radiosurgery for cavernous malformations. Prog Neurol Surg. 2019. 34: 260-6

18. Mooney MA, Zabramski JM. Developmental venous anomalies. Handb Clin Neurol. 2017. 143: 279-82

19.

20. Nguyen MH, Smets EM, Bol N, Bronner MB, Tytgat KM, Loos EF. Fear and forget: How anxiety impacts information recall in newly diagnosed cancer patients visiting a fast-track clinic. Acta Oncol. 2019. 58: 182-8

21. Park J, Son W, Park KS, Kang DH, Lee J, Oh CW. Educational and interactive informed consent process for treatment of unruptured intracranial aneurysms. J Neurosurg. 2017. 126: 825-30

22. Ramos CL, Williams JE, Bababekov YJ, Chang DC, Carter BS, Jones PS. Assessing the understandability and actionability of online neurosurgical patient education materials. World Neurosurg. 2019. 130: e588-97

23. Roberts H, Zhang D, Dyer GS. The readability of AAOS patient education materials: Evaluating the progress since 2008. J Bone Joint Surg Am. 2016. 98: e70

24. Sabharwal S, Badarudeen S, Unes Kunju S. Readability of online patient education materials from the AAOS web site. Clin Orthop Relat Res. 2008. 466: 1245-50

25. Schoof ML, Wallace LS. Methods Materials and Procedures Readability of American Academy of Family Physicians Patient Education Materials Assessment of Reading Demands. Brief Report. 2014. 46:

26. Stenberg U, Vågan A, Flink M, Lynggaard V, Fredriksen K, Westermann KF. Health economic evaluations of patient education interventions a scoping review of the literature. Patient Educ Couns. 2018. 101: 1006-35

27. Tang EW, Go J, Kwok A, Leung B, Lauck S, Wong ST. The relationship between language proficiency and surgical length of stay following cardiac bypass surgery. Eur J Cardiovasc Nurs. 2016. 15: 438-46

28. Walsh TM, Volsko TA. Readability assessment of internet-based consumer health information. Respir Care. 2008. 53: 1310-5

29. Weiss BD.editors. Health Literacy: A Manual for Clinicians. Chicago: American Medical Association; 2003. p.

30. Zipfel GJ, Derdeyn CP, Dacey RG. Current status of manpower needs for management of cerebrovascular disease. Neurosurgery. 2006. 59: S261-70