Preliminary development of a deep learning-based automated primary headache diagnosis model using Japanese natural language processing of medical questionnaire
- Department of Neurosurgery, Kesennuma City Hospital, Kesennuma, Miyagi, Japan,
- Sendai Headache and Neurology Clinic, Sendai, Miyagi, Japan,
- Department of Neurosurgery, Tohoku University Graduate School of Medicine, Aobaku, Sendai, Miyagi, Japan.
Department of Neurosurgery, Tohoku University Graduate School of Medicine, Aobaku, Sendai, Miyagi, Japan.
DOI:10.25259/SNI_827_2020Copyright: © 2020 Surgical Neurology International This is an open-access article distributed under the terms of the Creative Commons Attribution-Non Commercial-Share Alike 4.0 License, which allows others to remix, tweak, and build upon the work non-commercially, as long as the author is credited and the new creations are licensed under the identical terms.
How to cite this article: Masahito Katsuki1, Norio Narita1, Yasuhiko Matsumori2, Naoya Ishida1, Ohmi Watanabe1, Siqi Cai1, Teiji Tominaga3. Preliminary development of a deep learning-based automated primary headache diagnosis model using Japanese natural language processing of medical questionnaire. 29-Dec-2020;11:475
How to cite this URL: Masahito Katsuki1, Norio Narita1, Yasuhiko Matsumori2, Naoya Ishida1, Ohmi Watanabe1, Siqi Cai1, Teiji Tominaga3. Preliminary development of a deep learning-based automated primary headache diagnosis model using Japanese natural language processing of medical questionnaire. 29-Dec-2020;11:475. Available from: https://surgicalneurologyint.com/surgicalint-articles/10493/
Background: Primary headaches are widespread and costly public health problems. However, there are insufficient medical resources for their treatment in Japan due to two reasons. First, the numbers of headache specialists and clinics remain insufficient. Second, neurologists and neurosurgeons mainly treat headaches in Japan. However, they mainly work as general stroke neurologists, so they cannot focus on primary headache treatment. To solve these problems, we preliminarily developed a deep learning (DL)-based automated diagnosis model from patients’ Japanese unstructured sentences in the medical questionnaire using a DL framework. We hypothesized that the model would reduce the time and burden on both doctors and patients and improve their quality of life.
Methods: We retrospectively investigated our primary headache database and developed a diagnosis model using the DL framework (Prediction One, Sony Network Communications Inc., Japan). We used age, sex, date, and embedding layer made by the medical questionnaire’s natural language processing (NLP).
Results: Eight hundred and forty-eight primary headache patients (495 women and 353 men) are included. The median (interquartile range) age was 59 (40–74). Migraine accounted for 46%, tension-type headache for 47%, trigeminal autonomic cephalalgias for 5%, and other primary headache disorders for 2%. The accuracy, mean precision, mean recall, and mean F value of the developed diagnosis model were 0.7759, 0.8537, 0.6086, and 0.6353, which were satisfactory.
Conclusion: The DL-based diagnosis model for primary headaches using the raw medical questionnaire’s Japanese NLP would be useful in performing efficient medical practice after ruling out the secondary headaches.
Keywords: Deep learning, Japanese natural language processing, Migraine, Primary headache, Tension-type headache
Headache is a widespread and costly public health problem.[
We think that these insufficient medical resources for headache treatment in Japan are due to two reasons. First, the numbers of headache specialists and headache clinics remain insufficient.[
Furthermore, the World Health Organization declared the COVID-19 pandemic in March 2020, which led to challenges in health-care systems and societies worldwide. This pandemic has been producing a rapid shift in favor of telemedicine instead of an in-person consultation. Kristoffersen reported that 86% of the neurological department changed their headache practice during the lockdown into telephone consultations or video consultations in Denmark and Norway.[
Therefore, an automated diagnosis system is still expected, in addition to the telemedicine system. Different from the common diseases, headaches are diagnosed mainly based on the personal interview,[
Therefore, ahead of the world, we preliminarily tried to develop a deep learning (DL)-based diagnosis model from patients’ Japanese unstructured sentences in the medical questionnaire at outpatient using DL framework, Prediction One (Sony Network Communications Inc., Tokyo, Japan)[
The study was approved by our hospital’s research ethics committee (KCHE-2020-5) on October 30. We gained written informed consent for this study from all of the patients, the legally authorized representative of the patients, or the deceased patients’ next of kin. All methods were carried out following relevant guidelines and regulations (Declaration of Helsinki). All personal patient information were deleted from the database for this study to protect patient privacy.
This retrospective study included 848 consecutive new patients aged over 15 years of age at our neurosurgical outpatient who were revealed as a primary headache after neurological examination, personal interview, laboratory tests, head computed tomography, and magnetic resonance angiography between 2015 and 2019. The primary headaches were defined in the ICHD 3rd edition (beta version) (ICHD-3 beta),[
Making the diagnosis model by Prediction One
We used Prediction One framework to make an automated diagnosis model. We used four variables for making the model by Prediction One; age, sex, date of consultation, and the patients’ Japanese unstructured description in the medical questionnaire. Prediction One read the dataset and automatically divided them into almost half as internal training and cross-validation datasets. It automatically adjusted and optimized the numerical variables in a way that is easy to process statistically and mathematically. The unstructured description Japanese sentences were analyzed through NLP using an original Japanese morphological analysis library made by Sony Network Communications Inc. Predefined unimportant words or those that appeared too infrequently were treated as “stop words,” excluded from the analysis. After making the embedding layer from the sentences, using the embedding layer, age, sex, and date, Prediction One selected an appropriate algorithm with ensemble learning and made the best diagnosis model by an artificial neural network (ANN) with internal cross-validation. The details of ANN are trade secrets and could not be provided [
Chart of making deep learning-based automated diagnosis model. The Japanese unstructured description in the medical questionnaire was processed using Japanese natural language processing with Japanese morphological analysis, and the embedding layer was produced. Using the embedding layer, age, sex, and date, a deep learning framework (Prediction One) made an automated diagnosis model. DL: Deep learning, EOS: End of sentences, SW: Stop word, NLP: Natural language processing, TTH: Tension-type headache.
Results are shown as median (interquartile range) because the variables were not with a normal distribution. Mann–Whitney U-test and Chi-square test were used. These statistical analyses were performed with the SPSS software version 24.0.0. (IBM, New York, USA). A two-tailed P < 0.05 was considered statistically significant.
Clinical characteristics of the 848 patients (495 women and 353 men) with primary headaches are summarized in [
Characteristics of the Japanese unstructured description in the medical questionnaire
The median (interquartile range) number of Japanese words was 105 (77–155). Predefined stop words were 188 types of word, but those new were not provided by Prediction One framework. We could not obtain the NLP embedding layer nor identify the specific word contributing to the model accuracy.
DL-based automated diagnosis model
Prediction One produced the DL-based automated diagnosis model using our primary headache patients’ database. [
We herein report 848 primary headache patients who newly came to our hospitals. We preliminarily made the DL-based automated diagnosis model for primary headaches using Japanese NLP. Our model had satisfactory performance based on only four variables. This suggested the possibility of creating a DL-based automated diagnosis model for primary headaches with Japanese NLP analysis after ruling out secondary headaches.
DL and Japanese NLP
Recently, AI is attracting, and it is a transitional period regarding AI from machine learning to DL.[
We think that the automated diagnosis model for primary headaches is suitable for Japanese NLP analysis because patients often write their headache characteristics in detail in the medical questionnaire compared with the answers to the medical staff ’s closed questions. If we can know the diagnosis almost only from the patients’ medical questionnaire, we could save the times and burdens of doctors, nurses, and patients without an in-person consultation during this COVID-19 and telemedicine era. Furthermore, many Asian languages, including Japanese, have onomatopoeia.[
Despite the easiness, advantages, and future potential of DL, the majority of medical staff cannot treat DL software.[
Limitation of this study
First, the number of the sample was small, and this study performed in a single hospital. Other home doctors or internal medicine hospitals also treat headaches, so not all headache patients in our medical area could necessarily be studied. Second, we did not separate chronic migraine and TTH (15 or more days per month) and those nonchronic (less than 15 days per month). Its mechanism is supposed to be different in each type.[
We preliminarily made a DL-based automated diagnosis model for primary headaches. The accuracy, mean precision, mean recall, and mean F value were 0.7759, 0.8537, 0.6086, and 0.6353, respectively. In the future, after ruling out the secondary headaches, the DL-based automated diagnosis model using Japanese NLP of unstructured sentences in the medical questionnaire may lead to perform efficient medical practice, reduce the burden on both doctors and patients, and improve their quality of life.
1. Afzal N, Sohn S, Abram S, Liu H, Kullo IJ, Arruda-Olson AM. Identifying peripheral arterial disease cases using natural language processing of clinical notes. IEEE EMBS Int Conf Biomed Health Inform. 2016. 2016: 126-31
2. . Clinical Practice Guideline for Chronic Headache 2013. 2015. p.
3. Gatti F, Manneschi L. Headache relevance in outpatient activity during COVID-19 pandemic. Neurol Sci. 2020. 41: 481-2
4. Jensen R, Stovner LJ. Epidemiology and comorbidity of headache. Lancet Neurol. 2008. 7: 354-61
5. Katsuki M, Kakizawa Y, Nishikawa A, Yamamoto Y, Uchiyama T. Easily created prediction model using deep learning software (Prediction One, Sony Network Communications Inc.) for subarachnoid hemorrhage outcomes from small dataset at admission. Surg Neurol Int. 2020. 11: 374
6. Kristoffersen E, Faiz K, Sandset E, Storstein A, Stefansen S, Winsvold B. Hospital-based headache care during the COVID-19 pandemic in Denmark and Norway. J Headache Pain. 2020. 21: 132
7. Kudo K, Kishimoto T. Corpus analysis of psychiatric disorders utilizing natural language processing and neurolinguistics. Brain Nerve. 2019. 71: 765-70
8. Kwon J, Lee H, Cho S, Chung CS, Lee MJ, Park H. Machine learning-based automated classification of headache disorders using patient-reported questionnaires. Sci Rep. 2020. 10: 14062
9. López-Bravo A, García-Azorín D, Belvís R, González-Oria C, Latorre G, Santos-Lasaosa S. Impact of the COVID-19 pandemic on headache management in Spain: An analysis of the current situation and future perspectives. Neurologia. 2020. 35: 372-80
10. Nakatani H, Nakao M, Uchiyama H, Toyoshiba H, Ochiai C. Predicting inpatient falls using natural language processing of nursing records obtained from Japanese electronic medical records: Case-control study. JMIR Med Inform. 2020. 8: e16970
11. Okuma H, Kitagawa Y. Epidemiology of headache. Nihon Rinsho. 2005. 63: 1705-11
12. Olesen J, Bes A, Kunkel R, Lance JW, Nappi G, Pfaffenrath V. The international classification of headache disorders, 3rd edition (beta version). Cephalalgia. 2013. 33: 629-808
13. Sakai F, Igarashi H. Prevalence of migraine in Japan: A nationwide survey. Cephalalgia. 1997. 17: 15-22
14. Shibahara I, Osawa SI, Kon H, Morita T, Nakasato N, Tominaga T. Increase in the number of patients with seizures following the Great East-Japan Earthquake. Epilepsia. 2013. 54: e49-52
15. Shirai M, Soshi T. Why is heartache associated with sadness? Sadness is represented by specific physical pain through verbal knowledge. PLoS One. 2019. 14: e0216331
16. Sony Network Communications, Prediction One. Available from: https://www.predictionone.sony.biz [Last accessed on 2020 Feb 29].
17. Staartjes VE, Stumpo V, Kernbach JM, Klukowska AM, Gadjradj PS, Schröder ML. Machine learning in neurosurgery: A global survey. Acta Neurochir (Wien). 2020. 162: 3081-91
18. Suzuki N, Ishikawa Y, Gomi S, Ito N, Watanabe S, Yokoyama M. Prevalence and characteristics of headaches in a socially active population working in the Tokyo metropolitan area-surveillance by an industrial health consortium. Intern Med. 2014. 53: 683-9
19. Online Medical Care System for Pirmary Headache During COVID-19 Era. Available from: https://www.jhsnet.net/dl/20200508.pdf [Last accessed on 2020 Jul 14].
20. Wi CI, Sohn S, Ali M, Krusemark E, Ryu E, Liu H. Natural language processing for asthma ascertainment in different practice settings. J Allergy Clin Immunol Pract. 2018. 6: 126-31
21. Yokota S, Shinohara E, Ohe K. Can staff distinguish falls: Experimental hypothesis verification using Japanese incident reports and natural language processing. Stud Health Technol Inform. 2018. 250: 159-63