Preliminary study of eye tracking to investigate the differences in gaze behaviors depending on the experience of neuroendovascular therapy
- Department of Neurosurgery and Center for Cerebrovascular Intervention, Saitama Medical University Saitama Medical Center, Kawagoe, Saitama, Japan.
- Division of Medical Devices, National Institute of Health Sciences, Kawasaki, Kanagawa, Japan.
- Department of Neurosurgical Engineering and Translational Neuroscience, Tohoku University Graduate School of Bioengineering, Japan.
- Biomedical Flow Dynamics Laboratory Creative Flow Research Division, Tohoku University Institute of Fluid Science, Sendai, Miyagi, Japan.
- Department of Neurosurgery, The University of Tokyo Hospital, Japan.
- Department of Bioengineering, The University of Tokyo Graduate School of Engineering, Japan.
- Medical Device Development and Regulation Research Center, The University of Tokyo Graduate School of Engineering, Bunkyo-ku, Tokyo, Japan.
- Department of Neurosurgery, Kobe City Medical Center General Hospital, Kobe, Hyogo, Japan.
- Department of Neurosurgery, Tohoku University Graduate School of Medicine, Sendai, Miyagi, Japan.
Department of Neurosurgery, Tohoku University Graduate School of Medicine, Sendai, Miyagi, Japan.
DOI:10.25259/SNI_543_2020Copyright: © 2020 Surgical Neurology International This is an open-access article distributed under the terms of the Creative Commons Attribution-Non Commercial-Share Alike 4.0 License, which allows others to remix, tweak, and build upon the work non-commercially, as long as the author is credited and the new creations are licensed under the identical terms.
How to cite this article: Masaaki Shojima1, Yoshihiro Okamoto2, Kuniyasu Niizuma3, Makoto Ohta4, Osamu Ishikawa5, Ayano Fujisawa6, Hiroyuki Tsukihara7, Nobuyuki Sakai8, Teiji Tominaga9. Preliminary study of eye tracking to investigate the differences in gaze behaviors depending on the experience of neuroendovascular therapy. 21-Oct-2020;11:351
How to cite this URL: Masaaki Shojima1, Yoshihiro Okamoto2, Kuniyasu Niizuma3, Makoto Ohta4, Osamu Ishikawa5, Ayano Fujisawa6, Hiroyuki Tsukihara7, Nobuyuki Sakai8, Teiji Tominaga9. Preliminary study of eye tracking to investigate the differences in gaze behaviors depending on the experience of neuroendovascular therapy. 21-Oct-2020;11:351. Available from: https://surgicalneurologyint.com/surgicalint-articles/10344/
Background: Neuroendovascular therapy is now the choice for the management of many neurovascular pathologies, and physicians with endovascular skills are in high demand. In addition to the traditional method of practicing hand movements to learn skills, a new strategy of practicing eye movements to learn skills is also attracting attention. This preliminary study explored the differences in gaze behavior depending on experience with endovascular procedures to be facilitated in future skill training in neuroendovascular therapy.
Methods: Four physicians with experience of 3-412 neuroendovascular procedures wore eye-tracking devices during coil embolization of swine cervical arteries. Gaze metrics with direct correlations to the expertise of endovascular procedures were explored.
Results: Gaze metrics with a positive direct correlation to experience included the proportion of fixation durations (PFD) in the screen area and the native images. Those with a negative direct correlation included the PFD in the off-screen area and the roadmap images and the average fixation durations in the off-screen and coil areas. During the parent artery occlusion procedure with detachable coils, more experienced operators preferred to look at native images rather than roadmap images and that less experienced operators tended to look down at their hands more frequently.
Conclusion: This preliminary study demonstrated the feasibility of eye tracking to identify the differences in gaze behavior depending on the experience of endovascular procedures and may guide future eye-tracking studies in neuroendovascular therapy.
Keywords: Coil embolization, Experience, Eye tracking, Gaze behavior, Skill learning
While neuroendovascular therapy (NET) was considered a second-line approach to treat neurovascular pathologies before 2000, it is now the approach of choice;[
Eye movement characteristics are thought to differ between experts and nonexperts.[
While expert eye movement characteristics likely have useful applications to skill learning in NET, eye movement characteristics cannot be generalized across different domains of expertise.[
The ethical aspects of this study were reviewed and approved by the ethical committee of the research institute. All participants provided written informed consent. All experimental procedures were conducted according to the Guidelines for the Care and Use of Laboratory Animals of the research institute.
Study subjects were recruited from neurosurgeons who belonged to the affiliated institutions of the researchers. Among the 20 board-certified neurosurgeons informed of the study, four chose to participate. The participant experience level for endovascular procedures was quantified as the number of NET cases in which he or she was the operator.
Preparation of the animal
A 35 kg male swine was obtained from Tokyo Laboratory Animals Science Co., Ltd. (Tokyo, Japan). The habituated animal was inducted into anesthesia by intramuscular injections of ketamine and xylazine. After intubation, anesthesia was maintained with isoflurane inhalation. Under sterile conditions, both sides of the groin were incised to expose the femoral arteries. Six-French sheath introducers were cannulated into the bilateral femoral arteries under direct vision. After the 8 h experiment, euthanasia was performed using an intravenous injection of a pentobarbital overdose.
Each study participant was assigned the task of interrupting antegrade blood flow at a bifurcation of a swine cervical artery by coil embolization under X-ray fluoroscopy. The X-ray fluoroscopic apparatus had a single flat detector and a 60” monitor (INFX-8000V, Canon Medical Systems Corporation, Tochigi, Japan).
The vessel diameters and appearances were nearly identical at the site where the maxillary artery gave off the facial artery and where the subclavian artery bifurcated into the axillary and subscapular arteries. Therefore, each subject took charge of one of these four bifurcations for the task. The subjects were instructed to completely interrupt the antegrade flow by putting three detachable coils compactly within the area designated by the researchers. Each subject could freely choose detachable coils from the lineup with varying sizes and lengths.
Recording of eye movements
For recording eye movements, each subject put on a head-mounted, eye-tracking device (Tobii Pro Glasses 2, Tobii AB, Stockholm, Sweden), as shown in [
Eye-tracking device. (a) The eye-tracking device used in this study consisted of a glasses unit, a recording unit, and a mini-HDMI cable. (b) The operator (right) is wearing the eye-tracking device, and the assistant is wearing ordinary glasses. The recording device is attached to the operator’s back and is not seen in this figure. Informed consent was obtained from the subjects for the posting of this photo.
The coil insertion period, from when the coil arrived at the tip of the microcatheter to when the delivery wire was removed after the coil was detached, was determined for each coil. Only the data from that period were further analyzed. The time required for coiling was defined for analysis as the sum of the coil insertion time for each subject.
Using gaze data recorded at 100 Hz, gaze points that fell within a narrow area for a short period were grouped as fixations using the Tobii I-VT (attention) filter.[
Areas of interest for analysis (upper) and gaze plots (lower). The upper figure presents the area of interest set up for subject 1. The rectangular area with orange lines defines the screen area of interest. The rectangular areas with yellow continuous and dotted lines define the roadmap fluoroscopic image and the native fluoroscopic image. The circular areas painted in blue and red represent the coil area of interest and the microcatheter area of interest, respectively. The lower figure shows gaze plots of subject 1 during the coil insertion procedure. The yellow circle represents the fixation site, and the diameter of the circle is proportional to the fixation time.
Areas of interest
Areas of interest (AOI) were drawn on the still images, wherein the fixations had been mapped [
The diameters of the coil AOI and microcatheter AOI were 16 times and 8 times that of the 6-Fr guiding catheter, respectively. The area outside the screen was defined as off-screen AOI.
The duration of each fixation was summed in each AOI as the total fixation duration. For between-subject comparisons, the proportion of fixation duration (PFD) was calculated as the ratio of the total fixation duration to the time required for coiling for each area of interest. This metric has been reported to reflect the degrees of the attention of subjects; the importance, informativeness, and visibility of targets; and the difficulty of tasks.[
Each fixation had its duration, and the average fixation durations (AFDs) were calculated for each AOI. This metric has been reported to increase or decrease based on the efficiency of visual intake, informativeness of targets, cognitive workload to be processed, and visibility of targets.[
Spearman’s rank correlation coefficients (ρ) were calculated between experience level and the gaze metrics to identify specific gaze metrics related to experience level (i.e., the number of NET cases as an operator). An absolute value of ρ = 1.0 meant a direct correlation between experience level and a gaze metric.
The number of NET cases as an operator varied from 3 to 412 cases in the study participants. All subjects succeeded in completely interrupting antegrade blood flow by coil embolization. The average time required for coiling was 590.8 s.
The time required for coiling tended to decrease as the experience level increased, though the correlation with experience level was not direct (ρ = −0.8).
Relationships between the experience level of operators and gaze metrics are shown in [
Gaze metrics directly correlating to experience. Among the gaze metrics analyzed in this study, only the gaze metrics that demonstrated direct correlations with experience are shown in the line charts. The figures between parentheses along the horizontal axis are the number of experienced cases. AFD: Average fixation duration, PFD: Proportion of fixation duration.
On average, the PFD in the screen area was 89.5%, with the remaining 10.5% being devoted to the off-screen area. When subjects were watching the off-screen area, they were mainly looking down at their own hands. There were direct relationships between experience level and PFD in the screen area (ρ = 1.0) and PFD in the off-screen area (ρ = −1.0) [top of
PFD in the native images and PFD in the roadmap images correlated directly with the experience level (middle of
There was no direct relationship between the PFD in the coil and microcatheter areas and the experience level; however, the PFD in the coil area tended to decrease slightly as the experience level increased (ρ = −0.8).
AFD tended to decrease as the experience level increased when compared between subjects. In particular, experience level correlated directly with AFD in the roadmap images and AFD in the coil area in a negative manner (bottom of
This preliminary study performed eye tracking of four neurosurgeons during coil embolization of swine cervical arteries and identified several gaze metrics directly correlated with experience levels of endovascular procedures.
It is generally expected that more experienced operators will make quicker decisions. More experienced operators would be more likely to distribute their visual attention over a broader area if so. Part of this expectation was demonstrated by the shorter AFD of the more experienced operators. However, the experienced subjects’ gaze behavior differed from these expectations in that their visual attention was biased toward the native image rather than being broadly distributed between the roadmap and native images.
The characteristic deflection of the gaze to the native fluoroscopy by experienced subjects may be attributed to the simplicity of the task employed in this study. The roadmap image provides a fluoroscopic image of the coil overlaid on the vessel image, but the digital imaging process leads to the coil’s slightly unnatural appearance. In the parent artery occlusion procedure, once the first coil is placed, an experienced operator can see the vessels’ approximate course by merely looking at the native image. Besides, almost all operator considerations are to ensure that the coils are densely inserted. For this reason, a more experienced operator may have concentrated on native fluoroscopy with more natural coils’ appearance. If the task was complicated, such as the aneurysm embolization procedure, experienced operators’ gaze behavior may have differed from that obtained in this study. Experienced operators may have frequently switched their gazes between the roadmap and native images because the positional relationship between the coils and the vessels is also essential to avoid the aneurysm rupture or the coil protrusion into the parent vessel. The gaze behavior that serves as a learning model may differ from procedure to procedure. To further this study, it seems necessary not only to generalize the findings by increasing the number of subjects but also to investigate variations in gaze behavior in various procedures.
Less experienced operators spent more time looking out of the screen than experienced operators, and they tended to look down at their hands during that time. The PFD in the off-screen area was 17.5% in subject 1, with the least amount of experience, and was only 5.6% in subject 4, who had the largest amount of experience. In a study by Law et al.,[
In all areas of interest, AFD was negatively correlated with experience levels. Particularly, AFD in the roadmap image and coil area had a direct negative correlation with the experience level. Whether the AFD grows longer or shorter with experience depends on the type of task. Experts in inspection work are supposed to have acquired effective templates for processing scenes fast and implicitly, and AFD is reported to get shorter with experience.[
This study had several limitations. First, due to the small number of subjects, the results of this preliminary study need to be reconfirmed in studies with increased numbers of subjects before applying to actual skill learning in neuroendovascular therapy. Second, the blood vessels assigned to each subject were different. However, the diameter and the shape of the bifurcation were almost identical in the three-dimensional images. Thus, the influence of this difference may have been not significant. Third, the task adopted in this study might have been too easy compared to the actual endovascular procedures. In future studies, it may be better to devise more complex tasks or perform eye tracking in clinical cases while giving due consideration to safety. Further, there are some inevitable errors when measuring gaze points with eye-tracking devices. In this study, the eye-tracking device was calibrated to keep the spatial errors less than a 1.15° visual angle; however, the manual mapping procedure for fixations could also have been a source of spatial errors of fixation. The estimation of errors during the manual mapping procedure was not available in this study.
In actual neuroendovascular procedures, the four screens of the biplane fluoroscopy are referenced, and operator-assistant pairs work together to distribute their visual attention to the four screens. Simultaneous eye tracking of operator-assistant pairs to investigate intraoperative cooperation in terms of visual attention sharing will be a future issue to be investigated.
This preliminary study demonstrated the feasibility of eye tracking to identify the differences in the gaze behavior depending on the experience of endovascular procedures and may guide future eye-tracking studies in neuroendovascular therapy.
Institutional Review Board (IRB) permission obtained for the study.
This research was financially supported by AMED [grant Number JP17he1802004]; ImPACT Program of Council for Science, Technology and Innovation (Cabinet Office, Government of Japan) [Bionic Humanoids Propelling New Industrial Revolution].
There are no conflicts of interest.
1. Brams S, Ziv G, Levin O, Spitz J, Wagemans J, Williams AM. The relationship between gaze behavior, expertise, and performance: A systematic review. Psychol Bull. 2019. 145: 980-1027
2. Eivazi S, Hafez A, Fuhl W, Afkari H, Kasneci E, Lehecka M. Optimal eye movement strategies: A comparison of neurosurgeons gaze patterns when using a surgical microscope. Acta Neurochir (Wien). 2017. 159: 959-66
3. Holmqvist K, Nyström N, Andersson R, Dewhurst R, Jarodzka H, van de Weijer J.editors. Eye Tracking: A Comprehensive Guide to Methods and Measures. Oxford, UK: Oxford University Press; 2011. p.
4. Specialist Quantification System-JSNET. Available from: http://www.jsnet.website/documents.php?id=647 [Last accessed on 2020 Mar 27].
5. Law B, Atkins MS, Kirkpatrick AE, Lomax AJ, Mackenzie CL.editors. Eye Gaze Patterns Differentiate Novice and Experts in a Virtual Laparoscopic Surgery Training Environment, in ETRA ’04: Proceedings of the 2004 Symposium on Eye Tracking Research and Applications. New York, USA: Association for Computing Machinery; 2004. p. 41-8
6. Lévêque L, Bosmans H, Cockmartin L, Liu H. State of the art: Eye-tracking studies in medical imaging. IEEE Access. 2018. 6: 37023-34
7. Olsen A. The Tobii I-VT Fixation Filter Algorithm Description. Available from: https://www.tobiipro.com/siteassets/tobii-pro/learn-and-support/analyze/how-do-we-classify-eyemovements/tobii-pro-i-vt-fixation-filter.pdf/?v=2012 [Last accessed on 2020 Jan 18].
8. Sander CE, Alejandro AR, Ricardo CJ, Colin PD, Jacques D, Randall TH. Guidelines for the management of aneurysmal subarachnoid hemorrhage: A guideline for healthcare professionals from the American heart association/ American stroke association. Stroke. 2012. 43: 1711-37
9. Vine SJ, Moore LJ, Wilson MR. Quiet eye training: The acquisition, refinement and resilient performance of targeting skills. Eur J Sport Sci. 2014. 14: S235-42
10. William JP, Alejandro AR, Teri A, Opeolu MA, Nicholas CB, Kyra B. Guidelines for the early management of patients with acute ischemic stroke: 2019 update to the 2018 guidelines for the early management of acute ischemic stroke: A guideline for healthcare professionals from the American heart association/ American stroke association. Stroke. 2019. 50: e344-418
11. Wilson MR, Vine SJ, Bright E, Masters RS, Defriend D, McGrath JS. Gaze training enhances laparoscopic technical skill acquisition and multi-tasking performance: A randomized, controlled study. Surg Endosc. 2011. 25: 3731-9