Abstract

Background:The complexity of neurosurgical interventions demands innovative training solutions and standardized evaluation methods that in recent times have been the object of increased research interest. The objective is to establish an education curriculum on a phantom-based training system incorporating theoretical and practical components for important aspects of brain tumor surgery.

Methods:Training covers surgical planning of the optimal access path based on real patient data, setup of the navigation system including phantom registration and navigated craniotomy with real instruments. Nine residents from different education levels carried out three simulations on different data sets with varying tumor locations. Trainings were evaluated by a specialist using a uniform score system assessing tumor identification, registration accuracy, injured structures, planning and execution accuracy, tumor accessibility and required time.

Results:Average scores improved from 16.9 to 20.4 between first and third training. Average time to craniotomy improved from 28.97 to 21.07 min, average time to suture improved from 37.83 to 27.47 min. Significant correlations were found between time to craniotomy and number of training (P P P

Conclusion:The training system is evaluated to be a suitable training tool for residents to become familiar with the complex procedures of autonomous neurosurgical planning and conducting of craniotomies in tumor surgeries. Becoming more confident is supposed to result in less error-prone and faster operation procedures and thus is a benefit for both physicians and patients.

Keywords: Neurosurgical education, phantom, training system, tumor resection

INTRODUCTION

During brain tumor surgeries, the interactions between the surgeon and numerous technical components are complex and should ideally be learned during a standardized training that can be verified objectively.

Traditional concepts in neurosurgical training are live surgeries[ 23 ] or training on animal cadavers.[ 7 ] Teaching during surgery results in longer operating times and may increase the overall risk to the patient.[ 2 ] In contrast, surgical simulation and skill training offer an opportunity to teach and practice in a nonrisk environment where surgeons can develop and refine skills through harmless repetition.[ 16 ] Surgical organizations are calling for methods to ensure the maintenance of skills, advance surgical training, and credential surgeons as technically competent.

The state of the art simulation systems in neurosurgery are virtual reality-based systems,[ 3 10 22 27 ] which use force feedback[ 14 15 24 25 ] partly in combination with augmented reality.[ 1 ] Some models assist in procedure planning, augment the visual-spatial learning of complex surgical approaches and simulate technical components of neurosurgical procedures.[ 12 16 19 ] Other simulation environments combine a graphic interface with a graphic display and the corresponding software, such as Dextroscope,[ 13 ] Cranial Base Surgery Simulators,[ 5 26 ] ROBO-SIM,[ 20 21 ] and ImmersiveTouch.[ 15 ] Unfortunately virtual reality-based systems are currently limited by the computational complexity, the arduous process of manually segmenting volume-rendered models, the great expense of sophisticated haptic interfaces[ 16 ] and the restricted hand–eye coordination.

Harrop et al.[ 11 ] reported that the apprenticeship model of neurosurgical training and education created lengthy work days, altered sleep patterns, and potentially a limited educational environment, which may have an adverse effect on surgical proficiency. The resulting work hour restriction for residents by the Accreditation Council for Graduate Medical Education (ACGME) reduced the available time for teaching and education.[ 9 ] These increasing challenges led to a higher demand for the incorporation of simulation into the educational curriculum.[ 2 4 6 8 13 16 18 ] Phantom-based training systems already exist, for example, for spinal simulation as recently described in.[ 11 ] In contrast, at the onset of this project no established training system for navigated craniotomies was available. The convincing results of a first qualitative evaluation in the framework of a feasibility study were published in 2014.[ 17 ] That qualitative survey aimed to identify weaknesses and strengths by means of a questionnaire with questions to all used materials, to the handling and ergonomic of the system. This new manuscript describes a more quantitative evaluation approach of the same system. The focus of this study was more on the achievable learning effect of repetitive exercises on different datasets by same participant.

MATERIALS AND METHODS

Due to the lack of phantom-based simulation devices for cranial surgeries, a training system was developed in an iterative process in close collaboration with the specialist company PHACON GmbH (Leipzig, Germany),[ 17 ] which also developed the recently introduced cervical spine simulator.[ 11 ] The system comprises a tactile head phantom with a changeable frontotemporal module, a two-camera tracking system and a laptop with the corresponding navigation software [ Figure 1 ]. To satisfy the requirement to have a visualization of a patient's dataset that is suitable to the phantoms hardware, a method was developed for integrating real patient datasets into the predefined structures of the Montreal neurological institute (MNI) template.[ 17 ] An important question to ask is whether human performance can be improved through the use of a neurosurgical training environment and whether that improvement can be measured.[ 2 ] To quantitatively examine the practicability of the training system, a group of nine residents were asked to perform three simulations on different patient data sets on varying tumor locations in the frontotemporal region [ Figure 2 ]. For each tumor location, a standard was defined concerning the minimal distance to surrounding risk structures and landmarks, the length of the skin incision and the size of the craniotomy in relation to the tumor size and the acceptable number of drilling holes. Depending on the tumor location and the most likely access path, a minimal distance to surrounding risk structures was defined for each dataset, for example, distance to the ear, to the marked hairline, or to the sinus sagittalis. To estimate the standard for the length of the skin incision length and the size of the craniotomy, the maximum expansion of the tumor was measured in the corresponding magnetic resonance imaging (MRI) dataset on two perpendicular directions in the axial slices. The range for a valid length of the skin incision was defined to be between two and four times of the average of those two maximum values. Likewise the range for a valid craniotomy size was defined to be between that square average value and the square of the average value plus 2 cm. The number of acceptable drilling holes was defined to be between one and two for all data sets.

Figure 1

Phantom-based training system with head phantom fixed with a ball joint in a plastic tray, changeable module, two-camera tracking system and laptop with navigation software, original instruments from Aesculap (Microspeed uni, Aesculap AG, Tuttlingen, Germany) for drilling and milling

Figure 2

Three different tumor locations; upper row: metastases temporal, 24 × 22 × 16 mm; middle row: Two metastases precentral, 33 × 19 × 27 mm; bottom row: Three metastase (frontal, precentral, pons) frontal metastases 20 × 23 × 22 mm. The datasets were arranged with an increasing level of difficulty

Every simulation was supervised and evaluated by the same specialist and assisted by the same research associate to avoid a possible imbalance in favor of any participant.

Before starting the first simulation, a short introduction to the training system was given to every participant, comprising the handling of the ball joint for head positioning, the handling of the camera device for proper adjustment as well as the operation of the navigation software. The same specialist always gave the explanation exactly in the same way. During the simulation, all participants were left on their own and questions were answered only afterwards to guarantee equality of opportunities.

Training started with a didactic component with questions on tumor identification and surrounding risk structures in the patient's dataset. The subsequent simulation was carried out with real instruments and covered surgical planning of the optimal access path, the setup of the navigation system including a marker-based registration of the head phantom and the navigated craniotomy with preparation of simulated skin and possibly also muscle structure as well as drilling and milling of the simulated bone structure. The craniotomy was performed with a trepanning tool and a milling device from Aesculap (Microspeed uni, Aesculap AG, Tuttlingen, Germany), both accompanied by water flushing [ Figure 3 ]. Simulation finishes with the removal of the bony lid.

The specialist awarded the points during the simulation using a standardized protocol with reference values for every dataset. Criteria to be met were the correct identification of the tumor(s) and surrounding risk structures, phantom positioning, setup of the navigation system, registration accuracy, planning of the optimal access path, positioning and length of the skin incision, distance to risk structures, muscle and dura preparation, positioning and size of the craniotomy and finally for skin suture and tumor accessibility [ Table 1 ]. In total, a score of 23 could be obtained for each simulation. Scores were graded mainly as either correct, receiving a point, or not correct, receiving a zero. Exceptions were made with a graded score of more than one point for registration accuracy or accessibility of the tumor. A registration accuracy of less than 1 mm in the first trial was rated with two points and in the second try with one point. Tumor accessibility was rated with three points if the tumor was within the perpendicular projection from the margin of the craniotomy in anterior–posterior expansion, in latero-lateral expansion as well as inferior–superior expansion. If it failed to be within one/two of the three expansion directions, two one point(s) were given. Scores were awarded without interrupting the simulation. Afterwards, participants were informed of the obtained score and the reasons for not achieved points. Recommendations were made by the specialist with specific focus on the access path to the trained tumor location, camera positioning, use of the navigation support and potential complications resulting of the way of skin incision, placing of drilling holes and craniotomy.

To get a further measurement for the hypothesized learning curves, achievable by repetitive exercises on different datasets with increasing difficulty factor, time to craniotomy and time to suture were recorded for each simulation. Time to craniotomy includes all steps from planning until the removal of the bone lid, time to suture includes additionally the time needed to finish the suture. Both times were recorded in order to exclude the impact of different skin incision lengths on the outcome. Time distances between the single trainings were tried to arrange nearly comparable within all participants. Unfortunately the variance between the single trainings ranged between 5 days and 8 weeks at the end due to lacks of availability of the residents.

Table 1

Score system for a uniform evaluation of the simulations. The maximum of achievable points was 23. One specialist awarded the points in all trainings

Figure 3

Milling device in action (Microspeed uni, Aesculap AG, Tuttlingen, Germany), accompanied by water flushing

RESULTS

Nine neurosurgery residents completed three simulations on different data sets with varying tumor locations, all in the frontotemporal region. There was a preponderance of male residents (7 males to 2 females). The individual's level of training varied with: 3 post graduate year (PGY)-2, 2 PGY-3, 1 PGY-5, 1 PGY-6, 1 PGY-7, and 1 PGY-8.

Between the first and the third training, an average improvement of 3.6 score points was achieved and the average score increased from training to training [ Table 2 and Figure 4 ]. Time to craniotomy decreased continuously during the trainings, between first and third training in average 7.9 min [ Table 2 and Figure 5 ]. Time to suture decreased as well, in average 9.1 min [ Table 2 and Figure 5 ]. Injuries of the muscle during the skin incision decreased from 67% (6/9) in the first simulation to 33% (3/9) in the second simulation and finally 0% in the third simulation, while injuries of the dura barely decreased from 56% (5/9) in the first simulation to 44% (4/9) in the second and third simulation.

Table 2

Average results of score and time to craniotomy/time to suture for three trainings and 9 participants

Figure 4

Boxplot: Improvement of score during three trainings. The significant correlation between number of training and score is also reflected in the Boxplot

Figure 5

Boxplot: Improvement of time to craniotomy/time to suture in min during three trainings. The significant correlation between number of training and time to craniotomy/time to suture is also reflected in the Boxplot

For the adjustment and usage of the navigation support during the simulation, including the registration process, a maximum of four points was awarded. In the first simulation, an average score of 58% (2.3/4) was achieved that improved to 75% (3/4) in the second simulation and finally to 97% (3.9/4) in the third simulation. The Pearson's coefficient shows significant correlations between number of training and time to craniotomy (r = 0.46, P = 0.01), between number of training and time to suture (r = 0.45, P = 0.01) as well as between number of training and score (r = 0.62, P < 0.001).

The Pearson's coefficient shows well significant correlations between level of training (PGY) and time to craniotomy (r = 0.61, P < 0.001) as well as between level of training (PGY) and time to suture (r = 0.61, P < 0.001). No significant correlation could be found between level of training (PGY) and score (r = 0.07, P = 0.69).

A two-factor analysis of variance (ANOVA) for repeated measurements was performed to quantify the impacts of repeated simulation and the level of training (PGY) on the score and on the time to craniotomy/suture. The Mauchly's test of sphericity did not show a violation of the criteria of homogeneity of variance between the factor levels in any case. However, a Greenhouse–Geisser correction was calculated for each ANOVA since the sample size was quite small. The results confirm significant impacts of repeated simulation on time to craniotomy (P = 0.01, Greenhouse–Geisser correction P = 0.03), on time to suture (P = 0.03, Greenhouse–Geisser correction P = 0.03) as well as on score (P = 0.003, Greenhouse-Geisser correction P = 0.02). The interaction between the factors repeated simulation and level of training (PGY) did not show a significant impact neither on time to craniotomy (P = 0.57) and time to suture (P = 0.48), nor on score (P = 0.12).

DISCUSSION

The significant correlations between number of training and time to craniotomy/time to suture/score indicate learning curves that can be achieved by repeated trainings on different datasets with varying tumor locations in the frontotemporal area. The improvement is probably due in part to the familiarization with the training system. But since all participants also stated subjectively that training increases the confidence level in dealings with the interactions between navigation support, instrument use, and surgeon,[ 17 ] an actual learning curve is also hypothesized. This assumption is also reflected by the constant improvement of score values corresponding to the navigation system. The difference of the improvement related to detected injuries of muscle and dura may be explained by material characteristics and the fact that the Dura obtained the poorest rating in the survey.[ 17 ] The problem during the construction process was the low material thickness in combination with the challenge to attach it to the bony structure. All tested adhesives did not give a satisfactory result concerning the adhesive residues. That is why the decision was made in favor of a direct casting onto the bony structure for the modules used in the second study. The result is that the silicone mix needs to be adapted in a way that makes the material more elastic before it tears.

The correlation coefficients between level of training (PGY) and time to craniotomy/time to suture show significant results, that do not coincide with the results of the Anova. Six different levels of training (PGY) within the small sample size of nine residents could give a coherent explanation. Anyway, the significant correlation coefficients between PGY and time to craniotomy/suture confirm the assumption that residents with higher education level are more experienced and thus faster in conducting the simulations.

Since different tumor locations are associated with different access paths, the requirements for tissue preparation vary from dataset to dataset. Thus, the comparability of the simulations is affected. To exclude a positive impact on the results, datasets were arranged such that the complexity increased from case to case. For example, the tumor in the third dataset required the greatest effort in muscle preparation due to its location in the frontal lobe [ Figure 2 , lower row].

The evaluation study shows a learning effect that is expressed in score improving and time decreasing, which is probably due in part to the familiarization with the system. As well it is a legitimate question to ask, if the training on the simulator improves surgical skills during real life procedures. Of course the haptic feedback is different on a phantom and as well the psychological situation is not comparable with the scenario in the operating room while treating a patient. It is hardly possible to find a suitable instrument to measure the transferability of the acquired competences. However, the study showed that the training environment offers certain advantages in comparison to the operating room. During the initial phase of resident training, the main part consists of assisting surgeries and thus does not offer much learning potential in autonomous decision making or draw conclusions from mistakes, such as, for example, of a incorrect planning of the access path. The concepts for correct head positioning depending on the tumor location can be assumed to be transferable to the operating room and offer the opportunity of autonomous positioning by the resident during real life procedures. The handling of the navigation device, including camera positioning at the beginning, using the navigation during the planning and as well the navigated control before placing the drilling holes, are as well quite similar to the procedures during real surgeries and may lead to more confidence and understanding.

Despite the usage of a standardized score system, an automatic software-based evaluation would be more objective and independent of the assessment of a specialist. Another limitation of the study is the small size of participants. An extended study with more participants from different neurosurgical clinics would provide a more reliable base for the results. Current development is focusing on implementing a software-based evaluation based on tracked surgical instruments and deposited master access paths for every dataset.

After the implementation of this automatic evaluation method, a further study with more participants is planned, that will try to show that the learning effect is not only due to getting familiar with the system by comparing the results of the training of a resident group with a specialist group. As well a further study will try to show the transferability of the learning effect by developing an evaluation method for measuring the surgical skills directly in the operating room to get the chance to compare them before and after repetitive training on the phantom. A didactic posttest with the same questions than a didactic pretest could be an efficient tool for measuring the improvement concerning the theoretical concepts, as recently proposed by Harrop et al.[ 11 ]

In general, the further development of phantom-based training systems may have the potential to improve surgical education in order to address risk management concerns, patient safety, and operating room management by more effective training methods. The direct recognition of the consequences of autonomous decision making is supposed to result in less error-prone and faster operation procedures.

We have artists with no scientific knowledge and scientists with no artistic knowledge, and both with no spiritual sense of gravity at all, and the result is not just bad, it's ghastly.[ 4 ]

Every beginning is difficult; man is born with little knowledge and skills and develops these through lifetime training. Furthermore, it is evident that we cannot teach all trainees using the standard apprenticeship, combined with our reduced working hours, in the future.[ 7 ]

Simple statements such as “We have the ambition to give the participants, trainees or specialists an update on the practical as well as the theoretical aspects of microvascular anastomosis techniques, with special relevance to neurosurgery” found in the Scandinavian Manual for Microsurgery are insufficient for a modern training system. Skill learning or dexterity such as constructing a microvascular anastomosis is to develop an increase in spatial and temporal accuracy of hand/finger movements with practice and involves the acquisition of new skilled movements. It refers to the ability of an individual trainee to acquire the temporal and spatial characteristics of movement patterns, so that pre-programmed processes will increasingly characterize their execution. Abilities are the prerequisites for the training and performance of motor skills.

Surgical education is based on a combination of developing both skills and clinical patient handling. With skills we must understand the correct use of the hands, the effect of eventual tremor, the use of appropriate instruments, spatial awareness, organization, and control of environment. These skills are programmed into the basal ganglia in order to become automatic.[ 5 ] So, tested in a simple way, manual dexterity among physicians and surgeons showed no significant differences between medical and surgical residents and psycho-motor skill was not the major factor in distinguishing the proficient surgical performance from the mediocre one.[ 6 ] Developing dexterity, we start moving our fingers and instruments using our primary cortex, but it changes slowly to automatism. Swedenborg stated that all processes moving toward greater perfection move from general things to particulars (details). This process is according to “divine order” and holds true in playing the piano. Notes and keys have to be learned first, but their knowledge is not enough. When we first attempt to play the piano, we only have control over general groups of muscles in the fingers. Through years of practice, these general groups of muscles offer less and less resistance until we gain greater control of the particular muscle fibers within those groups. This gives our fingers both greater dexterity and greater responsiveness (pliability) to the knowledge and will of the brain. Finally, success leads the piano practitioner to a state of joy and happiness that simulates reaching a kind of “heaven.” This is the reason why a concert pianist trains for hours daily to perfect his piano playing.[ 2 ] Therefore, measuring speed, precision, knot-binding, etc., cannot be used as the sole monitoring of a neurosurgeon's standard. It seems obligatory that one needs to specify the specific trainee's personal abilities in order to combine these abilities in the best possible way within a training program. On the contrary, we must also agree that a neurosurgeon without abilities to use instruments in the surgical field can never become a proficient surgeon.

Environment is important. We know that brain surgery involves the risk of losing patient's lives. Therefore, we must be able to handle stress during surgery. A surgeon who thinks he can carry out a microsurgical procedure just by grabbing the microscope (that was just presented at a scientific meeting) and the entire toolkit of microinstruments has not understood this paradigm and surgical disasters are bound to follow. The complex neurosurgical brain operations, therefore, make it mandatory to create an innovative training set-up and a kind of evaluation system that can provide the teacher and the trainee with progress. The use of Virtual Reality (VR) systems has been suggested for many years, but is still not a standard part of a neurosurgical curriculum.[ 2 ] It has been found that surgeon's belief in them increases as they learn and perfect their job over a long period. This learning process could probably become more efficient and quicker by effective use of role models, imagery training, and teaching mental skills. By deliberately making “mistakes,” experiences grow and perfection is developed.[ 3 ] Ethically speaking, this is only allowed in a VR scenario. In today's life, a young person enjoys game playing and spends daily hours playing games on their iPad or mobile, hours that could be used better. By introducing VR training–as demonstrated in this paper–the authors implement basic surgical techniques in a fun way including possibilities for competition among trainees. Both they and their teachers can thus set individual goals and the trainee can continuously receive feedback of his/her development. This is much better than examinations.[ 1 ] I, therefore, wholeheartedly agree with the conclusion that VR training, as demonstrated here, will be a tool to develop better neurosurgeons in the future.

ACKNOWLEDGMENT

The described project was co-financed by the European Union under the European Regional Development Fund (EFRE, project number 14220/2466), while the PHACON GmbH (Leipzig, Germany) was involved as project partner. All authors state that there is no financial or personal conflict of interest regarding the material discussed in the article.

References

1. Alaraj A, Charbel FT, Birk D, Tobin M, Luciano C, Banerjee PP. Role of cranial and spinal virtual and augmented reality simulation using immersive touch modules in neurosurgical training. Neurosurgery. 2013. 72: 115-23

2. Alaraj A, Lemole MG, Finkle JH, Yudkowsky R, Wallace A, Luciano C. Virtual reality training in neurosurgery: Review of current status and future applications. Surg Neurol Int. 2011. 2: 52-

3. Arnold P, Farrell MJ. Can virtual reality be used to measure and train surgical skills?. Ergonomics. 2002. 45: 362-79

4. Auer LM, Auer DP. Virtual endoscopy for planning and simulation of minimally invasive neurosurgery. Neurosurgery. 1998. 43: 529-37

5. Bernardo A, Preul MC, Zabramski JM, Spetzler RF. A three dimensional interactive virtual dissection model to simulate transpetrous surgical avenues. Neurosurgery. 2003. 52: 499-505

6. Chan S, Conti F, Salisbury K, Blevins NH. Virtual reality simulation in neurosurgery: Technologies and evolution. Neurosurgery. 2013. 72: 154-64

7. Csókay A, Papp A, Imreh D, Czabajszky M, Valálik I, Antalfi B. Modelling pathology from autolog fresh cadaver organs as a novel concept in neurosurgical training. Acta Neurochir (Wien). 2013. 155: 1993-5

8. Cusimano MD. Virtual reality surgery: Neurosurgery and the contemporary landscape a three-dimensional interactive virtual dissection model to simulate transpetrous surgical avenues. Neurosurgery. 2003. 53: 1010-1

9. Durkin ET, McDonald R, Munoz A, Mahvi D. The impact of work hour restrictions on surgical resident education. J Surg Educ. 2008. 65: 54-60

10. Haase J, Musaeus P, Boisen E, Andersen P, Qvortrup L.editors. Virtual reality and habitats for learning microsurgical skills. Virtual Applications. London: Springer; 2004. p. 29-48

11. Harrop J, Rezai AR, Hoh DJ, Ghobrial GM, Sharan A. Neurosurgical training with a novel cervical spine simulator: Posterior foraminotomy and laminectomy. Neurosurgery. 2013. 73: 94-9

12. Kockro RA, Stadie A, Schwandt E, Reisch R, Charalampaki C, Ng I. A collaborative virtual reality environment for neurosurgical planning and training. Neurosurgery. 2007. 61: 379-91

13. Kockro RA, Serra L, Tseng-Tsai Y, Chan C, Yih-Yian S, Gim-Guan C. Planning and simulation of neurosurgery in a virtual reality environment. Neurosurgery. 2000. 46: 118-35

14. Lemole GM, Banerjee PP, Luciano C, Neckrysh S, Charbel FT. Virtual reality in neurosurgical education: Part-task ventriculostomy simulation with dynamic visual and haptic feedback. Neurosurgery. 2007. 61: 142-9

15. Luciano C, Banerjee P, Lemole GM, Charbel F. Second generation haptic ventriculostomy simulator using the immersive touch system. Stud Health Technol Inform. 2006. 119: 343-8

16. Malone HR, Syed ON, Downes MS, D’Ambrosio AL, Quest DO, Kaiser MG. Simulation in neurosurgery: A review of computer-based simulation environments and their surgical applications. Neurosurgery. 2010. 67: 1105-16

17. Müns A, Mühl C, Haase R, Möckel H, Chalopin C, Meixensberger J. A neurosurgical phantom-based training system with ultrasound simulation. Acta Neurochir (Wien). 2014. 155: 1237-43

18. Neubauer A, Wolfsberger S. Virtual endoscopy in neurosurgery: A review. Neurosurgery. 2013. 72: 97-106

19. Radetzky A, Rudolph M. Simulating tumour removal in neurosurgery. Int J Med Inform. 2001. 64: 461-72

20. Radetzky A, Rudolph M, Starkie S, Davies B, Auer LM. Robo-sim: A simulator for minimally invasive neurosurgery using an active manipulator. Stud Health Technol Inform. 2000. 77: 1165-9

21. Radetzky A, Schröcker F, Auer LM. Improvement of surgical simulation using dynamic volume rendering. Stud Health Technol Inform. 2000. 70: 272-8

22. Robison RA, Liu CY, Apuzzo ML. Man, mind, and machine: The past and future of virtual reality simulation in neurologic surgery. World Neurosurg. 2011. 76: 419-30

23. Roser F, Pfister G, Tatagiba M, Ebner FH. Live surgery in neurosurgical training courses: Essential infrastructure and technical set-up. Acta Neurochir (Wien). 2013. 155: 541-5

24. Sato D, Kobayashi R, Kobayashi A, Fujino S, Uchiyama M. Soft tissue pushing operation using a haptic interface for simulation of brain tumor resection. J Robot Mechatron. 2006. 18: 634-42

25. Spicer MA, van Velsen M, Caffrey JP, Apuzzo ML. Virtual reality neurosurgery: A simulator blueprint. Neurosurgery. 2004. 54: 783-98

26. Stredney D, Wiet GJ, Bryan J, Sessanna D, Murakami J, Schmalbrock P. Temporal bone dissection simulation–an update. Stud Health Technol Inform. 2002. 85: 507-13

27. Sure U, Miller D, Bozinov O. Neurosurgical training in Europe, problems and possible solutions. Surg Neurol. 2007. 67: 626-8

28. Haase J. European Association of Neurosurgical Societies. The European examination: Its present status and potential development. Acta Neurochir Suppl. 2004. 90: 107-14

29. Haase J, Lumenta CB, Di Rocco C, Haase J, Mooij JJ.editors. Basic training in technical skills: Introduction to a learning “surgical skills” in a constructive way. (European Manual of Medicine). Neurosurgery. USA: Springer; 2010. p. 17-23

30. McDonald JM.editors. Mental readiness and its links to performance excellence in surgery. Ottawa, Canada: Kinetek; 1993. p.

31. Pirsig RM.editors. Zen and the art of motorcycle maintenance. An inquiry into values. USA: William Morrow and Company; 1974. p.

32. Schiltz C.editors. Human skill learning studied with PET: The effect of training on the brain activity related to simple visuo-motor tasks.(Thesis). Belgium: University Catholique de Louvain; 2000. p.

33. Schueneman AL, Pickleman J, Hesslein R, Freeark RJ. Neuropsychologic predictors of operative skill among general surgery residents. Surgery. 1984. 96: 288-95

34. Sure U, Miller D, Bozinov O. Neurosurgical training in Europe, problems and possible solutions. Surg Neurol. 2007. 67: 626-8