- Department of Neurosurgery, Evidence and Impact, McMaster University, Hamilton, Ontario, Canada.
- Department of Health, Evidence and Impact, McMaster University, Hamilton, Ontario, Canada.
Radwan Ibrahim Takroni, Department of Neurosurgery, McMaster University, Hamilton, Ontario, Canada.
DOI:10.25259/SNI_1032_2021Copyright: © 2022 Surgical Neurology International This is an open-access article distributed under the terms of the Creative Commons Attribution-Non Commercial-Share Alike 4.0 License, which allows others to remix, transform, and build upon the work non-commercially, as long as the author is credited and the new creations are licensed under the identical terms.
How to cite this article: Radwan Takroni1, Sunjay Sharma1, Kesava Reddy1, Nirmeen Zagzoog1, Majid Aljoghaiman1, Mazen Alotaibi1, Forough Farrokhyar2. Randomized controlled trials in neurosurgery. 26-Aug-2022;13:379
How to cite this URL: Radwan Takroni1, Sunjay Sharma1, Kesava Reddy1, Nirmeen Zagzoog1, Majid Aljoghaiman1, Mazen Alotaibi1, Forough Farrokhyar2. Randomized controlled trials in neurosurgery. 26-Aug-2022;13:379. Available from: https://surgicalneurologyint.com/surgicalint-articles/11832/
Randomized controlled trials (RCTs) have become the standard method of evaluating new interventions (whether medical or surgical), and the best evidence used to inform the development of new practice guidelines. When we review the history of medical versus surgical trials, surgical RCTs usually face more challenges and difficulties when conducted. These challenges can be in blinding, recruiting, funding, and even in certain ethical issues. Moreover, to add to the complexity, the field of neurosurgery has its own unique challenges when it comes to conducting an RCT. This paper aims to provide a comprehensive review of the history of neurosurgical RCTs, focusing on some of the most critical challenges and obstacles that face investigators. The main domains this review will address are: (1) Trial design: equipoise, blinding, sham surgery, expertise-based trials, reporting of outcomes, and pilot trials, (2) trial implementation: funding, recruitment, and retention, and (3) trial analysis: intention-to-treat versus as-treated and learning curve effect.
Keywords: Neurosurgery, Randomized controlled trials, Research methodology
The term randomized controlled trial (RCT) refers to a type of study in which people are allocated randomly to receive one or more clinical interventions. One of these interventions is usually the standard of care, also known as the control. The control may be a standard practice, a placebo (e.g., sugar pill), or no intervention at all.[
The history of RCTs in neurosurgery is relatively young compared to other medical and some surgical specialties. Neurosurgery as a specialty did not start until the late 19th century, which could explain why it is one of the specialties that had a late start in exploring new treatments/ interventions through controlled trials.[
To illustrate the poor quality and low quantity of neurosurgical RCTs during that time, Haines reviewed 51 neurosurgical RCTs after searching the literature published between 1945 and 1981.[
Years later, Mansouri et al. reviewed 61 neurosurgical RCTs published between 2000 and 2014.[
Despite advancements over the past 50 years, the quality of designing and reporting of neurosurgical RCTs remains suboptimal. In this section, we outline the important aspects commonly encountered during designing, implementing, and analyzing RCTs in neurosurgery. While focusing on some of the most critical challenges and obstacles that face investigators, we will attempt to provide suggestions for potential solutions.
Equipoise refers to the situation in which there is no clear evidence that one intervention is superior or inferior to another, which constitutes the rationale behind conducting RCTs. This is common when comparing surgical to medical treatment. However, some trialists fail to make the distinction between two related but completely different concepts: Clinical Equipoise and Individual Equipoise. Individual Equipoise exists when the clinician involved in the research study has no preference or is truly uncertain about the overall benefit or harm offered by the treatment.[
Individual Equipoise, however, introduces inherent bias and, as such, may not equate to Clinical Equipoise, which is typically based on the best available literature at that time. However, when Clinical Equipoise is utilized as a base for designing a clinical trial, the participating neurosurgeons will have to set aside their individual biases and agree a priori on the study design and methodology, especially the inclusion and exclusion criteria for a research study. They will also have to be bound by the consensus opinion, which will enable the physician involved in the trial to make decisions efficiently.
Effective employment of the principle of Clinical Equipoise by the clinicians participating in the trials will result in the recruitment of a larger number of patients with more homogenous baseline characteristics. Consequently, this will deliver more clearly interpretable and valid results that will help in standardizing the practice.
In placebo-controlled studies of medical interventions, a double-blinded design is always encouraged. Simply put, this means that neither the subject nor the investigator are aware of which study group a particular subject is placed in. In this case, masking treatment assignment is generally considered ethically acceptable provided that the “shared ignorance” has been made clear in the consent process. However, when comparing two different surgical interventions, blinding physicians and patients are not possible. In sham surgery trials, only the patient is “blinded” to the treatment he/she receives. The clinician, who can distinguish between active and inactive treatment, may be required to engage in active deception.[
A common area in neurosurgery where the concept of sham surgery has been applied is in the management of Parkinson’s disease (PD). Two famous randomized double-blinded controlled trials by Gross et al.[
Since the aim of sham surgery is to ensure that both the investigator and the patient are blinded to the type of intervention, a possible solution is to consider using the prospective, randomized, open-label, and blinded-endpoint (PROBE) study design.[
The term blinding refers mainly to keeping investigators and trial participants unaware of the assigned intervention, in addition to outcomes assessors and/or analysts.[
As mentioned previously, RCTs in neurosurgery have had issues with blinding with regard to quantity and quality of reporting. A systematic review of 82 neurosurgical RCTs by Martin et al. showed that most trials were open label (59.8%), and double-blinded trials were relatively rare (8.5%).[
The definition of expert and expertise can vary between specialties, even within the same surgical specialty. Expertise can be understood as the ability to consistently reproducible good performance involving a given procedure.[
Many neurosurgical procedures are not as stereotypical and as easily classified compared to other surgical specialties, given the significant variability in the surgical approach used to deal with a specific operative target. This lack of surgical intervention standardization among neurosurgeons has led to differences in their preferred approach based on their expertise, which could introduce expertise bias in trials that compare two different surgical approaches/ techniques.[
Another problem with surgeons’ expertise is the generalizability of the results. When a procedure is performed by a highly expert surgeon, the results may differ when comparing them to those of a surgeon with less exposure to a similar procedure (which constitutes the majority of practicing surgeons, especially outside tertiary care centers, where the volume of cases is expected to be less). An example is the Barrow Ruptured Aneurysm Trial which compared the safety and efficacy of microsurgical clipping and endovascular coil embolization for the treatment of acutely ruptured cerebral aneurysms.[
A clear definition of expertise threshold is needed in neurosurgical RCTs. The North American Symptomatic Carotid Endarterectomy Trial (NASCET trial) has set a good example of defining a surgeon’s expertise before participating in a trial.[
Definition and Reporting of Outcomes
Outcomes (also called events or endpoints) are the variables monitored during a study to determine the impact of a given intervention or exposure on the health of a specific population.[
Patient-reported outcomes (PROs) are defined by the U.S. Food and Drug Administration as “a report that comes directly from the patient about the status of a patient’s health condition without amendment or interpretation of the patient’s response by a clinician or anyone else.”[
Another method of reporting outcomes is in the form of a composite. The use of composite outcomes increases the event rate and reduces the sample size but harbors the risk that clinically less relevant but typically more frequent outcomes may drive the trial’s main results or that the individual components move in different directions, thus generating uncertainty.[
Finally, a careful description of the outcome measure, which includes key criteria used to adjudicate it, is a crucial factor to ensure the external validity of a successful RCT. Furthermore, the inclusion of PROs through the use of PROMs with clinical outcomes in research and clinical practice provides a more complete understanding of the impact of an intervention, therapy, and/or service on the patient. We advise following the recommendations of the 2010 CONSORT statement, which points out that all outcome measures, whether primary or secondary, should be identified and completely defined.[
A pilot study is a small-scale, preliminary study that evaluates the feasibility, duration, cost, sampling strategy, and other research techniques before conducting a large, definitive clinical trial.[
Pilot studies are very important to conduct before commencing large-scale surgical trials for many reasons, including testing the feasibility of performing a new procedure or administering an experimental therapy. As mentioned earlier, surgical trials usually face some unique challenges compared to medical trials, making the pilot study an essential stage in any research project to identify potential problems and deficiencies in the research instrument and protocol.
The neurosurgical literature has many examples of pilot studies that tested the feasibility of recruiting patients to conduct larger clinical trials. These examples are distributed among different subspecialties, such as neurotrauma, neurooncology, neurovascular, neuropediatric, and perioperative seizure management.[
In conclusion, despite their limitations, pilot trials remain extremely instructive and helpful, especially when planning a large, multicenter trial in a common neurosurgical problem such as managing traumatic brain injury, subarachnoid hemorrhage, or spinal stenosis.
Funding is one of the key challenges to the success of any RCT. Obtaining adequate funding is more difficult in surgical trials compared to medical trials. A review published by Rangel et al.[
Several types of bias can be associated with industry-funded trials. Radcliff et al.[
Recruitment and trial discontinuation
RCTs require a sufficient number of participants to be adequately powered. This is necessary for the trial to answer the particular research question.[
Intention-to-treat (ITT) versus per-protocol analysis
Randomization allows us to compare the trial arms for all measured and unmeasured characteristics if the sample size of the randomized patients is large enough. It also allows for proper causal inference. Over the course of a trial, many factors, including crossover or withdrawal/loss to follow-up, disrupt the randomization process, ultimately producing groups that are imbalanced compared to the original randomization. This form of noncompliance causes a loss of statistical power.[
Statistical methodology to mitigate the bias introduced by crossover and loss to follow-up includes instrumental variable analysis, imputation techniques that account for potentially differential drop-out across study arms, marginal structural models, and other models to account for informative censoring. Applying the ITT principles yield an unbiased estimate of the efficacy of the intervention on the primary study outcome at the level of adherence observed in the trial. For instance, when the treatment under study is effective, but there is substantial nonadherence, the ITT analysis will underestimate the magnitude of the treatment effect that will occur in adherent patients. Although an underestimate of effective therapy, it will be unbiased.[
Learning curve effect
A learning curve (or experience curve) is a graphical representation that provides a visual assessment of the surgeon’s skills with time and experience.[
Neurosurgical interventions are complex, which complicates their rigorous assessment through randomized clinical trials. Hierarchical models, time-dependent covariates, splines, and subgroup analyses by cumulative surgeon volume or period of recruitment can be applied to account for the learning curve. The use of concurrent controls is vital to account for the time effect.
Evidence-based medicine is informed by hierarchical evidence, and this hierarchy informs clinical decision-making. RCTs form the basis of today’s evidence-based approach to medicine and play an important role in guidelines development, as well as novel drugs and device approval processes. RCTs in neurosurgery have dramatically improved over the past 60 years since the McKissock’s group published their first trial in 1960. However, several issues led to the quality of neurosurgical RCT being suboptimal. Quality of blinding and reporting, bias, funding, recruitment, and learning curve are among the challenges investigators usually face when planning an RCT. In this contribution, we reviewed the history of neurosurgical trials, presented the most critical issues commonly encountered in designing, implementing, and analyzing RCTs in neurosurgery, and provided potential solutions.
Patient’s consent not required as there are no patients in this study.
There are no conflicts of interest.
We would like to thank Mr. Anmar Attar (University of Toronto) for his contribution to the paper.
1. Abraha I, Montedori A. Modified intention to treat reporting in randomised controlled trials: Systematic review. BMJ. 2010. 340: c2697
2. Alderson P. Equipoise as a means of managing uncertainty: Personal, communal and proxy. J Med Ethics. 1996. 22: 135-9
3. Azad TD, Veeravagu A, Mittal V, Esparza R, Johnson E, Ioannidis JP. Neurosurgical randomized controlled trials-distance travelled. Neurosurgery. 2018. 82: 604-12
4. Black N. Patient reported outcome measures could help transform healthcare. BMJ. 2013. 346: f167
5. Chalmers I, Glasziou P, Godlee F. All trials must be registered and the results published. BMJ. 2013. 346: f105
6. Cooper DJ, Rosenfeld JV, Murray L, Wolfe R, Ponsford J, Davies A. Early decompressive craniectomy for patients with severe traumatic brain injury and refractory intracranial hypertension a pilot randomized trial. J Crit Care. 2008. 23: 387-93
7. Daltroy LH, Cats-Baril WL, Katz JN, Fossel AH, Liang MH. The North American spine society lumbar spine outcome assessment instrument: Reliability and validity tests. Spine (Phila Pa 1976). 1996. 21: 741-9
8. Dayyani M, Mohammadi EM, Ashoorion V, Sadeghirad B, Yekta MJ, Grotta JC. Aneurysmal subarachnoid haemorrhage-cerebral vasospasm and prophylactic ibuprofen: A randomised controlled pilot trial protocol. BMJ Open. 2022. 12: e058895
9. Delgado AF, Delgado AF. The association of funding source on effect size in randomized controlled trials: 2013-2015 a cross-sectional survey and meta-analysis. Trials. 2017. 18: 125
10. Desai B, Desai V, Shah S, Srinath A, Saleh A, Simunovic N. Pilot randomized controlled trials in the orthopaedic surgery literature: A systematic review. BMC Musculoskelet Disord. 2018. 19: 412
11. Devereaux PJ, Bhandari M, Clarke M, Montori VM, Cook DJ, Yusuf S. Need for expertise based randomised controlled trials. BMJ. 2005. 330: 88
12. Donovan EK, Greenspoon J, Schnarr KL, Whelan TJ, Wright JR, Hann C. A pilot study of stereotactic boost for malignant epidural spinal cord compression: Clinical significance and initial dosimetric evaluation. Radiat Oncol. 2020. 15: 267
13. English SW, Fergusson D, Chassé M, Turgeon AF, Lauzier F, Griesdale D. Aneurysmal subarachnoid hemorrhagered blood cell transfusion and outcome (SAHaRA): A pilot randomised controlled trial protocol. BMJ Open. 2016. 6: e012623
14. Ericsson KA. Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. Acad Med. 2004. 79: S70-81
15. Esene IN, Baeesa SS, Ammar A. Evidence-based neurosurgery. Basic concepts for the appraisal and application of scientific information to patient care (Part II). Neurosciences (Riyadh). 2016. 21: 197-206
16. Ferguson GG, Eliasziw M, Barr HW, Clagett GP, Barnes RW, Wallace MC. The North American symptomatic carotid endarterectomy trial: Surgical results in 1415 patients. Stroke. 1999. 30: 1751-8
17. Ferreira JC, Patino CM. Types of outcomes in clinical research. J Bras Pneumol. 2017. 43: 5-5
18. Frank JI, Schumm LP, Wroblewski K, Chyatte D, Rosengart AJ, Kordeck C. Hemicraniectomy and durotomy upon deterioration from infarction-related swelling trial: Randomized pilot clinical trial. Stroke. 2014. 45: 781-7
19. Freed CR, Greene PE, Breeze RE, Tsai WY, DuMouchel W, Kao R. Transplantation of embryonic dopamine neurons for severe Parkinson’s disease. N Engl J Med. 2001. 344: 710-9
20. Freemantle N, Calvert M, Wood J, Eastaugh J, Griffin C. Composite outcomes in randomized trials: Greater precision but with greater uncertainty?. JAMA. 2003. 289: 2554-9
21. Gallagher MJ, Durnford AJ, Wahab SS, Nair S, Rokade A, Mathad N. Patient-reported nasal morbidity following endoscopic endonasal skull base surgery. Br J Neurosurg. 2014. 28: 622-5
22. Gawlitza M, Soize S, Barbe C, le Clainche A, White P, Spelle L. Aneurysm characteristics, study population, and endovascular techniques for the treatment of intracranial aneurysms in a large, prospective, multicenter cohort: Results of the analysis of recanalization after endovascular treatment of intracranial aneurysm Study. AJNR Am J Neuroradiol. 2019. 40: 517-23
23. Gross RE, Watts RL, Hauser RA, Bakay RA, Reichmann H, von Kummer R. Intrastriatal transplantation of microcarrier-bound human retinal pigment epithelial cells versus sham surgery in patients with advanced Parkinson’s disease: A double-blind, randomised, controlled trial. Lancet Neurol. 2011. 10: 509-19
24. Haines SJ. Randomied clinical trials in neurosurgery. Neurosurgery. 1983. 12: 259-64
25. Hansson L, Hedner T, Dahlöf B.editors. Prospective randomized open blinded end-point (PROBE) study. A novel design for intervention trials. Prospective randomized open blinded endpoint. Blood Press. 1992. 1: 113-9
26. Hassan ZA, Schattner P, Mazza D. Doing a pilot study: Why is it essential?. Malays Fam Physician. 2006. 1: 70-3
27. Jamjoom AA, Gane AB, Demetriades AK. Randomized controlled trials in neurosurgery: An observational analysis of trial discontinuation and publication outcome. J Neurosurg. 2017. 127: 857-66
28. Khan NR, Saad H, Oravec CS, Rossi N, Nguyen V, Venable GT. A review of industry funding in randomized controlled trials published in the neurosurgical literature-the elephant in the room. Neurosurgery. 2018. 83: 890-7
29. Kiehna EN, Starke RM, Pouratian N, Dumont AS. Standards for reporting randomized controlled trials in neurosurgery. J Neurosurg. 2011. 114: 280-5
30. Lancaster GA, Dodd S, Williamson PR. Design and analysis of pilot studies: Recommendations for good practice. J Eval Clin Pract. 2004. 10: 307-12
31. Leon AC, Davis LL, Kraemer HC. The role and interpretation of pilot studies in clinical research. J Psychiatr Res. 2011. 45: 626-9
32. Lim DA, Tarapore P, Chang E, Burt M, Chakalian L, Barbaro N. Safety and feasibility of switching from phenytoin to levetiracetam monotherapy for glioma-related seizure control following craniotomy: A randomized phase II pilot study. J Neurooncol. 2009. 93: 349-54
33. Mansouri A, Cooper B, Shin SM, Kondziolka D. Randomized controlled trials and neurosurgery: The ideal fit or should alternative methodologies be considered?. J Neurosurg. 2016. 124: 558-68
34. Martin E, Muskens IS, Senders JT, DiRisio AC, Karhade AV, Zaidi HA. Randomized controlled trials comparing surgery to non-operative management in neurosurgery: A systematic review. Acta Neurochir (Wien). 2019. 161: 627-34
35. McCoy CE. Understanding the intention-to-treat principle in randomized controlled trials. West J Emerg Med. 2017. 18: 1075-8
36. McDougall CG, Spetzler RF, Zabramski JM, Partovi S, Hills NK, Nakaji P. The barrow ruptured aneurysm trial. J Neurosurg. 2012. 116: 135-44
37. Mckissock W, Richardson A, Walsh L. Posterior-communicating aneurysms. A controlled trial of the conservative and surgical treatment of ruptured aneurysms of the internal carotid artery at or near the point of origin of the posterior communicating artery. Lancet. 1960. 275: 1203-6
38. Miller FG, Kaptchuk TJ. Sham procedures and the ethics of clinical trials. J R Soc Med. 2004. 97: 576-8
39. Misra S. Randomized double blind placebo control studies, the “gold standard” in intervention based studies. Indian J Sex Transm Dis AIDS. 2012. 33: 131-4
40. Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux PJ. CONSORT 2010 explanation and elaboration: Updated guidelines for reporting parallel group randomised trials. BMJ. 2010. 340: c869
41. Mohr JP, Parides MK, Stapf C, Moquete E, Moy CS, Overbey JR. Medical management with or without interventional therapy for unruptured brain arteriovenous malformations (ARUBA): A multicentre, non-blinded, randomised trial. Lancet. 2014. 383: 614-21
42. Mouw TJ, Hong SW, Sarwar S, Fondaw AE, Walling AD, Al-Kasspooles M. Discontinuation of surgical versus nonsurgical clinical trials: An analysis of 88,498 trials. J Surg Res. 2018. 227: 151-7
43. Patel SK, Kashyrina O, Duru S, Miyabe M, Lim FY, Peiro JL. Comparison of two-and three-dimensional endoscopic visualization for fetal myelomeningocele repair: A pilot study using a fetoscopic surgical simulator. Childs Nerv Syst. 2021. 37: 1613-21
44. Polgar S, Mohamed S. Evidence-based evaluation of the ethics of sham surgery for Parkinson’s disease. J Parkinsons Dis. 2019. 9: 565-74
45. Probst P, Zaschke S, Heger P, Harnoss JC, Hüttner FJ, Mihaljevic AL. Evidence-based recommendations for blinding in surgical trials. Langenbecks Arch Surg. 2019. 404: 273-84
46. Radcliff K, Siburn S, Murphy H, Woods B, Qureshi S. Bias in cervical total disc replacement trials. Curr Rev Musculoskelet Med. 2017. 10: 170-6
47. Rangel SJ, Efron B, Moss RL. Recent trends in national institutes of health funding of surgical research. Ann Surg. 2002. 236: 277-86
48. Reponen E, Tuominen H, Hernesniemi J, Korja M. Patient-reported outcomes in elective cranial neurosurgery. World Neurosurg. 2015. 84: 1845-51
49. Rosenthal R, Kasenda B, Dell-Kuster S, von Elm E, You J, Blümle A. Completion and publication rates of randomized controlled trials in surgery: An empirical study. Ann Surg. 2015. 262: 68-73
50. Schulz KF, Grimes DA. Blinding in randomised trials: Hiding who got what. Lancet. 2002. 359: 696-700
51. Smith DH, Neutel JM, Lacourcière Y, Kempthorne-Rawson J. Prospective, randomized, open-label, blinded-endpoint (PROBE) designed trials yield the same results as double-blind, placebo-controlled trials with respect to ABPM measurements. J Hypertens. 2003. 21: 1291-8
52. Stolker JM, Spertus JA, Cohen DJ, Jones PG, Jain KK, Bamberger E. Rethinking composite end points in clinical trials: Insights from patients and trialists. Circulation. 2014. 130: 1254-61
53. . U.S. Department of Health and Human Services FDA Center for Drug Evaluation and Research, U.S. Department of Health and Human Services FDA Center for Biologics Evaluation and Research, U.S. Department of Health and Human Services FDA Center for Devices and Radiological Health. Guidance for industry: Patient-reported outcome measures: Use in medical product development to support labeling claims: Draft guidance. Health Qual Life Outcomes. 2006. 4: 79
54. Van Gompel JJ, Marsh WR, Meyer FB, Worrell GA. Patient-assessed satisfaction and outcome after microsurgical resection of cavernomas causing epilepsy. Neurosurg Focus. 2010. 29: E16
55. Vranos G, Tatsioni A, Polyzoidis K, Ioannidis JP. Randomized trials of neurosurgical interventions: A systematic appraisal. Neurosurgery. 2004. 55: 18-25
56. Ware JE, Sherbourne CD. The MOS 36-item short-form health survey (SF-36). I Conceptual framework and item selection. Med Care. 1992. 30: 473-83
57. Weijer C, Shapiro SH, Glass KC, Enkin MW. For and against: Clinical equipoise and not the uncertainty principle is the moral underpinning of the randomised controlled trial. Br Med J. 2000. 321: 756-8
58. Weinstein GS, Levin B. Effect of crossover on the statistical power of randomized studies. Ann Thorac Surg. 1989. 48: 490-5
59. Weinstein JN, Lurie JD, Tosteson TD, Skinner JS, Hanscom B, Tosteson AN. Surgical vs nonoperative treatment for lumbar disk herniation. JAMA. 2006. 296: 2451
60. Wickens AP. A History of the Brain: From Stone Age Surgery to Modern Neuroscience. Available from: https://books.google.ca/books?id=piKcBQAAQBAJ&pg=PT59&lpg=PT59&dq=history+of+neurosurgery+incas&redir_esc=y&hl=en#v=onepage&q&f=false [Last accessed on 2020 May 08].
61. Yelle LE. The learning curve: Historical review and comprehensive survey. Decis Sci. 1979. 10: 302-28