Investigating Factors Predictive of High Performance in a Multidisciplinary Pain Medicine Team

An expert interview with Robert Bolash, Departments of Pain Management and Evidence Based Pain Research, Cleveland Clinic Lerner College of Medicine, Cleveland, OH, USA

In this expert interview, Robert Bolash, an Assistant Professor of Anesthesiology in the Departments of Pain Management and Evidence Based Pain Research at the Cleveland Clinic Lerner College of Medicine, OH, USA, discusses the topic of his award-winning research poster entitled Analysis of the Validity of Factors Predicting Performance in a Multidisciplinary Pain Medicine Fellowship Program, as presented at the American Academy of Pain Medicine (AAPM) 36th Annual Meeting, National Harbor, MD, USA on February 28, 2020.1 

Increasing both the quality and quantity of scientific pain research is a primary goal of the AAPM Annual Meeting. Dr Bolash’s poster submission was considered one of the six highest ranking and most novel. Here, Dr Bolash highlights the basis for the research project, design and findings of the study and ultimately how it could impact selection of multidisciplinary pain fellowship candidates in the future.

Q. What are the limitations of current selection methods for multidisciplinary pain fellowship candidates?

At Cleveland Clinic, we believe in continuous improvement not only in patient care, but also in education. We therefore recognize that there are opportunities to improve our selection of pain fellowship applicants. This is exemplified by our data which shows correlation between particular factors, though not clear universal predictors. If there is a formula for the perfect fit for a program, we could probably eliminate interviews and meeting the candidates altogether. We’re far away from that today and will likely continue to rely on meeting the human behind the application for the foreseeable future. 

As a top fellowship program for pain management, we are fortunate to have only “champagne problems” with higher numbers of excellent candidates then we can accommodate. Currently, we consider academic achievements, interpersonal skills, research accomplishments, interview presentation, personal statement and letters of reference. We also consider future potential for leadership in our field, and a more subjective category of how well the candidate will fit with our program. We occasionally have candidates who come in for an interview who are exceptionally qualified, but don’t embody the culture of Cleveland Clinic, and frankly will excel further at another institution. We are honest with them and suggest an alternative program where we think they will excel.

Q. Could you tell us a little about your selection methodology at the Cleveland Clinic?

We consider professional credentials, academic achievements, research involvement and general presentation during the interview. Quite simply, is this someone who we want to work with for a year, is likely to develop into an excellent pain physician, and do they exemplify the values needed to care for patients with chronic pain diagnoses now and in the future? 

Interviews are conducted in-person with faculty members, nurses, current house officers and administrative leadership. All of these data points are considered together to create a “rank list” where the most desirable candidate is positioned first, and the least desirable positioned at the bottom. Then, a computerized system takes our rank list and compares it to a similarly created rank list of each applicant. Ultimately, the applicant has the greatest control in selection, as the list is considered by the computer algorithm before ours. 

Q. Could you tell us about the design and findings of your study evaluating these assessments?

We are always looking to improve our fellow selection process. We receive more than 300 applications per year, so need to whittle down that list to a few dozen who we invite for interview, and ultimately settle on 10 individuals to spend the year with us. The application is multifaceted, and considers multiple domains of clinical patient care, leadership, research accomplishments, and self-expression in the form of a personal statement. We don’t know if each of these are equally important, or if we should be placing more significant weight on some, and discounting others.  

Our program director, and senior author on this research, Jianguo Cheng, MD PhD, thought to critically assess these factors, and embarked on a statistical evaluation of correlations between various objective data points and positioning on an initial rank list. He even translated subjective data points, such as letters of reference, into objective assessments by looking for keywords, and superlatives within these narrative submissions. Dr. Cheng also tasked the faculty to evaluate our initial rank list from the interview against the second rank list the faculty made at the end of year. This asked a question of whether or not our initial feelings were concordant with the assessment we made throughout the year. It was a real self-study in process improvement. 

Q. How will the findings impact on future selection processes?

Perhaps the most interesting finding was an inverse correlation between research accomplishments prior to commencing the fellowship program, and ultimate rank at the end of the clinical fellowship.  We had always looked for candidates who demonstrated excellence in not only patient care, but also research. We asked ourselves, “is this a candidate who sought to excel the discipline of pain management, by engaging in scientific inquiry?” The surprising finding was that those with the most accomplishments in the research domain were actually the lowest ranked fellows when they completed the fellowship! The correlation was inverse. This is perhaps the area that deserves the most appraisal in the coming years, as we had historically given preference to those with lengthy research accomplishments, because we thought they would be potential future leaders in advancing our field. 

Other factors such as objective scores on an examination taken during residency training programs, the “in-training exam”, had no correlation with performance during the fellowship. When we look at data like this, it suggests that we should perhaps minimize reliance on these data points, and give additional consideration to assessments which had a stronger correlation coefficient.

Q. What future studies are planned?

Several members of our department were struck by the communications shared by Angela Duckworth, the author of Grit,2,3 when she visited our campus. She spoke about attributes of perseverance called grit and how her research indicated that these “gritty” individuals had success throughout their careers. It’s much more difficult to measure these, but we’d love to include a “grit” assessment in future fellowship interviews.

References

1. Bolash R, Basi H, Minzter B, et al. Analysis of the Validity of Factors Predicting Performance in a Multidisciplinary Pain Medicine Fellowship Program. Presented at the American Academy of Pain Medicine (AAPM) 36th Annual Meeting, National Harbor, MD, USA, February 28, 2020. Poster no. 507.
2. Duckworth, A. 2016. Grit: The power of passion and perseverance. New York, NY, USA: Scribner/Simon & Schuster.
3. Angela Duckworth. Grit. Available at: https://angeladuckworth.com/grit-book/ (accessed March 12, 2020)

Author Profile:

Robert Bolash, MD, is an Assistant Professor of Anesthesiology in the Departments of Pain Management and Evidence Based Pain Research at the Cleveland Clinic Lerner College of Medicine, and is an Enterprise Physician Advisor for Medical Operations at Cleveland Clinic, OH, USA. He received his medical degree from the University of Miami School of Medicine, FL, USA, and trained in Interventional Pain Management at the Cleveland Clinic. He is a Fulbright Scholar and is board certified in both pain management and anesthesiology.

Published: March 19, 2020

Support: No funding was received in the publication of this article.