Center for Education Research, Technology, and Innovation
About CERTAIN
The Center for Education Research, Technology and Innovation (CERTAIN) in the Department of Anesthesia, Critical Care and Pain Medicine creates an environment of educational excellence through a process of continuous learning, assessment, and feedback, under the leadership of Dr. Robina Matyal. CERTAIN conducts wide-ranging educational research, uses findings to improve our programs, and evaluates those programs to maintain heightened learning and satisfaction. We use state-of-the-art technology to create innovative teaching and assessment methods that allow us to tailor educational efforts to individual trainees and focus on their learning needs. Our ongoing cycle of quality improvement ensures that our anesthesia educational programs are among the best in the country. In addition, the Center also fosters a supportive, collaborative environment for faculty to engage in educational scholarship. We provide mentorship, guidance, and support in areas such as study design, grant preparation, methodology, IRB correspondence, and manuscript preparation and submission.
Mission Statement
The Center seeks to develop, test, and deploy a dynamic educational environment that optimizes learning for all department members and enhances patient outcomes. Through collaborative research that uses state-of-the-art technology to identify the best educational methods, CERTAIN aims to continually enhance our existing programs and create new and innovative systems for teaching and learning.
Goals
- Advance research to create an optimal learning environment for our department members.
- Develop and apply innovative metrics to evaluate and enhance our educational programs.
- Implement systems to quantify and improve communication and professionalism skills in our department members.
- Design and assess systems to teach faculty to provide effective and high-quality feedback to department members.
- Use state-of-the-art technology to create, assess, and deploy new learning tools and methods.
- Refine techniques to teach complex technical skills to our department members.
- Support other departmental educational efforts to improve patient care and satisfaction.
- Provide mentorship, support, and guidance to faculty pursuing educational research.
Faculty
Robina Matyal, MD
Staff Anesthesiologist
Director, Center for Education Research, Technology and Innovation
Vice Chair, Education
Associate Professor of Anaesthesia
Dr. Robina Matyal is a distinguished anesthesiologist, educator, and research investigator. She received her medical degree from the Army Medical College in Pakistan and did her initial internship and residency training at the Combined Military Hospital in Kharian, Pakistan. She came to Beth Israel Deaconess Medical Center in 1977 and did Research Fellowships in Surgery and Pulmonary Critical Care. Dr. Matyal subsequently did an Anesthesia Residency at BIDMC, followed by fellowships in both Anesthesia and Vascular Anesthesia. She is an Associate Professor of Anaesthesia at Harvard Medical School, Director of Vascular Anesthesia at BIDMC, and Co-Director of the Valve Research Group at Harvard Medical School.
Lauren K. Buhl, MD, PhD
Staff Anesthesiologist
Associate Program Director, Anesthesia Residency Program
Instructor in Anaesthesia
Lauren K. Buhl, MD, PhD received her medical degree from Harvard Medical School and her PhD in Molecular Neuroscience from Massachusetts Institute of Technology as part of the Medical Scientist Training Program at Harvard Medical School. Dr. Buhl did her anesthesia residency at Beth Israel Deaconess Medical Center, where she is an Associate Program Director of the Anesthesia Residency Program. She completed a Neuroanesthesia Fellowship at BIDMC and a two-year fellowship in Medical Education Research through The Academy at Harvard Medical School. Dr. Buhl’s research interests in medical education center around curriculum development for undergraduate and graduate medical education, optimization of intra-operative teaching, maximizing learning opportunities through targeted scheduling, and promoting learner resilience through shared failure.
Cullen D. Jackson, PhD
Research Scientist
Associate Director, Center for Education Research, Technology and Innovation
Instructor in Anaesthesia
Dr. Jackson’s research focuses on applying cognitive science and human factors engineering theories and methods to challenges in the surgical setting with particular focus on individual, team, and system performance assessment and augmentation to improve quality and patient safety. He frequently teaches on using game-based training to teach non-technical skills, and he has 25 scholarly works in print, as well as numerous conference posters and talks, focused on simulation-based training and education, virtual reality, performance assessment, and information visualization. Dr. Jackson completed his postdoctoral training in the Visualization Research Lab in the Department of Computer Science at Brown University, and he earned his Ph.D. and Sc.M. degrees in Experimental Psychology from Brown University in Providence, RI, and a B.S. in Computer Science and a B.A. in Psychology from Trinity University in San Antonio, TX.
Lior Levy, MD
Staff Anesthesiologist and Intensivist
Director, Anesthesia Resident Simulation
Instructor in Anaesthesia
Lior Levy, MD received her BS in Biochemistry from Brandeis University and her medical degree from Chicago Medical School. She completed her intern year in OBGYN at George Washington University and her anesthesia residency at NYU, followed by an obstetric anesthesia fellowship at Brigham and Women's, after which she joined Beth Israel Medical Center. She is the Director of Resident Simulation for the BIDMC Anesthesia Residency Program. Her research interests are in obstetric anesthesiology and in medical education. She was one of the recipients of the Shapiro Scholarship in 2021, which further sparked her interest in education and Med Ed research. She enjoys international traveling back to Israel and France and also to tropical destinations where she can scuba dive and explore.
Haobo Ma, MD, MS
Staff Anesthesiologist and Intensivist
Instructor in Anaesthesia
Haobo Ma, MD, MS received his medical degree at Peking Union Medical College in Beijing, China and his MS in Clinical Informatics at the University of Pittsburgh. He finished his anesthesia residency at Massachusetts General Hospital and a critical care fellowship at Stanford. Dr. Ma is triple Board-certified in Anesthesiology, Critical Care Medicine, and Clinical Informatics and is a staff anesthesiologist and intensivist. He has completed a Harvard Medical School Medical Education Research Fellowship. Dr. Ma is applying innovative approaches such as video-based feedback and 3D modeling to improve the effectiveness of procedural teaching.
Kadhiresan R. Murugappan, MD
Staff Anesthesiologist and Intensivist
Instructor in Anaesthesia
Kadhiresan R. Murugappan, MD received his BA in Latin American Studies, Spanish, and History at Tulane University in New Orleans, LA and his medical degree at Rush Medical College in Chicago, IL. He did his anesthesia residency at Rush University Medical Center and completed a critical care fellowship at Beth Israel Deaconess Medical Center prior to joining the department as a staff anesthesiologist and intensivist. His research interests include developing tools to improve patient outcomes and aid in clinical decision making, analyzing weight cycling in combat sports athletes through collaboration with the California State Athletic Commission (CSAC), and developing ultrasound education for medical trainees at all levels. In his teaching, he passes on a respect for self-care, emphasizing mindfulness, exercise, and healthy living, so that our future physicians can practice longer while maintaining a healthy work/life balance.
Sara E. Neves, MD
Staff Anesthesiologist and Intensivist
Director, Anesthesia Residency Program
Instructor in Anaesthesia
Sara Neves, MD is the Program Director of the BIDMC Anesthesia Residency Program and the Director of the Resident Root Cause Analysis Program. Dr. Neves received her MD at Sidney Kimmel Medical College of Thomas Jefferson University, did her residency at Yale New Haven Hospital, spent a year as Chief Resident, and then completed a critical care fellowship at Brigham and Women’s Hospital. She is also a graduate of the BIDMC Physician Leadership Program and has been selected as a 2021-2022 Rabkin Fellow in Medical Education for her project on creating and evaluating a skills-based pre-clinical curriculum for invasive procedures. Dr. Neves’ research interests include medical education, patient safety, and critical care. She has helped with the Center’s efforts in developing machine learning to rate the quality and utility of feedback and also does work on physician wellness and occupational health.
Ameeka Pannu, MD
Staff Anesthesiologist and Intensivist
Program Director, Critical Care Medicine Fellowship
Instructor in Anaesthesia
Ameeka Pannu, MD received her BS at Brandeis University and her medical degree on the global health track at George Washington University in Washington, DC. She did her anesthesia residency and critical care fellowship at Beth Israel Deaconess Medical Center, where she was a Chief Resident for a year. Dr. Pannu is the Program Director of the Critical Care Medicine Fellowship at BIDMC. Her research interests include the optimization of critical care and subspecialty education and impact of curricular design.
Lindsay A. Rubenstein, MD
Staff Anesthesiologist
Associate Program Director, Anesthesia Residency Program
Instructor in Anaesthesia
Lindsay Rubenstein, MD received her medical degree from the University of Rochester School of Medicine and Dentistry in Rochester, NY and her MA in Medical Science from Boston University. She did her anesthesia residency and a pain medicine fellowship at Beth Israel Deaconess Medical Center. She is an Associate Program Director for the Anesthesia Residency Program. Dr. Rubinstein’s research interests include neuroaxial and regional anesthesia as well as the implementation of wellness and equity programming within the department.
Aidan Sharkey, MD
Staff Anesthesiologist and Intensivist
Instructor in Anaesthesia
Aidan Sharkey, MD graduated from the School of Medicine at Trinity College Dublin and undertook his residency training in Anesthesia from the College of Anesthetists of Ireland. After completion of his residency training, he undertook both Cardiac and Obstetric Anesthesia fellowships in Canada at Mount Sinai and Toronto General Hospitals after which he joined Beth Israel Deaconess Medical Center in Boston as a staff anesthesiologist. His research interests include transesophageal echocardiography and point-of-care ultrasound.
Shahla Siddiqui, MBBS, MSc
Staff Anesthesiologist and Intensivist
Assistant Professor in Anaesthesia
Shahla Siddiqui, MBBS, DABA, MSc, FCCM received her medical degree from Aga Khan University in Pakistan and her MSc in Medical Ethics from the University Singapore. Dr. Siddiqui did her anesthesia residency at the University of Maryland and a critical care fellowship at Columbia University. In addition to her clinical work, she has had an active research career and has many areas of interest, including inter-professional education, health-science qualitative research, ICU burnout, sepsis in the ICU, the genomics of sepsis and ICU outcomes. She is also very interested in medical ethics and related topics such as compassion in medicine, an area where she has several active projects.
Daniel P. Walsh, MD
Staff Anesthesiologist and Intensivist
Associate Program Director, Anesthesia Residency Program
Medical Director, Intensive Care Unit, Beth Israel Deaconess Hospital - Plymouth
Instructor in Anaesthesia
Daniel P. Walsh, MD received his BS in Biochemistry and Physiology and his medical degree at the University of Minnesota in Minneapolis. He did his anesthesia residency at the University of Nebraska Medical Center in Omaha, was Chief Resident for a year, and then came to Beth Israel Deaconess Medical Center to complete a critical care fellowship. He is an Associate Program Director for the BIDMC Anesthesia Residency Program. His research interests include curriculum design and development of ultrasound education. He also enjoys baking, strength sports, and international travel.
Staff
Vincent Baribeau
Clinical Research Assistant
Vincent is a Research Assistant at Beth Israel Deaconess Medical Center. He is an active member of both the Valve Research Group (VRG) and Center for Education Research, Technology and Innovation (CERTAIN). He specializes in 3D printing and motion tracking technology, for use in both educational and clinical applications.
Michael J. Chen
Education Research Assistant
Michael Chen works with the Center leadership and research investigators to coordinate both research and educational activities. He is active in all stages of the research process, including protocol submission, enrolling patients, and data collection and analysis. He is particularly interested in incorporating technology such as machine learning and 3D printing into education. He is a graduate of Northeastern University, where he received his BS in Health Science in 2014, and while in college he tutored in organic chemistry. In his spare time, Michael likes to read Sci-Fi and fantasy fiction.
Dustin Lin
Clinical Research Assistant
Dustin Lin is a research assistant who is an active member of the Center for Education Research, Technology and Innovation (CERTAIN). His research interests include medical education, curriculum development and the clinical applications of gross anatomy structure and function.
Vanessa T. Wong
Education Project Coordinator
Vanessa T. Wong, BS received her BS in Business Administration at Boston University and works with department members in developing curricular tools for learners as well as the department’s online learning management system. She also works with the Center in coordinating education research studies, from scheduling data collection sessions to collecting and analyzing data to preparing and revising grant applications, abstracts, and manuscripts. She works closely with Dr. Walsh in integrating the Center’s studies with our residents' curriculum so that our residents can benefit from novel ideas in education.
Non-Technical Skills
Non-technical skills, such as professionalism, communication, and teamwork, are essential for physicians to work successfully with colleagues and co-workers and communicate empathically with patients. Led by Director Dr. John D. Mitchell, CERTAIN does innovative and groundbreaking work in developing these skills in trainees.
After conducting a national needs assessment survey, we designed and launched a system to provide daily feedback to residents and an accompanying educational program for faculty on how to give feedback. With the support of a grant from the Foundation for Anesthesia Education and Research (FAER), we then conducted a multicenter project to assess whether this educational program can increase the quantity, quality, and utility of faculty feedback to residents. We saw evidence of improvement in newer faculty and are conducting follow-up work to gauge the extent of the impact.
Our team also developed an ambulatory surgical patient survey to assess residents’ professionalism and communication skills. We used survey results to construct a curriculum for trainees that resulted in overall improved performance on the survey. We later adapted the paper survey into a tablet-based survey administered in the post-anesthesia care unit (PACU) to improve compliance and accuracy. Using this new data, we developed an individualized curriculum for trainees that targeted specific areas for improvement based on survey results. We continue our work in this area to devise better ways to collect and use patient feedback to enhance residents’ professional and communication skills.
Under the mentorship of Dr. Mitchell, Dr. Sara Neves and Mr. Michael Chen, we completed a project on using a machine learning model to predict feedback quality. They lead a group of investigators from multiple Anesthesiology Departments at different institutions in this important endeavor. The model predicts when open-ended feedback is of low quality and utility in a way that agrees substantially with trained human raters. This is the first step in using machine learning to help screen feedback and provide timely guidance and support to feedback providers.
In addition to these studies, our team is investigating methods to teach skills to improve communication, emotional intelligence, and teamwork. Dr. Cullen Jackson leads a project on determining whether the cooperative board game Pandemic (Z-Man Games) can be used to teach teamwork skills as effectively as traditional simulation-based training. Dr. Shahla Siddiqui launched a study on developing a discussion-based interprofessional curriculum to improve emotional intelligence in anesthesia and surgical trainees and nurses in order to improve teamwork in the PACU.
Technical Skills
Perioperative ultrasound, including transesophageal echocardiography (TEE), transthoracic echocardiography (TTE), ultrasound guided regional anesthesia, ultrasound guided vascular access, and abdominal and lung exams, is a critical technique in the modern practice of anesthesiology. These procedures require a complex combination of technical and cognitive skills, so it can be challenging to provide a training experience that helps providers to master these procedures.
CERTAIN collaborates with the Valve Research Group at BIDMC to identify the best methods for ultrasound training. Our efforts span various ultrasound procedures, different levels of learners, and multiple departments and institutions. We aim to:
- create specific metrics to objectively assess proficiency in different types of procedures
- improve ultrasound proficiency through multimodal teaching approaches
- ensure proficiency retention
We have created and tested training approaches for both TEE and TTE for trainees in Anesthesiology, Emergency Medicine, Cardiology, Critical Care, Surgery, and Internal Medicine. These approaches demonstrated improvements in cognitive and manual skills for these procedures. We developed and tested similar teaching approaches for a wide range of perioperative ultrasound procedures, including lung exams, abdominal exams, and ultrasound guided regional anesthesia and vascular access. We are also leading studies on using hand-motion analysis to evaluate manual skills when performing ultrasound procedures, such as placing central lines.
We also are working with military to create a standardized ultrasound training program for military medical personnel to triage and manage acute battlefield injuries. The program focuses on point-of-care ultrasound (POCUS) and includes metrics to assess proficiency and implementation of the training program, and follow-up assessments to ensure skill retention. We have piloted this program and conducted several sessions with military medics.
In addition to our work in teaching ultrasound, we are exploring methods for teaching other skill sets, including airway management techniques. Dr. Haobo Ma is leading our efforts at exploring the teaching of fiberoptic intubation, and Dr. Cullen Jackson is leading the team in studying hand motions in intubations using direct laryngoscopy.
Pedagogy
CERTAIN combines the most current technology and educational modalities to create, test, and evaluate innovative methods of teaching and learning. We aim to individualize learning with systems and curricula that allow trainees to target efforts in specific areas where they can improve. We are national leaders in this area and have many ongoing projects that involve multiple teaching methods.
We seek to create learning tools that are easy to use, targeted, and effective. In order to improve intraoperative learning, Dr. Daniel Walsh developed an “E-journal club” that supplies access to journal articles and other educational resources based on weekly topics and allows trainees and faculty to initiate discussion both through online forums and in the intraoperative setting. He is also working with former and senior residents to develop an online “keywords” curriculum with push notifications to help prepare residents for Board exams. In addition, Dr. Lauren Buhl is leading an effort to create and evaluate a series of branched learning scenarios for medical students prior to their anesthesia rotation to familiarize them with the processes of anesthesia care. She also leads a study on using and assessing a system to more evenly distribute low-volume neuroanesthesia cases mandated by the Accreditation Council of Graduate Medical Education (ACGME) among residents as well as a project on developing a curriculum to help faculty share their failures with trainees as a teaching tool.
The Center also participates in multi-institutional studies on pedagogy. Dr. John Mitchell is the site Principal Investigator for a project with Baylor College of Medicine and Baystate Health to develop a tool to assess transfusion medicine knowledge and ultimately a transfusion medicine curriculum for anesthesiologists. He is also part of a project with Stanford University to assess and improve the current anesthesia milestone evaluation system. In a project with Boston Children's Hospital, he is helping to evaluate residents’ responses to reflective questions on different clinical scenarios as well as their perceptions of reflection. He also is working with the University of Virginia, University of Kentucky, and University of North Carolina at Chapel Hill to investigate the effect of mindful self-CARE (Compassion, Appreciation, Rest and Exercise) education on anesthesiology resident wellness.
Analyzing the Anesthesia Milestone Evaluation System
Description: Multi-center project with Stanford University to learn more about and improve the current anesthesia milestone evaluation system
Enhancing feedback in anesthesia residency programs
Description: Multicenter project (with the University of Kentucky, University of Rochester Medical Center, and University of California, San Diego) on teaching faculty how to provide feedback to residents using a video-based tool; funded by the Foundation for Anesthesia Education and Research (FAER)
Ensuring patient satisfaction with resident communication skills
Description: Departmental project using patient feedback to develop and assess curricular activities aimed at improving anesthesia residents' communication skills
Project Pandemic - Efficient and effective teamwork and communication skills Training Using a Tabletop Game
Description: Departmental project on determining whether a tabletop board game (Pandemic) can teach communication/teamwork skills as effectively as simulation; funded by the Society for Education in Anesthesia (SEA)
Virtual operating room team experience (VORTeX)
Description: Multicenter project (with Rensselaer Polytechnic Institute and Albany Medical Center) to develop a shared virtual environment to enable OR teams to train together over the Internet using head-mounted VR displays to develop their nontechnical skills; funded by the NIH/NIBIB
Interprofessional education for residents
Description: Development of an interprofessional curriculum to improve emotional intelligence in anesthesia and surgical trainees and nurses in order to improve teamwork in the PACU
Fundamentals of ultrasound curriculum
Description: Departmental project on developing a curriculum to teach ultrasound to trainees, nurses, and faculty, and developing an index to measure proficiency in ultrasound
Measuring intubation skills in the operating room (MINTOR)
Description: Departmental project on developing a method to teach intubation skills to novice residents using hand motion metrics
Incorporating video-based feedback and deliberate practice in teaching fiberoptic intubation
Description: Multicenter project (with Peking Union Medical College Hospital) on developing a video-based teaching method for fiberoptic intubation
Hand motion assessment for objective evaluation of central line placement: From simulation to real-world application
Description: Multi-departmental project on creating a central line placement training and assessment program using objective hand motion metrics; funded by CRICO
Creation and Validation of an Examination on Transfusion Practices for Anesthesiologists
Description: Multi-center project with Baylor College of Medicine and Baystate Health to develop a tool to assess transfusion medicine knowledge and (later) a transfusion medicine curriculum for anesthesiologists
Developing an Anesthesia Keywords Curriculum
Description: Departmental project to develop an online keywords curriculum for residents with push notifications
Developing an Online Journal Club (e-Journal Club) Curriculum for Anesthesia Residents
Description: Departmental project to develop an online Journal Club curriculum for residents
Improving residents perceptions of fairness and balance with targeted scheduling
Description: Departmental project on using the neuroanesthesia rotation as a model to test a system to more evenly distribute low-volume, ACGME-mandated cases among residents and assess the effects on their perceptions of their experience on the rotation
Preparing clinicians for the ICU care of COVID-19 patients
Description: Multi-departmental project on developing a website with educational resources/materials for preparing clinicians to care for COVID-19 patients in the ICU
Preparing staff to safely care for COVID-19 suspected/confirmed patients
Description: Departmental project to evaluate the efficacy of COVID-19 resources developed by the Quality, Safety, and Innovation Division, including simulations and website resources
Characterization of anesthesia residents’ reflective abilities in learning
Description: Multi-center project with Boston Children's Hospital to analyze residents’ perceptions of reflection and their responses to reflective questions on different clinical scenarios
Teaching decision making in anesthesia to medical students with a branched-chain learning module
Description: Departmental project on developing a branched-chain learning module for medical students before they start their anesthesia rotation to better acquaint them with all the choice points in an anesthetic plan; funded by the Academy at Harvard Medical School Fellowship in Medical Education Research
The power of shared failure
Description: Departmental project on identifying barriers to faculty sharing experiences of failure with trainees, determining the benefits of shared failure, and developing a curriculum to help faculty use failure as a teaching tool