I recently was a faculty member on a panel at an international simulation meeting involving a number of senior leaders from the simulation community. One senior leader was ridiculing the collection of what he deemed to be insignificant data. He was critical of spending time and resource in the collection of such data points as number of trainees served by the program last year, number of room hour’s utilized amount of money spent on catering, did the student like the simulation etc. I think it was a sad display of a misunderstanding of the importance of data in the modern era of simulation program start up, existence, and expansion.
Data surrounding simulation centers classically is thought of as metrics that are often reported in educational research studies that may include examples such as test scores, participant feedback, various scoring metrics associated with performances observed during simulation etc. But as a senior administrator I recognize that it is so much more.
I was fortunate enough to be mentored by the founding director of WISER, Dr John J. Schaeffer to understand the power of data in a generic sense. We look at data in a very inclusive way. A unit of data may be the answer to test question number six, how many minutes it took to perform a cricothyrotomy on a simulated patient, how many times room one was used last year, how many hours Dr Jones facilitated in our center last year, or how the student “liked” her simulation efforts.
While some of the metrics mentioned are certainly important and often worthy of research publications, the operational examples given in the first paragraph above are equally as important to a program’s existence, sustainment and justification to grow. Admittedly, they may not be important for publishing in educational research, however the program has to exist to be able to allow the educational research to move forward. While we certainly don’t need another peer-reviewed, scientific publication that states “students like simulation”, this type of information can be critically important to the Dean, Hospital Executive or funding authority of a burgeoning simulation program. Don’t underestimate the importance of data for your simulation program. Having valid, reliable data that is easy to access is a powerful element in the day-to-day decision-making that it takes to run a successful simulation program.
A fundamental key to making data useful is to develop a systematic, embedded way attempt to standardize and streamline the data collection involved with your center’s day-to-day operations. It requires far more effort to determine “the number of times simulator room number one was used last year” or “how many students hours of simulation you provided last year”, with a manual retrospective review process, then if you have a system that provided that information automatically. Yes of course when you’re running a specialized research project there might be additional data metrics necessary to collect. I would argue that if the core fundamental data set that assists you in the running of your center, including the common research metrics are built-in to the underlying simulation curricular development processes it will seem like significantly less work overall to reports on such activities. Developing this automaticity in data collection will continue to build a robust data set for your center.
As the data amasses, combined with systems that allow easy access, analytics, and cross-referencing, your center could potentially soon be sitting on a mountain of data that is right for any type of data mining research necessary. The possibilities include but are not limited to justification for new employees, in-depth education simulation research, quality assurance mechanisms to evaluate faculty, facilities and program development processes.
Go forth and collect the data. All of it 🙂