Category Archives: assessment

Its January 2019 and I am flying to San Antonio, Tx to attend the International Meeting for Simulation in Healthcare. While traveling (in coach) I cannot help but to ponder where we are in simulation and where we are going. While I feel that simulation has a bright future and will earn a deservedly important role in healthcare it feels as if it is taking longer then it should.

In my overly simplistic view of simulation I envision two primary user groups. Those who utilize simulation to teach students of various healthcare professions (schools) and those who use simulation to somehow improve the quality of the delivery of healthcare. The latter of which likely includes education of individuals as well, but more of the ilk of practicing healthcare professionals and those in the apprentice phases of training such as resident physicians.

For the purpose of this post, I will be focusing on simulation efforts associated with healthcare delivery. Toward the end, I will circle back to the “school” environment again.

As healthcare dollars for the delivery of healthcare continue to be under more pressure and harder to come by there is great interest in controlling spending and increasing vigilance by corporate overlords on money being spent on investments. Investments or capital purchases are under higher levels of scrutiny than ever before. Simulationists must bear in mind that simulation is an investment, or at the least a capital expense for healthcare systems. This realization is accompanied by the stark reality that whatever you want to purchase for your simulation efforts whether it be a single simulator, or a suite of training equipment is competing against other “things” also associated with the delivery of care. Pesky things such as CT Scanners, ultrasound machines, laproscopic surgical equipment for the operating room or dialysis machines.

Why pesky? From my view as a simulation and safety leader I am envious. I am flat our jealous that it is so easy for the purchasers of the above listed examples, it is so easy for them to justify their return on investment (RPOI). Huh? What’s that? In simple terms the ROI is the business term and calculations that allow spreadsheet drivers to determine how much profit an investment of dollars in a “thing” will bring back to the

Perhaps looking at an overly simplistic explanation will help. Let’s say someone wants to put in a new CT Scanner. The costs of the scanner and installation, maintenance, staffing, and operational expenses are calculated. Then how much can be charged for each scan, how many scans can be done by the hour, and how many hours per day the scanner will be running calculates the revenue that the new CT scan will bring in. After the install is paid for, all of the rest of the revenue coming in once the expenses are deducted is profit. Thus at least when justifying the new CT scanner a requester of funds will create a fancy business proposal with colors and graphs that show money flowing in as a result of the purchase after a given period time. Purchase approved!

Now let’s take a typical cost justification scenario discussion between a simulationist (sim) and a Chief Finance Officer CFO of a healthcare system:

Simulationist (Sim): I’d like $250,000 to buy a simulator.

CFO: How is that going to make us more money?

SIM: To educate people and make them smarter and reduce mistakes?

CFO: We have lots of smart doctors and nurses working here. You should be reducing mistakes anyway.

SIM: There is a study showing a reduction of central line infections saves money.

CFO: Save who money? We still make money when the patient is in the hospital. And besides, your not asking for central line simulator.

SIM: But insurers are not going to pay for errors and hospital acquired infections anymore

CFO: Maybe not. We still make money when the patient is in the hospital. What’s your return on investment for this doll?

SIM: We are buying the simulator to train people to work together better. To work as highly functional interdisciplinary teams.

CFO: Right. We have lots of smart doctors and nurses working here. You should be reducing mistakes anyway. They are smart enough to work as teams. They do it every day.

SIM: But we can make the teams work better and make people enjoy working together more and improve patient care.

CFO: People like working here. You should be improving patient care. Where is the proof that simulation is needed to train teams AND that team training improves patient care?

SIM: The airlines have been doing it for years.

CFO: Where is the proof that airline simulation improves the airlines?

SIM: everybody just knows. It makes sense. And planes don’t crash as much as they used to.

CFO: Hospitals don’t burn down either. You know, we bought the new CT Scanner last year, and we have been able to make money on it. Its just like radiology predicted in their purchase proposal. Let me think about your request and I’ll get to you.

While the above scenario is somewhat tongue in cheek, sadly, I think it is closer to real life then many simulations we conduct. The fact of the matter is the true ROI of simulation is buried in nuances, potential opportunities, mired by anecdotal enthusiasm with a scant amount of hard-core evidence that provides the black and white spreadsheet numbers that make the bean counters excited.

It is upon us to figure out ways to describe the ROI of simulation more coherently, accompanied by facts and figures that make a difference to the leaders of healthcare systems. Let me give you a hint……. It aint about education.

We must transcend long hold belief and common assumption that the value of simulation is the education. I think the realization and yet unlocked true potential of simulation remains ties up in the ability to assess. It is tough to pivot from thinking that simulation is primarily an education methodology. But I encourage you to do so. Now before you get your hair on fire and leave me nasty comments, I’m not suggesting that we abandon simulation which we know to be an incredibly powerful education platform/modality. I just believe it you think it the main power is education first and foremost its becomes difficult to strategically plan, document, and provide leadership in other directions.

I think in the healthcare delivery space a more powerful argument that can contribute to the ROI of simulation is to harness the ability of simulation to identify the best deployment of judicious resources. So, what does this mean? Stop teaching with simulation? No, of course not.

Focusing more on the use of simulation as an assessment and surveillance tool can help to create bigger value. When teaching with simulation, conducting assessments of what people or perhaps units are doing well, what they are struggling with in a more quantitative way can help to identify the true needs of the organization. Understanding the local struggle and perhaps what the local community is not struggling with allows for a smarter utilization strategy for simulation.

Now before the heads pop off of the safe learning environment people, I’m not suggesting we need to turn every simulation into a summative performance assessment and give passing and failing grades that will ruin peoples lives. However consideration should be given to the gathering of data to show improvement is critically important as you do all of your great education work. After you collect the data is a systematic way have the courage to abandon what participants always do well on, focus or increase in the areas of greatest improvements.

Carefully collect the data if you use your simulation activities for on-boarding. Don’t ask if they liked the simulation. That’s not the data you need for your ROI justifications. Can you shorten aspects of on-boarding through the use of simulation? Showing credible evidence that nursing on-boarding can be shortened by x number of days or weeks through the strategic and judicious use of simulation will bring music to the ears of the bean counting crew who don’t fancy paying for the training of people when they could be working.

Other thoughts…. Using simulation as an evaluation tool in a human factors applications can assist other departments in increasing efficiency, and improving throughput. Think about the importance of that. What????? Not your cup of tea? Think back a few paragraphs on calculations leading to justifying the need for the new CT Scanner.

Carefully documenting that simulation trained anesthesiologists, CRNA’s , endoscopists and surgeons for example may shorten OR time which means more surgeries can occur, which generate lots of revenue is part of the ROI that should be in capital letters. This is the data that matters for the ROI justifications.

In-situ programs can give valuable feedback to hospital safety and quality leaders to demonstrate volatilities in the system with regard to both process, staffing, human performance etc. It can also demonstrate where the strengths lie. If there is unnecessary training going on where the strengths lie, then redeploy or readjust to the actual needs of your system. Additionally, formulating such relationships with the quality and safety leaders of your institution and letting them know of you true capabilities that are more then making people happy and smarter through education, can win you some powerful allies in the corporate leadership suites.

Lastly circling back to the schools……. Looking past the education benefits of simulation to use it as a tool to create data that can lead to information the underpins significant change, cost savings, and allocations of precious resources (people and money) will do you well. With the exception of more students,  I  don’t think it is likely that windfalls of money are coming your way either…….

So is you are carefully assessing you simulation efforts and activities in a thoughtful manner, you can help to reduce redundancy, unnecessary training intervals, or repetitions and on and on. Doing less of that which is ineffective save money. Saving money is a variable of the ROI that your CFO will pay attention to.