Tag Archives: simulation

August 24, 2023 · 3:11 pm

What is Simulation? The question that caught me off guard!

I was having an exit interview meeting with one of my graduating simulation fellows, and he asked me an interesting question for his last day. He said, “Dr. Paul, what is simulation?” I thought this was perplexing after a year-long intense study of simulation with us at our Institute! It was quite insightful, though. One of his observations was that there are many ways to do simulations right. He had many experiences throughout the year, visiting other simulation centers, attending international meetings, and teaching with us at different facilities. He realized many different vantage points, missions, visions, and purposes for implementing healthcare simulation.

I took a deep breath, thought about it, and said, “Simulation is a methodology by which we re-create a portion of the healthcare delivery experience with a goal of education and/or assessment of people, groups of people, teams, and/or environments of care.” Then, I drew a rough sketch of my vantage point of simulation that divided into two major subgroups, including methods/modes on one side and primary purpose on the other. I recreated it in the accompanying figure.

Methods/Modes

I think of the methods or modes of simulation based on the primary simulator technology employed to generate the goals of an intended program. Of course, mixed modality simulations often incorporate a spectrum of technologies.

I don’t mean this list to be exhaustive by any stretch of the imagination, and some may argue an oversimplification. The general categories that come to my mind are as follows:

  1. High-technology manikins generally presents the form factor of an entire human being complemented with electronics, pneumatics, and computer equipment that helps the manikin represent various aspects of anatomy and or physiology. (As you have undoubtedly heard me opine in the past, the word FIDELITY does not belong in any descriptor of a simulator. It muddles the water and confuses the overall strategies associated with simulation, although it is a popular industry buzzword that has somehow worked its way into academic definitions inappropriately.)
  2. Low-technology manikins generally have the form factor of an entire human being but with significantly less electronics or infrastructure to allow physiologic or anatomic changes that occurred during the simulation encounter.
  3. Standardized people/patients, meaning live people playing various roles ranging from patients, family members, and other healthcare team members to help bring a simulation encounter to life.
  4. Task trainers represent a re-creation of a portion of the human being oftentimes created to accomplish goals of completing skills or procedures. Depending on the purpose, they may or may not have a significant amount of augmenting technology.
  5. Screen-based simulations are computerized case or situation representations of some aspects of patient care that change in response to the stimulus provided by participants.
  6. Role-play includes designs that utilize peers and/or select faculty to engage in a simulated conversation or situation to accomplish learning outcomes.
  7. Virtual reality/augmented reality are high technology recreations or supplements that re-create reality through the lens of a first-person engaging in some sort of healthcare situation and have the capacity to change in response to the stimulus provided by the participant or participants.

Primary Purpose/Goals

Again, looking at a given simulation’s primary purpose and goals will lead one to quickly find overlaps and that the categories did not exist in complete isolation. However, for this discussion, it helps to think of the different categories of intent.

Education

When I think of simulation programs primarily focusing on education, it comes down to helping participants gain or refine knowledge, skills, competence, or other measures that allow them to become better healthcare providers. In general, a teaching exercise. This can apply to simulation scenarios that are directed at one person, groups of people (all learning the same thing), or perhaps teams that have learning goals of competencies associated with the interaction between the groups of people similar to that that occurs in the care of actual patients in the healthcare environment.

Assessment

The simulation encounter is primarily designed as an assessment. This means there is a more formal measurement associated with the performance of the simulation, often employing scoring tools, with the primary focus of measuring the competency of an individual, groups of individuals, or similar to the above teams of individuals functioning as teams. Further, assessment can measure aspects of the environment of care and/or the systems involved in supporting patients and the healthcare workforce.  (For example, an in-situ code blue response simulation may measure the response of the local care team, the response of a responding team, the engagement of the hospital operator, the location and arrival of necessary equipment, etc.)

Research

There are many approaches to the use of modern healthcare simulation in research. At a crude level, I subdivided into looking at the outcomes of the simulation; meaning did the simulation encounter help to improve the participant’s performance? At the next level, you can evaluate if the simulation improves patient care.

The next category is using simulation as a surrogate of the patient care environment but not measuring the effect of the simulation. For example, we might set up an ICU patient care environment for human factors experiments to figure out the ideal location of a piece of equipment, the tone of an alarm, the interaction of caregivers with various equipment, etc. Such an example of simulation often helps to determine optimal environments and systems of care in the primary planning stages or the remodeling of healthcare delivery processes and procedures.

So, the next time I orient an incoming simulation fellow, I will start with this discussion. I am thankful that my fellow who just graduated provided such a simple but deeply probing question to help wrap his arms around the various simulations he has been experiencing over the last year while he studied with us.

Having put some more thought into this, I think it’s a useful exercise for those of us in leadership positions within the simulation world; it is probably good to stop and think about this a couple of times a year to refresh, reset, and ensure that we are remaining mission-driven to our purpose.

Until next time, Happy Simulating!

Leave a comment

Filed under simulation

Tagged as , ,

August 8, 2023 · 11:07 am

HUMBLE: Six Traits That Will Make You a Better Simulation Educator and Lead Effective Debriefings

HUMBLE: Six Traits That Will Make You a Better Simulation Educator and Lead Better Debriefings

Excelling as a educator in the healthcare simulation field goes beyond just imparting knowledge; it requires a unique set of qualities that can truly make a difference in students’ learning experiences. The acronym HUMBLE focuses on six key traits that can help educators better design, facilitate, and lead more effective debriefings. These traits include Humility, Understanding, Mindfulness, Balance, Learning, and Engaging. In this blog post, I will delve into these traits and explore how they can enhance your abilities as an educator, ultimately leading to more impactful and engaging debriefing sessions.

H – Humility

This is one of my favorites and the most important in my humble opinion! Approaching teaching responsibilities in simulation from a perspective of humility goes a long way. Instructors, with humility, acknowledge that they don’t know everything and remain open to continuous learning. This attitude is also imparted to the participants, encouraging them to adopt the same approach throughout their careers.

An instructor who demonstrates humility creates a more approachable and non-threatening atmosphere, allowing students to feel comfortable admitting to and learning from their errors. This also contributes to a milieu that helps maintain a safe learning environment and a perspective of a level playing field that helps to allow participants of the simulation to share their thoughts. This, in turn, gives us as faculty a privileged glimpse into their thought processes. Interestingly, it is also well-known in business literature that leaders who demonstrate humility are often perceived as more credible and trustworthy.

U – Understanding

Understanding the fact that each participant of your simulation is a person that has their individual lives, challenges, successes, experiences, and strong and weak skills is key to understanding the fact that there are varying amounts of knowledge and/or abilities for the person to apply that knowledge in the simulated session. In other words, many factors contribute to why someone knows something or can apply knowledge in each situation. We should maintain an understanding that everyone has gaps in knowledge and attempt to remain nonjudgmental as to why those gaps exist.

M – Mindfulness

It is incredibly important that we are mindful of our presence during the simulation as well as the debriefing. Educators need to be attentive, focused, immersed, and committed to the learning objectives to expertly facilitate and then lead a high-quality debriefing that contributes to the learning outcomes. We need to work to identify tips and challenges that help maintain our mindfulness, focus, and attention during these activities.

While I am not suggesting a prescriptive approach, it is important to introspect and determine how you enhance your mindfulness associated with the simulation-based education process. For some, it means being well rested; for others, it means appropriately titrated doses of caffeine, and yet for others, exhaustive preparation the day before. Reflect on your performance by thinking about when your concentration may have waxed and waned and what you can do to improve. I find it particularly challenging to remain cognitively sharp throughout the entire series when running the same scenario repeatedly with different groups of learners!

B – Balance

Creating the mindset of balance in any one simulation session helps participants discover what they need to improve upon and what they did well in each simulation encounter. There is an old saying, “The negative screams, while the positive only whispers….. ” that I think you would agree applies when we are facilitating a simulation and about to go into the debriefing. If you think about it from the learner’s perspective, exploring a laundry list of their failures without recognizing the contributions that went well can be demoralizing and interfere with the faculty/participant relationship. While I’m not suggesting that we gloss over egregious errors, it is important to find a balance between those activities that went well and those that need improvement.

L – Limited, Lifelong Learning

This may be my second favorite! When conducting the debriefing, faculty should avoid trying to comment or debrief on every single thing in every scenario. It is important to remember that the journey of healthcare, whether in a simulated environment, attending lectures, attending workshops, or generating experiences by taking care of real patients, is a lifelong learning process. Each encounter along the way provides the potential for learning, albeit limited by the amount of cognitive transfer that can occur at a given time. During simulation, there is a natural tendency to want to cram everything into every scenario. I think this emanates from the fact that we are so excited about the simulation modality and get a small opportunity with each participant! Admittedly, I need to keep myself in check during such encounters. It’s important to think of the human brain as a sponge. Once it is saturated, the sponge cannot effectively take on more water.

E – Engagement

Engaging the learners in the conversation, as well as designing the scenarios to engage learners actively, is part and parcel of the basis of the idea that simulation, through active learning, is a high-quality opportunity. Think about this during the design process of your scenarios as well as the debriefings, insofar as how you assign roles, what your observers are required to do, and how you rotate people in and out of the scenario.

During the debriefing, remember that engaging your learners so that they are responding to the prompts you provide during the debriefing will elicit the responses. As the learners are engaged in the conversation, you can listen to their thought processes and make evaluations of the depth of their knowledge around a particular topic. Additionally, you can identify gaps that exist, either in knowledge or the application of knowledge, that can help them improve for the future. So often, when training others in debriefing, I observe faculty members dropping into a mode of “mini-lecture” during what is supposed to be a debriefing. This deviates from active cognitive engagement and sometimes transcends into (a well-intentioned) one-way conversation. It is important to remember that if your participants are not engaged, you are potentially squandering some of the learning opportunities. At a minimum, you are giving up the ability to hear what they are thinking.

In summary, as you continue to develop your skills as a healthcare simulation educator, I invite you to use HUMBLE as an acronym that helps to reflect upon positive traits, actions, and good guiding principles, that provide learners with an optimized environment for improvement.  I truly think that healthcare simulation educators have powerful opportunities for assisting with the transfer of knowledge, and experience and creating opportunities for reflection, and by being HUMBLE we can ensure a more effective and empathetic learning environment for all participants.

Until Next Time,

Happy Simulating!

Leave a comment

Filed under debriefing, simulation

Tagged as , ,

July 27, 2023 · 11:52 am

The Importance of the Psychological Contract in Healthcare Simulation: Six Fundamental Elements

Simulation is a powerful tool in healthcare education to enhance learning and improve patient outcomes. Through simulation-based learning encounters, participants can engage in hands-on experiences that mimic real-life situations, allowing them to develop critical skills and knowledge.

The success of healthcare simulation educational encounters relies on the participants and the facilitators who guide and support the learning process. Understanding the psychological contract that needs to exist between participants, facilitators, and content designers, is crucial in creating a positive and effective learning environment. In this blog post, we will explore the importance of this psychological contract and discuss strategies to enhance it, ultimately leading to enhanced learning and improved outcomes in healthcare simulation.

While most discussions of the psychological contract are in the context of facilitating a simulation in real time, some elements are critically important to consider during the design process associated with simulation-based education encounters. How we structure our briefings, pre-briefings, and course schedules can dramatically influence our relationship with the participants to enhance the learning potential in the simulated environment.  

I like to think of six essential elements when designing and facilitating simulations.

Professionalism: We agree to treat each other as professionals throughout simulation-based education encounters. The learner agrees to attempt to interact in the scenario as if they were taking care of an actual patient, and the simulation facilitator agrees that the scenario will be directed to respond with a reasonable facsimile of how an actual patient will respond to the care being delivered.

Confidentiality: The simulation program agrees to keep the performance assessment of participants confidential to the extent possible. The simulation participant should be apprised of the fate of any audio, video, or still photographic media generated from the simulation. If, by programmatic design, there is the intent to share any performance results, the participant should be aware of this before engagement in the program.

Time: The simulation facilitator commits to creating an environment of learning that respects the participant’s time. The simulation program commits to the intent that the simulation encounter and all associated time spent will help provide the participant with relevant, professional education and growth potential.

Realism/Deception: Both the participant and the facilitator acknowledge that the environment is not real and will contain varying degrees of realism. The simulation environment’s primary intent is to provide a reasonable facsimile of a healthcare encounter to serve as the background for the participant to demonstrate their clinical practice proficiency to the best of their knowledge in exchange for feedback that highlights areas of success and identifies areas of potential improvement. Our simulation-based scenario designs are modeled after actual patient encounters or close representations of cases that may occur within your practice domain. While the case may represent areas of diagnostic mystery or other unknowns, the scenarios are not designed to deceive or mislead the learner deliberately. The facilitator acknowledges there may be facsimiles of the simulation that may be misinterpreted by the learner as a matter of simulation scenario design limitations and will address them as appropriate, as they occur.

Judgment: While there will be an assessment of the learner’s performance to carry out effective feedback, it will be based upon known best practices, guidelines, algorithms, protocols, and professional judgment. No judgment will be associated with why a gap in knowledge or performance was identified. The facilitators agree to maintain a safe learning environment that invites questions, explorations, and clarifications as needed to enhance learning potential.

Humbleness: Healthcare is a complicated profession regardless of the practice domain. It requires the engagement of lifelong learners to learn and retain a significant amount of knowledge and skill. Additionally, there is a constant refinement of knowledge, best practices, and procedures. The facilitator acknowledges that they are imperfect and engage in the same lifelong learning journey as the participant.

While the descriptions associated with each element of the psychological contract in this post are more aligned with the engagement with senior learners or practicing professionals, it is easy to translate each category when working with students and other types of junior learners.

Educators and learners can establish a foundation of trust, collaboration, and active participation by understanding and embracing the tenants of psychological contracts in healthcare simulation. Careful consideration of these elements is beneficial during program design and when actively facilitating simulation-based learning encounters. This, in turn, enhances the learning outcomes, improves clinical practice, and prepares healthcare professionals to deliver high-quality care as they engage in real-world patient encounters and associated situations.

The next time you are designing or conducting simulation based education endeavors give careful consideration to the psychological contract!

Until next time, Happy Simulating!

Leave a comment

Filed under Curriculum, design, simulation

Tagged as , ,

May 26, 2023 · 1:17 pm

Simulation Professionals: Don’t let the Vocal Minority Get You Down!

The social psychologist Barbara Fredrickson coined the phrase, “The negative screams while the positive only whispers.” I don’t know about you, but this is extraordinarily true when reviewing course evaluations after simulation-based education programs!

Post-course evaluations are essential in measuring the program’s effectiveness and participant perceptions and are a tool to help with quality improvement initiatives. However, the feedback from vocal minorities can sometimes overshadow the opinions of the silent majority. After pouring blood, sweat, and tears into creating what you believe to be a successful simulation-based program, it can sometimes be a blow to your motivation when you receive negative evaluations.  At times the feedback can be pithy and personal and can sting.

Receiving negative feedback can be challenging for many reasons. First and foremost, it can feel like a personal attack on the hard work and effort you’ve put into a project or program. It’s natural to feel defensive or upset when someone criticizes something you’ve put so much time and energy into creating. Additionally, negative feedback can be difficult to process and use constructively. It’s easy to get caught up in the moment’s emotions and feel overwhelmed by the criticism. This can make it difficult to see the feedback as an opportunity for growth and improvement rather than a setback or failure.

This can be problematic as the feedback may not accurately represent the actual experiences of most participants, but it can certainly feel that way. It is also important to recognize the opportunities that come with critical feedback that could help you improve your program. It can help educators and course designers to identify areas for improvement and develop strategies for addressing these areas. Particularly when it is delivered constructively, and with a focus on improvement, negative feedback can be a powerful tool for enhancing the quality of simulation-based education programs and developing resilience in educators and learners alike. Critical feedback can help to identify areas for improvement, develop new strategies, and implement changes that can benefit future participants.

It is also important to remember that most participants with positive experiences may not feel the need to provide feedback. In contrast, those who have negative experiences may be more inclined to do so. So, I challenge you to go back and look at the designs of your course evaluation tools. It’s important to remember that the silent majority can be an important ally in the success of your program. By actively seeking out their feedback and insights, you can ensure that your program is meeting the needs of all participants, not just the most vocal. I’m not suggesting that we ignore the critical feedback; we just must find a way to balance it into a healthy model that contributes to resilience.

Developing a growth mindset is essential for developing resilience for those running simulation programs. It involves embracing challenges and staying motivated even when things get tough. Instead of seeing failures and setbacks as signs of inadequacy, individuals with a healthy mindset view them as opportunities for growth and learning. One powerful tool I use is remaining patient-centric in the decisions made regarding our simulations. Thinking about the downstream benefits that help raise the quality-of-care patients receive because of our efforts helps to keep my eye on the ball.

Lastly, remember that we can’t be all things to all people. While we remain excited and recognize the power of simulation-based education, not everyone will share our enthusiasm. As we move forward, remember that we can learn from the naysayers and the people unhappy that they are required to participate in some of our programs. Try to avoid the negative screaming in your ear, and you mistakenly believe that it represents the majority opinion. Stay focused on the idea that patients will benefit from our efforts, and many participants likely perceive value from our efforts.

Leave a comment

Filed under Curriculum, simulation

Tagged as , , ,

April 25, 2023 · 12:22 pm

When Simulation Is NOT the Answer: Own It!

Obviously, we are happy that simulation has become a popular method of education in healthcare. Simulation can provide a hands-on approach to learning that allows participants to experience real-life situations in a safe and controlled environment.

However, while simulation has many benefits, it’s not necessarily the best option for every type of education.  When we engage simulation as a modality, it is relatively complex, expensive and resource intensive compared to other educational methodologies. That all being said we all know at times it is an irreplaceable methodology that allows education, competency assessment, as well as system assessment information to be utilized in the improvement of healthcare.  The key is to have a stratification process/policy in place to evaluate opportunities to decide when simulation is the optimal deployment tool.

As leaders and managers of simulation programs we are charged with creating the return on investment for our programs. We are entrusted by the people who provide our funding to be good stewards of the investment and ongoing operational support of the simulation efforts.  It is up to us to hold the keys to the vault that we call simulation so that it gets engaged, deployed and/or utilized in the fashion that generates the expected outcomes with the highest amount of efficiency and effectiveness.

In short, don’t simulate because you can, simulate because you need to!

As your simulation center becomes a more recognized resource within your institution, there will often be an increase in request for services.  As this occurs it is critically important that leaders of programs are ensuring that the simulations are bringing value. 

For example, if someone wants you to do simulation training for an entire unit to rule out a new simple policy or procedure change, do not just say yes.  Instead, create a framework that advises the requester if simulation is the best modality.

When contemplating the value of simulation as a modality, I think it is best to go back to the creation of learning objectives for anticipated scenarios.  I always like to say that if you do knowledge, skills, and attitudes (KSA) analysis of your learning objectives and they all come up with K’s, you should reevaluate whether simulation is the best method.

Web-based education including courses, videos, lectures, or assigned reading may accomplish the same objectives as your planned simulation.  If this is the case, as a leader in simulation it is important that you recognize this and recommend modalities other than simulation.  It will likely save your organization time and money.  More importantly, it may increase the credibility of your advice and reputation moving forward as a problem solver for the institution as well as someone who is fiscally responsible.  Over time it can be valuable for a simulation program to enjoy a reputation of “the solution deployment” expert, not simply the “simulation” expert.

It is important to remember that the true value we provide is in the end-result of creating higher quality healthcare along with a safer environment for patients.  In this day and age, it has become increasingly important that our engagement is thoughtful, prudent with cost considerations in mind.  While we are all passionate about simulation, leaders of the future will garner success through a lens of efficiency and effectiveness in the programs that we deploy.

In conclusion, healthcare simulation is an important tool for education and patient safety, but it is not always the best tool. Simulation program managers and leaders should consider the specific learning outcomes they hope to achieve and carefully consider which educational modality is most appropriate for their learners. By doing so, they can ensure that they are providing the best possible, most cost-efficient training for their staff and ultimately improving patient outcomes.

Remember: Don’t simulate because you can, simulate because you need to!

Let me know what you think in the comments! If you enjoyed this post, please let me know by liking it, or subscribing to my Blog!

Until next time,

Happy Simulating!

Leave a comment

Filed under Curriculum, return on investment, simulation

Tagged as , ,

January 19, 2023 · 9:00 am

Not Every Simulation Scenario Needs to Have a Diagnostic Mystery!

It is quite common to mistakenly believe that there needs to be a diagnostic mystery associated with a simulation scenario. This could not be further from the truth.

Sometimes it arises from our clinical hat being confused with our educator hat (meaning we let our view of the actual clinical environment become the driving factor in the design of the scenario.) We must carefully consider the learning objectives and what we want to accomplish. One of the powerful things about simulation is that we get to pick where we start and where we stop, as well as the information given or withheld during the scenario.

Let us take an example of an Inferior Wall Myocardial Infarction (IWMI). Let us imagine that we desire to assess a resident physician’s ability to manage the case. Notice I said to manage the case, not diagnose, then manage the case. This has important distinctions on how we would choose to begin the scenario. If the objectives were to diagnose and manage, we might start the case with a person complaining of undifferentiated chest pain and have the participant work towards the diagnosis and then demonstrate the treatment. Elsewise, if we were looking to have them only demonstrate proficiency in the management of the case, we may hand them an EKG showing an IMWI (or maybe not even hand them the EKG) and start the case by saying, “your patient is having an IWMI” and direct them to start the care.  

What is the difference? Does it matter?

In the former example of starting the case, the participant has to work through the diagnostic conundrum of undifferentiated chest pain to come up with the diagnosis of IWMI. Further, it is possible that the participant does not arrive at the proper diagnosis, in which case you would not be able to observe and assess them in the management of the case. Thus, your learning objectives have become dependent on one another. By the way, there’s nothing wrong with this as long as it is intended. We tend to set up cases like this because that is the way that the sequencing would happen in the actual clinical environment (our clinical hat interfering). However, this takes up valuable minutes of simulation, which are expensive and should be planned judiciously. So, my underlying point is if you deliberately are creating the scenario to see the diagnostic reasoning and treatment, then the former approach would be appropriate.

The latter approach, however, should be able to accomplish the learning objective associated with demonstrating the management of the patient. Thus, if that is truly the intended learning objective, the case should be fast-forwarded to eliminate the diagnostic reasoning portion of the scenario. Not only will this save valuable simulation time it will also conceivably lead to more time to carefully evaluate the treatment steps associated with managing the patient. Additionally, it will eliminate the potential of prolonged simulation periods that do not contribute to accomplishing the learning objectives and/or get stuck because of a failure to achieve the initial objective (in this case, for example, the diagnosis.)

So, the next time you make decisions in the scenario’s design, take a breath and ask yourself, “Am I designing it this way because this is the way we always do it? Am I designing it this way because this is the way it appears in the real clinical environment?”

The important point is that one is asking themselves, “How can I stratify my design decisions so that the scenario is best crafted to accomplish the intended learning objectives?” If you do, you will be on the road to designing scenarios that are efficient and effective!

Leave a comment

Filed under scenario design, simulation

Tagged as , ,

August 30, 2022 · 10:40 am

Sherlock Holmes and the Students of Simulation

I want to make a comparison between Sherlock Holmes and the students of our simulations! It has important implications for our scenario design process. When you think about it, there’s hypervigilance amongst our students, looking for clues during the simulation. They are doing so to figure out what we want them to do. Analyzing such clues is like the venerable detective Sherlock Holmes’s processes when investigating a crime.

Video version of this post

This has important implications for our scenario design work because many times, we get confused with the idea that our job is to create reality when in fact, it is not that at all our job. As simulation experts, our jobs are to create an environment with the reality that is sufficient to allow a student to progress through various aspects of the provision of health care. We need to be able to make a judgment and say, “hey, they need some work in this area,” and “hey, they’re doing good in this area.”

To accomplish this, we create facsimiles of what they will experience in the actual clinical environment transported into the simulated environment to help them adjust their mindset so they can progress down the pathway of taking care of those (simulated) patient encounters.

We must be mindful that during the simulated environment, people engage their best Sherlock Holmes, and as the famous song goes, [they are] “looking for clues at the scene of the crime.”
Let’s explore this more practically.

Suppose I am working in the emergency department, and I walk into the room and see a knife sitting on the tray table next to a patient. In that case, I immediately think, “wow, somebody didn’t clean this room up after the last patient, and there’s a knife on the tray. I would probably apologize about it to the patient and their family.”

Fast forward…..

Put me into a simulation as a participant, and I walk into the room. I see the knife on the tray next to the patient’s bed, and I immediately think, “Ah, I’m probably going to do a crich or some invasive procedure on this patient.”

How does that translate to our scenario design work? We must be mindful that the students of our simulations are always hypervigilant and always looking for these clues. Sometimes when we have things included in the simulation, we might just have there as window dressing or to try to (re)create some reality. However, stop to think they can be misinterpreted as necessary to be incorporated into the simulation by the student for success in their analysis.

Suddenly, the student sees this thing sitting on the table, so they think it is essential for them to use it in the simulation, and now they are using it, and the simulation is going off the tracks! As the instructor, you’re saying that what happened is not what was supposed to happen!

At times we must be able to objectively go back and look at the scenario design process and recognize maybe just maybe something we did in the design of the scenario, which includes the setup of the environment, that misled the participant(s). If we see multiple students making the same mistakes, we must go back and analyze our scenario design. I like to call it noise when we put extra things into the simulation scenario design. It’s noise, and the potential for that noise to blow up and drive the simulation off the tracks goes up exponentially with every component we include in the space. Be mindful of this and be aware of the hypervigilance associated with students undergoing simulation.

We can negate some of these things by a good orientation, by incorporating the good practice into our simulation scenario design so that we’re only including items in the room that are germane to accomplishing the learning objectives.

Tip: If you see the same mistakes happening again and again, please introspect, go back, look at the design of your simulation scenario, and recognize there could be a flaw! Who finds such flaws in the story?  Sherlock Holmes, that’s who!

1 Comment

Filed under Curriculum, design, scenario design, simulation

Tagged as , ,

June 6, 2022 · 12:05 pm

5 Tips to Improve Interrater Reliability During Healthcare Simulation Assessments

One of the most important concepts in simulation-based assessment is achieving reliability, and specifically interrater reliability. While I have discussed previously in this blog every simulation is assessment, in this article I am speaking of the type of simulation assessment that requires one or more raters to record data associated with the performance or more specifically an assessment tool.

Interpreter reliability simply put is that if we have multiple raters watching a simulation and using a scoring rubric or tool, that they will produce similar scores. Achieving intermittent reliability is important for several reasons including that we are usually using more than one rater to evaluate simulations over time. Other times we are engaged in research and other high stakes reasons to complete assessment tools and want to be certain that we are reaching correct conclusions.

Improving assessment capabilities for stimulation requires a significant amount of effort. The amount of time and effort that can go into the assessment process should be directly proportional to the stakes of the assessment.

In this article I offer five tips to consider for improving into rate of reliability when conducting simulation-based assessment

1 – Train Your Raters

The most basic and overlooked aspect of achieving into rate and reliability comes from training of the raters. The raters need to be trained to the process, the assessment tools, and each item of the assessment that they are rendering an opinion on. It is tempting to think of subject matter experts as knowledgeable enough to fill out simple assessments however you will find out with detailed testing that often the scoring of the item is truly in the eye of the beholder. Simple items like “asked medical history” may be difficult to achieve reliability if not defined prior to the assessment activity. Other things may affect the assessment that require rater calibration/training such as limitations of the simulation, and how something is being simulated and/or overall familiarity with the technology that may be used to collect the data.

2 – Modify Your Assessment Tool

Modifications to the assessment tool can enhance interrelated reliability. Sometimes it can be extreme as having to remove an assessment item because you figure out that you are unable to achieve reliability despite iterative attempts at improvement. Other less drastic changes can come in the form of clarifying the text directives that are associated with the item. Sometimes removing qualitative wording such as “appropriately” or “correctly” can help to improve reliability. Adding descriptors of expected behavior or behaviorally anchored statements to items can help to improve reliability. However, these modifications and qualifying statements should also be addressed in the training of the raters as described above.

3 – Make Things Assessable (Scenario Design)

An often-overlooked factor that can help to improve indurated reliability is make modifications to the simulation scenario to allow things to be more “assessable”. We make a sizable number of decisions when creating simulation-based scenarios for education purposes. There are other decisions and functions that can be designed into the scenario to allow assessments to be more accurate and reliable. For example, if we want to know if someone correctly interpreted wheezing in the lung sounds of the simulator, we introduced design elements in the scenario that could help us to gather this information accurately and thus increase into rater reliability. For example, we could embed a person in the scenario to play the role of another healthcare provider that simply asks the participant what they heard. Alternatively, we could have the participant fill out a questionnaire at the end of the scenario, or even complete an assessment form regarding the simulation encounter. Lastly, we could embed the assessment tool into the debriefing process and simply ask the participant during the debriefing what they heard when I auscultated the lungs. There is no correct way to do this, I am trying to articulate different solutions to the same problem that could represent solutions based on the context of your scenario design.

4 – Assessment Tool Technology

Gathering assessment data electronically can help significantly. When compared to a paper and pencil collection scheme technology enhanced or “smart” scoring systems can assist. For example, if there are many items on a paper scoring tool the page can sometimes become unwieldy to monitor. Electronic systems can continuously update and filter out data that does not need to be displayed at a given point in time during the unfolding of the simulation assessment. Simply having previously evaluated items disappear off the screen can reduce the clutter associated with scoring tools.

5 – Consider Video Scoring

For high stakes assessment and research purposes it is often wise to consider video scoring. High stakes meaning pass/fail criteria associated with advancement in a program, heavy weighting of a grade, licensure, or practice decisions. The ability to add multiple camera angles as well as the functionality to rewind and play back things that occurred during the simulation are valuable in improving the scoring accuracy of the collected data which will subsequently improve the interrater reliability. Video scoring associated with assessments requires considerable time and effort and thus reserved for the times when it is necessary.

I hope that you found these tips useful. Assessment during simulations can be an important part of improving the quality and safety of patient care!

If you found this post useful please consider subscribing to this blog!

Thanks and until next time! Happy Simulating.

Leave a comment

Filed under assessment, Curriculum, design, scenario design

Tagged as , , ,

November 24, 2021 · 3:21 pm

Adjuncts to Enhance Debriefing

I wanted to discuss some ideas of using adjuncts as part of your debriefing.

When we think about debriefing, we often think about a conversation between faculty member or members and participants of simulation with a focus on everyone developing an understanding of what they did right as well as what they need to improve upon.  We rarely think about the possibility of including other “things” to enhance the learning that comes from the debriefing.

I tend to incorporate adjuncts into a many of the debriefings associated with courses that I design.  What I mean is things that added into the debriefing process/environment that can enhance the discussion.  Sometimes with deliberate purpose, and other times just to mix it up a little bit so that it is not just a dialogue between the participants and the faculty.  It may be something technical, it may be something as simple as a paper handout.

Simple Task Trainer as an Adjunct

Some ideas of adjuncts include PowerPoint slide deck or a few targeted slides that help to review a complex topic, one that requires a deeper understanding, or a subject that benefits from repetition of exposure.  Another type of adjunct is the simulator log file which can help set the stage for the debriefing and create a pathway of discussion that chronologically follows what happened during the simulation.  Another adjunct could be a partial task trainer or a model that helps to describe or demonstrate something.  For example, the students forgot to do a jaw-thrust or open the airway.  We can use a task trainer, or a teaching aide incorporated into the discussion during the debriefing.  

Example of an Algorithm Poster on the Wall

Other things that I use are charts, graphs, and algorithms that may represent best practices.  When I debrief during my difficult airway management course for physicians, I have the algorithm up on the wall hanging as a poster.  We use the algorithm posters as a pathway to compare the performance of the participants of the simulation with what the ideal case would be.  You can use the adjunct learning aid as a reference to standards.  This can help you to take yourself out of the direct argument of right vs. wrong.  This allows use of the adjunct as a third-party messenger of a reference to best practices excellence when I have the participants compare their performance against what appears on the algorithm.  This allows them to discover their own variations from the expected standard.  It tends to create powerful learning moments without the faculty having to be “the bearer of bad news!”

I think that if you start to strategically think about how to incorporate adjuncts into your debriefing you will find the students are more satisfied with the debriefing.  It also increases the stickiness of the learning and creates a more enjoyable experience for the faculty member as well as the participants.  Try it!  It does not have to be fancy!

Thanks, and as always,

Happy Simulating!

Leave a comment

Filed under Curriculum, debriefing

Tagged as , ,

September 21, 2021 · 4:28 pm

Five Tips for Creating Hybrid Curricula for Simulation Based Learning

For the purposes of this discussion, we will assume that hybrid curriculums in simulation combine online educational materials in advance of on-site activities involving (in person) simulation into one curriculum.  

Why Hybrid?

There are things that we want the student to obtain knowledge on from a perspective of knowing things, or cognitively loading, for an upcoming education event. This often lends itself to carefully created on-line course work.

The in-person side of the equation is best used for when we want to see people doing things, particularly doing things with an understanding of the knowledge that they had already studied during the pre-work described above.  Combining these two facets, or hybrid learning, are some of the most efficient and effective designs for simulation programs.

Accompanying Video Discussion

Advantages

It allows students to be fully prepared from a knowledge perspective before the simulation encounters. This will allow you to conduct your simulation encounters at a much higher level by “raising the tide” of the knowledge of the learners in advance.  Such a design can potentially reduce unnecessary (costly) time in the simulation center. It also allows for students to assimilate the knowledge portion of your program at their own pace on their own time. Further, it helps to set the expectations of what the learners will need to incorporate when they participate in the simulations. Conducting the online portion as pre-work allows the student to seek out additional instruction mediums to help enhance their knowledge base understanding of the materials.

Disadvantages

Curriculum planning will require more effort. It’s more complicated than just deploying a simulation or just creating online education in isolation. You’re doing both! Combining the two which means that there is a time investment in creating the online materials that didn’t exist before we decided to move into a hybrid curriculum. There may be additional skills or resources needed associated with the creation of the materials and/or the administration of some sort of learning management system to make the online curriculum available to your learner population.

Students may not do the work online and prepare like they should before they come to your simulation center. Thus, you need to consider building incentives into the program that creates a compelling reason to do the work.

Tip 1: Begin with the End in Mind

Start with a detailed list of exactly what we want them to know and exactly what they want them to do. Yes, folks it is creating learning objectives, just like we’re designing simulations. Then carefully decide what is knowledge, what is skills and what is application of skills to help parse out which of the curriculum can benefit from on-line (pre) learning.  

Tip 2: Create High Quality Learning Materials

You want your students to take the online materials seriously. So, it is important to ensure they are of high quality, contribute to the learning, and not distracting. Not everything in your pre-learning needs to be Hollywood quality. Many people now do cell phone or mobile phone videos, and that’s fine! However, I want to caution you on the audio. You must make the audio or sound as good as the picture looks. If not, it is distracting, and your students may not take things seriously.

Tip 3: Create Active Learning for the Pre-Course Material

Try to create components of active learning in your online materials. Just because it’s online material and delivered asynchronously doesn’t mean there can’t be an active component. Resist the urge to simply regurgitate one of your old lectures and then toss it up online!

Find small opportunities to have them DO something. It might be as simple as asking them to write out a list of the steps of a procedure, drawing a diagram that they see on the screen, or maybe connecting social media so that they are communicating and learning from and/or with their peers. Lastly, having them taking an on-line assessment or quiz can serve as an effective tool.  

Tip 4: Ensure Learner Expectations and Consequences are Clear

Make sure your learners are clear on their responsibilities associated with the completion of the online materials, and what the consequences are if they don’t. Additionally, ensure the learners understand how the pre-course content is linked to the expectations that will be encountered when they arrive for the simulation sessions.

Some design examples include having the learners take a written pretest when they arrive at the simulation center and determine whether they have adequately prepared for the simulation or not. Other examples make it clear that they will be called on and expected to know the answers for the content contained in the pre-course materials.  It is important that we are fair to the student, with hybrid education, we need to ensure that the learner expectations and consequences are very clear.

TIP 5: Link Your Online Materials Directly to Your Simulations

Work to create an integrated continuum of learning that carries forward from the online materials through the expectations that the learners will encounter during the simulations. This can be emphasized through the direct inclusion of online materials into your simulation sessions. 

Consider including exact diagrams, exact pictures, exact phrases and themes utilized during the online learning during your face-to-face instruction. It might be in the form of a mini lecture. It might be audio/visuals that are incorporated during the debriefing process that can trigger in their mind the lessons that were learned from the online material and how it’s being applied to the simulation session learning outcomes.

Conclusions

The words online and hybrid can cause educators to become nervous because of the amount of work that’s involved as well as not understanding how to make those linkages between the pre-course materials and the simulation sessions. Admittedly, it is more work, but I would argue that the outcomes are far superior then either modality alone. Think of it as an investment. Things that can be moved to the online portion of hybrid design can prepare the learners so the valuable on-site time with the faculty can be conducted at a higher level.

I think that by incorporating good hybrid design with these tips, you will find that you will be creating exceptional learning environments for your students.

Until the next time, happy simulating!

Leave a comment

Filed under Uncategorized

Tagged as ,