Category Archives: scenario design

January 19, 2023 · 9:00 am

Not Every Simulation Scenario Needs to Have a Diagnostic Mystery!

It is quite common to mistakenly believe that there needs to be a diagnostic mystery associated with a simulation scenario. This could not be further from the truth.

Sometimes it arises from our clinical hat being confused with our educator hat (meaning we let our view of the actual clinical environment become the driving factor in the design of the scenario.) We must carefully consider the learning objectives and what we want to accomplish. One of the powerful things about simulation is that we get to pick where we start and where we stop, as well as the information given or withheld during the scenario.

Let us take an example of an Inferior Wall Myocardial Infarction (IWMI). Let us imagine that we desire to assess a resident physician’s ability to manage the case. Notice I said to manage the case, not diagnose, then manage the case. This has important distinctions on how we would choose to begin the scenario. If the objectives were to diagnose and manage, we might start the case with a person complaining of undifferentiated chest pain and have the participant work towards the diagnosis and then demonstrate the treatment. Elsewise, if we were looking to have them only demonstrate proficiency in the management of the case, we may hand them an EKG showing an IMWI (or maybe not even hand them the EKG) and start the case by saying, “your patient is having an IWMI” and direct them to start the care.  

What is the difference? Does it matter?

In the former example of starting the case, the participant has to work through the diagnostic conundrum of undifferentiated chest pain to come up with the diagnosis of IWMI. Further, it is possible that the participant does not arrive at the proper diagnosis, in which case you would not be able to observe and assess them in the management of the case. Thus, your learning objectives have become dependent on one another. By the way, there’s nothing wrong with this as long as it is intended. We tend to set up cases like this because that is the way that the sequencing would happen in the actual clinical environment (our clinical hat interfering). However, this takes up valuable minutes of simulation, which are expensive and should be planned judiciously. So, my underlying point is if you deliberately are creating the scenario to see the diagnostic reasoning and treatment, then the former approach would be appropriate.

The latter approach, however, should be able to accomplish the learning objective associated with demonstrating the management of the patient. Thus, if that is truly the intended learning objective, the case should be fast-forwarded to eliminate the diagnostic reasoning portion of the scenario. Not only will this save valuable simulation time it will also conceivably lead to more time to carefully evaluate the treatment steps associated with managing the patient. Additionally, it will eliminate the potential of prolonged simulation periods that do not contribute to accomplishing the learning objectives and/or get stuck because of a failure to achieve the initial objective (in this case, for example, the diagnosis.)

So, the next time you make decisions in the scenario’s design, take a breath and ask yourself, “Am I designing it this way because this is the way we always do it? Am I designing it this way because this is the way it appears in the real clinical environment?”

The important point is that one is asking themselves, “How can I stratify my design decisions so that the scenario is best crafted to accomplish the intended learning objectives?” If you do, you will be on the road to designing scenarios that are efficient and effective!

Leave a comment

Filed under scenario design, simulation

Tagged as , ,

August 30, 2022 · 10:40 am

Sherlock Holmes and the Students of Simulation

I want to make a comparison between Sherlock Holmes and the students of our simulations! It has important implications for our scenario design process. When you think about it, there’s hypervigilance amongst our students, looking for clues during the simulation. They are doing so to figure out what we want them to do. Analyzing such clues is like the venerable detective Sherlock Holmes’s processes when investigating a crime.

Video version of this post

This has important implications for our scenario design work because many times, we get confused with the idea that our job is to create reality when in fact, it is not that at all our job. As simulation experts, our jobs are to create an environment with the reality that is sufficient to allow a student to progress through various aspects of the provision of health care. We need to be able to make a judgment and say, “hey, they need some work in this area,” and “hey, they’re doing good in this area.”

To accomplish this, we create facsimiles of what they will experience in the actual clinical environment transported into the simulated environment to help them adjust their mindset so they can progress down the pathway of taking care of those (simulated) patient encounters.

We must be mindful that during the simulated environment, people engage their best Sherlock Holmes, and as the famous song goes, [they are] “looking for clues at the scene of the crime.”
Let’s explore this more practically.

Suppose I am working in the emergency department, and I walk into the room and see a knife sitting on the tray table next to a patient. In that case, I immediately think, “wow, somebody didn’t clean this room up after the last patient, and there’s a knife on the tray. I would probably apologize about it to the patient and their family.”

Fast forward…..

Put me into a simulation as a participant, and I walk into the room. I see the knife on the tray next to the patient’s bed, and I immediately think, “Ah, I’m probably going to do a crich or some invasive procedure on this patient.”

How does that translate to our scenario design work? We must be mindful that the students of our simulations are always hypervigilant and always looking for these clues. Sometimes when we have things included in the simulation, we might just have there as window dressing or to try to (re)create some reality. However, stop to think they can be misinterpreted as necessary to be incorporated into the simulation by the student for success in their analysis.

Suddenly, the student sees this thing sitting on the table, so they think it is essential for them to use it in the simulation, and now they are using it, and the simulation is going off the tracks! As the instructor, you’re saying that what happened is not what was supposed to happen!

At times we must be able to objectively go back and look at the scenario design process and recognize maybe just maybe something we did in the design of the scenario, which includes the setup of the environment, that misled the participant(s). If we see multiple students making the same mistakes, we must go back and analyze our scenario design. I like to call it noise when we put extra things into the simulation scenario design. It’s noise, and the potential for that noise to blow up and drive the simulation off the tracks goes up exponentially with every component we include in the space. Be mindful of this and be aware of the hypervigilance associated with students undergoing simulation.

We can negate some of these things by a good orientation, by incorporating the good practice into our simulation scenario design so that we’re only including items in the room that are germane to accomplishing the learning objectives.

Tip: If you see the same mistakes happening again and again, please introspect, go back, look at the design of your simulation scenario, and recognize there could be a flaw! Who finds such flaws in the story?  Sherlock Holmes, that’s who!

1 Comment

Filed under Curriculum, design, scenario design, simulation

Tagged as , ,

June 6, 2022 · 12:05 pm

5 Tips to Improve Interrater Reliability During Healthcare Simulation Assessments

One of the most important concepts in simulation-based assessment is achieving reliability, and specifically interrater reliability. While I have discussed previously in this blog every simulation is assessment, in this article I am speaking of the type of simulation assessment that requires one or more raters to record data associated with the performance or more specifically an assessment tool.

Interpreter reliability simply put is that if we have multiple raters watching a simulation and using a scoring rubric or tool, that they will produce similar scores. Achieving intermittent reliability is important for several reasons including that we are usually using more than one rater to evaluate simulations over time. Other times we are engaged in research and other high stakes reasons to complete assessment tools and want to be certain that we are reaching correct conclusions.

Improving assessment capabilities for stimulation requires a significant amount of effort. The amount of time and effort that can go into the assessment process should be directly proportional to the stakes of the assessment.

In this article I offer five tips to consider for improving into rate of reliability when conducting simulation-based assessment

1 – Train Your Raters

The most basic and overlooked aspect of achieving into rate and reliability comes from training of the raters. The raters need to be trained to the process, the assessment tools, and each item of the assessment that they are rendering an opinion on. It is tempting to think of subject matter experts as knowledgeable enough to fill out simple assessments however you will find out with detailed testing that often the scoring of the item is truly in the eye of the beholder. Simple items like “asked medical history” may be difficult to achieve reliability if not defined prior to the assessment activity. Other things may affect the assessment that require rater calibration/training such as limitations of the simulation, and how something is being simulated and/or overall familiarity with the technology that may be used to collect the data.

2 – Modify Your Assessment Tool

Modifications to the assessment tool can enhance interrelated reliability. Sometimes it can be extreme as having to remove an assessment item because you figure out that you are unable to achieve reliability despite iterative attempts at improvement. Other less drastic changes can come in the form of clarifying the text directives that are associated with the item. Sometimes removing qualitative wording such as “appropriately” or “correctly” can help to improve reliability. Adding descriptors of expected behavior or behaviorally anchored statements to items can help to improve reliability. However, these modifications and qualifying statements should also be addressed in the training of the raters as described above.

3 – Make Things Assessable (Scenario Design)

An often-overlooked factor that can help to improve indurated reliability is make modifications to the simulation scenario to allow things to be more “assessable”. We make a sizable number of decisions when creating simulation-based scenarios for education purposes. There are other decisions and functions that can be designed into the scenario to allow assessments to be more accurate and reliable. For example, if we want to know if someone correctly interpreted wheezing in the lung sounds of the simulator, we introduced design elements in the scenario that could help us to gather this information accurately and thus increase into rater reliability. For example, we could embed a person in the scenario to play the role of another healthcare provider that simply asks the participant what they heard. Alternatively, we could have the participant fill out a questionnaire at the end of the scenario, or even complete an assessment form regarding the simulation encounter. Lastly, we could embed the assessment tool into the debriefing process and simply ask the participant during the debriefing what they heard when I auscultated the lungs. There is no correct way to do this, I am trying to articulate different solutions to the same problem that could represent solutions based on the context of your scenario design.

4 – Assessment Tool Technology

Gathering assessment data electronically can help significantly. When compared to a paper and pencil collection scheme technology enhanced or “smart” scoring systems can assist. For example, if there are many items on a paper scoring tool the page can sometimes become unwieldy to monitor. Electronic systems can continuously update and filter out data that does not need to be displayed at a given point in time during the unfolding of the simulation assessment. Simply having previously evaluated items disappear off the screen can reduce the clutter associated with scoring tools.

5 – Consider Video Scoring

For high stakes assessment and research purposes it is often wise to consider video scoring. High stakes meaning pass/fail criteria associated with advancement in a program, heavy weighting of a grade, licensure, or practice decisions. The ability to add multiple camera angles as well as the functionality to rewind and play back things that occurred during the simulation are valuable in improving the scoring accuracy of the collected data which will subsequently improve the interrater reliability. Video scoring associated with assessments requires considerable time and effort and thus reserved for the times when it is necessary.

I hope that you found these tips useful. Assessment during simulations can be an important part of improving the quality and safety of patient care!

If you found this post useful please consider subscribing to this blog!

Thanks and until next time! Happy Simulating.

Leave a comment

Filed under assessment, Curriculum, design, scenario design

Tagged as , , ,

July 7, 2020 · 2:57 pm

Cognitive Load Control and Scenario Design in Healthcare Simulation

As the design architects of simulation scenarios, we must remain cognizant of our ability to have influence over the cognitive load of those experiencing our simulations in the role of learners.

When caring for patients in real life, we expend cognitive energy in doing so to ensure we make the right decisions to provide the absolute best care for every patient. We engage in critical thought processes, that guide our interpretation of the enormous number of facts surrounding each patient so we can make further decisions to provide various therapies, or advice to the patient.

Headache brain in a clamp isolated grey background

When we design simulations for our learners, we are creating similar environments noted above that demand a significant amount of cognitive workload to be endured for the participant to successfully navigate the case and care the [simulated] patient. In addition, I argue that we are adding additional cognitive workload by subjecting someone to the simulated environment insofar as they are engaged in a conscious or perhaps subconscious pursuit of deciding what is simulated and what is not. I have previously written about this and dubbed it the cognitive third space of simulation.

Nonetheless, there is mental energy spent in the care of the patient as well as the interpretation of the simulation. We also must realize that our design choices inside of the scenario contribute to the adjustment of the cognitive load endured by the learner(s) associated with our simulations. It is important that we be deliberate in our design to ensure that we are allowing all involved to achieve the desired learning outcomes.

Some specific examples of this cognitive load influence may help to bring forth an understanding. Take a test result for example. If one looks in the electronic health record and sees the values reported for a simple test, like a basic metabolic profile (which consists of a sodium, chloride, potassium, CO2, BUN, creatinine and glucose) there is a certain amount of mental energy goes into the interpretation of the numeric data presented for each of the seven items of the basic metabolic profile. Some electronic health records may color-code the results to assist in the processing of normal versus normal, and some may not.

Such a decision involved in the human factors design of electronic health record actually influences the amount of cognitive spend on the interpretation of the given value. Further, as experienced clinicians are keenly aware, we must interpret the lab value in the context of the patient for whom the test has been ordered. What is normal for one patient, may not be normal for another. Thus, even in the interpretation of a simple test, there is a significant amount of cognitive process (critical thought) that should be applied.

How does this relate to simulation scenario design? We have the ability to engineer the scenario design to help the participants channel cognitive energy into those things that are important and away from those those things that are not. If we continue to run with the example of the basic metabolic profile as an example, we have choices on how said values are reported to the participants of our simulation.

We could have the participants look it up in the simulated electronic health record which takes time and cognitive processing as described above. We could give them a piece of paper or display the results on a screen demonstrating the seven values. This still takes significant cognitive processing to interpret the data. We could simply indicate that the basic metabolic profile result was “normal”.  This method significantly decreases the cognitive processing associated with the seven values of the basic metabolic profile and how it is to be interpreted into the context of the scenario. Also, one could make the argument that we are offering subtle, or perhaps not-so-subtle clues to the case that the basic metabolic profile is not a major part of what needs to be processed in the care of this particular patient.  

It is important to realize that all the examples above are viable options and there is not one that is superior to another. It is important that the decision is made during the design of the case that allows the participant(s) of the scenario to focus the appropriate cognitive spend on that which the designers of the scenario feel are most important. In other words, if it is part of the learning objectives that the participant should evaluate the actual values of the basic metabolic profile, then of course it would be appropriate to provide the requisite information at that level of detail. If, however, the results of the same test are perfunctory to the bigger picture of the case then one should consider a different mechanism of resulting values to the simulation participant.

A common misperception in the design of healthcare simulation scenarios is to try to re-create the realistic environment of the clinical atmosphere. While this is always a tempting choice, it is not without consequences. It comes from the mistaken belief that the goal of simulation scenarios is to re-create reality. Modern, successful simulationists need to recognize this outmoded, immature thought process.

In the context of a case where the basic metabolic profile is not significantly important that we should not design the “dance” (scenario) to include the steps of looking in the electronic health record and making determinations of the values associated with the test. It is a waste of time, and more importantly a waste of cognitive processing which is already artificially increased by the participant being involved in the simulation in the first place. It is in my opinion a violation of the learner contract between faculty and students.

While I am focusing on a simple example of a single test, I hope that you can imagine that this concept extrapolates to many, many decisions that are made in the scenario design process. For example, think about a chest x-ray. Do you result a chest x-ray as “normal”, “abnormal” or otherwise during the run time of the scenario? Or do you show an image of a chest x-ray and have your participants interpret the image? One answer is not superior to the other. It is just critically important that you evaluate what is best for the cognitive load of the learners involved in your scenario and how the decision relates to the details of the learning objectives you wish to achieve during the course of the simulation activity.

In moderate to complex cases associated with healthcare simulation the designer of the simulation, or architect, has a responsibility to craft the scenario to accomplish the learning objectives that are intended. In many scenarios, hundreds of decisions are made in terms of how participants extract data from the experience to incorporate into their performance of the simulation. It is critically important that as the designers of such learning events that we remain cognizant of the cognitive load placed upon our learner(s) that is associated with the normal care of patients, as well as the extra that is imposed upon them from participating in a simulation-based case.

Many of the decisions that we incorporate into the design of our scenarios have significant influence over this cognitive load, and the mental energy participants will spend to engage in the participation. We need to understand the impact of our choices and be deliberate with our design decisions to enhance the overall simulation-based learning process efficiency and effectiveness.

Leave a comment

Filed under Curriculum, design, scenario design, simulation, Uncategorized

May 7, 2020 · 1:16 pm

Exploring the Elements of Orientation and (Pre)Briefing in Simulation Based Learning Design

AdobeStock_119412077

I want to explore a little bit about orientation and (pre)briefing(s) associated with simulation based education design concepts. The words are often tossed about somewhat indiscriminately. However it is important to realize they are both important elements of successful healthcare simulation and serve distinct purposes.

When we look in the Healthcare Simulation Dictionary, we find that the definition of Orientation is aligned with an overview preparation process including “… intent of preparing the participants.” Examples include center rules, timing and the simulation modalities.

On the other hand, according to the same dictionary the definition of the word Briefing includes “An activity immediately preceding the start of a simulation activity where participants receive essential information about the simulation scenario….”

I look at orientation as the rules of engagement. I like to think of orientation linked to the overall educational activity in total. Some essential components include orientation to the simulation center, the equipment, the rules, and the overall schedule for the learning activity.

At a somewhat deeper level of thought I think the orientation is linked to the learning contract. What do I mean by that?

I think it is essential that we as the faculty are establishing a relationship with our learners and begin to establish trust and mutual respect. To that end, we can use orientation to minimize surprises. Adult learners do not like surprises!

We need to have the adult learner understand what they can expect. I always orient the learners as to what will feel real, and I am similarly honest with them about what will not feel real. If they will be interacting with a computerized simulator for example, I orient them to the simulator before the start of the program.

In the simulation world we throw around words like debriefing, scenario and task training. To clinical learners these terms may be unfamiliar, or have different contexts associated with them. This for example, can cause anxiety and during the orientation we need to walk them through the experience they are about to embark upon.

Some factors can influence the amount and depth of the orientation. Variables such as the familiarity your participants have with simulation, your simulation center, and your simulation-based encounters. For example, learners who come to your center on the monthly basis probably need less total orientation than those who are reporting for the first time. Learners familiar with the fact that debriefings occur after every simulation may already be acclimated to that concept, but people coming to the sim center for the first time may not be aware of that at all.

Participants just meeting you for the first time they might need a little bit more warming up and that an come in the form of orientation. Overall though it is not just about telling them what’s going on, as it is using the opportunity toward earning their trust and confidence in the simulated learning encounter(s) and the value associated to them as a professional.

BriefingGraphic3Switching the focus to the brief, briefing or (pre)briefing. The briefing is more linked to the scenario as compared to the orientation. The briefing should focus on the details of the case at hand introducing components of information that allow one to acclimate to what they going to need to accomplish during the simulation. What is their role and goals in this scenario they are about to embark upon? If you are going to ask people to play different roles then they are in real life, it is very important that this fact is crystal clear in the briefing.

I think that the briefing should also bring the context to the healthcare experience. It is important to orient the learner for the impending encounter what they are to perceive and think of as real as they are experiencing what is in the simulation. You as a simulation faculty may think that it is obvious that a room in your simulation center is an ICU bed. The participant may not and deserves clarity prior to the start of the simulation so they do not feel like they are being tricked or duped. During the briefing the statement “You are about to see a patient in the ICU…..” can remove such ambiguity.

Another critical briefing point is to clarify the faculty-student engagement rules that should be expected during the scenario runtime if it was not covered in the orientation. There are many correct ways to conduct simulation scenarios. There are varying levels of interaction between faculty members running the simulation and the learners that are participating. This should be clarified before the scenario starts.

For example, are you going to let the learners ask questions of the of the faculty member during the simulation? Or not? This should be upfront and covered in the briefing, and perhaps even aspects of that in the orientation.

While not a requirement I think that parameters associated with time expectations are always good to give in a briefing. For example stating “You are going to have 10 minutes in the scenario to accomplish X,Y and Z, and then we will have a ten minute debriefing before the next scenario.”

Remember our adult learners don’t like surprises! I always use the briefing before a scenario to remind the participant(s) that afterward we are going to have a debriefing. I remind them of that so that they know that they should collect her thoughts and ideas and be ready to have this discussion. Secondly, I am saying in any unspoken way, that if they are uncomfortable about something, or have questions, that there will be an opportunity for discussion during the debriefing. (In other words, your sort of giving some control back to the learner…. Helping to build the trusting relationship.)

Some of the variations of the briefing are similar to that of the orientation mentioned above. People who are more familiar to simulation, your particular programs, your style, may require slightly less of a briefing than others. Additionally, if you are running multiple scenarios as part of a simulation-based course, after the first couple of scenarios you will find that the briefing can be shortened as compared to the beginning of the day.

So, in summary, orientation and briefings are different elements of simulation-based learning that are useful for different things that will contribute to the success of your simulations.

Think of orientation linked to the bigger picture and the learner contract that contributes to making the relationship comfortable between the participants and the faculty. The orientation is the rules of engagement and orientation to the technology and being explicit as to what is to be expected of the participant. Think of the briefing as linked more to the scenario roles, goals, and introduction to patient and environment information to help the participant mentally acclimate to what they are about to dive into.

Leave a comment

Filed under Curriculum, scenario design, simulation, Uncategorized

Tagged as ,

November 5, 2019 · 8:36 am

Simulation, Music, and Dancing

Many of you know of my crazy thoughts and ideas to try to connect things together with contrasts and comparisons to help people understand concepts and ideas. Well…. Here goes another one of them!Dance

I find that people continuously struggle with understanding the true relationship of the scenario (defined as the collective information, tools, and techniques that are presented to participants of simulations) to the outcomes of the simulation. The confusion arises from the fact that people get inappropriate messaging during the formative times of their simulation careers.

People gain the idea that the scenario must be as real as possible, or perfect mimic some aspect of real life in healthcare in order to be effective instead of recognition that the sole purpose is to create a script and stage that allows participants to perform. Some people believe that the overall goal of simulation is to recreate reality. The sad part is, those misguided thoughts often lead to over-production of the scenario and that the scenario is the primary focus of the activity. This can lead to the unintended consequences of increasing the workload of the simulation relative to the value of performance improvement and/or introduce confusion to the participants of the scenario.  Neither of which are desirable.

It occurred to me recently that a terrific analogy can be made by evaluating the relationship of music, to competitive dance. As it turns out the scenario is simply the music.

Thinks about it. When a dancer or group of dancers are going to compete, a number of things must be in place. First, there is an understanding that the dance will be carried out with the playing of music. The activity will last a certain length of time, involve one or more people who are supposed to do certain things at certain times and that various details will be assessed or evaluated along the way. At times the evaluation maybe be structured to focus on improvement (formative) and perhaps feedback is shared along the way (deliberate practice preparing for a competition), while other times may it may be a high-stakes evaluation (summative) resulting in only a score (the actual competition).

Now let’s focus on the music. What is its purpose in a dance competition? If you think about it, the music providers the framework or backdrop against which the dancing activity occurs. It helps to coordinate the tone, the tempo, and the activities associated with the dance. If the objective is to assess a pair of dancers doing a waltz, then a waltz is played. So the learning objective would read, at the conclusion of this five minute activity, the participants will demonstrate the ability to perform a waltz. If we wanted to evaluate a Latin dance, we would play Latin music and have an appropriate assessment criterion by which to guide the improvement of the activity.

While it is technically possible for the assessment to occur in the absence of the music, it would be awkward for the participants and the evaluators as well. Further, a piece of music may be specifically chosen to encourage a certain dance move that would facilitate the evaluation of the activity, let’s say a twirl or a flip. If we needed to evaluate or score how well one performed a flip, a flip would need to occur during the dance.

When using the methods of simulation in the healthcare world, we need to see people dance. The dance we need to see is often a complex one involving the delivery of healthcare, but it is a dance none the less involving specific movements, communications, and other activities toward a specific goal There are times that we need to see individuals dance, other times teams.

If we are to evaluate a certain element of healthcare, then we must have carefully composed the music that propagated the desired activity to have occurred during the dance. As they dance, we perform an assessment with a goal of helping them improve through various feedback mechanisms. Such feedback may occur through active reflection and facilitated discussion (debriefing), self-reflection, peer to peer engagement, or perhaps in the delivery of a more formal score in the case of summative feedback.

The bigger point is, the scenario is constructed and executed (composed) to provide the background milieu to form the basis of the dance, i.e. have participants perform the activity that we wish to assess. We choose different types [of music] to play that is concordant with the activity we wish to evaluate. At times we play a tune that accentuates the evaluation of critical thinking skills, perhaps the performance of a complex skill, or maybe one that allows a whole team to dance together requiring teamwork that will benefit from feedback.

So, the next time you are composing your scenario, give careful consideration to the moves that you desire to evaluate. The music that plays should allow/encourage your dancers to perform the steps and activities that will be evaluated and turned into useful information to facilitate improvement.

Compose, have people dance and help them get better!

If you enjoy or find my musings helpful, please sign up for my blog!

Leave a comment

Filed under Curriculum, scenario design, Uncategorized

Tagged as ,

July 23, 2018 · 11:08 am

The First Four Steps of Healthcare Simulation Scenario Design

How can you make your scenario design process more consistent and efficient? One way is by following a step-by-step method to create your masterpieces!

In this post I cover the first four steps of a proven scenario design process.
There are four core steps that must be done in order. After the first four are accomplished you can branch out and be a little bit more variable in your approach to scenario design.

4 Success Steps, business concept

Step One: Pick A Topic

Picking a topic may seem like common sense but there is a lot to think about.

In healthcare simulation we have many topics to choose from. But in step one we want to a little bit specific and figure out that the major topic is that will be covered. We may be cover the teaching of physiologic, diagnostic or treatment where people are going to be making critical decisions, ordering medications, and other therapy, or perhaps our primary focus going to be on team training, teamwork, communications, team leadership. You get to pick!

Step Two: Define the Learner(s)

This is really important because in order to go to the next step which is designing the learning objectives we have to understand our learner population. For example, what do you expect of a fourth-year medical student what you expected them in terms of being able to evaluate and treat a simulated patient that is complaining of chest pain? Now contrast that to if your learners are medical students that are in the second year of medical school and haven’t had any clinical experience. In other words, we can take the same topic but as applied to two different populations, our expectations and what we are going to be evaluating from them is very different.

Step Three: Designing  the Learning Objectives

This is where you want to go into detail, great painstaking detail, about what you’re trying to accomplish with the simulation scenario. It is very important to take time on this step. Many people tend to gloss over this step which can create confusion later.

Let’s take a topic example. Let’s say asthma in the emergency department. When you think about asthma in the emergency department there could be many sub topics or areas from which to choose. It could be focused on competence of managing a minor asthma attack, or it could be a first-ever asthma attack, or it could be management of chronic asthma, or it could be major could be a life-threatening situation.

Carefully consider what do we want this learner group that we have defined in step two. Do you want them to diagnose? To treat? To critical compare and contrast it to other cases of shortness of breath in an acute patient? You get to choose!

Perhaps we want to focus on the step-by-step history presentation or the physical exam or maybe we want to see the learners perform treatment. Or maybe we want to see the overall management or the critical thinking that goes on for managing asthma in the emergency department. There are many possibilities, largely driven by your intended learner group demographics.

So, in other words were taking the big topic of asthma and we are going to cone it down to answer the question of what exactly we want our learners to accomplish by the end of the scenario. We can’t just assume that what is supposed to happen in the real clinical environment will or should happen in the simulation environment. That rarely works. We actually want to later engineer the story and situation to allow us to be able to focus on the learning objectives.

Step Four: Define the Assessment Plan

How are you going to assess that each objective defined in step three was accomplished? That is the fundamental thought process for step four.

What are you going to be watching for when you the creator of this simulation scenario are watching the participants do their thing? What are you going to be focusing your attention on that you’re going to bring into the debriefing? What are you picking up on that you might be filling out assessment tools?

Define your assessment plan with specificity of what you’re looking for. This is different than designing the assessment tools that could come later. Or perhaps not at all. It is important that you remember every simulation is an assessment of sorts. See Previous Blog Post on this!

This doesn’t mean that every simulation needs assessment tool like a checklist, rating scale or formal grading scheme. It simply is referring to consideration of how to focus the facilitating faculty member, or teacher, or whatever you call them, who are observing the simulation. Remember, that to help the learner(s) of the simulation get better, the faculty need to be focused on certain things to ensure that the goals of the scenario are accomplished for our selected learner group, associated with the topic we chose in step one.

Lastly, what I want to point out to you is that you should notice something missing. The story!

The story comes later. Everybody wants to focus on the story because the story is fun. It’s often related to what we do clinically. It’s replicating things that are fun that brings in the theatrics of simulation! But what we really want to do is bring the theatrics of simulation to cause the actors on the stage (the participants) to so some activity. This activity gives us the situation to focus our observations on the assessment of the performance. This in turn allows us to accomplish the learning objectives of the scenario and help the participants improve for the future!

Until next time, Happy Simulating!

Leave a comment

Filed under Curriculum, design, scenario design, simulation, Uncategorized

Tagged as , ,

May 8, 2018 · 4:25 pm

Don’t be Confused! Every Simulation is an Assessment

 

Recently as I lecture and conduct workshops I have been asking people who run simulations how often they do assessments with their simulations. The answers are astounding. Every time there are a few too many people reporting that they are performing assessments less than 100% of the time that they run their simulations. Then they are shocked when I tell them that they do assessments EVERY TIME they run their simulations.

While some of this may be a bit of a play on words there should be careful consideration given to the fact that each time we run a simulation scenario we must be assessing the student(s) that are the learners. If we are going to deliver feedback, whether intrinsic to the design of the simulation, or promote discovery during a debriefing process, somewhere at some point we had to decide what we thought they did well and identify areas for needed improvement. To be able to do this you had to perform an assessment.

Kundenbewertungen - Rezensionen

Now let’s dissect a bit. Many people tend to equate the word assessment with some sort of grade assignment. Classically we think of a test that may have some threshold of passing or failing or contribute in some way to figure out if someone has mastered certain learnings. Often this may be part of the steps one needs to move on, graduate, or perhaps obtain a license to practice. The technical term for this type of assessment is summative. People in healthcare are all too familiar with such types of assessment!

Other times however, assessments can be made periodically with a goal of NOT whether someone has mastered something, but with more of a focus of figuring out what one needs to do to get better at what they are trying to learn. The technical term for this is formative assessment. Stated another way, formative assessment is used to promote more learning while summative assesses whether something was learned.

When things can get even more confusing is when assessment activities can have components or traits of both types of assessment activities. None the less, what is less important then the technical details is the self-realization and acceptance of simulation faculty members that every time you observe a simulation and then lead a debriefing you are conducting an assessment.

Such realization should allow you to understand that there is really no such thing as non-judgmental debriefing or non-judgement observations of a simulation-based learning encounter. All goal directed debriefing MUST be predicated upon someone’s judgement of the performance of the participant(s) of the simulation. Elsewise you cannot provide and optimally promote discovery of the needed understanding of areas that require improvement, and/or understanding of the topic, skills, or decisions that were carried out correctly during the simulation.

So, if you are going to take the time and effort to conduct simulations, please be sure and understand that assessment, and rendering judgement of performance, is an integral part of the learning process. Once this concept is fully embraced by the simulation educator greater clarity can be gained in ways to optimize assessment vantage points in the design of simulations. Deciding the assessment goals with some specificity early in the process of simulation scenario design can lead to better decisions associated design elements of the scenario. The optimizing of scenario design to enhance “assess-ability” will help you whether you are applying your assessments in a formative or summative way!

So, go forth and create, facilitate and debrief simulation-based learning encounters with a keen fresh new understanding that every simulation is an assessment!

Until Next Time Happy Simulating!

Leave a comment

Filed under assessment, Curriculum, design, scenario design, simulation

Tagged as , ,

March 23, 2018 · 12:00 pm

Three Things True Simulationists Should NEVER Say Again

From Wiktionary: Noun. simulationist (plural simulationists) An artist involved in the simulationism art movement. One who designs or uses a simulation. One who believes in the simulation hypothesis.

Woman taping-up mans mouth

 

After attending, viewing or being involved in hundreds if not thousands of simulation lectures, webinars, workshops, briefings and conversations there are a few things that I hear that make me cringe more than others. In this post I am trying to simmer it down to the top three things that I think we should ban from the conversations and vocabularies of simulationists around the globe!

1. Simulation will never replace learning from real patients!: Of course it wont! That’s not the goal. In fact, in some aspects simulation offers some advantages over learning on real patients. And doubly in fact, real patients have some advantages too! STOP being apologetic for simulation as a methodology. When this is said it is essentially deferring to real patients as some sort of holy grail or gold standard against which to measure. CRAAAAAAAZY……   Learning on real patients is but one methodology by which to attack the complex journey of teaching, learning and assessing the competence of a person or a team of people who are engaged in healthcare.  All the methodologies associated with this goal of education have their own advantages, disadvantages, capabilities and limitations. When we agree with people and say simulation will never replace learning from real patients, or allow that notion to go unchallenged, we are doing a short service to the big picture of creating a holistic education program for learners. See previous blog post on learning on real patients. 

2. In simulation, debriefing is where all of the learning occurs!: You know you have heard this baloney before. Ahhhhhhhhhhhhh such statements are purely misinformed, not backed up by a shred of evidence, kind of contrary to COMMON SENSE, as well as demeaning to the participants as well as the staff and faculty that construct such simulations. The people who still make this statement are still stuck in a world of instructor centricity. In other words, “They are saying go experience all of that…… and then when I run the debriefing the learning will commence.” The other group of people are trying to hard sell you some training on debriefing and then make you think it is some mystical power held by only a certain few people on the planet. Kinda cra’ cra’ (slang for crazy) if you think about it.

When one says something to articulate learning cannot occur during the simulation is confirming that they are quite unthoughtful about how they construct the entire learning encounter. It also hints at the fact that they don’t take the construct of the simulation itself very seriously. The immersive experience that people are exposed to during the simulation and before the debriefing can be and should be constructed in a way that provides built in feedback, observations, as well as experiences that contribute to a feeling of success and/or recognition of the need for improvement. See previous blog post  on learning beyond debriefing

3. Recreation of reality provides the best simulation! [or some variant of this statement]: When I hear this concept even eluded to, I get tachycardic, diaphoretic, and dilated pupils. My fight or flight nervous system gets fully engaged and trust me, I don’t have any planning on running. 😊

[disclaimer on this one: I’m not talking about the type of simulation that is designed for human factors, and/or critical environmental design decisions or packaging/marketing etc. which depend upon a close replication to reality.]

This is one of the signs of a complete novice and/or misinformed person or sometimes groups of people! If you think it through it is a rather ludicrous position. Further, I believe trying to conform to this principle is one of the biggest barriers to success of many simulation endeavors. People spent inordinate amounts of time trying to put their best theatrical foot forward to try to re-create reality. Often what is actually occurring is expanding the time to set up the simulation, expanding the time to reset the simulation and dramatically increasing the time to clean up from the simulation. (All of the after mentioned time intervals increase the overall cost of the individual simulation, thereby reducing the efficiency.) While I am a huge fan of loosely modeling scenarios off of real cases in an attempt to create an environment with some sense of familiarity to the clinical analog, I frequently see people going to extremes trying to re-create details of reality.

We have hundreds and thousands of design decisions to make for even moderately complex scenarios. Every decision we make to include something to try to imitate reality has the potential to potentially cause confusion if not carefully thought out. It is easy to introduce confusion in the attempts to re-create reality since learners engage in simulation with a sense of hyper-vigilance that likely does not occur in the same fashion when they are in the real clinical learning environment. See previous blog post on cognitive third space.

If you really think about it the simulation is designed to have people perform something to allow them to learn, as well as to allow observers to form opinions about the things that the learner(s) did well, and those areas that can be improved upon. Carefully selecting how a scenario unfolds, and/or the equipment that is used to allow this performance to occur is part of the complex decision-making associated with creating simulations. The scenario should be engineered to exploit the areas, actions, situations or time frames that are desired focal points of the learning and assessment objectives.  Attention should be paid to the specifics of the learning and assessment objectives to ensure that the included cache of equipment and/or environmental accoutrements are selected to minimize the potential of confusion, create the most efficient pathway that allows the occurrence of the assessment that contributes improving the learning.

Lastly, lets put stock into the learning contract we are engaging in with our learners. We need to treat them like adult learners. (After all everybody wants to throw in the phrase adult learning principles…. Haha).

Let’s face it: A half amputated leg of a trauma patient with other signs and symptoms of hemorrhagic shock that has a blood-soaked towel under it is probably good enough for our adult learners to get the picture and we don’t actually need blood shooting out of the wound and all over the room. While the former might not be as theatrically sexy, the latter certainly contributes to the overall cost (time and resource) of the simulation. We all need to realistically ask, “what’s the value?”

While my time is up for this post, and I promised to limit my comments to only three, I cannot resist to share with you two other statements or concepts that were in the running for the top three. The first is “If you are not video recording your scenarios you cannot do adequate debriefing”, and the second one is “The simulator should never die.” (Maybe I’ll expand the rant about these and others in the future 😉).

Well… That’s a wrap. I’m off to a week of skiing with family and friends in Colorado!

Until next time,

Happy Simulating!

8 Comments

Filed under Curriculum, debriefing, scenario design, simulation

Tagged as , , ,

August 23, 2017 · 8:20 pm

Recreating Reality is NOT the goal of Healthcare Simulation

Discussing the real goals of Healthcare Simulation as it relates to the education of individuals and teams. Avoiding the tendency to put the primary focus into recreating reality, and instead providing the adequate experience that allows deep reflection and learning should be the primary focus. This will help you achieve more from your simulation efforts!

 

Leave a comment

Filed under scenario design

Tagged as , ,