Monthly Archives: May 2015

Are Routine Pre-and Post Simulation Knowledge Tests Ethical? Useful? To whom?

shutterstock_77554009X_aDisclaimer (before you read on): This post is not referring to research projects that have been through an institutional review board or other ethics committee reviews.

What I am actually referring to is the practice of many simulation programs that do routine written pre-test, followed by written posttest to attempt to document a change in the learner’s knowledge as a result of participating in the simulation. This is a very common practice of simulation programs. It seems the basis of such testing would be to eventually be able to use the anticipated increase in knowledge as a justification for the effectiveness of the simulation-based training.

However we must stop and wonder if this ethical? I believe as described in some of my previous posts that there is a contract that exists between participants of simulation encounters, and those who are the purveyors of such learning activities. As part of this contract we are agreeing to utilize the time of the participating in a way that is most advantageous to their educational efforts that help them become a better healthcare provider.

With regard to pretesting, we could argue from an educational standpoint that we are going to customize the simulation education to help tailor of the learning to the needs of the learners as guided by the results of some pretest. I.e. using to pretesting some sort of needs analysis fashion. But this argument requires that we actually used the results of said pre-test in this fashion.

The second argument and one that we embark upon in several of the programs of which I have designed is that we are assessing the baseline knowledge to evaluate the effectiveness of pre-course content, or pre-course knowledge that participants are programs to do either complete or possess prior to coming to the simulation center.  I.e.  A readiness assessment of sorts. In other words the question being is this person cognitively prepared to engage in the simulation endeavors that I am about to ask them to participate in.

Finally another argument from an educational standpoint for pretesting could be made that we would like to point out to the participants of the simulation areas of opportunity to enhance their learning. We could essentially say that we are helping the learner direct where they will pay close attention and focus on during the simulation activities or participation in the program. Again this is predicated on the fact that there will be a review of the pretest answers, and/or at least feedback to the intended participants of the simulation program on the topic areas, questions or subjects of which they did not answer the questions successfully.shutterstock_201601739-a

The posttest argument becomes a bit more difficult from an ethical perspective outside of the aforementioned justification of the simulation-based education. I suppose we could say that we are trying to continue to advise the learner on areas that we believe there are opportunity for improvement and hopefully inspire self-directed learning.

However my underlying belief is if we look at ourselves in the mirror, myself included, we are trying to collect the data over time so that we can perform some sort of retrospective review and hopefully uncover there was a significant change in pretest versus posttest testing scores that we can use to justify our simulation efforts in whole or in parts.

This becomes more and more concerning if for no other reason than it can lead to sloppy educational design. What I mean is if we are able to ADEQUATELY assess the objectives of a simulation program with a given pair written tests, it is likely more knowledge-based domain items we are assessing and we always have to question is simulation the most efficient and effective modality for this effort. I.e. if this is the case may be every time I give a lecture I should give a pre-and posttest (although this would make the CME industry happy) to determine the usefulness of my education and justify the time of the participants attending the session. Although in this example if I was lecturing and potentially enhancing knowledge, perhaps one could argue that a written test is the correct tool. However the example is intended to put out the impracticality and limited usefulness of such an endeavor.

As we continue to attempt to create arguments for the value of simulation and overcome the hurdles that are apparent as well as hidden, I think that we owe it to ourselves to decide whether such ROUTINE use of pre-and post-testing is significantly beneficial to the participants of our simulation, or are we justifying the need to do so on the half of the simulation entity. Because we owe it to our participants to ensure that the answer reflects the former in an honest appraisal.

1 Comment

Filed under Uncategorized

The Contract Essential to the Parties of Simulation

If you think about it an agreement needs to exist between those whom facilitate simulation and those who participate. Facilitate the purposes of this discussion is referring to those who create and execute simulation based learning encounters. Sometimes the agreement is more formal other times more implied. This phenomenon has been described in many ways over the yearsshutterstock_226296865 having been branded by such descriptors as fiction contract, psychological contract, or learning contract.

Why does this need to be the case? A contract or agreement is generally called for when two or more parties are engaging in some sort of collaborative relationship to accomplish something. Often times these type of contracts spell out the responsibilities of the parties involved. If you think about simulation at a high level the facilitator side is agreeing to provide learning activities using simulation to help the participant(s) become better healthcare providers. The participants are engaged at the highest level because they want to become better healthcare providers. While not trying to hold a comprehensive discussion, let’s explore this concept and the responsibilities of each party a bit further.

Facilitators are designing simulation activities with a variety of tools and techniques that are not perfect imitators of actual healthcare. They are crafting events for which the participant to a greater or lesser extent immerse themselves in, or at a minimum simply participate. Some of these activities are designed to contain diagnostic mystery, some demand specific knowledge, skills and attitudes be known or developed to successfully complete the program. Facilitators are also putting participants in situations that the must perform in front of others and that can create feelings of vulnerability. So all toll, the role of the facilitator comes with enormous responsibility.

Facilitators are also asking the participants to imagine part of what they are engaging in is a reasonable facsimile of what one may encounter when providing actual healthcare. Therefore another tenet of the agreement is that the facilitator will provide an adequate orientation to the simulation environment pointing out what is more and less real including the role that the participant may be playing and how their role interacts with the environment outside of the simulation, if at all. (I.e. define any communications that may occur during the simulation between the participants and the facilitator.

Facilitators trained in simulation know that mistakes occur sometimes due to a lack of knowledge, incorrect judgement or unrelated issues such as a poorly designed simulation. Facilitators thereby commit to not judge the participant in anything other than their performance during the simulation. While diagnostic conundrums are inevitable in many types of simulations the facilitator should not try to unnecessarily trick or mislead the participant in any way that is not directly contributing to helping the participant(s) improve. The facilitator must attempt to use the time of the participants wisely and responsibly.

The role of the participant shares responsibilities as a part of the agreement as well. Participants agree to a commitment to become better healthcare providers through continuous learning and improvement. This is inherent in a professional, but there are some likely good reasonsshutterstock_147464348 to remind participants of this important premise.

Participants must agree to the use of their time to participate in the simulation. The participants are also agreeing to an understanding that they know the environment of the simulation is not real, and that there will be varying levels of realism employed to help them perform in the simulation. But to be clear they agree to this tenet predicated on the trust that that facilitators are having the participant experience simulations that are relevant to what they do, with an underlying commitment to help them get better. In simulations involving multiple participants, they must also agree to similarly not judge others on what occurs in the simulation, as well as keeping the personal details of what they experience in the simulation confidential.

So in closing, successful simulation or other immersive learning environments require an agreement of sorts between those who create and execute the simulation based learning environments as well as those who participate in them. Each party brings a set of responsibilities to the table to help to ensure a rich learning environment with appropriate professional decorum and commitment to improvement. The agreements range from implicit to explicit, but when they exist and are adhered to will continue to allow the recognition of value that can arise from simulation to help improve the care ultimately delivered to our patients. After all, isn’t that our highest goal?

1 Comment

Filed under Uncategorized