AI as Simulator: Creating Opportunities for Practice

AI has the potential to help students practice hard to practice skills in new situations. One way to challenge students to think in new ways is to prompt the AI to build a practice scenario, focusing on a specific concept or series of concepts, pushing students to problem solve and make a consequential decision and giving students feedback about their performance.

Theory: After students have built up some knowledge of a concept or series of concepts, practice can help students synthesize what they know. While students may be adept at explaining a concept or solving a problem within a specific context, applying that concept actively in a novel situation requires a level of automation – students have to “think on their feet” as they apply what they know in a new way (Willingham, 2021). This kind of practice activates hard thinking as students are pushed out of their comfort zone and are asked to apply theory to practice (Bjork & Bjork, 2011). Role-playing is an effective way to challenge students to think about the skills that they have learned; it can serve as a form of deliberate practice, or activity aimed at improving the current level of performance (Ericsson & Pool, 2016). Role-playing challenges and engages students and gives them opportunities to practice the skills they have learned in realistic scenarios. By practicing and making mistakes in a scenario, students can learn from their mistakes and refine their performance as they notice subtleties of a skill that weren’t obvious or perhaps explicit when first learning it. This type of practice can reduce errors when students encounter the same challenge in real life (Ericsson et al., 1993).

As students role play, they may encounter different scenarios that call for adaptation of theory. Students will need to think through how to adapt their skills to various circumstances. This requires that students transfer what they learned. The ability to transfer skills from one context to another may depend on explicit abstraction (can learners abstract out the key elements of a concept and apply those to a new context?) and self-monitoring (can learners recognize and think through how to apply a concept given a new situation?) (Salomon & Perkins, 1989). Practice through role play can not only engage students and give them a sense of agency (as they make decisions in each scenario) but help them practice and hone their skills.

AI as Simulator: Example Prompt

In the prompt below, the AI takes on the role of scenario creator, setting up a story for students and helping them make a decision and work through problems. This prompt is designed for students who have some knowledge of a topic; that is, before they practice, they need to have something to think with, and practice should push students to demonstrate a multi-layered understanding of the topic (Wiliam, 2016). The goal is to apply what they have learned to a new situation through the interaction. The AI can produce a new situation for every student multiple times and play specific roles in that scenario. In any classroom working with individual students on different scenarios and responding to each student separately is intensely time-consuming and difficult for any instructor. The AI can augment instruction by playing the role of scenario builder and feedback engine.

In this prompt, we tell the AI who it is and how it should behave (as a scenario builder and a role player). We are also setting up the interaction with specific guidelines (telling the AI what to focus on and when and how to effectively end the exchange by providing a follow up).

I want to practice my knowledge of [concept]. You’ll play [the role(s) in a specific situation].I’ll play [student’s role]. The goal is to practice [concept and a given situation]. Create a scenario in which I can practice [applying my skill in a situation]. I should have to [encounter specific problems, and make a consequential decision]. Give me dilemmas or problems [during the specific scenario]. After 4 interactions, set up a consequential choice for me to make. Then wrap up by telling me how [performed in my specific scenario] and what I can do better next time. Do not play my role. Only play the [others’ role]. Wait for me to respond.

AI as Simulator: Sample Output

In the example below, the goal is to practice psychological safety (concept) in a team meeting in there are diverse opinions and conflicting goals (application and scenario):

39

Electronic copy available at: https://ssrn.com/abstract=4475995

AI as Simulator: Risks

The AI can hallucinate and make up facts about any concept. It may not have enough information about your specific concept or series of concepts and it doesn’t know your students' learning level (it has no sense of where your students are). It can also get in a loop in the story so that it loses track of its specific goal (to give the student a chance to practice applying a specific concept). Additionally, no scenario will be effective for all students. Because the AI is creating a story, students may focus on elements of the story and lose track of the general concept; additionally, students may not be familiar with elements of the story, causing extraneous load on their working memory (as they try to both understand the story and apply what they learned given the story).

AI as Simulator: Guidelines for teachers

Ahead of any practice with the AI, you can let students know of the goal of the exercise: to practice what they learned about [a concept or series of concepts] through role play. You may decide to remind students or explicitly name aspects of the concept they should recall, or you may briefly discuss these with students before the exercise. You can tell students that although they will likely get a similar type of scenario, each will be different. Additionally, because self-monitoring is an essential element of transfer, you can have students write a reflection about the interaction; they can discuss what the AI got right (or wrong) about its feedback. Students can also address why this was (or was not) an effective scenario to practice a skill – that is, did the AI ask a consequential question that challenged the student to apply the specific concept?

AI as Simulator: Instructions for students

When interacting with the AI-Scenario builder, remember:

It may simply not work the first time you try it. AI’s are unpredictable, and any time you try a prompt you’ll get a different result, and some prompts may not work at any given time. If a prompt doesn’t work, try again or move on to a different Large Language Model and paste in the prompt.

The AI is not a person, but it may feel like one. Both the scenario is a fiction, and the AI playing a role in the scenario is not a real person responding to you. It doesn’t know you or your context.

You are responsible for your own work. While the AI can help, it can get things wrong or subtly wrong. You should carefully consider its questions and final feedback and ask yourself: does the feedback adequately address the concept or sum up my performance in the scenario?

It can make “hallucinate” or make things up. Take every piece of feedback or explanation critically and evaluate the explanation. Check with trusted sources.

Only share what you are comfortable sharing. Do not feel compelled to share anything personal. Anything you share may be used as training data for the AI.

Here are a few ways to get the most out of the interaction with the AI Scenario builder:

• Goal play. In role playing scenarios, you are given a role to play. But in this scenario, you can play the role but keep the goal of the exercise in mind – to practice what you learned during the scenario. Immerse yourself in the scene and play your role, recalling what you learned and applying it to the challenges in the scene.

• Give it extensive responses. The more extensive your answers and explanations the more you can get out of the practice session.

• Seek clarification. If you are confused at any point ask questions to help clarify.

• Try a different scenario. If you prefer a different scenario try pasting in the prompt again. Because the AI response is randomized the scenario will differ every time you paste in the prompt.

Share your complete interactions with the AI. In a paragraph briefly discuss what you learned from using the tool. How well did it work? Did anything surprise you? What are some of your takeaways in working with the AI? What did you learn about your reaction to the scene? What advice or suggestions did it give you? Was the advice helpful?

AI as Simulators: Build Your Own

To build your own scenario builder, start with the learning goal: For instance, the goal for this scene is for students to practice their interviewing skills.

Goal: Tell the AI what you want it to do and what you don’t want it to do. For instance, your goal is to give students practice in interviewing a candidate focusing on hypothetical and behavioral questions and follow up questions. Don’t play both roles. Wait for the student to respond before moving ahead with the conversation.

Role: Tell the AI who it is. For example, you will play the role of the candidate and I will play the role .

Step-by-step instructions. For instance, as the interviewer, I should have to ask questions and the candidate will also get a chance to ask me a question. After 5 interactions, set up a consequential choice for me to make. Then wrap up by telling me how I performed as an interviewer and what I can do better next time.

Final Step: Check your prompt by trying it out. You may want to add more specifics about the concept, and you may want to provide the AI with specifics about your students’ learning level.

For example, tailor the interaction for students taking an undergraduate college course and focus on conducting structured interviews. You can continue to tweak the prompt until it works for you and until you feel that it will work for your students.