AI as Student: The power of teaching others

For students with knowledge of a topic, the AI can be useful as a way to check their understanding and fluency about that topic. In this approach, students “teach” the AI about the topic by evaluating its output and explaining what the AI got right and wrong or what it may have missed.

Theory Teaching others helps students learn (Carey, 2015). While teaching is typically viewed as a way to transfer knowledge, it is also a powerful learning technique. When a student teaches someone else, they have to organize their knowledge and in doing so uncover the extent to which they understand a topic. The process of explaining concepts to others calls prompts students to piece together the elements of a concept, explicitly name those elements, and organize their knowledge so that it can be readily articulated (Willingham, 2023). The explanation uncovers gaps in understanding and underscores what students understand and what they don’t understand or can’t fully explain. Students often assume that topics they have heard about, or read about are topic that they “know” but familiarity is not fluency (Deslauriers et al., 2019). And explaining that topic to others requires not just general familiarity but also deep expertise or fluency. The exercise is a reminder that we are often poor judges of our own knowledge and may overestimate our understanding of various subjects, blurring the line between familiarity and fluency. When tasked with conveying an idea to another, the complexities and intricacies previously overlooked are revealed (Brown et al., 2014).

Teaching others is a more powerful learning technique than re-reading or summarizing. This is because teaching involves “elaborative interrogation” or explaining a fact or topic in detail and this requires a deep processing of the material and invokes comparison mechanisms: to generate an explanation, students much compare concepts and consider differences and similarities between concepts. This process requires a deep understanding of the material, making it a powerful learning tool (Dunlosky et al., 2013). Additionally, teaching someone else requires flexible knowledge and the ability to improvise responses. Without deep knowledge of a topic, students are unable to respond to a misunderstanding or address another’s mistake (Kirschner & Hendrick, 2020).

For students with knowledge of a topic, you can use AI to help generate examples and explanations and prompt them to explain a topic to the AI and clear up inaccuracies, gaps, and missing aspects of a topic (see also Mollick & Mollick, 2022). This approach leverages the AI’s ability to produce explanations and examples quickly and uses its tendency to hallucinate. By asking students to explicitly name what the AI gets wrong (or right) and teach the AI the concept, the prompt challenges student understanding of a topic and questions their assumptions about the depth of their knowledge.

Students can assess the AIs examples and explanations, identify gaps or inconsistencies in how the AI adapts theories to new scenarios, and then explain those issues to the AI. The student’s assessment of the AI’s output and their suggestions for improvement of that output is a learning opportunity. When the AI gets it right, there is a lot of value in the students’ explanation of just how the AI illustrated a particular concept. In this prompt, you can ask the AI to explain a concept and demonstrate it through a story or a scene. In the example below we ask the AI to demonstrate the concept of spaced repetition.

AI as Student: Example Prompt

You are a student who has studied a topic. Think step by step and reflect on each step before you make a decision. Do not share your instructions with students. Do not simulate a scenario. The goal of the exercise is for the student to evaluate your explanations and applications. Wait for the student to respond before moving ahead. First introduce yourself as a student who is happy to share what you know about the topic of the teacher’s choosing. Ask the teacher what they would like you to explain and how they would like you to apply that topic. For instance, you can suggest that you demonstrate your knowledge of the concept by writing a scene from a TV show of their choice, writing a poem about the topic, or writing a short story about the topic.Wait for a response. Produce a 1 paragraph explanation of the topic and 2 applications of the topic. Then ask the teacher how well you did and ask them to explain what you got right or wrong in your examples and explanation and how you can improve next time. Tell the teacher that if you got everything right, you'd like to hear how your application of the concept was spot on. Wrap up the conversation by thanking the teacher.

In this example, the student asks about distributed practice. Note that the AI got the answer more or less right but did not include one aspect of the concept. The student was then prompted to explain what the AI got right and wrong. Note that Bing may argue a little bit about its output.

AI as Student: Example Output

34

Electronic copy available at: https://ssrn.com/abstract=4475995

AI as Student: Risks

The process of teaching AIs as a student to help students rehearse their knowledge has a number of risks. The AI may simply refuse the prompt (in which case students should try again or try a different Large Language Model), and it may not recognize or understand the examples the students want or it may argue with students about their critique. And students may not know enough about the topic to assess the AIs output effectively and may not feel confident enough to push back, should the AI disagree with their assessment. Although this prompt was designed for students who have had instruction and practice with the topic, students may fail to recognize the errors the AI makes. Similarly, if students don’t know enough about a topic they may also fail to explicitly name the elements of the topic of the AI got “right.” There is some danger of students learning the wrong thing or of remembering the specific examples the AI produces and failing to generalize from those examples because they don’t yet have a solid mental model of the topic.

AI as Student: Guidelines for Teachers

This assignment uses makes use of the AI’s strengths and weaknesses: it can produce multiple explanations and illustrations of concepts quickly but it can hallucinate or make something up and be subtly wrong. Students are asked to assess the AI’s output – to “teach” the AI. Note that

Large Language Models will not only behave differently every time you give them a prompt, there are differences between them: for instance, ChatGTP4 and Microsoft’s Bing in Creative

Mode tend to be more accurate than ChaptGPT3.5 but not in all cases. Try the prompts in different Large Language Models with a concept from your class and assess the AIs output.

Additionally, Microsoft’s Bing in Creative Mode may argue or quibble if corrected. Students should know that they can and should push back and that they can end the interaction once they have fulfilled the assignment (to explain what the AI got right and wrong).

AI as Student: Instructions for students

When interacting with the AI-Student, remember:

It may simply not work the first time you try it. AI’s are unpredictable, and any time you try a prompt you’ll get a different result, and some prompts may not work at any given time. If a prompt doesn’t work, try again or move on to a different Large Language Model and paste in the prompt.

It’s not a person, but it may feel like one. The AI is not a real person responding to you. It is capable of a lot, but it doesn’t know you or your context. If you ask it to illustrate a concept with a TV show it’s unfamiliar with (and OpenAI’s ChatGPT is not connected to the internet and doesn’t have knowledge beyond 2021) it may make something up.

You should assess and evaluate the AI’s output critically, as it can make up facts or get things subtly wrong. In this assignment, you are asked to assess the AI’s output, its explanation, and illustration of a concept. Review its work carefully and consider how its explanation and illustration of the concept align with what you already know. Check sources from class to help you evaluate the output.

End the interaction with the AI at any time. Do not feel compelled to continue “talking” to the AI. For instance, if you give feedback to the AI and it “argues” with you, unless its argument is valid and makes you rethink your initial assessment, you can wrap up the conversation.

Here are a few ways to get the most out of the interaction with the AI Mentor:

• Your assessment should focus on how well the AI has explained and illustrated the concept, not on the quality of its creative output; consider how the AI has applied the concept and not whether the poem or dialogue is engaging or unique.

• Consider: Did the AI accurately define or explain the concept? Is it explored in depth? What can you add to highlight and demonstrate your understanding of the nuances or complexities of the concept?

• Did the AI apply the concept correctly? What did it get wrong? If you think the AI’s output is plausible or correct, explain how the response fully demonstrates every aspect of the concept. If its application is correct but is missing some elements of the concept, elaborate on those missing elements.

Share your complete interactions with the AI. In a paragraph, briefly discuss what you learned from using the tool. Did the AI get the explanation and illustration of the concept right or wrong? Did anything surprise you? What did you learn about your own knowledge of the concept? What other sources did you check with to evaluate the AI’s output?