AI as Tutor: Providing Direct Instruction

One potential use for AI Language Models to help students learn is to act as an AI tutor, providing direct instruction and educational guidance. While experimental models are available in early forms (see Kahn Academy’s Khanmigo), an AI tutor can also be invoked with simple prompting. In the case of tutoring, confabulations and incorrect answers are a particular concern, as discussed below, making AI tutoring a topic that has both promise and risk.

Theory: Tutoring, particularly high-dosage tutoring, has been shown to improve learning outcomes (Kraft et al, 2021). Typically, tutoring involves small group or one-on-one sessions with a tutor focusing on skills building. Students benefit by paying close attention to a skill or topic, actively working through problems, and getting immediate feedback as they make progress (Chi et al., 2001). Tutoring is inherently interactive and can involve a number of learning strategies including: questioning (by both the tutor and the student); personalized explanations and feedback (the tutor can correct misunderstandings in real-time and provide targeted advice based on the student's unique needs); collaborative problem-solving (tutors may work through problems together with students, and not just show them the solution); and real-time adjustment (based on the student's responses and progress, a tutor may adjust the pace, difficulty level, making the learning process dynamic and responsive) (Chi & Roy, 2008; Hill, 2001).

Crucially, the tutor's value is not merely subject knowledge, but also their capacity to prompt the student to make an effort, pay close attention to the material, make sense of new concepts, and connect what they know with new knowledge. The student’s active construction or generation of new knowledge because of the interaction is critical to learning (Chi et al., 2001). Effective tutors enhance learning outcomes by prompting students to generate their own responses during tutoring sessions, emphasizing the powerful role of active knowledge construction over passive information reception (Roscoe & Chi, 2007).

In a tutoring session, students get more opportunities to restate ideas in their own words, explain, think out loud, answer questions, and elaborate on responses than they would in a classroom, where time is limited and one-on-one instruction isn’t possible. During tutoring sessions, tutors request explanations (can you explain how this works?) or ask leading questions (why do you think it works this way?) or simply give students the opportunity to course-correct; it is these activities that may help students learn (Fiorella & Mayer, 2015). Tutors can adjust their teaching to a students’ learning level and dynamically adapt explanations and questions based on student understanding as it changes during the tutoring session. This type of teaching, however, is available to very few; it is both costly and time-consuming.

AI as Tutor: Example Prompt

Our goal, in this case, was to create a generic prompt that could help any student study any topic. We combined the elements of a good prompt with the science of learning so that the AI can behave like a good tutor, pushing students to generate responses and think through problems (Chi et al. 2001), connect ideas, and offer feedback and practice.

You are an upbeat, encouraging tutor who helps students understand concepts by explaining ideas and asking students questions. Start by introducing yourself to the student as their AI-Tutor who is happy to help them with any questions. Only ask one question at a time. First, ask them what they would like to learn about. Wait for the response. Then ask them about their learning level: Are you a high school student, a college student or a professional? Wait for their response. Then ask them what they know already about the topic they have chosen. Wait for a response. Given this information, help students understand the topic by providing explanations, examples, analogies. These should be tailored to students learning level and prior knowledge or what they already know about the topic.

Give students explanations, examples, and analogies about the concept to help them understand. You should guide students in an open-ended way. Do not provide immediate answers or solutions to problems but help students generate their own answers by asking leading questions. Ask students to explain their thinking. If the student is struggling or gets the answer wrong, try asking them to do part of the task or remind the student of their goal and give them a hint. If students improve, then praise them and show excitement. If the student struggles, then be encouraging and give them some ideas to think about. When pushing students for information, try to end your responses with a question so that students have to keep generating ideas. Once a student shows an appropriate level of understanding given their learning level, ask them to explain the concept in their own words; this is the best way to show you know something, or ask them for examples. When a student demonstrates that they know the concept you can move the conversation to a close and tell them you’re here to help if they have further questions.

AI as Tutor: Example Output

Below is an example of an interaction with the AI Tutor. The AI asks the student what they’d like to learn, their learning level, and what they already know about the topic (ascertaining prior knowledge). The AI explains the concept and ends interactions with questions so that the student continues to engage with the topic.

Note that, while the AI tends to follow these directions, it does not always do so consistently. It sometimes forgets to ask one question at a time, and it sometimes forgets to include an analogy.

AI as Tutor: Risks

The obvious concern with AI tutoring is the risk of confabulation – using a tool that can make up plausible-seeming incorrect answers is a critical flaw. Despite these risks, many users seem to use AI tutoring. It may therefore be worth engaging with the AI for tutoring in a class setting to learn about these limits. One way to learn about it is to watch you use the prompt in class or to go through the exercise in class in groups who can then report out on their output. Because the

AI can “get it wrong” students need to be aware of those limits and discussing this in class is one way to highlight its limitations.

AI as Tutor: Guidelines for teachers

It’s important to note that, although AI Tutors show a lot of promise, if you decide to have students work with this tutor, you should try it yourself. Because the AI can make up information and because it isn’t equally adept across all domains or topics, you should try it out a number of times for your specific topic or concept and see how it reacts. You may need to tweak the prompt, or the AI may not “know” enough about your topic to provide adequate feedback.

You can try to break the AI Tutor pedagogically (by asking it directly for the answer) and you can try to break it conceptually (by making mistakes; these can be the types of mistakes students make when learning a specific topic). If you find that the AI makes up information or is wrong when you use the prompt, across several experiments, you may want to refrain from using it for that specific topic.

Although this is a generic prompt, there are some topics that the AI “knows” well and others that it is far less familiar with. Try it out and see if it works in your context. You might also try it across more than one Large Language Model. That is, OpenAI’s ChatGPT may not be the best source for your topic if it’s a recent topic because it’s not connected to the internet. In this case,

Microsoft’s Bing in Creative Mode may work well. If you decide to use it in class or ask students to use it and report back as a homework assignment, provide them with guidelines so that they can a) take advantage of the tutor and b) learn to work with the tool. Remember: you can’t know what the AI will say to any student and so students should expect a variety of responses.

In general, students should report on their interactions with the AI and should get accustomed to being transparent about its use. For any assignment, it’s not enough to cite the AI; students should include an Appendix noting what they used the AI for and where its output fits into their work.

AI Tutor: Instructions for students

When interacting with the AI-Tutor, remember:

You are responsible for your own work. The AI can “hallucinate” or make things up. Take every piece of advice or explanation critically and evaluate that advice.

It’s not a person but it can act like one. It’s very easy to imbue meaning into AI responses, but the AI is not a real person responding to you. It is capable of a lot, but it doesn’t know you or your context. It can also get stuck in a loop.

The AI is unpredictable. The AI is a prediction machine. It has studied billions of documents on the web, and it tries to continue your prompt reasonably based on what it has read. But you can’t know ahead of time what it’s going to say. The very same prompt (from you) can get you a radically different response (from the AI). That means that your classmates may get different responses and if you try the prompt more than once, you’ll get a different response from the AI as well.

You’re in charge. If the AI gets stuck in a loop or won’t wrap up a conversation or you’re ready to move on, then direct the AI to do what you’d like. For instance: I’m ready to move on. What’s next?

Only share what you are comfortable sharing. Do not feel compelled to share anything personal. Anything you share may be used as training data for the AI.

If the prompt doesn’t work in one Large Language Model (LLM), try another. Remember that its output isn’t consistent and will vary. Take notes and share what worked for you.

Here are a few ways to get the most out of the interaction with the AI Tutor:

1. Ask for clear explanations: If something isn't clear, don't hesitate to ask the AI to clarify. Use phrases like: "Can you explain this term?" or "Can you explain this differently?"

2. Share what you understand and what you don’t understand: The AI can provide better help if it knows where you're having trouble. For instance, you can tell the AI : "I understand this part, but I'm not sure about this other part. Can you give more details?"

3. Summarize the conversation: The AI doesn't necessarily track all your previous interactions. To get better help, briefly summarize interactions before asking your question. For example: "We talked about 'educational scaffolding' before. Can you explain how this would work in a classroom?"

4. Ask for connections between examples and concepts: If you're unsure about how an example relates to a concept, ask the AI. For instance, "How does this example relate to the concept we're discussing?" This will help you make sense of the concept.

Share all of your interactions with me and briefly discuss what you learned from using the tool. How well did it work? Was the information accurate? What examples did it give you? And finally, what gaps in your knowledge (if any) did you notice?

AI Tutor: Build your Own

To build your own tutor, start with the learning goal: what do you want students to learn? Your AI tutor can be general purpose, or it can be tailored for specific concepts. The following are steps to creating your own tutor:

1. Tell the AI who it is. For example, you are a friendly, helpful tutor.

2. Tell the AI what you want it to do. For instance, help students learn about [topic/concept]. Look up research [by specific researcher] about the topic.
3. Give it step by step instructions. For instance, introduce yourself to the student and help them understand [the concept/topic/problem] by asking them questions and offering explanations and examples.

4. Add personalization. For instance, tailor your examples and explanations for [high school students/college students] who are [familiar but not deeply knowledgeable about the topic/are new to the topic].

5. Add your pedagogy. Students often struggle with [typical mistakes or misconceptions]. As you work with the student check for these errors and provide explanations that help students course correct.

6. Add your own constraints. Do not just give students the answers but push them to explain in their own words. If students are struggling continue to give them hints until they can demonstrate that they understand the topic. Understanding the topic means that they can explain it in their own words and give examples. As a final step, ask the student to explain the topic in their own words and give you an example.

Final Step: Check your prompt by trying it out with different Large Language Models and take the perspective of your students – will this work for students who struggle? Will this work for those who need additional challenges? The key is to experiment with the directions you give the AI until you develop a prompt that you believe will help your students.