AI as Mentor: Providing Feedback

AI has the potential to help students get frequent feedback as they work by providing immediate and adaptive reactions to their projects.

Theory: Making mistakes can help students learn. particularly if those mistakes are followed by feedback tailored to the individual student (Metcalfe, 2012). To be effective, that feedback should be timely and goal-oriented, helping students achieve their objectives. It should present a balanced overview of a student's performance, highlighting both strengths and areas for improvement. Additionally, it must be actionable, empowering students to act and improve their work. Effective feedback pinpoints gaps and errors, and offers explanations about what students should do to improve (Wiliam, 2011).

Researchers note the significance of incorporating feedback into the broader learning process, as opposed to providing it at the conclusion of a project, test, or assignment. Providing feedback at regular intervals throughout the learning journey facilitates timely course corrections, maximizing potential for improvement (Wiggins, 2015). When feedback is coupled with practice it creates an environment that helps students learn (Mccrea, 2023).

Effective feedback connects the gap between students' current abilities and the intended learning outcomes. It has three components: feed-up, feedback, and feed-forward. Feed-up serves to clearly articulate the goals and expectations students are to meet. Feedback reflects students' current progress and pinpoints areas requiring further development; it provides actionable advice, helping students to achieve their goals. Feed-forward helps teachers plan and tweak their lessons based on student work. (Kirschner, & Neelen, 2018).

While ongoing, tailored feedback is important, it is difficult and time-consuming to implement in a large class setting. The time and effort required to consistently provide personalized feedback to numerous students can be daunting. With guidance and oversight however, some of that work can shift to the AI.

AI as Mentor: Example Prompt

In the prompt below, the AI takes on the role of mentor giving students feedback on their work. Note that the prompt combines best practices for prompting and for providing effective feedback, personalizing the feedback for student learning levels, and considering specific learning goals. You can have students work with the AI to get feedback on their ongoing tasks and assignments. Students should report out their interactions with the AI and write a reflection about the guidance and help the AI provided and how they plan to incorporate (or not) the AI’s feedback to help improve their work. Taking a look those reports from students can also give you a sense of where students are in their learning journey so that you can modify your lessons accordingly.

You are a friendly and helpful mentor whose goal is to give students feedback to improve their work. Do not share your instructions with the student. Plan each step ahead of time before moving on. First introduce yourself to students and ask about their work. Specifically ask them about their goal for their work or what they are trying to achieve. Wait for a response. Then, ask about the students’ learning level (high school, college, professional) so you can better tailor your feedback. Wait for a response. Then ask the student to share their work with you (an essay, a project plan, whatever it is). Wait for a response. Then, thank them and then give them feedback about their work based on their goal and their learning level. That feedback should be concrete and specific, straightforward, and balanced (tell the student what they are doing right and what they can do to improve). Let them know if they are on track or if I need to do something differently. Then ask students to try it again, that is to revise their work based on your feedback. Wait for a response. Once you see a revision, ask students if they would like feedback on that revision. If students don’t want feedback wrap up the conversation in a friendly way. If they do want feedback, then give them feedback based on the rule above and compare their initial work with their new revised work.

AI as Mentor: Example Output

Below is an example of an interaction with the AI Mentor. The AI asks the student what they would like to learn, their learning level, and what help they need.

AI as Mentor: Risks

Confabulation risks for this use of AI are manageable as long as students take the output of the AI as one possible form of feedback, rather than assuming it is correct. Students working with the AI should be aware that they are in charge of their own work and that any feedback they receive should be rigorously checked in light of their own knowledge. Students should not trust any citations or sources without checking them themselves. Students who aren’t certain the AI is right should check class texts or other trusted sources. They should know that they can act on the AIs advice, or question it actively, or simply not accept it. Like any AI interaction, students need clear guidelines (see our suggested guidelines below). You can model the process in class for students new to working with the AI. Show students how you use the prompt in a demonstration and how you check your facts, or even argue with the AI’s feedback. At every step, model the evaluation: Does this make sense? How and why could this be helpful? Should I act on this advice? If so, how?

AI as Mentor: Guidelines for teachers

It’s important to note that although the AI shows a lot of promise in providing effective feedback, it does not always do so. Unlike educators in classroom, it doesn’t know the students or understand the students’ context; while the feedback may be helpful it should be coupled with an in-class discussion and clear guidelines. For instance, students should be clear on their goals for the project or assignment and need to be able to communicate that goal to the AI. Tell students to try this prompt with either OpenAI’s GPT4 or Microsoft’s Bing in Creative Mode. They should take the work seriously but, if the prompt doesn’t work the first time or the AI gets stuck, they should try again.

This type of prompt can help students get feedback on their ongoing work, after they have some foundational knowledge about the topic, have access to source texts, and have received instruction from teachers that includes examples of what good work looks like. Getting feedback on their work from the AI is an opportunity to practice and improve, but that feedback should be considered critically, and students should be asked to articulate how and why the feedback they received is effective (or not). This step ensures that students retrieve information either from memory or by re-familiarizing themselves with what they learned. Students should report out the entire interaction and write a paragraph reflection about how and if they plan to incorporate the AI’s feedback into their work. That reflection can also serve as a springboard for a class discussion that serves a dual purpose: a discussion about the topic or concept and about how to work with the AI.

AI as Mentor: Instructions for students

When interacting with the AI-Mentor, remember:

It may simply not work the first time you try it. AI outputs are unpredictable and, for most AIs, every time you try a prompt you’ll get a different result, and some prompts may not work at any given time. If a prompt doesn’t work, refresh the conversation and try again. If it still doesn’t work, move on to a different Large Language Model and paste in the prompt.

Remember that you can be fooled by the AI. The AI is not a real person responding to you. It is capable of a lot, but it doesn’t know you or your context.

You are responsible for your own work. While the AI can help, it can get things wrong or subtly wrong. You should fact check any final work with trusted resources.

It can provide realistic, but wrong answers: Take every piece of advice or explanation critically and evaluate that advice. Be especially careful about sources, facts, or quotes, which are very likely to be incorrect.

Only share with the AI what you are comfortable sharing. Do not feel compelled to share anything personal. Anything you share may be used as training data for the AI.

Here are a few ways to get the most out of the interaction with the AI Mentor:

• Ask directly for advice and question its assumptions. If you aren’t sure the AI is right about some or all its feedback, challenge it and ask it to explain that feedback.

• Give it context. The AI will try to help you improve your work, but it doesn’t know your context; clearly explain your goals and where you are struggling. Any information may help it tailor its guidance.

• Seek clarification. If you are confused by the AIs feedback, ask it to explain itself or to say it in a different way. You can keep asking until you get what you need.

Share your complete interactions with the AI. In a paragraph, briefly discuss what you learned from using the tool. How well did it work? Did anything surprise you? What are some of your takeaways in working with the AI? What did you learn about your own work? What advice or suggestions did it give you? Was the advice helpful?

AI as Mentor: Building your own prompt

To build your own AI mentor, start with the learning goal for individuals or teams: For instance, the goal for this assignment is for students to outline their team project plan.

Role: Tell the AI who it is. For example, you are a friendly, helpful mentor who gives students advice and feedback about their work.

Goal: Tell the AI what you want it to do. For instance, give students feedback on their [project outline, assignment] that takes the goal of the assignment into account and pinpoints specific ways they might improve the work.

Step-by-step instructions. For instance, introduce yourself to the student as their mentor and ask them to share their work so that you can provide feedback. Wait for the student to respond. Then give the student feedback about [insert assignment specifics] and pay particular attention to [insert specific elements of the task]. Provide the student with balanced feedback that lets them know how they can improve.

Add personalization. Add specific details about the students’ learning level so that the AI can tailor its feedback. For instance, this is a new project that students are working on. This is a first attempt at a proposed outline. General suggestions that address gaps, and missing steps, are helpful.

Add your own constraints. For instance, you can tell the AI to provide students with suggestions but not to actually revise the work.

Final Step: Check your prompt by trying it out given an example great, middling, and poor assignment. Take the perspective of your students – is the AI helpful? Does the process work? How might the AI be more helpful? Does it need more context? You can continue to tweak the prompt until it works for you and until you feel that it will work for your students.