By: Alice Tesar

At the start of his English elective mod “Ethical Standards for AI in Education” at Steamboat Mountain School, instructor Jed Donnel’s  goal seemed straightforward: work with students to suggest ways and reasons to redesign the school’s AI policy to make it more thoughtful, flexible, and less tied to individual assignments. In particular, the course aimed to use a student-centric perspective to help steer the school’s practices as we adjust to the rapidly changing world of generative AI. Such technologies provide useful tools, though they can also be easily misused, and often faculty have difficulty adjusting our own modes of thought to consider how students may ethically explore what a responsible use of AI may look like. In a class of just four students, the project quickly evolved into something far bigger. 

Instead of only debating policy, students decided to build their own AI bot, which they’ve dubbed Penguino. As they developed Penguino using Google’s AI Studio software, they also created its ethical framework, writing the rules for how it should respond to prompts, what kinds of help it could provide, and where the boundaries should exist. They wanted to ensure that it sustained critical thinking, that it was objective and credible, that it cited all of its sources using credible sites, that it would not hallucinate, and that would admit whenever it could not answer any prompt as written. Then came the most interesting part: students intentionally tried to “break” the model. By testing loopholes, edge cases, and confusing prompts, then rewriting additional instructions for it to adjust its coding, they strengthened Penguinio’s safeguards and sharpened their understanding of how AI systems work. 

As a culminating exercise, the students presented their revised policy to school administrators and let them try to break the model, too. While the administrators weren’t able to trick the bot, they did find some ways students could improve the model to better support the faculty. Head of School, Samantha Coyne Donnel pointed out a fascinating tension within many school AI policies. While student handbooks often restrict AI-generated work, they also permit the hiring of a tutor and, in this case, Penguino wasn’t functioning like a cheating tool at all; it operated more like a tutor offering structure, asking guiding questions, providing feedback, and helping students think through writing prompts and math equations without simply giving answers. Penguino was supporting the learning process rather than replacing it. 

That realization shifted the conversation: Dean of Students Matt Thomas noted that tools like Penguino could help level the playing field for students who may not have access to private tutors or outside academic support. Instead of asking, “Should students use AI?” the class began asking a more meaningful question: “How can AI be designed to support authentic learning?”

In the end, the module became far more than a policy-writing exercise. It became an exploration of ethics, equity, and critical thinking, and the evolving relationship between education and AI, led entirely by students willing to ask difficult questions and rethink outdated assumptions.