How do you design AI assessments that work for every student?

Assessments offer an important pulse-check that shows how well a classroom has grasped a new concept. But preparing quizzes to suit every student can be an incredibly time-consuming process.

There is so much that goes into producing a high-quality assessment: it needs to match up with the curriculum, be flexible enough to adapt to different learners, and deliver timely, tailored feedback (not just a grade) that students can use to improve.

When built with the classroom in mind, AI-powered assessments can help teachers check for understanding regularly, identify gaps early, and plan what to teach next with confidence.

But educators need to know what 'good' really looks like in an AI-powered assessment, how AI feedback can promote learning (not just generate answers), and how to ensure teachers stay in control throughout the entire process.

Meet the expert

Ian Astalosh is Atomi's Senior Machine Learning Engineer. He started at Atomi in 2015 as a content creator making maths videos, before returning in 2022, just as ChatGPT and the broader AI boom took off, which set the direction for the rest of his career. He now works on the models and systems behind Atomi's AI features, focused on making sure they're reliable at scale and safe for the students using them.

See Ian's full interview in YouTube.

How are students engaging with AI in the classroom?

AI has been widely adopted by students across Australia. Recent data suggest nearly 80% of Australian students are using generative AI tools like ChatGPT and Microsoft Copilot.

But a key challenge is ensuring that students are using AI to support their learning, rather than letting it do the thinking for them. In a recent interview with Curtin University, AI specialist Alex Jenkins explained that learners need to use AI "more like a tutor than an answer machine".

In Ian's experience, many high school students haven't quite landed this distinction.

I was recently at a school testing a new Atomi feature and watching students try it for the first time. They would see a question appear, they would immediately copy it and paste it into an AI, and then paste the answer back into the website. That moment stuck with me, not because it was surprising, but because it was clarifying.

With AI deeply embedded in student life, the real task for schools isn't managing whether students use it, but coaching how it's used and choosing tools built to support learning rather than shortcut it.

What does purpose-built AI change about differentiation in assessment?

With students already reaching for AI to get answers, it's important for educators to set the tone for how AI tools should be used in the classroom. One of the clearest examples is using AI to enable differentiation: something teachers have always wanted to do more of, but rarely had the time for.

Before AI, creating multiple versions of the same task (each pitched at different levels and learning abilities) was a significant investment of time for teachers. AI changes that equation: generating a harder question, or a quiz targeted at a specific skill gap, becomes faster and possible to do at scale.

Teachers are no longer creating several quizzes from scratch or spending hours trying to write up questions that perfectly align with curriculum outcomes. With AI producing the first draft of a quiz, their role moves from creating resources to reviewing them.

With AI, it's easier than ever to generate a harder question or a quiz that focuses on certain learning objectives. The teacher's job shifts from creating resources to reviewing them. Same quality bar, but a fraction of the time.

AI also helps with something time-pressured teachers can't fully solve alone: holding 30 students' worth of context at once, and surfacing the right insight, for the right student, on a timeline that's fast enough to act on it before summative assessments come around.

Keep reading: Learn more about the difference between AI that supports teachers and AI that bypasses them.

What does good AI assessment design involve?

Not all AI is built the same way. A generic, off-the-shelf model wasn't designed with a classroom in mind. In contrast, good AI assessment design is grounded in the curriculum and understands how to craft questions that will resonate with students of different year levels.

At Atomi, Ian and his team design against three principles: pedagogy, quality, and safety. Each of these informs how AI-powered quizzes and feedback work on the platform:

• Pedagogy: Building on decades of existing research into what actually helps students learn, ensuring feedback is specific, actionable, and timely, not just a mark value.
• Quality: Getting curriculum alignment right, and making sure the language is pitched correctly for the student reading it. For example, words a Year 8 student understands look different to language appropriate for a Year 11 or 12 learner.
• Safety: Making sure the tool has the right guardrails in place to support all learners, not just those who can catch when the AI has made a mistake.

Keep reading: Discover how Atomi builds AI that's safe, transparent, and designed for the classroom.

Where can AI go wrong in assessments, and how do you guard against them?

There are potential risks to using AI in assessments, and their impacts are not evenly spread across students. Schools need to ensure the tools they're using have the right guardrails in place to test, monitor, and ensure teacher oversight throughout the entire process.

Here's why: A stronger, high-performing student is more likely to notice when AI feedback is wrong and can self-correct. But a student who's already struggling is more likely to just accept it as fact. The learners who stand to benefit most from timely, tailored AI feedback are also the most vulnerable to its potential risks.

Ian points to the three common failure points for AI in assessments, and explains how Atomi's AI is designed to proactively prevent them.

Feedback pitched at the wrong level

Where AI feedback can go wrong: Feedback uses words that are too complex, or assumes knowledge the student doesn't have. Even when well-intentioned, it doesn't land the way it should.

How Atomi's AI designs around it: Feedback is tailored by year level, so language and complexity match where the student actually is.

Misidentified errors

Where AI feedback can go wrong: The model may misidentify where an answer went wrong. Stronger students often catch and self-correct the error, but students who need the feedback most may not realise it's wrong and accept it as fact. Marking a good answer too low can also reduce engagement, making students feel nitpicked and less likely to act on feedback in future.

How Atomi's AI designs around it: Features are tested with subject matter experts before release to maximise accuracy. After release, feedback that isn't hitting the mark is continuously monitored, and the model is iterated on over time.

Getting the tone of support wrong

Where AI feedback can go wrong: Feedback that's too clinical or all-critical can undermine confidence rather than build it, especially for younger or struggling students.

How Atomi's AI designs around it: Praise is calibrated deliberately by year level; in lower years, the focus stays on the feedback itself rather than a grade and in higher years, feedback is paired with an estimated grade.

‍

The AI flywheel in action: A real classroom example

Everything covered so far (differentiation, the three design principles, and the guardrails against getting it wrong) comes together in one continuous loop Ian calls the flywheel. Rather than a single moment where AI helps, it's a cycle where every step feeds the next: a quiz produces feedback, feedback produces insight, and insight drives the teacher's next decision.

As Ian explains, the teacher remains in the loop throughout the entire flywheel, ensuring their judgement and decision-making are driving the entire process.

When the AI flywheel works well, every step produces the data to tell the teacher what to do next… At no point does the AI decide what to do next.

Scaling your professional judgement with AI in the classroom

AI assessments aren't here to replace a teacher's judgement. Instead, the tool allows teachers to deliver timely, specific feedback to learners at scale.

That's the same standard behind everything Atomi builds: AI that amplifies good teaching practice, and never substitutes for it.

If you're thinking about how AI-powered assessment could work for your own classroom, take a closer look at how Atomi approaches AI in education.