32 Comments
User's avatar
Mark Fraser's avatar

Brilliantly clear and balanced, Tawnya. Thank you. It gave me lots to think about

Don Zhang's avatar

Most educators are also in zone 2 with AI use.

Social Cybernetics's avatar

This is awesome, thank you for sharing. I agree that academia is not ready, but they easily could be. HITL has to be made explicit. I am working on this myself on the research methods side. https://brandtasmith.substack.com/p/the-sausage-of-science-just-got-uglier?r=1ltdgg

Dr. Peter Sheehy's avatar

Thanks for laying this out so clearly. I agree with your comments re portfolio work and oral exams but by my estimation this adds an unsustainable burden of grading and feedback work for teachers. Unless parts of that process are reviewed by AI?

Tawnya Means's avatar

There is a lot of feedback and review work that can be provided to students while leaving place for the instructor to review and finalize. Emerging evidence suggests that AI-mediated feedback, when guided by rubrics and reviewed by instructors, can provide more comprehensive, consistent, and timely feedback than instructor-only workflows, while preserving human oversight for fairness and judgment.

Evidence:

Dai et al. (2024) — GPT feedback compared with human instructors

Dai and colleagues compared GPT-3.5/GPT-4 feedback with human instructor feedback on higher-ed writing tasks. They found that GPT models produced feedback that was more readable and more consistent than human instructors, and that GPT-4 outperformed both GPT-3.5 and human instructors on several dimensions of effective feedback, including feed-up, feed-forward, process-level, and self-regulation feedback. This is a strong source for comprehensiveness and consistency, but I would treat “fairness” as implied through consistency rather than directly proven.

Dai, W., Tsai, Y.-S., Lin, J., Aldino, A., Jin, H., Li, T., Gašević, D., & Chen, G. (2024). Assessing the proficiency of large language models in automatic feedback generation: An evaluation study. Computers and Education: Artificial Intelligence, 7, 100299. https://doi.org/10.1016/j.caeai.2024.100299

Lu et al. (2026) — randomized trial of AI-mediated feedback

This randomized controlled trial in a large undergraduate economics course compared AI-mediated feedback with human-only feedback. Students receiving AI-mediated feedback produced significantly higher-quality revisions; teaching assistants retained control by adopting, editing, or dismissing AI suggestions. The study also reports that TAs found the AI useful for spotting gaps and clarifying rubrics, and the university summary notes that AI-mediated feedback covered more rubric items than human-only feedback. This is especially strong because it tests AI as an instructor-support tool, not a replacement.

Lu, X., Ju, K. P., Dudley, M., Sano, L., & Wang, X. (2026). AI-mediated feedback improves student revisions: A randomized trial with FeedbackWriter in a large undergraduate course. arXiv. https://doi.org/10.48550/arXiv.2602.16820

Korbin Greeno's avatar

So, crazy idea here, we make sure the education teachers are receiving before they become teachers is adequate enough that they can provide efficient, concise feedback consistently! The answer isn’t “give it to something else”, the answer is “make ourselves smarter”. I’m sorry but we are radically shifting the goalposts of what even being educated and intelligent means.

Dr. Peter Sheehy's avatar

Thanks!

Dr. Barry Gelston's avatar

This is a really important article. Well done.

Dr. James Ottavio Castagnera's avatar

This is a fascinating article. And I certainly see, and embrace, your points regarding your Zone 2 and Zone 3. I'm having a little trouble with grasping your point about Zone 1. My field is the law. Let's consider a first-year law student in her research & writing (legal methods) course. It's not unusual for the first legal memorandum that is assigned to include a small group of cases, selected by the instructor, since the class has not yet gained much experience in researching relevant cases for themselves. I'm trying to envision where AI would fit into this assignment/assessment. I don't think I would want the students to assign the "substantive" work of reading, digesting, and summarizing the individual cases to AI. I'm hard pressed to know just what I would want AI to be asked to do. Perhaps this traditional assignment has lost its efficacy, since one can hardly prevent the students from assigning these "substantive" tasks to AI. Perhaps your response is that in Zone 3, the students would be expected to evaluate AI's review and summation of the cases and document that process. If I got that right, I wonder if this exercise would be cognitively superior to the students performing the "substantive" work themselves. At first blush, I might prefer a Zone 1 approach. And since I can't prevent the students from assigning the analytical tasks ---e.g., reading and briefing the assigned cases---to AI, I might require them to perform that exercise in class rather than outside of class. Am I on point here, or have I totally misunderstood you?

Tawnya Means's avatar

Thank you for this thoughtful response, and for the concrete legal methods example – it actually illustrates the tension at the heart of the article quite well.

I should clarify one thing: what I labeled as Zone 1 was not a recommendation. It was a description of where most educators instinctively land – constrained, restricted AI use – and an argument that staying there is neither sustainable nor educationally sufficient. Your instinct to move the assignment into class (blue books) to prevent AI use is a common response, and I understand the appeal. But it addresses the symptom rather than the underlying question, which is what the learning goal actually is.

Your legal methods example is worth considering. If the goal is for students to practice reading and briefing cases, then yes, assigning that to AI defeats the purpose. But the more interesting question is whether the goal itself should evolve. What if students briefed the cases themselves AND then interrogated what AI produced – identifying where it flattened nuance, missed procedural posture, or got the holding subtly wrong? That requires a deeper command of the skill, not a shallower one.

That is closer to what I see as the transformation space, drawing on the Wang and Zhang framework. It is not about evaluating AI instead of doing the work. It is about doing the work and then using AI to push your thinking further – which, in legal education, has real professional relevance given how quickly AI tools are entering legal practice.

You have not misunderstood me entirely – you are asking exactly the right questions. The next challenge is to decide whether we want the deeper learning or if we want to push to keep the same constraints we have had in the past.

Dr. James Ottavio Castagnera's avatar

I'm not sure that I concede "the same restraints we have had in the past." I suppose I fancy that the legal research and writing course (Legal Methods) enabled the students to learn precisely the skills they would practice as new associate attorneys in the law firms and judicial chambers they would join after graduation. Having said that seems to beg the question of AI's impact on those entry level jobs. Attorneys were quick to use chatbots to draft briefs... too quick, as you probably have read. A few such "early adapters" :) suffered national embarrassment in the news media when the cases they cited proved to be chatbot hallucinations. I've queried senior attorneys whom I know about what will happen to the traditional apprenticeship in the legal profession, if and when (and I think 'when' is more likely than 'if') first-year associates are no longer needed to do research and writing. One thought I have is that law schools might place much greater emphasis on other skills: litigation, mediation, negotiation, conciliation...to name a few. Such skills training was available 45 years ago, when I attended law school, but it wasn't required, nor was it much emphasized by faculty advisors. Consequently, a student could get through three years of law school with almost no experience in those skill sets, if s/he so desired. The legal methods course could look quite different ala some of the "deeper command of the skill" to which you refer. I hope that's the direction we go, as GenAI chews its way up the professional-workforce food chain.

Tawnya Means's avatar

I appreciate these further thoughts and I agree that we need to build more skill building into our classroom experiences (experiential learning anyone?).

Tina Austin's avatar

an interesting take—one thing I noticed when I went back to the paper is that the “zones” framing isn’t actually in Wang & Zhang’s analysis (it comes from a post-hoc interpretation of a curve).

I wrote a deeper breakdown here if helpful, especially on the difference between orientation vs. offloading: https://tinaaustin.substack.com/p/how-cognitive-offloading-ai-research?r=4a98uc&utm_campaign=post-expanded-share&utm_medium=web

Tawnya Means's avatar

Thank you Tina! This is very helpful and I appreciate the thorough discussion of the nuances. I will adjust my writing moving forward.

Joseph Cosgriff's avatar

Great points. My colleague Daryl Privott and I have found the LOVE framework (Logic, Originality, Verifiability, and Ethics) useful for helping faculty and students engage with AI more intentionally.

Your discussion of Zone 3 strategic delegation is especially helpful, because LOVE teaches students a structure for deciding what to delegate, what to keep as human work, and how to evaluate AI output critically. In that sense, it can support the shift from scattered AI use toward the purposeful, reflective engagement you describe.

Your point about the Blind Spot is equally important. We have found Verifiability especially useful for making AI use more visible, discussable, and tied to learning goals, which can shift faculty from trying to police AI toward designing assessments that support deeper reasoning.

I’d be eager to see more on assignment design, disclosure, and policy. Pedagogical problems deserve pedagogical solutions.

Tawnya Means's avatar

Thank you for the feedback and sharing what you do! Are you looking for disclosure and policy as it relates to assignment design? Or are you thinking of these as three separate topics?

Joseph Cosgriff's avatar

That’s a great question. I see them as deeply interconnected. Effective assignment design essentially provides the structure where disclosure becomes a meaningful part of the process rather than just a boilerplate statement.

When an assignment is designed to require a 'Logic' or 'Verifiability' check, policy shifts from a set of restrictive rules into a collaborative agreement on how to use the tool ethically. I'd be especially interested in your thoughts on how to move disclosure beyond a simple syllabus statement and into the actual metadata or reflective components of a student's work.

Tawnya Means's avatar

This is spot on. I always recommend that we have real conversations on an ongoing basis with students. Model the behavior we want to see. Show them how you are using AI. Show them when it is right and when it is wrong. If they ask you a question, show them how you think through getting answer. Blow their minds with how AI can empower your thinking. It should normalize as just part of how we work and think.

Jason Wright's avatar

Zone 3 is the right destination and you've named the pedagogical path to it precisely.

The deeper structural question underneath it: Zone 3 learning produces a fundamentally different kind of human capability than the credential system was designed to measure or reward. A student who has spent three years developing genuine judgment, questioning assumptions, and directing AI toward original synthesis has built something real and valuable. But it doesn't show up on a transcript the way mastery of syntax does.

The assessment redesign you're recommending — oral exams, portfolios, AI transcripts — is the right direction within existing institutional frameworks. The more fundamental shift is building the attribution infrastructure that makes the Zone 3 learner's body of work directly legible and verifiable outside the institution entirely.

When the portfolio credential becomes a verifiable record of what a productive entity has actually produced — not curated by the student, not evaluated by the faculty, but structurally encoded at the point of production — the Johari Window blind spot closes by architecture rather than by awareness.

The equity point is the most urgent one. Zone 3 as a privilege of the resourced is the productive entity two-tier economy playing out in the classroom before the legal infrastructure even exists. Universal baseline access to capable AI agents isn't just an economic equity question. It's an educational equity question with the same urgency.

Kevin Wright's avatar

It is about Human - AI collaboration (not unlike other intelligent tools that have emerged in the past).

Lots of discussions around "Will AI take your job, the better question is "Will someone who knows how to collaborate with AI take your job?".

There is a concept used in financial analysis, namely “The Efficient Frontier” that gives us a perspective in the implementation of AI.

The Efficient Frontier:

It’s the set of “best” portfolios where:

• You can’t get higher returns without taking more risk, and

• You can’t reduce risk without lowering returns

Think of it as the “sweet spot curve” of optimal choices.

Using this concept of “The Efficient Frontier” to example how humans and AI should collaborate would lead us to the question:

Instead of organizations asking: How much AI should we use?

The better question is: What combination of human judgment and AI capability puts us on the efficient frontier of performance?

Human only and AI only are both suboptimal. The real advantage comes from operating on the efficient frontier of Human–AI collaboration and determining within a specific implementation where judgment and scale are combined, not traded off.

Tawnya Means's avatar

I love this! Thank you for extending the idea.

TeacherAI's avatar

The "measuring the wrong zone" point is so important. Most of the research on AI and cognitive offloading measures whether students perform tasks with or without AI, which misses the question of whether they're developing the right thinking in the right moments. The zone of proximal development framing would actually reframe the whole debate. Offloading the stuff you've already mastered to focus cognitive effort on the edge of your competence isn't dangerous, it's just good learning.

Elizabeth Sturgeon's avatar

Great research! Really helpful for those in the AI trenches of teaching, learning, and assessment. Thx much!

Diego Bonifacino's avatar

The Zone 2 point stands out: scattered, half-committed use costs the most and gets the least. It matches what I see with teams, where delegation is rarely decided on purpose. One lens I use: every task gets a deliberate label (automate, augment, add, or avoid), and the room argues about the label. That makes the freed capacity visible and keeps the choice human. Thanks for this.

wess trabelsi's avatar

I need to push back against the studies cited. The MIT study gave participants 20 minutes for a random SAT-style essay; using ChatGPT to quickly finish this chore is highly rational and doesn't prove diminished capability. Wang and Zhang's results, relying on self-reported Likert scales, are encouraging but not proof. We use the same sources but reach entirely different conclusions, especially when looking beyond the classroom. If you want to see the alternative takeaway—including what a massive KPMG study reveals about the behavioral patterns of true AI power users—check out my breakdown here: https://wesstrabelsi.substack.com/p/cognitive-superloading

Tawnya Means's avatar

I appreciate the continued dialogue, but I think we may be closer than it first appears.

I agree with your methodological pushback. The MIT study does not prove broad diminished capability, and Wang and Zhang should not be treated as causal proof. The MIT task was constrained and artificial; Wang and Zhang relies heavily on self-reported constructs. Those are real limitations.

But that is actually the point I am trying to make. The evidence does not support a simple “AI makes students worse” conclusion. It supports a more precise claim: different patterns of AI use produce different cognitive conditions.

A student using ChatGPT to comply with a course requirement for a 20-minute SAT-style essay may be acting rationally, but that rationality is exactly what should worry educators. If the assignment feels like a chore, students will optimize for completion. That tells us something important about assessment design.

And I think the KPMG findings strengthen this argument rather than overturn it. The power users were not just using AI more. They were framing problems, iterating, scrutinizing outputs, delegating with intent, and applying domain understanding. That is very close to what I am calling strategic cognitive offloading.

For readers who have not explored the KPMG study, here is the TL;DR: KPMG and researchers at UT Austin analyzed more than 1.4 million workplace AI interactions from 2,597 KPMG users over eight months. The key distinction was not simply who used AI the most. The strongest users showed patterns of frequency, persistence, ambition, and intentionality. In practice, that meant they used AI regularly, brought more complex problems to it, iterated instead of accepting first outputs, and chose tools and prompting strategies deliberately. Importantly, only a small share of users consistently demonstrated these sophisticated behaviors.

That is where I think your “cognitive superloading” framing is useful, but I would be careful not to overstate what the KPMG study proves. I read it less as causal evidence that AI automatically enhances cognition and more as evidence that sophisticated AI use is possible, observable, and teachable. It shows what strong human-AI collaboration can look like in practice, but it does not show that all or most users will get there simply by having access to the tools.

So I do not think the question is offloading versus superloading. The real question is: what conditions move learners from shortcutting to sophisticated partnership?

That is where higher education is under-designed. We cannot assume students will become KPMG-style power users through exposure alone. We have to teach the discernment, metacognition, domain knowledge, and accountability structures that make that kind of use possible.

In that sense, I do not see your argument as a rebuttal so much as an important extension. The goal should not be to scare people away from AI or to celebrate all AI use as cognitive enhancement. The goal should be to design learning experiences that help students move from unexamined offloading to intentional, accountable, and intellectually demanding collaboration with AI.

wess trabelsi's avatar

Agreed. My focus is initially more on K–12 education, where the incentive structure to perform is slightly different than in higher ed (although...). As you point out, the KPMG profiling of power users reveals patterns of use that are indeed teachable. However, my contention is that where most people fail is treating AI "skills" as a technical capability, when the challenge is almost purely metacognitive.

The technical aspects of understanding what AI is good at and what it's not—what Mollick dubbed the jagged frontier—are evolving too quickly to ever be formalized in a curriculum. Trial and error are the only paths to mastery on the technical side. So, when it comes to teaching this proficiency, accepting it as metacognitive shapes the strategies we employ, but it also highlights the difficulty. It's a commitment that requires intrinsic motivation. At least, that's my argument..

The old wisdom of playing by the rules to get a stable, comfortable life simply isn't true anymore, especially for K–12 kids, but even for college students. Therefore, extrinsic motivation will lead to reward hacking in most cases, not to the development of metacognitive abilities—especially in adults, if they haven't already developed those skills.

Foundation's Edge's avatar

The U-curve reframes this well, but Zone 3 has a prerequisite the design discussion stops short of: what the AI actually does with the work you delegate to it. The model evaluations behind our series show systematic directional preferences — institutional sources surface over practitioner knowledge, quantified formats outrank the quality of an argument — and those preferences compound quietly across every layer of AI-assisted work. That makes the freed capacity in Zone 3 more constrained than it looks: it has to go toward auditing what the delegation itself changed, or the higher-order reflection inherits the same bias. I mapped where that compounding shows up here: https://seldondance.substack.com/p/when-ai-helps-you-think

#CriticalThinking #AIEducation