AI helped people perform better while they were using it.
Then the AI disappeared—and their performance got worse.
That is the uncomfortable result of a peer-reviewed study involving nearly 1,000 high-school students. With access to a standard GPT-4 assistant, students performed 48% better during practice than students working without AI. But when they took an exam without assistance, they performed 17% worse than the control group.
The tool improved the visible output while weakening the learning underneath it.
That should concern more than teachers and parents.
Every day, marketers ask AI to create positioning. Ecommerce teams use it to analyse reviews, write product pages and design experiments. Managers ask it to summarise documents they have not read. Developers accept code they could not explain five minutes later.
The work gets finished. The dashboard looks productive. But are we becoming more capable—or merely more dependent?
Welcome to the AI productivity trap.
First, what did the study actually prove?
The paper, Generative AI Without Guardrails Can Harm Learning, was published in the journal Proceedings of the National Academy of Sciences.
Researchers conducted a randomized controlled trial at a large high school in Turkey during the 2023–2024 academic year. Nearly 1,000 students across grades 9, 10 and 11 participated in four 90-minute mathematics sessions. Classes were assigned to one of three groups:
- Control: Students used their normal books and notes without generative AI.
- GPT Base: Students had a standard GPT-4 chat interface similar to ChatGPT.
- GPT Tutor: Students used GPT-4 with teacher-designed safeguards. It was instructed to provide hints, ask students to show their work and avoid giving away complete answers.
Each session contained assisted practice followed by an exam completed without AI or other resources.
The results were striking:
- GPT Base improved assisted practice performance by 48% compared with the control group.
- GPT Tutor improved assisted practice performance by 127%.
- On the unassisted exam, the GPT Base group performed 17% worse than the control group.
- The GPT Tutor group performed about the same as the control group on the exam. Its learning penalty was essentially removed, but it produced no statistically detectable exam improvement.
The researchers also examined how students interacted with the systems. GPT Base users frequently asked for answers and copied solutions. GPT Tutor users were more likely to attempt answers, ask for help and work through the problem.
Perhaps the most worrying finding was that students did not recognise the damage. Those using the standard assistant did not believe they had learned less or would perform worse.
Their output gave them a feeling of mastery that their independent performance could not support.
Important fact check: this does not prove AI lowers intelligence
“AI is making us dumber” is a deliberately provocative headline. The study did not measure IQ, permanent cognitive decline or the long-term impact of AI on adults at work.
It studied short-term mathematics learning in one Turkish high school. The exam followed the practice session, and the authors explicitly noted that long-term learning remains a subject for future research.
We should therefore avoid turning one strong experiment into a universal law.
What the study does demonstrate is narrower—and highly relevant:
AI can increase performance while the tool is present without building the user’s underlying ability. When the tool provides answers too easily, independent performance can become worse.
That is not proof that AI makes everyone less intelligent. It is evidence that unguarded AI use can replace the mental effort required to learn.
For businesses, that distinction is more useful than the headline.
The difference between performance and capability
We often treat these words as if they mean the same thing.
They do not.
Performance is the quality of what you produce today.
Capability is what you can understand, judge and produce tomorrow—even when the tool is unavailable, wrong or facing a situation it has never seen.
AI can raise performance instantly. It can write a polished email, summarise a report, produce ten campaign ideas or generate a complete product description in seconds.
But polished output can hide weak understanding.
If you cannot explain why a recommendation is correct, recognise when it is wrong or reproduce the reasoning in a new context, the capability may belong to the tool rather than to you.
That is the trap: borrowed competence feels like personal competence while the AI is present.
How the AI productivity trap appears in marketing
Imagine a marketer asks AI:
“Create a campaign strategy for our new skincare product.”
The model produces personas, hooks, channel recommendations and a 30-day content calendar. The result looks comprehensive. The marketer cleans up the language, puts it into slides and presents it.
But ask three follow-up questions:
- Why is this the right audience?
- Which customer evidence supports this positioning?
- What would make you abandon this strategy after launch?
If the marketer cannot answer without reopening the chatbot, AI did not multiply their strategy. It substituted for it.
Over time, this changes the job. The marketer becomes skilled at requesting and formatting answers but less practised at customer research, positioning and making trade-offs.
The deliverables continue. The thinking weakens quietly.
How it appears in ecommerce
The same risk exists across ecommerce operations.
Product descriptions
AI can create hundreds of descriptions quickly. But if nobody understands the customer’s objections, product differentiators or regulatory boundaries, the catalogue becomes fluent and generic.
Customer-review analysis
An AI summary may identify recurring complaints. But the operator who never reads the original reviews may miss sarcasm, emerging edge cases or the emotional language customers actually use.
Conversion optimisation
AI can recommend tests, but it does not automatically understand your traffic quality, technical constraints, commercial margins or brand history. A team that accepts experiments without forming its own hypothesis is generating activity, not learning.
Reporting
AI can explain why revenue changed. If the analyst cannot trace that explanation back to reliable data, it becomes a confident story attached to a chart—not analysis.
In each case, the immediate task becomes easier. The organisation’s ability to question, interpret and decide can still become weaker.
The AI Crutch Test
You do not need to stop using AI. You need a quick way to recognise when assistance has become dependence.
Ask yourself these three questions.
1. Could I explain the result without reopening the AI?
You do not need to reproduce every sentence from memory. You should be able to explain:
- What conclusion was reached
- What evidence supports it
- Which assumptions it depends on
- Where it might fail
If you can only defend the output by saying “the AI suggested it,” you have an answer but not understanding.
2. Did I think before I prompted?
Write down your initial hypothesis, outline or decision criteria before asking the model.
For a campaign, define the customer, problem and desired action. For an ecommerce test, state what you expect to happen and why. For research, list what evidence would change your mind.
Then use AI to challenge, expand or improve that thinking.
If the blank prompt box is always the beginning of your thought process, you are outsourcing the most valuable part.
3. Can I detect a confident but wrong answer?
The study’s ordinary GPT interface sometimes produced incorrect mathematics. Yet the researchers found that copying—not merely exposure to incorrect answers—was the main mechanism behind the learning penalty.
The lesson is important: AI errors are most dangerous when the user has stopped engaging deeply enough to notice them.
Before using an output, ask:
- Which claims require verification?
- Does this conflict with our data or experience?
- What information might the model be missing?
- Would I be comfortable putting my name behind this decision?
If you lack enough domain knowledge to evaluate the answer, involve someone who has it.
If you answer “no” to any part of the AI Crutch Test, change how you are using the tool—not necessarily the tool itself.
Use AI as a coach, not an answer machine
The most encouraging part of the study was the guarded GPT Tutor.
It used the same underlying GPT-4 technology, but its behaviour was different. It had correct, teacher-supplied problem information. It was instructed to give hints rather than full solutions and to ask students to attempt the work first.
That design eliminated the detectable exam penalty.
For professionals, we can reproduce some of those safeguards through better workflows.
Instead of asking:
“Write the strategy.”
Try:
“Here is my proposed strategy and the customer evidence behind it. Challenge my assumptions, identify the weakest argument and ask me three questions before recommending changes.”
Instead of:
“Analyse this performance report.”
Try:
“I believe conversion fell because mobile traffic shifted toward a lower-intent source. Test this hypothesis against the data, show contradictory evidence and separate facts from inference.”
Instead of:
“Give me the answer.”
Try:
“Do not solve this immediately. Give me one hint, let me attempt it, then critique my reasoning.”
The goal is to keep yourself inside the cognitive loop.
A practical framework: Think, Ask, Verify, Rebuild
For work that develops an important professional skill, use this four-step process.
Think
Form your first view without AI. Even five minutes of independent thinking forces you to retrieve knowledge, notice uncertainty and create a position worth testing.
Ask
Use AI for counterarguments, options, missing evidence and feedback—not only for finished deliverables.
Verify
Check material claims against original sources, company data and domain experts. Separate what the model knows from what it inferred.
Rebuild
Close the AI and summarise the conclusion in your own words. State the decision and why you made it. If you cannot, you are not finished.
This takes longer than copying the first response. That small amount of productive friction is the point.
Not every task needs to teach you something
There is an important counterargument: we routinely use calculators, search engines, templates and automation without practising the underlying task every time.
That is sensible.
You do not need to preserve your ability to manually rewrite 500 product titles or format meeting notes. Offloading low-value work is one of AI’s biggest benefits.
The real question is whether you are automating execution or surrendering judgement.
Use AI aggressively for:
- Reformatting and repetitive transformations
- First-pass transcription and summarisation
- Translation drafts followed by appropriate review
- Generating variations after the core strategy is decided
- Routine documentation
Keep humans actively involved in:
- Defining the problem
- Evaluating evidence
- Making strategic trade-offs
- Understanding customers
- Approving high-impact decisions
- Recognising when the situation has changed
The boundary will differ by role, but every team should define it intentionally.
The biggest risk is the skill you stop practising
AI dependence will not always feel like decline.
It may feel like speed.
You complete more tasks, respond faster and produce work that looks more polished. Meanwhile, the moments that once forced you to struggle, remember, compare and decide slowly disappear from your day.
That struggle was not always inefficiency. Sometimes it was the mechanism through which expertise developed.
The study’s students were not lazy or unintelligent. They reacted naturally to a system that offered an easier path to the answer. Professionals will do the same—especially when every workplace metric rewards output today rather than capability next year.
That is why individual discipline is not enough. Leaders must design workflows and incentives that preserve thinking.
Do not ask only, “How much faster did AI make the team?”
Also ask:
- Can the team explain and defend the output?
- Are people improving at their core discipline?
- Can they work when the model fails?
- Is AI producing learning—or merely deliverables?
AI should make your judgement more powerful, not less necessary.
Final thought
The future will not belong to people who refuse AI. It will not automatically belong to the people who use it most either.
It will belong to those who know when to delegate to AI, when to challenge it and when to close the window and think for themselves.
Use the tool to increase your range.
Just make sure the intelligence in the workflow still includes yours.
Frequently asked questions
Does research prove that AI is making people dumber?
No. The cited study did not measure intelligence or permanent cognitive decline. It found that unguarded GPT-4 assistance improved immediate practice performance but reduced short-term unassisted exam performance in a specific high-school mathematics setting.
What was the 48% and 17% AI study?
In a randomized trial involving nearly 1,000 Turkish high-school students, a standard GPT-4 assistant increased performance during assisted practice by 48% relative to the control group. The same group later performed 17% worse than the control group on an immediate exam without AI.
Did the guarded AI tutor harm learning?
The guarded GPT Tutor improved assisted practice performance by 127% and performed approximately the same as the control group on the unassisted exam. It removed the statistically detectable penalty but did not create a statistically significant exam improvement.
How can professionals prevent AI skill atrophy?
Think before prompting, use AI to challenge rather than replace your reasoning, verify important claims and explain the final decision without relying on the original AI response.
Should marketers and ecommerce teams stop using AI?
No. They should automate repetitive execution while retaining human ownership of customer understanding, strategy, evidence evaluation and final decisions.
