Most forecasts about AI search and website traffic rely on observational data, which cannot separate cause from coincidence. A new preprint tried something different: it assigned real Google users to different search experiences and measured what they did. This article explains what that experiment found, where its limits sit, and how a business planning around Generative Engine Optimization (GEO) and AI search should use it.
- In a preregistered experiment, forcing searches into Google’s AI Mode reduced external click-through by 18.8 percentage points versus standard Google Search.
- AI Mode was almost unused at baseline (0.6% of searches), so the result describes forced adoption, not what happens as people opt in gradually.
- The sample was small, short (seven days) and skewed younger and more educated than the US population.
- Treat the figure as a stress-test scenario for traffic plans, and measure your own referrals and inquiries instead of copying it.
How the experiment worked
The study, by Stephanie Wang, Jeffrey Gleason, Yakov Bart, Christo Wilson and Danae Metaxa of the University of Pennsylvania and Northeastern University, was posted to arXiv on August 18, 2026, as a preprint. It was preregistered, meaning the researchers fixed their hypotheses before collecting data. They enrolled 1,444 participants, mostly through Prolific, and installed a browser extension to record search behavior. Of those, 1,100 made at least one search during the experiment.
Each participant was randomly assigned to one of three conditions for seven days. One group kept normal Google Search. Another had AI Overviews and AI Mode hidden. The third had every search redirected into AI Mode. Random assignment is what lets the authors describe the differences as effects of the interface rather than differences between kinds of users, and it is the main reason this study deserves more weight than a traffic correlation.
Click-through rate (in this study): the share of search activity that led to a click on an external website, rather than staying inside Google.
What it found
The headline result is that assignment to AI Mode reduced external click-through by 18.8 percentage points, with a 95% confidence interval of -22.2 to -15.3 points. The authors report that daily search sessions also fell by 0.92 sessions. Clicks fell across site types: the share of users clicking through to news sites dropped 12.5 points, to Reddit 21.2 points and to Wikipedia 9.9 points.
Hiding AI Overviews moved clicks the other way, raising click-through by 8.8 points. That estimate carries more uncertainty, because a Google interface change during the study left compliance in that group at roughly 50%, so the authors report a local average treatment effect. Users in the AI Mode group also spent 0.43 more minutes per session, which suggests they stayed engaged inside Google rather than leaving. The authors note these changes did not come with better trust or experience.
| Condition (7 days) | Change in external click-through | Reliability note |
|---|---|---|
| AI Mode forced for all searches | -18.8 points (95% CI -22.2 to -15.3) | Direct effect of assignment |
| AI Overviews and AI Mode hidden | +8.8 points (95% CI 2.3 to 15.3) | Compliance fell to about 50% after a Google interface change |
| Normal Google Search | Baseline for comparison | AI Overviews appeared on 36% of baseline searches |
What it cannot tell you
The authors state their own limits clearly, and a careful reader should repeat them. Before the experiment, participants used AI Mode for only 0.6% of their searches. That means the AI Mode group reflects a forced, full-adoption scenario, not the gradual, opt-in shift most users would experience. The paper says effects of organic adoption may differ.
The sample was recruited from Prolific and a university work-study program. In the analysis sample, 75% of participants were under 45, 87% had at least some college education, and 58% identified as Democrat versus 20% as Republican. Seven days can capture short-term shifts but not long-run adaptation. Also, the 42.7-point drop in the share of users clicking ads is not evidence about advertising performance, because the authors note AI Mode did not surface ads at the time of the experiment.
For a business, the safest reading is that this is a well-designed upper-bound scenario, not a forecast for your site.
How to use it in planning
The practical value is in how it frames risk. If AI Mode becomes the default for a large share of your audience, the study suggests click-based reporting could fall sharply even if your visibility inside answers is unchanged. That makes the choice of metrics a planning decision, not a reporting detail.
- Pull twelve months of organic landing-page sessions and split them by commercial intent, so you know how much revenue depends on clicks from informational queries.
- Run a stress test: model the effect of an 18.8-point drop in click-through on those pages as a worst-case scenario, and label it that way in the plan.
- Set up referral tracking for AI sources, following our guide to tracking AI referral traffic.
- Report visibility across a fixed prompt set, as explained in how to measure AI search visibility, alongside clicks.
- Add inquiries, demo requests and assisted conversions as primary success measures, since they hold up when clicks fall.
This fits with earlier evidence on click-through. Our write-up of what Pew and Ahrefs measured covers observational data, and this experiment adds a causal check. The budget implications are covered in how much to invest in AI search, and the question of visitor quality in whether AI search traffic converts better.
Frequently Asked Questions
Does this study prove AI Mode will cut my traffic by 18.8%?
No. The figure comes from a forced seven-day scenario with a young, educated sample. Real adoption will be gradual, and the effect on your site depends on your mix of queries.
Should I stop investing in SEO if clicks may fall?
No. The experiment measured clicks, not visibility or brand influence inside answers, and Google’s Search Central documentation says no special optimizations are needed for its AI features. The sensible change is to measure more than clicks.
Has the paper been peer reviewed?
It was posted as an arXiv preprint on August 18, 2026. The design was preregistered, but readers should treat the findings as provisional until peer review.
Planning around a moving search interface is easier with a clear baseline and a measurement plan you control. At Grow with Hafsa we help US and UK businesses set up AI search measurement and build agentic reporting workflows with human review, as part of our AI marketing and performance marketing services.