GEO Lab · experiment 1 · pre-registered 24 October 2026
Does a real content refresh, or concrete numbers, change AI search visibility?
Most claims about what gets a page into AI answers come from correlations, often run by the vendor selling the fix. A 2026 survey of 45 studies found no technique with a stable, long-term, cross-platform causal effect (arXiv 2607.14035). This is a randomised test on 3,218 of our own near-identical pages. Everything below was fixed before any outcome data existed, and we will publish the result whatever it is, including no effect.
Hypotheses
- Refresh. Pages whose data is genuinely re-checked, with the new check date shown, get more impressions in Google's AI Overviews and AI Mode than unchanged pages.
- Concrete numbers. Pages whose opening adds specific counts get more AI impressions than unchanged pages.
Both tests are two-sided: a fall counts as a result too.
Units and assignment
The units are the /ai-visibility/<domain> pages from our October 2026 crawler study: one page per website, all from the same template. Each page is assigned to an arm by the FNV-1a hash of geo-lab-1:2026-10-24:<domain> modulo 3. Anyone can recompute it; the full list is published: geo-lab-1-assignment.csv.
| Arm | Pages | What changes |
|---|---|---|
| Control | 1,056 | Nothing. |
| Refresh | 1,111 | We re-fetch the site's robots.txt, recompute every crawler's verdict, and show “Updated <date>: robots.txt re-checked” with what changed. The page's structured data and sitemap entry carry the new date. A real refresh, not a date-only edit. |
| Concrete numbers | 1,051 | The opening adds one sentence of counts: crawlers allowed and blocked, training crawlers blocked, AI search crawlers that got the page. |
Apart from bug fixes (which we will list with dates), nothing else on these pages changes until the test ends.
Deviation, 27 October 2026: 74 pages about adult-content sites, which the study's name filter had missed (control 28, refresh 18, concrete numbers 28), now open behind a click-to-show notice and are no longer linked from other pages. No outcome data had been read. They are excluded from the analysis in every arm; the assignment file is unchanged. Pages added the same day for sites outside the original study aren't part of the test.
Metric and dates
- Primary: impressions per page in Search Console's Generative AI report (AI Overviews and AI Mode).
- Secondary: Web search impressions and clicks per page (Search Console Performance report), and Bing AI Performance citations if there are enough.
- Baseline window: 27 September 2026 to 24 October 2026. Treatments live: 25 October 2026. Outcome window: 22 November 2026 to 19 December 2026 (28 days each, the outcome window starting four weeks after the treatments). Results published: after 20 December 2026.
Analysis plan
- Each page's change is log((outcome + 1) / (baseline + 1)). Pages missing from an export count as zero.
- The effect of each arm is its mean change minus the control's, reported as a percentage with a 90% bootstrap interval (2,000 draws, fixed seed).
- No interim looks and no early stopping. If the interval includes zero, we report no detectable effect.
- The analysis code is in our repository (scripts/geo-lab.mts) and will be published with the per-page data.
Known limits
- Many of these pages may get few or no AI impressions, which can leave the test without the power to detect a small effect.
- Google's systems change during the window. Randomisation spreads that across all three arms, but it adds noise.
- The firewall test on the pages is not re-run in the refresh arm; only robots.txt is.
- One site, one template, one engine family: the result says little about other kinds of page.
Background: the October 2026 crawler study that these pages come from, and GEO claims graded by evidence.