Does leboncoin.fr block AI crawlers?
leboncoin.fr couldn't be firewall-tested for OAI-SearchBot, Claude-SearchBot and PerplexityBot because it refused an ordinary browser request too. Its robots.txt blocked 1 of 7 AI training crawlers. We tested it on 13 October 2026 as part of our study of the top 5,000 websites, where leboncoin.fr ranked #1,210.
Can AI search engines cite it?
ChatGPT search · OAI-SearchBot
Firewall test inconclusive
The site answered our ordinary browser request with HTTP 403, so a crawler test can't tell bot blocking apart.
Claude · Claude-SearchBot
Firewall test inconclusive
The site answered our ordinary browser request with HTTP 403, so a crawler test can't tell bot blocking apart.
Perplexity · PerplexityBot
Firewall test inconclusive
The site answered our ordinary browser request with HTTP 403, so a crawler test can't tell bot blocking apart.
Firewall results come from requests that identified as each crawler from our own connection, not from the crawlers' IP addresses, so the real crawler may be treated differently.
This is a snapshot from 13 October 2026.
Run a live audit to see what AI crawlers get from leboncoin.fr today, plus 50 more checks.
At a glance
- AI training crawlers blocked
- 1 of 7 (Bytespider)
- robots.txt
- Published, with rules
- CDN
- Amazon CloudFront
- Content-Signal
- None
- Home page to a browser
- HTTP 403
All 31 crawlers in leboncoin.fr's robots.txt
| Crawler | Job | leboncoin.fr | Sites blocking it |
|---|---|---|---|
| OAI-SearchBot | AI search index | Allowed (named rule) | 7.5% |
| Claude-SearchBot | AI search index | Allowed (named rule) | 7.1% |
| PerplexityBot | AI search index | Allowed (named rule) | 10% |
| MistralAI-Index | AI search index | Allowed (named rule) | 4.1% |
| YouBot | AI search index | Allowed (no rule applies) | 10% |
| ChatGPT-User | Fetches when a user asks | Allowed (named rule) | 9.5% |
| Claude-User | Fetches when a user asks | Allowed (named rule) | 7.5% |
| Perplexity-User | Fetches when a user asks | Allowed (named rule) | 7.6% |
| Google-GeminiNotebook | Fetches when a user asks | Allowed (no rule applies) | 3.9% |
| Meta-ExternalFetcher | Fetches when a user asks | Allowed (no rule applies) | 7.9% |
| Amzn-User | Fetches when a user asks | Allowed (no rule applies) | 4.8% |
| DuckAssistBot | Fetches when a user asks | Allowed (named rule) | 7.9% |
| MistralAI-User | Fetches when a user asks | Allowed (named rule) | 7.1% |
| Googlebot | Search engine | Allowed (named rule) | 2.1% |
| Bingbot | Search engine | Allowed (named rule) | 2.3% |
| Meta-WebIndexer | Search engine | Allowed (no rule applies) | 6.3% |
| Amzn-SearchBot | Search engine | Allowed (no rule applies) | 5.1% |
| Applebot | Search and training under one name | Allowed (named rule) | 3.5% |
| Meta-ExternalAgent | Search and training under one name | Allowed (no rule applies) | 12% |
| Amazonbot | Search and training under one name | Allowed (no rule applies) | 9.8% |
| GPTBot | Model training | Allowed (named rule) | 14% |
| ClaudeBot | Model training | Allowed (named rule) | 13% |
| Google-Extended | Model training | Allowed (named rule) | 12% |
| Applebot-Extended | Model training | Allowed (named rule) | 11% |
| MistralAI-Training | Model training | Allowed (no rule applies) | 4.2% |
| Bytespider | Model training | Blocked (named rule) | 15% |
| CCBot | Model training | Allowed (named rule) | 15% |
| Google-Agent | Browses and acts for a user | Allowed (no rule applies) | 4.2% |
| OAI-AdsBot | Ads checks and site-owner services | Allowed (no rule applies) | 4.0% |
| Google-CloudVertexBot | Ads checks and site-owner services | Allowed (no rule applies) | 6.9% |
| Meta-ExternalAds | Ads checks and site-owner services | Allowed (no rule applies) | 3.9% |
“Sites blocking it” is the share of the 2,332 live sites with readable robots.txt rules in the same study. Matching follows RFC 9309: a crawler obeys the group that names it, otherwise the rule for all crawlers.
Run leboncoin.fr?
Blocking training crawlers doesn't remove you from AI answers; blocking the search crawlers does. To keep AI search and opt out of training, use the AI robots.txt generator, then check your CDN with a live audit. Every crawler's user agent is in the AI crawler list, and the full method and data are in the study.
Free to run. Pay only for the fixes you want.
Free
No login
Your score, every check's result, and the full fix for the top 3 issues.
★ Starter
$9 / ₹499 once
Every issue with its exact fix for one site, PDF/Markdown/JSON exports, re-audit for 30 days. Nothing renews.
Pro
$29 / ₹1,499 / month
Full reports on every site, weekly alerts when AI access breaks, competitor compare.
Agencies: white-label widget, branded reports and lead capture on Agency.