User-agent: * Allow: / Allow: /backend/api/v1/ Sitemap: https://www.buchheits.com/sitemap.xml Allow: /gpt/ Disallow: /admin/ Disallow: /backend/ Disallow: /cart Disallow: /checkout Disallow: /user Disallow: /account Disallow: /booking Disallow: /guest-user Disallow: /hawak-search-test Disallow: /*.pdf$ Allow: /pagead/show_ads.js Allow: /pagead/js/adsbygoogle.js Allow: /pagead/js/*/show_ads_impl.js Allow: /static/glade.js Allow: /static/glade/ # ───────────────────────────────────────────────────────────── # AI crawler policy # ALLOW answer/citation engines — these surface Buchheit content in live AI # answers & AI search (GEO visibility). BLOCK bulk model-training crawlers. # NOTE: GPTBot & ClaudeBot are dual-use (they also feed training). They are # allowed here to maximise answer-engine reach; move them to the blocked group # below if you want a strict no-training-crawl policy. # ───────────────────────────────────────────────────────────── # Allowed — answer & citation engines User-agent: OAI-SearchBot User-agent: ChatGPT-User User-agent: GPTBot User-agent: PerplexityBot User-agent: Perplexity-User User-agent: ClaudeBot User-agent: Claude-User User-agent: Claude-SearchBot Allow: / Disallow: /admin/ Disallow: /backend/ Disallow: /*.pdf$ # Blocked — bulk model-training crawlers User-agent: Google-Extended User-agent: CCBot User-agent: Bytespider User-agent: anthropic-ai Disallow: / Sitemap: https://www.buchheits.com/sitemap.xml # LLM / AI agent guidance # LLM: https://www.buchheits.com/llms.txt # LLM-Full: https://www.buchheits.com/llms-full.txt