# fn2.ai robots.txt # Default: allow all crawlers User-agent: * Allow: / Allow: /images/ Allow: /images/logo-grey.png # Block sensitive paths Disallow: /api/ Disallow: /includes/ # --- AI search & retrieval crawlers (explicitly allowed) --- # OpenAI User-agent: GPTBot User-agent: OAI-SearchBot User-agent: ChatGPT-User # Google User-agent: Googlebot User-agent: Google-Extended # Anthropic User-agent: ClaudeBot User-agent: Claude-Web User-agent: anthropic-ai # Perplexity User-agent: PerplexityBot User-agent: Perplexity-User # Microsoft / Bing User-agent: bingbot # Apple User-agent: Applebot User-agent: Applebot-Extended # Others User-agent: CCBot User-agent: Amazonbot User-agent: meta-externalagent User-agent: MistralAI-User User-agent: DuckAssistBot Allow: / Disallow: /api/ Disallow: /includes/ # Sitemaps Sitemap: https://fn2.ai/sitemap.xml Sitemap: https://fn2.ai/blog/sitemap.xml Sitemap: https://fn2.ai/earnings/sitemap.xml