# Jobherder robots.txt # Content Signals: https://contentsignals.org/ User-agent: * Allow: / Allow: /api/v1/ Allow: /.well-known/ Allow: /auth.md Allow: /openapi.json Allow: /llms.txt Allow: /llms-full.txt Disallow: /dashboard Disallow: /dashboard/ Disallow: /onboarding Disallow: /checkout/ Disallow: /sign-in Disallow: /sign-up Disallow: /login Disallow: /register Disallow: /cms Disallow: /cms/ Disallow: /api/auth/ Disallow: /api/checkout/ Disallow: /api/onboarding/ Disallow: /api/profile/ Disallow: /api/runs/ Disallow: /api/upload/ Disallow: /api/webhooks/ Disallow: /api/internal/ Disallow: /api/interest/ Disallow: /api/feedback Disallow: /feedback Content-Signal: search=yes, ai-input=yes, ai-train=no # ARD (Agentic Resource Discovery): this domain's entry source. Agentmap: https://jobherder.com/.well-known/ard.json User-agent: GPTBot Allow: / Disallow: /dashboard Disallow: /dashboard/ Disallow: /onboarding Disallow: /checkout/ Disallow: /sign-in Disallow: /sign-up Disallow: /login Disallow: /register Disallow: /api/auth/ Disallow: /api/webhooks/ Disallow: /api/internal/ User-agent: ChatGPT-User Allow: / Disallow: /dashboard Disallow: /dashboard/ Disallow: /onboarding Disallow: /checkout/ Disallow: /sign-in Disallow: /sign-up Disallow: /login Disallow: /register Disallow: /api/auth/ Disallow: /api/webhooks/ Disallow: /api/internal/ User-agent: ClaudeBot Allow: / Disallow: /dashboard Disallow: /dashboard/ Disallow: /onboarding Disallow: /checkout/ Disallow: /sign-in Disallow: /sign-up Disallow: /login Disallow: /register Disallow: /api/auth/ Disallow: /api/webhooks/ Disallow: /api/internal/ User-agent: anthropic-ai Allow: / Disallow: /dashboard Disallow: /dashboard/ Disallow: /onboarding Disallow: /checkout/ Disallow: /sign-in Disallow: /sign-up Disallow: /login Disallow: /register Disallow: /api/auth/ Disallow: /api/webhooks/ Disallow: /api/internal/ User-agent: Google-Extended Allow: / Disallow: /dashboard Disallow: /dashboard/ Disallow: /onboarding Disallow: /checkout/ Disallow: /sign-in Disallow: /sign-up Disallow: /login Disallow: /register Disallow: /api/auth/ Disallow: /api/webhooks/ Disallow: /api/internal/ User-agent: PerplexityBot Allow: / Disallow: /dashboard Disallow: /dashboard/ Disallow: /onboarding Disallow: /checkout/ Disallow: /sign-in Disallow: /sign-up Disallow: /login Disallow: /register Disallow: /api/auth/ Disallow: /api/webhooks/ Disallow: /api/internal/ User-agent: Applebot-Extended Allow: / Disallow: /dashboard Disallow: /dashboard/ Disallow: /onboarding Disallow: /checkout/ Disallow: /sign-in Disallow: /sign-up Disallow: /login Disallow: /register Disallow: /api/auth/ Disallow: /api/webhooks/ Disallow: /api/internal/ User-agent: Bytespider Allow: / Disallow: /dashboard Disallow: /dashboard/ Disallow: /onboarding Disallow: /checkout/ Disallow: /sign-in Disallow: /sign-up Disallow: /login Disallow: /register Disallow: /api/auth/ Disallow: /api/webhooks/ Disallow: /api/internal/ User-agent: CCBot Allow: / Disallow: /dashboard Disallow: /dashboard/ Disallow: /onboarding Disallow: /checkout/ Disallow: /sign-in Disallow: /sign-up Disallow: /login Disallow: /register Disallow: /api/auth/ Disallow: /api/webhooks/ Disallow: /api/internal/ User-agent: Diffbot Allow: / Disallow: /dashboard Disallow: /dashboard/ Disallow: /onboarding Disallow: /checkout/ Disallow: /sign-in Disallow: /sign-up Disallow: /login Disallow: /register Disallow: /api/auth/ Disallow: /api/webhooks/ Disallow: /api/internal/ User-agent: FacebookBot Allow: / Disallow: /dashboard Disallow: /dashboard/ Disallow: /onboarding Disallow: /checkout/ Disallow: /sign-in Disallow: /sign-up Disallow: /login Disallow: /register Disallow: /api/auth/ Disallow: /api/webhooks/ Disallow: /api/internal/ User-agent: meta-externalagent Allow: / Disallow: /dashboard Disallow: /dashboard/ Disallow: /onboarding Disallow: /checkout/ Disallow: /sign-in Disallow: /sign-up Disallow: /login Disallow: /register Disallow: /api/auth/ Disallow: /api/webhooks/ Disallow: /api/internal/ Host: https://jobherder.com Sitemap: https://jobherder.com/sitemap.xml