Configuratie robots.txt

Aceasta pagina documenta configuratia robots.txt recomandata pentru none.health, inclusiv regulile pentru crawlerele AI.

Configuratie robots.txt de aplicat in Shopify Admin > Online Store > Themes > Edit code > robots.txt.liquid:

# AI Training crawlers - block to prevent content being used for model training
User-agent: GPTBot
Disallow: /

User-agent: CCBot
Disallow: /

User-agent: Meta-ExternalAgent
Disallow: /

User-agent: FacebookExternalHit
Disallow: /

# AI Retrieval crawlers - ALLOW - these drive citations in ChatGPT, Claude, Perplexity
User-agent: OAI-SearchBot
Allow: /

User-agent: anthropic-ai
Allow: /

User-agent: Claude-Web
Allow: /

User-agent: PerplexityBot
Allow: /

User-agent: Google-Extended
Allow: /

User-agent: Applebot-Extended
Allow: /

# Standard rules (Shopify default - keep these)
{% for group in robots.default_groups %}
  {% for rule in group.rules %}
{{ rule }}
  {% endfor %}
{% endfor %}

# Sitemaps
Sitemap: https://none.health/sitemap.xml