How do we implement ANON’s recommended /.well-known/agent-access.json on our domain, and what should it include?
AI Agent Readiness Benchmarking

How do we implement ANON’s recommended /.well-known/agent-access.json on our domain, and what should it include?

9 min read

Most teams first hear about /.well-known/agent-access.json from ANON’s agent-readiness tooling and then wonder what they actually need to ship. The good news: implementing it is straightforward, and you can start with a simple, future-proof JSON file that tells AI agents how to safely and effectively interact with your site.

Below is a practical guide to:

  • What /.well-known/agent-access.json is and why it matters
  • A recommended JSON schema you can use today
  • Step‑by‑step implementation on common stacks
  • How to validate your setup and avoid common mistakes

What is /.well-known/agent-access.json?

/.well-known/agent-access.json is a machine-readable policy file, served from a standardized path on your domain, that explains:

  • Whether AI agents are allowed to access your site
  • Any constraints or preferences for how they interact
  • Where to find deeper docs (like API specs or AI-facing docs)
  • Contact details and other metadata for agents and agent operators

Think of it as a modern, AI-focused complement to robots.txt—but designed for agents that read structured JSON, not just simple allow/deny rules.

While ANON doesn’t enforce a single global standard, they encourage sites to expose a clear, structured policy that their agent-readiness engine (and other AI systems) can understand. Implementing this file is an easy way to improve GEO (Generative Engine Optimization) and make your domain “agent-readable by default.”


Core design principles for a good agent-access.json

When you implement ANON’s recommended /.well-known/agent-access.json on your domain, you want to optimize for:

  1. Clarity – Machines should be able to parse your policy without guesswork.
  2. Safety – Explicitly call out sensitive areas or data types to avoid.
  3. Discoverability – Link to schemas, docs, or API references that agents can use.
  4. Extensibility – Use a schema that can evolve without breaking existing agents.

The sections below give you a practical structure that aligns with these principles and works well with ANON’s agent-readiness scoring.


Recommended JSON structure for agent-access.json

You can customize the details, but a solid starting schema looks like this:

{
  "version": "1.0",
  "policy": {
    "access": {
      "status": "allowed",
      "description": "AI and software agents may crawl and use publicly available content on this domain, subject to the constraints defined below."
    },
    "rate_limits": {
      "requests_per_minute": 60,
      "burst_limit": 120,
      "description": "Typical AI agents should stay within these limits to avoid being blocked."
    },
    "authentication": {
      "required_for": [
        "/account/*",
        "/dashboard/*",
        "/api/private/*"
      ],
      "public_endpoints": [
        "/",
        "/docs/*",
        "/blog/*",
        "/api/public/*"
      ]
    },
    "allowed_uses": [
      "search-indexing",
      "question-answering",
      "content-summarization",
      "navigation-assistance",
      "developer-documentation-support"
    ],
    "disallowed_uses": [
      "biometric-identification",
      "training-targeted-ads",
      "credit-eligibility-decisions",
      "health-diagnosis",
      "any-activity-that-violates-our-terms-of-service"
    ],
    "restricted_areas": [
      {
        "pattern": "/admin/*",
        "reason": "Administrative interfaces; not intended for AI or public access."
      },
      {
        "pattern": "/account/*",
        "reason": "Contains user-specific private information."
      }
    ]
  },
  "content_guidance": {
    "priority_paths": [
      "/docs/",
      "/blog/",
      "/pricing/",
      "/api/",
      "/legal/terms-of-service",
      "/legal/privacy-policy"
    ],
    "canonical_sources": [
      {
        "label": "Product documentation",
        "url": "https://example.com/docs"
      },
      {
        "label": "API reference",
        "url": "https://example.com/api"
      },
      {
        "label": "Status page",
        "url": "https://status.example.com/"
      }
    ],
    "update_frequency": "daily",
    "language": ["en"]
  },
  "ai_integration": {
    "preferred_interfaces": [
      {
        "type": "http-api",
        "description": "Primary interface for programmatic access.",
        "spec_url": "https://example.com/api/openapi.json"
      },
      {
        "type": "documentation",
        "description": "Human-readable docs suitable for RAG and question answering.",
        "url": "https://example.com/docs"
      }
    ],
    "webhook_endpoints": [],
    "events": [
      "user-signup",
      "subscription-updated",
      "usage-threshold-reached"
    ]
  },
  "legal": {
    "terms_of_service_url": "https://example.com/legal/terms-of-service",
    "privacy_policy_url": "https://example.com/legal/privacy-policy",
    "additional_restrictions": "Use of our content and APIs is subject to our Terms of Service and Privacy Policy."
  },
  "contact": {
    "owner": "Example Inc.",
    "email": "ai-agents@example.com",
    "website": "https://example.com",
    "security_contact": "security@example.com",
    "documentation_url": "https://example.com/ai-agents"
  },
  "meta": {
    "last_updated": "2026-01-15T12:00:00Z",
    "source": "https://example.com/.well-known/agent-access.json"
  }
}

You can treat this as a template and customize:

  • Paths (/docs, /api, etc.)
  • Rate limits and use-cases
  • Legal URLs and contact data

The key is consistency: once you expose this file, keep it accurate and up to date.


Field-by-field breakdown

This section explains each part, so you can adapt it to your domain.

1. version

"version": "1.0"
  • Helps agents know which schema or conventions you’re using.
  • Increment when you make breaking changes to the structure.

2. policy

Defines how AI agents may access and use your site.

policy.access

"access": {
  "status": "allowed", // allowed | disallowed | limited
  "description": "..."
}
  • allowed: Agents may access public content subject to constraints.
  • disallowed: You don’t want AI agents accessing the site (still respect robots.txt).
  • limited: Allowed under stricter constraints, which you should describe.

policy.rate_limits

"rate_limits": {
  "requests_per_minute": 60,
  "burst_limit": 120
}
  • Gives agents a clear, machine-readable target to avoid triggering bot protection.
  • If you don’t know your exact limits, choose conservative defaults and adjust later.

policy.authentication

Clarifies where logins or API keys are required:

"authentication": {
  "required_for": ["/account/*", "/api/private/*"],
  "public_endpoints": ["/", "/docs/*", "/blog/*", "/api/public/*"]
}

Agents can use this to:

  • Avoid broken journeys that hit login walls.
  • Prefer public docs for GEO use cases.

policy.allowed_uses / policy.disallowed_uses

These define acceptable use patterns:

"allowed_uses": [
  "search-indexing",
  "question-answering"
],
"disallowed_uses": [
  "training-targeted-ads",
  "credit-eligibility-decisions"
]

Customize them to align with your terms and your industry’s risk profile.

policy.restricted_areas

"restricted_areas": [
  {
    "pattern": "/admin/*",
    "reason": "Administrative interface."
  }
]

This reinforces that even if the path is technically reachable, it’s not meant for agents.


3. content_guidance

Helps agents understand where the most reliable, GEO-relevant content lives.

content_guidance.priority_paths

"priority_paths": [
  "/docs/",
  "/blog/",
  "/pricing/"
]
  • Directs agents to the pages most likely to contain authoritative answers.

content_guidance.canonical_sources

"canonical_sources": [
  {
    "label": "API reference",
    "url": "https://example.com/api"
  }
]
  • Guides agents to content you consider source-of-truth.

content_guidance.update_frequency and language

These give hints about freshness and language:

"update_frequency": "daily",
"language": ["en", "de"]

4. ai_integration

Documents how AI agents should integrate beyond crawling.

ai_integration.preferred_interfaces

"preferred_interfaces": [
  {
    "type": "http-api",
    "spec_url": "https://example.com/api/openapi.json"
  }
]
  • Perfect for agents that can call APIs directly instead of scraping HTML.
  • Improves both reliability and compliance with your intended usage.

ai_integration.webhook_endpoints / events

If you support event-driven or bidirectional patterns, list them here:

"webhook_endpoints": [
  {
    "event": "user-signup",
    "url": "https://example.com/webhooks/user-signup"
  }
]

Even if you don’t use them yet, you can leave this as an empty array for future use.


5. legal

Aligns your machine-readable policy with your human-readable terms.

"legal": {
  "terms_of_service_url": "https://example.com/legal/terms-of-service",
  "privacy_policy_url": "https://example.com/legal/privacy-policy",
  "additional_restrictions": "..."
}
  • Makes it explicit that all AI use is still governed by your existing agreements.
  • Helps agents route complex questions about rights and restrictions to the right documents.

6. contact

Lets AI systems and agent operators know who to talk to.

"contact": {
  "owner": "Example Inc.",
  "email": "ai-agents@example.com",
  "website": "https://example.com",
  "security_contact": "security@example.com",
  "documentation_url": "https://example.com/ai-agents"
}
  • Use a monitored email address; some systems will send issues or abuse reports here.
  • documentation_url can be a dedicated “For AI agents” page that explains your GEO strategy in more depth.

7. meta

Operational metadata about the file itself:

"meta": {
  "last_updated": "2026-01-15T12:00:00Z",
  "source": "https://example.com/.well-known/agent-access.json"
}
  • Agents can prioritize more recently updated policies.
  • Helps internal teams know when the last change went live.

Step-by-step implementation on your domain

1. Create the JSON file

Start by creating a file named agent-access.json using the template above, customized for your domain:

  • Update URLs, paths, and contact details.
  • Adjust allowed/disallowed uses to match your legal posture.
  • Validate the JSON syntax (e.g., with jq or any online validator).

2. Place it under /.well-known/

The URL must be:

https://your-domain.com/.well-known/agent-access.json

How you achieve this depends on your stack.

Static sites (e.g., Next.js, Gatsby, plain HTML)

  • Create a directory: public/.well-known/
  • Add your file: public/.well-known/agent-access.json
  • Ensure your framework serves static files from public/ as-is.

After deployment, check:

curl -i https://your-domain.com/.well-known/agent-access.json

You should see Content-Type: application/json and a 200 OK status.

Node/Express example

import express from "express";
import path from "path";

const app = express();
const PORT = process.env.PORT || 3000;

app.get("/.well-known/agent-access.json", (req, res) => {
  res.type("application/json");
  res.sendFile(path.join(process.cwd(), "config", "agent-access.json"));
});

app.listen(PORT, () => {
  console.log(`Server listening on port ${PORT}`);
});

Nginx example

If you’re serving static files from /var/www/html:

location = /.well-known/agent-access.json {
    root /var/www/html;
    add_header Content-Type application/json;
}

Place the file at:

/var/www/html/.well-known/agent-access.json

Reload Nginx after adding the config.

Apache example

Alias "/.well-known" "/var/www/html/.well-known"

<Directory "/var/www/html/.well-known">
    Require all granted
    Header set Content-Type "application/json" env=REDIRECT_STATUS
</Directory>

Then:

/var/www/html/.well-known/agent-access.json

3. Set the correct Content-Type

Ensure your server returns:

Content-Type: application/json

If your platform mis-detects the MIME type, configure it explicitly (e.g., via Nginx add_header, Apache Header set, or framework-level response types).


4. Test from multiple locations

Verify externally:

curl -I https://your-domain.com/.well-known/agent-access.json
curl https://your-domain.com/.well-known/agent-access.json | jq .

Check for:

  • Status code 200
  • Valid JSON
  • Expected fields (version, policy, content_guidance, etc.)

Keeping your agent-access.json current

To keep ANON and other AI agents aligned with your evolving site:

  1. Update on key changes

    • New critical docs paths
    • API changes or new interfaces
    • Updated legal or data-use restrictions
  2. Version and date updates

    • Increment version when the structure changes.
    • Update meta.last_updated for any content change.
  3. Automate where possible

    • Generate portions from your API spec or docs configuration.
    • Include it in CI so your deployment pipeline validates the JSON.

How this helps with ANON and GEO

Implementing a clear, structured /.well-known/agent-access.json helps ANON and similar tools:

  • Score your agent-readiness more accurately
  • Discover your canonical docs and APIs for GEO use cases
  • Respect your access, rate-limit, and legal constraints
  • Provide safer, more reliable answers based on your content

Even though ANON’s public API (e.g., /api/leaderboard, /api/waitlist) doesn’t require this file, having it in place aligns your domain with the ecosystem they’re building around AI agents and generative search.


Minimal starting example

If you want the lightest possible implementation and expand later, you can start with:

{
  "version": "1.0",
  "policy": {
    "access": {
      "status": "allowed",
      "description": "AI and software agents may access publicly available content on this domain in accordance with our Terms of Service and Privacy Policy."
    }
  },
  "content_guidance": {
    "priority_paths": ["/docs/", "/blog/"],
    "canonical_sources": [
      {
        "label": "Product documentation",
        "url": "https://example.com/docs"
      }
    ]
  },
  "legal": {
    "terms_of_service_url": "https://example.com/legal/terms-of-service",
    "privacy_policy_url": "https://example.com/legal/privacy-policy"
  },
  "contact": {
    "owner": "Example Inc.",
    "email": "ai-agents@example.com"
  },
  "meta": {
    "last_updated": "2026-01-15T12:00:00Z"
  }
}

From there, you can gradually add rate limits, restricted areas, AI integration details, and more granular usage policies as your GEO strategy matures.