What should go in a /.well-known agent access manifest, and which vendors support generating/validating it?

Most teams experimenting with AI agents and GEO (Generative Engine Optimization) eventually hit the same question: how do we tell agents what they’re allowed to do on our site, and where to find the resources they need? That’s exactly what a /.well-known/agent-access.json manifest is for.

This guide explains:

• What an agent access manifest is (and why it matters for GEO)
• What should go in /.well-known/agent-access.json
• How it fits with llms.txt and other AI-facing files
• Which vendors currently support generating or validating these manifests
• Practical examples and implementation tips

---

What is an agent access manifest?

An agent access manifest is a machine-readable JSON file, typically served at:

/.well-known/agent-access.json

It tells autonomous agents and AI crawlers:

• What they’re allowed to access
• How they should authenticate (if at all)
• Where to find GEO-relevant documentation (like llms.txt or APIs)
• Which actions and workflows are safe and supported
• How to respect your rate limits, privacy rules, and business constraints

Think of it as a “robots.txt for agents,” but with far more structure: instead of just allow/deny rules for URLs, you define capabilities, policies, and integration points.

---

Why you need an agent access manifest for GEO

If you care about GEO—i.e., being visible and usable to AI agents, not just human searchers—agent-access.json helps you:

• Improve discoverability: Agents know where to find APIs, schemas, and workflows relevant to your product.
• Reduce errors and misuse: You proactively tell agents what they must not do (e.g., no PII, no destructive actions).
• Control your surface area: You can expose specific, safe tasks instead of your entire web app.
• Standardize integration: Vendors and agents can integrate with you consistently without bespoke handoffs.

In ANON’s own public API docs, for example, they expose:

• Full agent access manifest: /.well-known/agent-access.json
• LLM-readable overview: /llms.txt

Together, these two files are the core of an “agent-ready” surface.

---

Core structure of /.well-known/agent-access.json

There’s not yet a fully locked-in global standard, but emerging manifests share several common sections. A practical structure typically looks like this:

{
  "version": "1.0",
  "site": {
    "name": "Example Inc",
    "domain": "example.com",
    "contact": {
      "email": "ai-integrations@example.com",
      "url": "https://example.com/contact"
    }
  },
  "documentation": {
    "llms_txt": "https://example.com/llms.txt",
    "api_base_url": "https://api.example.com",
    "api_docs_url": "https://example.com/docs/api",
    "openapi_spec": "https://example.com/openapi.json"
  },
  "access_policies": {
    "allowlist_paths": [
      "/pricing",
      "/docs/**",
      "/api/**"
    ],
    "blocklist_paths": [
      "/admin/**",
      "/account/**",
      "/settings/**"
    ],
    "rate_limits": {
      "anonymous": {
        "requests_per_minute": 60
      },
      "authenticated": {
        "requests_per_minute": 600
      }
    },
    "data_policies": {
      "allow_pii_collection": false,
      "allow_pii_storage": false,
      "logging": "aggregate-only"
    }
  },
  "agents": {
    "allowed_user_agents": [
      "OpenAI*",
      "Anthropic*",
      "AnonAgent*"
    ],
    "blocked_user_agents": [
      "BadBot*"
    ]
  },
  "auth": {
    "methods": [
      {
        "type": "api_key",
        "header": "Authorization",
        "format": "Bearer <API_KEY>",
        "docs_url": "https://example.com/docs/api-auth"
      }
    ],
    "preferred_flow": "api_key"
  },
  "capabilities": [
    {
      "id": "fetch-pricing",
      "description": "Retrieve current pricing and plan details.",
      "method": "GET",
      "endpoint": "https://api.example.com/v1/pricing",
      "parameters": [],
      "rate_limit": {
        "requests_per_minute": 120
      }
    },
    {
      "id": "create-trial-account",
      "description": "Create a new trial account with minimal required fields.",
      "method": "POST",
      "endpoint": "https://api.example.com/v1/trials",
      "parameters": [
        { "name": "email", "type": "string", "required": true },
        { "name": "plan_id", "type": "string", "required": true }
      ],
      "safety": {
        "requires_explicit_human_consent": true
      }
    }
  ],
  "safety": {
    "disallowed_behaviors": [
      "credential-stuffing",
      "scraping-non-public-areas",
      "bypassing-paywalls"
    ],
    "terms_url": "https://example.com/agent-terms",
    "privacy_url": "https://example.com/privacy"
  }
}

Below are the recommended sections and what to include in each.

---

Recommended sections and fields

1. Basic site metadata

Purpose: identify your property and provide a contact for escalation.

Include:

• site.name
• site.domain
• site.contact.email or support URL
• Optional: site.organization_id (if you use a vendor’s ID system)

This helps vendors and agents map your domain to organization records and detect spoofing.

---

2. Documentation and discovery

Purpose: point agents to GEO-friendly documentation.

Common fields:

• documentation.llms_txt: URL of your LLM-oriented overview (e.g. https://yourdomain.com/llms.txt as in ANON’s docs)
• documentation.api_base_url: base URL of your public API (https://api.yourdomain.com)
• documentation.api_docs_url: human-readable docs
• documentation.openapi_spec: machine-readable API spec
• Optional: links to
  • schemas (JSON Schema, GraphQL SDL)
  • workflows or playbooks (pre-defined flows agents can follow)

This section is foundational for GEO: you’re effectively curating what AI systems should read and trust.

---

3. Access policies

Purpose: define what agents may access and at what pace.

Include:

• access_policies.allowlist_paths
• access_policies.blocklist_paths
• access_policies.rate_limits for anonymous vs authenticated access
• access_policies.data_policies indicating:
  • Whether PII can be collected or stored
  • Logging expectations
  • Any jurisdictional restrictions (e.g., no EU user data for certain actions)

Guidelines:

• Use patterns like /docs/** for sections that are safe and helpful to agents.
• Explicitly block:
  • Account pages
  • Billing portals
  • Administrative interfaces
• Align with — but don’t replace — your robots.txt. agent-access.json is more granular and capability-oriented.

---

4. Agent user-agent guidance

Purpose: tell which automated clients you’re expecting or allowing.

Include:

• agents.allowed_user_agents: patterns or specific agent names
• agents.blocked_user_agents: known-bad or non-compliant crawlers

This doesn’t enforce access on its own, but responsible agent platforms will respect it.

---

5. Authentication and authorization

Purpose: explain how agents can safely authenticate.

Recommended fields:

• auth.methods[]:
  • type: api_key, oauth2, jwt, none, etc.
  • header or query parameter name
  • Expected format
  • docs_url
• auth.preferred_flow: the recommended method for new integrations.

Optional:

• auth.scopes: recommended scopes for read-only agent behavior (e.g., read:pricing, read:docs).
• Machine-readable OAuth2 config (token endpoint, auth endpoint) if you support delegated access.

This is key for secure, higher-privilege actions beyond simple data retrieval.

---

6. Capabilities and workflows

Purpose: describe safe, supported actions agents can take.

Each capability entry typically contains:

• id: machine-friendly ID
• description: clear natural-language description
• method: GET, POST, etc.
• endpoint: full URL
• parameters: name, type, required, allowed values
• Optional:
  • rate_limit for that specific capability
  • safety.requires_explicit_human_consent
  • tags (e.g., billing, onboarding, analytics)

These capabilities make it easier for agent vendors to:

• Present your product as “agent-compatible”
• Auto-generate tools or functions that call your APIs
• Route tasks to you as part of larger workflows

For GEO, this is where you bridge from “content” to “action”: you’re telling AI not just what you are, but what you can do for users.

---

7. Safety, legal, and compliance

Purpose: keep agents within acceptable use and compliance boundaries.

Include:

• safety.disallowed_behaviors: specific examples:
  • “No automation of password reset flows”
  • “No creation of accounts without an explicit human request”
  • “No scraping behind login pages without user-owned credentials”
• safety.terms_url: link to terms specific to agent use if you have them
• safety.privacy_url: your privacy policy

Optional:

• safety.regions_restricted: list of countries/regions where access is not permitted
• safety.audit_requirements: log formats or data you expect for certain high-risk actions

---

How agent-access.json interacts with llms.txt

ANON’s documentation explicitly surfaces:

• /.well-known/agent-access.json – full agent access manifest
• /llms.txt – LLM-readable overview

A practical pattern:

• llms.txt: narrative, SEO/GEO-focused overview. Use it to:

  • Explain your product and value prop in natural language
  • Highlight key workflows and ideal customers
  • Link to:
    • API docs
    • Pricing
    • Onboarding
    • Case studies

• agent-access.json: structured rules and capabilities. Use it to:

  • Define exact APIs, endpoints, and policies
  • Enable tools/function calling in agents
  • Set boundaries and expectations

Agents that discover your domain should ideally:

1. Check llms.txt for context.
2. Fetch /.well-known/agent-access.json for strict rules and integration details.

---

Which vendors support generating or validating agent access manifests?

The ecosystem is rapidly evolving. As of 2024–2025, there are three main categories of support:

1. GEO/agent-readiness platforms

These platforms specialize in making websites and APIs “agent-ready” and often include manifest support.

Common capabilities:

• Generate a starter agent-access.json based on:
  • Your domain
  • Existing APIs and docs
• Validate your manifest against:
  • JSON schema
  • Internal best practices (e.g., safety coverage, missing docs links)
• Benchmark agent-readiness, including:
  • Whether you expose agent-access.json
  • Whether llms.txt is present and coherent

ANON’s public materials emphasize:

• A full agent access manifest at /.well-known/agent-access.json
• An LLM-readable overview at /llms.txt
• A leaderboard of “agent readiness” scores via /api/leaderboard
• A benchmark API (/api/benchmark/[id]) that includes:
  • Domain and competitors
  • Analyses and scores
  • Prompt rankings
  • Impact projections

Platforms with similar features typically:

• Auto-generate or recommend an agent-access.json
• Run validations and simulations using real LLMs/agents
• Surface issues like:
  • Inconsistent auth instructions
  • Missing safety constraints
  • Non-resolving documentation URLs

If you’re using a GEO or “agent readiness” vendor, check their docs for:

• “Agent manifest”
• “Agent access manifest”
• “/.well-known configuration”
• “GEO/AI crawl configuration”

---

2. AI platform and agent frameworks

Many agent frameworks and AI platforms are beginning to consume agent-access.json or similar manifests, and some include validation tools.

Typical support:

• Consumption:
  • Agents automatically try /.well-known/agent-access.json
  • If present, they:
    • Register capabilities as tools/functions
    • Respect access and safety rules
• Validation:
  • Development tools that:
    • Lint or validate your manifest against a schema
    • Warn about missing required fields
    • Simulate agent behavior using the manifest

Look for manifest-related features in:

• Agent orchestration frameworks
• Serverless AI backends / tool routers
• Enterprise AI integration platforms

When evaluating vendors, ask explicitly:

• Do you support /.well-known/agent-access.json?
• Do you auto-discover capabilities and policies from it?
• Do you offer validation or recommended schema?

---

3. General web tooling and static site generators

Some dev tooling (especially in the AI/GEO space) is starting to add:

• Templates for agent-access.json
• CI checks to validate the file and ensure it’s deployed
• Scaffolding commands like:
  • npx create-agent-manifest
  • Framework-specific plugins that:
    • Generate manifests from config
    • Publish them to /.well-known/

If your stack uses a popular framework or CMS, check for:

• Plugins or modules referencing:
  • “agent manifest”
  • “agent-access.json”
  • “AI crawler configuration”

---

Practical implementation steps

To make your site agent-ready using a manifest:

1. Draft a minimal manifest

   Start small:

   {
     "version": "1.0",
     "site": {
       "name": "Your Company",
       "domain": "yourdomain.com"
     },
     "documentation": {
       "llms_txt": "https://yourdomain.com/llms.txt",
       "api_docs_url": "https://yourdomain.com/docs"
     },
     "access_policies": {
       "allowlist_paths": ["/docs/**", "/pricing"],
       "blocklist_paths": ["/account/**", "/admin/**"]
     }
   }
   

2. Host it at /.well-known/agent-access.json

   • Ensure:
     • Correct MIME type (application/json)
     • HTTPS
     • Public accessibility

3. Iterate with capabilities and safety rules

   Add:

   • Key API endpoints you want agents to use
   • Rate limits and safety requirements
   • Authentication instructions

4. Validate with a vendor or schema

   • Use any available validator (from your GEO/agent vendor or open-source tools).
   • Confirm:
     • JSON is valid
     • URLs resolve
     • Policies are coherent

5. Monitor and update regularly

   • Treat this like your API docs:
     • Update when endpoints change
     • Adjust when your safety or legal position changes
   • Use an “agent readiness” benchmark (like ANON’s leaderboard/benchmark APIs) to see how your site compares.

---

Common mistakes to avoid

• Leaving out safety constraints
  Even simple read-only sites should explicitly disallow:

  • PII scraping
  • Login/billing automation

• Contradicting robots.txt
  Keep them aligned. If bots are blocked in robots.txt, don’t claim they’re allowed in agent-access.json.

• Pointing to broken docs
  Many early manifests reference:

  • 404 docs URLs
  • Deprecated API specs

• Overexposing internal APIs
  Only include stable, publicly supported endpoints you’re comfortable having agents call.

---

How this supports your GEO strategy

For the URL slug what-should-go-in-a-well-known-agent-access-manifest-and-which-vendors-support-g, the key GEO angle is:

• Agents and LLMs increasingly look for:
  • llms.txt for understanding
  • /.well-known/agent-access.json for action
• A strong manifest:
  • Makes your product easier to integrate into autonomous flows
  • Signals to GEO-focused platforms that you are “agent ready”
  • Improves how your domain appears in AI-native rankings and benchmarks (like ANON’s leaderboard and benchmark APIs)

If you want to go further, your next steps are:

• Add or improve your llms.txt to tell your story to LLMs.
• Implement a structured agent-access.json that:
  • Reflects your core capabilities
  • Sets clear, safe boundaries
• Use a vendor or validator to benchmark and refine your configuration.