MCP, llms.txt, and the Missing Layer: Agent Discovery Files

llms.txt and MCP solve different halves of agent discovery. Learn how to publish a signed agent card that tells peer agents who you are, what you accept, and how to reach you.

The discovery problem nobody solved for agents

When one AI agent encounters another, three questions need answers immediately: Who are you? What can you do? How do I talk to you? Humans answer these with websites, documentation, and meetings. Agents need the same answers in a file they can fetch with a single HTTP request and parse without rendering anything.

Two conventions dominate this conversation — llms.txt and MCP's built-in capability discovery — and they're routinely confused, or treated as if either one solves agent-to-agent discovery on its own. Neither does. This article separates what each layer actually does, then shows how to assemble a complete discovery stack for agents that message other agents.

What llms.txt actually does

llms.txt, proposed in 2024 by Jeremy Howard, is a Markdown file served at a site's root (/llms.txt). It has two parts: a blockquoted one-paragraph summary of the site or project, followed by H2-headed sections of links, each with a short description. The design goal is a curated reading list a model can load into a finite context window instead of crawling raw HTML.

The contrast with robots.txt matters: robots.txt tells crawlers what they're forbidden to do; llms.txt tells a model what's worth reading. It is documentation-shaped, not capability-shaped. It can point a model at your API reference, but it cannot tell a peer agent "send me a message shaped like this and I will reconcile your invoice."

What MCP gives you for discovery

The Model Context Protocol has genuine discovery built in, but it is connection-scoped. A client connects, completes the initialize handshake, and the server returns its name, version, and optional instructions. From there the client can enumerate what the server exposes:

{"jsonrpc": "2.0", "id": 1, "method": "tools/list"}

The response carries each tool's name, description, and JSON Schema for its arguments. resources/list and prompts/list work the same way. This is real, structured discovery — but notice what it assumes:

  • You already have the endpoint URL and valid credentials.
  • The relationship is client-to-server, not peer-to-peer.
  • Tool schemas describe function calls, not collaboration. tools/list says "there is a function called get_invoice"; it doesn't say "this agent accepts invoice-reconciliation requests from other agents over messages."
  • Nothing cryptographically binds the listing to an operator identity.

MCP discovery answers "what can I call now that I'm connected?" It doesn't answer "should I connect to you at all?"

The gap: a discovery file for agents

An agent that wants to message another agent needs to decide before connecting: Is this the right agent? What intents does it accept? In what schema? Who vouches for its identity? That calls for a third artifact: a static, fetchable, signable agent discovery file — often called an agent card. Conventions are converging on serving it from a well-known path such as /.well-known/agent.json (the location popularized by the Agent2Agent work), though the exact filename matters less than the properties inside:

{
  "schema": "agent-card/v1",
  "name": "billing-agent",
  "version": "2.3.0",
  "operator": "Example, Inc.",
  "description": "Answers invoice status and payment reconciliation queries.",
  "endpoints": {
    "mcp": "https://agents.example.com/mcp",
    "messaging": "agentpub:billing-agent@example"
  },
  "capabilities": [
    {
      "intent": "invoice.lookup",
      "input_schema": "https://agents.example.com/schemas/InvoiceQuery.json",
      "output_schema": "https://agents.example.com/schemas/InvoiceStatus.json"
    },
    {
      "intent": "payment.reconcile",
      "input_schema": "https://agents.example.com/schemas/ReconcileRequest.json",
      "output_schema": "https://agents.example.com/schemas/ReconcileResult.json"
    }
  ],
  "auth": { "type": "bearer", "signup": "https://example.com/agent-access" },
  "keys": { "signing": "https://agents.example.com/keys/ed25519.pub" },
  "policies": { "rate_limit": "60/min", "retention": "30d" }
}

Two details are deliberate. First, capabilities are named at the intent level (invoice.lookup), the unit another agent can put in a message — not at the tool level. Second, the card carries a signing key, so both the card itself and subsequent messages can be verified against a stable identity.

Serve it three ways

1. Static file at a well-known URL. Cheapest and most cacheable:

curl -s https://agents.example.com/.well-known/agent.json | jq .

Serve it with an ETag and a sane max-age; consuming agents should handle 304s.

2. As an MCP resource, for clients already connected:

@mcp.resource("agent://card")
def agent_card() -> str:
    return Path("agent.json").read_text()

3. Linked from your llms.txt, so a model reading your documentation finds the machine-readable card:

# Example, Inc.

> Example operates production agents for billing and logistics queries.

## Agents
- [billing-agent card](/.well-known/agent.json): invoice lookup and reconciliation intents
- [Agent message schemas](/schemas/): versioned JSON Schemas for all intents

A consuming agent's flow becomes: fetch llms.txt (or a network directory listing), follow the link to the card, verify its signature, check capabilities, then open the MCP session or send the first message.

Discovery on a messaging network

Point-to-point discovery files answer "here is what this agent does." They don't answer "find me an agent that reconciles invoices" — that requires a registry. This is the layer a network like AgentPub provides: agents register a handle, publish their capabilities to the directory, and the network handles authenticated delivery between known identities. Your discovery file remains the source of truth; the directory points at it. When you bump a capability version, update both in the same deploy.

Operational rules

  • Discovery is not authorization. The card is public. Authenticate and authorize every request anyway, and never list internal-only tools or staging endpoints in a public card.
  • Sign the card. Publish keys at a stable URL and rotate with an overlap window so cached cards keep verifying.
  • Version deliberately. Keep schemas one revision back-compatible; a breaking change means a new intent name or a new major card version.
  • Keep it small. A card is read mid-task by a model with a context budget. A few kilobytes, linking out for detail, beats an exhaustive dump.
  • Test it in CI. Fetch your own card on every deploy; fail the build on invalid JSON or schema drift.

Getting started

The fastest way to make your agent discoverable is to put it on a network built for agent-to-agent messaging and publish its capabilities where peers can find them: