AI Agents and the Web¶

AI agents are software that reads, navigates and acts on websites on a person's behalf: assistants inside browsers that click and type in the user's own session, cloud services that browse from remote machines, and chat assistants that call your backend through a protocol instead of your UI. For a PWA this is a new kind of client, sitting somewhere between a human user, a search crawler and an API consumer, and it rewards the same engineering that already makes web apps good: semantic markup, stable URLs, server-rendered content and a clean service layer. This page describes the agents that exist as of September 2026, how they perceive your app, the emerging standards for cooperating with them (WebMCP, the Model Context Protocol, MCP Apps, llms.txt, Web Bot Auth), and how to handle authentication, consent, bot detection and crawler controls without breaking either humans or agents.

Four kinds of agent traffic¶

"AI agent" covers clients that behave very differently. Sort them before deciding what to allow, block or build.

The first two are covered by your existing crawler strategy (SEO for PWAs) plus the controls later on this page. The last two are genuinely new, and they are where a PWA can do more than defend itself.

Agentic browsers and assistants in September 2026¶

This field changes month to month. The table lists only products whose status could be confirmed from vendor or primary sources at the time of writing; check the linked sources before depending on any detail.

Two things are common to all of them. First, they act inside a real browser with the user's state: cookies, storage, service workers, installed PWAs and saved passwords. From your server's point of view the requests are the user's requests. Second, all vendors acknowledge prompt injection as an unsolved risk; independent research such as Brave's disclosure about Comet showed page content steering an agent into exfiltrating account data. Your page is part of that attack surface, both as a possible victim and as a possible carrier of injected text from user-generated content.

How agents perceive a web app¶

Browser agents combine three views of a page, in different proportions per product:

1. The accessibility tree: roles, accessible names, states and relationships computed from HTML and ARIA. It is compact, semantic and the cheapest for a model to reason over.
2. The DOM and text content, often simplified or distilled.
3. Screenshots, with actions targeted by coordinates.

Chrome's documentation for its new Lighthouse Agentic Browsing category states that agents "rely on the accessibility tree as their primary data model" and audits a subset of accessibility checks that matter for machine interaction (names and labels, valid roles and parent-child relationships, interactive content not hidden from the tree), layout stability via CLS, the presence of llms.txt, and WebMCP tool registration and schema validity. The category is experimental, requires Chrome 150 or later, reports a pass ratio instead of a 0 to 100 score, and its WebMCP audits need the origin trial.

What breaks agents, and what fixes it¶

Everything in this table also breaks assistive technology, which is why the accessibility page is the best preparation for agents.

A form built this way is already agent-ready, and adding WebMCP's declarative attributes (covered below) is a small step from here:

<form id="shipping" action="/checkout/shipping" method="post" novalidate>
  <h2 id="shipping-heading">Shipping address</h2>

  <label for="name">Full name</label>
  <input id="name" name="name" autocomplete="name" required>

  <label for="postal">Postal code</label>
  <input id="postal" name="postal" autocomplete="postal-code" inputmode="numeric"
         required aria-describedby="postal-error">
  <p id="postal-error" class="field-error" hidden></p>

  <fieldset>
    <legend>Delivery speed</legend>
    <label><input type="radio" name="speed" value="standard" checked> Standard (3 to 5 days)</label>
    <label><input type="radio" name="speed" value="express"> Express (next day)</label>
  </fieldset>

  <button type="submit">Continue to payment</button>
</form>

autocomplete tokens help both browsers and agents fill fields correctly, the fieldset/legend pair names the radio group, and the error paragraph is programmatically tied to its field.

Stable URLs, server rendering and structured data¶

URLs are the agent's handles¶

Agents plan in terms of pages: "open the order page, find order 1234, click Return". Every meaningful state of your app (a product, a search result, a filtered list, an order, a settings pane) should have its own URL that can be loaded directly, including in a fresh browser profile. Deep links are what let a cloud agent resume a task, let a chat assistant cite your page, and let a user verify what an agent did. The mechanics for PWAs, including status codes in single-page apps, are on the SEO page; protocol and link handling covers how links open installed PWAs.

Server-render the content that matters¶

Browser agents execute JavaScript because they run in a browser. Many other agent clients do not: user-triggered fetchers, retrieval pipelines and search crawlers often fetch the raw HTML or render it only in a limited way. An app-shell PWA that returns an empty <div id="app"> to a first request is invisible to them. Server-render (or statically render) the primary content and metadata of every public URL, and let the client take over for interactivity. App shell and SPA vs MPA discuss how to keep an app-like experience while doing so.

Two service worker details matter for agents:

• A browser agent operating an installed PWA gets your service worker's responses, including cached app shells and offline fallbacks. An offline page served with status 200 for a real URL, or a stale cached page, will mislead an agent just as it misleads a user. Keep server HTML authoritative for navigations when online (fetch handling).
• Cloud agents and fetchers start from a clean profile: no service worker, no cache, no storage. They see your first-visit experience, including cookie banners and interstitials. Make sure those are dismissible with a labeled button and do not block the content in the accessibility tree.

Structured data with JSON-LD¶

Schema.org markup gives any machine reader an unambiguous description of the entities on a page. It predates AI agents and is documented as a search feature, but it is equally useful to any agent that reads the page source. Describe the application and the page's primary entity:

<script type="application/ld+json">
{
  "@context": "https://schema.org",
  "@graph": [
    {
      "@type": "WebApplication",
      "@id": "https://shop.example/#app",
      "name": "Example Shop",
      "url": "https://shop.example/",
      "applicationCategory": "ShoppingApplication",
      "operatingSystem": "Any",
      "browserRequirements": "Requires a modern browser with JavaScript enabled",
      "offers": { "@type": "Offer", "price": "0", "priceCurrency": "EUR" }
    },
    {
      "@type": "Product",
      "@id": "https://shop.example/p/travel-mug#product",
      "name": "Insulated travel mug, 350 ml",
      "sku": "MUG-350-BLK",
      "gtin13": "4006381333931",
      "image": ["https://shop.example/img/mug-350-black.avif"],
      "description": "Double-walled stainless steel travel mug with leak-proof lid.",
      "brand": { "@type": "Brand", "name": "Example" },
      "offers": {
        "@type": "Offer",
        "url": "https://shop.example/p/travel-mug",
        "price": "24.00",
        "priceCurrency": "EUR",
        "availability": "https://schema.org/InStock",
        "itemCondition": "https://schema.org/NewCondition",
        "shippingDetails": {
          "@type": "OfferShippingDetails",
          "shippingDestination": { "@type": "DefinedRegion", "addressCountry": "DE" },
          "deliveryTime": {
            "@type": "ShippingDeliveryTime",
            "handlingTime": { "@type": "QuantitativeValue", "minValue": 0, "maxValue": 1, "unitCode": "DAY" },
            "transitTime": { "@type": "QuantitativeValue", "minValue": 1, "maxValue": 3, "unitCode": "DAY" }
          }
        },
        "hasMerchantReturnPolicy": {
          "@type": "MerchantReturnPolicy",
          "applicableCountry": "DE",
          "returnPolicyCategory": "https://schema.org/MerchantReturnFiniteReturnWindow",
          "merchantReturnDays": 30
        }
      }
    }
  ]
}
</script>

Rules that keep structured data trustworthy: it must match the visible content (price, availability), it must be in the server-rendered HTML rather than injected later, and it must be updated with the same data source as the page. Validate with the Schema Markup Validator and Google's Rich Results Test. Agentic commerce is also getting dedicated protocols; for example, Google announced that Chrome will support its Universal Commerce Protocol for agentic checkout. Treat such protocols as additions on top of good product pages, not replacements.

WebMCP: tools for in-browser agents¶

Browser agents that operate pages by "actuation" (reading the accessibility tree or screenshots, then simulating clicks and keystrokes) are slow and error-prone on complex UIs. WebMCP lets the page declare what it can do: named tools with descriptions, JSON Schema inputs and JavaScript implementations that run in the page, visible to the user, using the app's own state and code. Chrome's documentation describes it as a progressive enhancement for sites and positions it as usable by "any browser with agentic capabilities" (WebMCP overview).

The API has moved during incubation. Early previews exposed navigator.modelContext with methods such as provideContext(); the current specification draft and Chrome's documentation use document.modelContext, on the reasoning that tools belong to a document. Feature-detect both while origin-trial users may still run older builds.

The imperative API¶

The current WebIDL (from the specification draft):

partial interface Document {
  [SecureContext, SameObject] readonly attribute ModelContext modelContext;
};

[Exposed=Window, SecureContext]
interface ModelContext : EventTarget {
  Promise<undefined> registerTool(ModelContextTool tool, optional ModelContextRegisterToolOptions options = {});
  Promise<sequence<RegisteredTool>> getTools(optional ModelContextGetToolOptions options = {});
  Promise<DOMString> executeTool(RegisteredTool tool, optional any inputObject, optional ModelContextExecuteToolOptions options = {});
  attribute EventHandler ontoolchange;
  attribute EventHandler ontoolactivated;
  attribute EventHandler ontoolcancel;
};

dictionary ModelContextTool {
  required DOMString name;
  USVString title;
  required DOMString description;
  object inputSchema;
  required ToolExecuteCallback execute;
  ToolAnnotations annotations;
};

dictionary ToolAnnotations {
  boolean readOnlyHint = false;
  boolean untrustedContentHint = false;
  boolean consequentialHint = false;
  boolean debugging = false;
};

callback ToolExecuteCallback = Promise<any> (object inputObject, ToolExecuteCallbackOptions options);
dictionary ToolExecuteCallbackOptions { required AbortSignal signal; };
dictionary ModelContextRegisterToolOptions { sequence<USVString> exposedTo; AbortSignal signal; };

Key behaviors, per the spec draft and Chrome's imperative API guide:

• Registration is per document. registerTool() adds one tool; pass an AbortSignal in the options to unregister it later by aborting. Since Chrome 153, unregistering does not cancel executions already in flight.
• Execution calls your execute(input, { signal }). The return value is serialized and given to the model; plain strings work well. The signal aborts when the user or agent cancels, so pass it to fetch() and other async work.
• Annotations are hints: readOnlyHint (no state change), consequentialHint (significant or irreversible; lets the browser or agent require confirmation), untrustedContentHint (output contains user-generated or external content, a prompt-injection risk), and debugging (developer tooling only, from Chrome 156).
• Scope and security. WebMCP needs a secure context, and registerTool(), getTools() and executeTool() reject with a SecurityError in documents whose agent cluster is not origin-keyed (for example, pages that send Origin-Agent-Cluster: ?0 to keep document.domain working). It is gated by the tools Permissions Policy, whose default allowlist is self: cross-origin iframes need allow="tools", and tools are only visible to other origins listed in exposedTo that also request them with getTools({ fromOrigins }).
• Input format. Chrome deprecated passing JSON-stringified arguments from Chrome 155; tools receive an object.

A production registration module for a single-page PWA, reusing the same functions the UI calls:

import { catalog, cart, router, ui } from "../app.js"; // the app's own modules

/** Returns the ModelContext object, or null where WebMCP is unavailable. */
function getModelContext() {
  // Current spec and Chrome docs: document.modelContext.
  // Earlier previews: navigator.modelContext. Both are feature-detected.
  return document.modelContext ?? navigator.modelContext ?? null;
}

let routeController = null;

/**
 * Registers the tools that make sense on the current route. Call on every client-side
 * navigation: tools describe what the *current* page can do, and stale tools from a
 * previous view would let an agent act on state the user can no longer see.
 */
export async function registerToolsForRoute(route) {
  const mc = getModelContext();
  if (!mc || typeof mc.registerTool !== "function") return; // progressive enhancement

  routeController?.abort();              // unregister the previous route's tools
  routeController = new AbortController();
  const { signal } = routeController;

  const register = (tool) =>
    mc.registerTool(tool, { signal }).catch((err) => {
      // NotAllowedError in cross-origin frames without allow="tools", or a name clash.
      console.warn(`WebMCP: could not register ${tool.name}`, err);
    });

  await register({
    name: "search_products",
    title: "Search products",
    description: "Search the catalog and show results on screen. Returns up to 10 matches with id, name, price in EUR and stock status.",
    inputSchema: {
      type: "object",
      properties: {
        query: { type: "string", description: "Words to search for, e.g. 'travel mug'" },
        maxPrice: { type: "number", description: "Optional maximum price in EUR" },
      },
      required: ["query"],
    },
    annotations: { readOnlyHint: true, untrustedContentHint: true }, // product names come from sellers
    async execute({ query, maxPrice }, { signal }) {
      // Navigate like a user would, so the person watching sees the same results.
      await router.navigate(`/search?${new URLSearchParams({ q: query, ...(maxPrice ? { max: maxPrice } : {}) })}`);
      const results = await catalog.search({ query, maxPrice, limit: 10, signal });
      if (results.length === 0) return `No products match "${query}".`;
      return results.map((p) => `${p.id} | ${p.name} | ${p.price.toFixed(2)} EUR | ${p.inStock ? "in stock" : "out of stock"}`).join("\n");
    },
  });

  if (route.name === "product") {
    await register({
      name: "add_to_cart",
      title: "Add to cart",
      description: "Add the product shown on this page to the cart. Reversible: the user can remove it.",
      inputSchema: {
        type: "object",
        properties: { quantity: { type: "integer", minimum: 1, maximum: 10, description: "Number of items" } },
        required: ["quantity"],
      },
      async execute({ quantity }, { signal }) {
        const line = await cart.add(route.params.productId, quantity, { signal });
        ui.highlight("#cart-count");                  // make the change visible
        return `Added ${quantity} x ${line.name}. Cart total: ${line.cartTotal.toFixed(2)} EUR.`;
      },
    });
  }

  if (route.name === "checkout") {
    await register({
      name: "place_order",
      title: "Place order",
      description: "Place the order currently shown on the checkout page. Charges the saved payment method. The user must confirm on screen.",
      inputSchema: { type: "object", properties: {} },
      annotations: { consequentialHint: true },
      async execute(_input, { signal }) {
        // consequentialHint is a hint to the browser and agent. The page still asks the
        // user itself, and the server still enforces confirmation (see "Consent").
        const confirmed = await ui.confirmDialog({
          title: "Place this order?",
          body: `Total ${cart.total().toFixed(2)} EUR will be charged.`,
          confirmLabel: "Place order",
          signal,
        });
        if (!confirmed) return "The user declined to place the order.";
        const order = await cart.checkout({ signal });
        await router.navigate(`/orders/${order.id}`);
        return `Order ${order.id} placed. Confirmation page is open.`;
      },
    });
  }
}

Guidance from Chrome's tool security page worth adopting: keep tool names under about 30 characters, descriptions under about 500, parameter descriptions under about 150, and outputs under about 1,500 characters; mark user-generated output with untrustedContentHint; and only use exposedTo for origins you would share the data with directly. Chrome also notes that extensions can query and execute WebMCP tools through content scripts, so do not treat a tool call as proof of a specific agent.

The declarative API: forms as tools¶

The declarative API turns an existing <form> into a tool by adding attributes; the browser derives the input schema from the form's controls (Chrome's declarative API guide):

<form id="support" action="/support" method="post"
      toolname="create_support_request"
      tooldescription="Open a support request about an existing order. The user reviews it before sending.">
  <label for="order">Order number</label>
  <input id="order" name="orderId" required pattern="[A-Z0-9-]{6,32}"
         toolparamdescription="Order number as shown in the order confirmation, e.g. A1B2-C3D4">

  <label for="topic">Topic</label>
  <select id="topic" name="topic" required>
    <option value="return">Return an item</option>
    <option value="delivery">Where is my package?</option>
    <option value="damaged">Item arrived damaged</option>
  </select>

  <label for="details">Details</label>
  <textarea id="details" name="details" maxlength="2000"></textarea>

  <button type="submit">Send request</button>
</form>

<script type="module">
  const form = document.getElementById("support");

  form.addEventListener("submit", async (event) => {
    event.preventDefault();
    const agent = event.agentInvoked === true; // undefined in browsers without WebMCP

    if (!form.checkValidity()) {
      form.reportValidity();
      if (agent) event.respondWith(Promise.resolve("Validation failed: check the order number format."));
      return;
    }

    const request = fetch(form.action, { method: "POST", body: new FormData(form) })
      .then(async (res) => {
        if (!res.ok) throw new Error(`Support request failed (${res.status})`);
        const { ticketId } = await res.json();
        showConfirmation(ticketId); // visible to the user either way
        return `Support request ${ticketId} created.`;
      });

    if (agent) event.respondWith(request.catch((err) => `Error: ${err.message}`));
    else request.catch(showError);
  });

  addEventListener("toolactivated", ({ toolName }) => {
    if (toolName === "create_support_request") form.scrollIntoView({ block: "center" });
  });
  addEventListener("toolcancel", ({ toolName }) => {
    if (toolName === "create_support_request") announce("The assistant stopped filling the support form.");
  });
</script>

/* Make agent activity obvious to the person watching. */
form:tool-form-active {
  outline: 2px dashed light-dark(#1a56db, #7aa2ff);
  outline-offset: 4px;
}
button:tool-submit-active {
  outline: 2px dashed light-dark(#b42318, #ff8a80);
}

showConfirmation, showError and announce stand for your app's existing UI helpers. Without toolautosubmit the user stays in control of the final click, which is the right default for anything that sends data on the user's behalf.

Enabling, testing and debugging WebMCP¶

• Local development: enable chrome://flags/#enable-webmcp-testing and relaunch.
• Production experiments: register your origin for the Chrome origin trial (and the Edge origin trial if needed) and deliver the token with an Origin-Trial response header or <meta http-equiv="origin-trial" content="TOKEN">. Tokens expire; build expiry monitoring into your deploy checks.
• Inspecting tools: Chrome's documentation points to the Model Context Tool Inspector extension, which lists registered tools, calls them manually, validates schemas and lets you chat with a test agent. Chrome DevTools also has experimental WebMCP support.
• Auditing: run the Lighthouse Agentic Browsing category; it records imperative registrations through the DevTools Protocol, so register tools early in page load or they may be missed.
• Installed PWAs: tools are registered per document, so they exist in an installed PWA's window exactly as in a tab. They are not available in the service worker (the interface is exposed to Window only).

MCP servers alongside your web app¶

The Model Context Protocol is the protocol chat assistants, IDEs and agent frameworks use to connect to external tools and data. The current specification revision is dated 2026-07-28; it uses JSON-RPC over stdio (local servers) or Streamable HTTP (remote servers), treats remote servers as OAuth 2.1 protected resources, and requires hosts to obtain user consent before invoking tools. Where WebMCP serves an agent that is looking at your page, an MCP server serves an agent that is not: in a chat window, a desktop assistant or a background job.

Chrome's own comparison (WebMCP versus MCP) frames them as complementary:

One service layer, three front ends¶

The architecture that avoids duplicated business logic: a service layer that enforces authorization and validation, consumed by the PWA's REST (or RPC) API, by WebMCP tools (through that API), and by an MCP server.

flowchart LR
    PWA["PWA UI"] --> REST["REST API /api/*"]
    WEBMCP["WebMCP tools in the page"] --> REST
    HOST["MCP host (chat assistant)"] -- "OAuth bearer token" --> MCP["MCP endpoint /mcp"]
    REST --> SVC["Service layer: authz, validation, audit"]
    MCP --> SVC
    SVC --> DB[("Database")]

import { db } from "../db.js";
import { audit } from "../audit.js";

export class ForbiddenError extends Error {}

/** Every function takes an explicit actor; nothing reads ambient request state. */
export const orders = {
  async list(actor, { status, limit = 20 }) {
    requireScope(actor, "orders:read");
    return db.orders.findMany({ userId: actor.userId, status, limit: Math.min(limit, 50) });
  },

  async get(actor, orderId) {
    requireScope(actor, "orders:read");
    const order = await db.orders.findOne({ id: orderId, userId: actor.userId });
    if (!order) throw new ForbiddenError("Order not found"); // same error for "not yours"
    return order;
  },

  async requestReturn(actor, orderId, { reason, confirmationToken }) {
    requireScope(actor, "orders:write");
    const order = await this.get(actor, orderId);
    // Consequential action: require a confirmation the *user* produced (see "Consent").
    await verifyConfirmation(actor, confirmationToken, { action: "return", orderId });
    const ret = await db.returns.create({ orderId: order.id, reason });
    await audit.log({ actor, action: "orders.requestReturn", orderId, via: actor.client });
    return ret;
  },
};

function requireScope(actor, scope) {
  if (!actor.scopes.includes(scope)) throw new ForbiddenError(`Missing scope ${scope}`);
}

async function verifyConfirmation(actor, token, expected) {
  const ok = await db.confirmations.consume({ token, userId: actor.userId, ...expected });
  if (!ok) throw new ForbiddenError("confirmation_required");
}

The PWA's API builds the actor from the session cookie ({ userId, scopes: ["orders:read", "orders:write"], client: "pwa" }); the MCP endpoint builds it from the verified OAuth token.

The MCP server¶

The code below uses version 2 of the official TypeScript SDK (@modelcontextprotocol/server, @modelcontextprotocol/express and @modelcontextprotocol/node), which implements the 2026-07-28 specification. Version 2 replaced the monolithic @modelcontextprotocol/sdk package and its per-request transport wiring with createMcpHandler(factory), where the factory builds a fresh server per HTTP request. Check the SDK documentation for changes before copying.

import {
  createMcpExpressApp,
  getOAuthProtectedResourceMetadataUrl,
  mcpAuthMetadataRouter,
  requireBearerAuth,
  type OAuthTokenVerifier,
} from "@modelcontextprotocol/express";
import { toNodeHandler } from "@modelcontextprotocol/node";
import {
  createMcpHandler,
  McpServer,
  requireScopes,
  OAuthError,
  OAuthErrorCode,
  type AuthInfo,
  type OAuthMetadata,
} from "@modelcontextprotocol/server";
import * as z from "zod/v4";
import { orders, ForbiddenError } from "./services/orders.js";
import { verifyJwt, authServerMetadata } from "./identity.js"; // your identity provider integration

const mcpServerUrl = new URL("https://shop.example/mcp");

// 1. Token verification: your MCP endpoint is an OAuth resource server. It never issues tokens.
const verifier: OAuthTokenVerifier = {
  async verifyAccessToken(token: string): Promise<AuthInfo> {
    const claims = await verifyJwt(token, { audience: mcpServerUrl.href }).catch(() => null);
    if (!claims) throw new OAuthError(OAuthErrorCode.InvalidToken, "Invalid or expired token");
    return {
      token,
      clientId: claims.client_id,          // which MCP client the user authorized
      scopes: String(claims.scope ?? "").split(" ").filter(Boolean),
      expiresAt: claims.exp,               // required: tokens without expiresAt are rejected
      extra: { userId: claims.sub },
    };
  },
};

const actorFrom = (auth?: AuthInfo) => {
  if (!auth) throw new ForbiddenError("unauthenticated");
  return { userId: String(auth.extra?.userId), scopes: auth.scopes, client: `mcp:${auth.clientId}` };
};

// 2. Per-request server factory: cheap, stateless, built around the verified caller.
function buildServer({ authInfo }: { authInfo?: AuthInfo }) {
  const server = new McpServer({ name: "example-shop", version: "1.4.0" });

  server.registerTool(
    "list_orders",
    {
      title: "List orders",
      description: "List the signed-in customer's recent orders with id, date, status and total.",
      inputSchema: z.object({
        status: z.enum(["open", "shipped", "delivered", "cancelled"]).optional(),
        limit: z.number().int().min(1).max(50).optional(),
      }),
      outputSchema: z.object({
        orders: z.array(z.object({ id: z.string(), date: z.string(), status: z.string(), total: z.string() })),
      }),
      annotations: { readOnlyHint: true, openWorldHint: false },
      scopeChallenge: requireScopes("orders:read"),
    },
    async ({ status, limit }) => {
      const list = await orders.list(actorFrom(authInfo), { status, limit });
      const output = {
        orders: list.map((o) => ({ id: o.id, date: o.createdAt.toISOString().slice(0, 10), status: o.status, total: `${o.total} ${o.currency}` })),
      };
      return {
        content: [{ type: "text", text: output.orders.map((o) => `${o.id} ${o.date} ${o.status} ${o.total}`).join("\n") || "No orders." }],
        structuredContent: output,
      };
    },
  );

  server.registerTool(
    "request_return",
    {
      title: "Request a return",
      description:
        "Start a return for an order. Requires a confirmation code the customer obtains by approving the return at https://shop.example/confirm. Ask the customer for it; never guess.",
      inputSchema: z.object({
        orderId: z.string().regex(/^[A-Z0-9-]{6,32}$/),
        reason: z.enum(["damaged", "wrong_item", "no_longer_needed", "other"]),
        confirmationCode: z.string().min(6).max(64),
      }),
      annotations: { readOnlyHint: false, destructiveHint: false, idempotentHint: true },
      scopeChallenge: requireScopes("orders:write"),
    },
    async ({ orderId, reason, confirmationCode }) => {
      try {
        const ret = await orders.requestReturn(actorFrom(authInfo), orderId, { reason, confirmationToken: confirmationCode });
        return { content: [{ type: "text", text: `Return ${ret.id} created for order ${orderId}. Label: https://shop.example/returns/${ret.id}` }] };
      } catch (err) {
        if (err instanceof ForbiddenError) {
          // A tool-level error the model can read and explain, not a protocol failure.
          return { content: [{ type: "text", text: `Not allowed: ${err.message}` }], isError: true };
        }
        throw err;
      }
    },
  );

  return server;
}

// 3. HTTP wiring: bearer auth in front, RFC 9728 protected resource metadata published.
const app = createMcpExpressApp({ host: "0.0.0.0", allowedHosts: ["shop.example"] });
const oauthMetadata: OAuthMetadata = await authServerMetadata(); // your IdP's RFC 8414 document
app.use(mcpAuthMetadataRouter({ oauthMetadata, resourceServerUrl: mcpServerUrl }));

const auth = requireBearerAuth({
  verifier,
  requiredScopes: ["orders:read"],
  resourceMetadataUrl: getOAuthProtectedResourceMetadataUrl(mcpServerUrl),
});
const node = toNodeHandler(createMcpHandler(buildServer));
app.all("/mcp", auth, (req, res) => void node(req, res, req.body));

app.listen(Number(process.env.PORT ?? 3000));

How the pieces fit:

• An MCP host that calls /mcp without a token receives 401 with a WWW-Authenticate: Bearer challenge whose resource_metadata parameter points at /.well-known/oauth-protected-resource/mcp. From there the host discovers your authorization server, runs the OAuth flow with the user, and retries with a token.
• requireScopes() on a tool registration issues a 403 insufficient_scope step-up challenge before the handler runs, so a client authorized for read access cannot call write tools.
• Annotations (readOnlyHint, destructiveHint, idempotentHint, openWorldHint) are hints for hosts deciding when to ask the user; the specification tells hosts to treat them as untrusted unless the server is trusted. Authorization lives in the service layer.
• createMcpExpressApp validates Host and Origin headers against DNS rebinding; when binding to all interfaces, list your public hostnames in allowedHosts.

Apps inside chat assistants: MCP Apps¶

The MCP Apps extension lets a tool declare an interactive HTML interface that the host renders inside the conversation, in a sandboxed iframe, communicating with the host over a JSON-RPC dialect on postMessage. A tool links to its UI with _meta.ui.resourceUri pointing at a ui:// resource; the resource is served with the MIME type text/html;profile=mcp-app, and _meta.ui.csp declares which external origins the UI may load. The MCP documentation lists Claude, Claude Desktop, VS Code GitHub Copilot, Microsoft 365 Copilot, Goose and Postman among supporting hosts, and OpenAI's documentation states that ChatGPT implements the MCP Apps standard, with its Apps SDK's window.openai object as a set of ChatGPT-specific extensions on top (OpenAI's guidance is to use the MCP Apps field or method whenever the shared specification covers a capability, and to feature-detect extensions rather than branch on the host name).

For a PWA team this is a distribution channel that reuses web skills: the app UI is an HTML document built with your usual components, and its data comes from the same MCP tools. The UI side uses the App class from @modelcontextprotocol/ext-apps:

import { App } from "@modelcontextprotocol/ext-apps";

const app = new App({ name: "Order status", version: "1.0.0" });
app.connect(); // handshake with the host over postMessage

const list = document.querySelector<HTMLUListElement>("#orders")!;

type OrderList = { orders?: { id: string; status: string; total: string }[] };

// The SDK types structuredContent as unknown; narrow it to the tool's outputSchema.
function render(result: { structuredContent?: unknown }) {
  const data = (result.structuredContent ?? {}) as OrderList;
  list.replaceChildren(
    ...(data.orders ?? []).map((o) => {
      const li = document.createElement("li");
      li.textContent = `${o.id}: ${o.status} (${o.total})`; // textContent: data may be untrusted
      return li;
    }),
  );
}

// The host pushes the result of the tool call that opened this UI.
app.ontoolresult = render;

// The UI can call server tools itself, through the host, subject to host policy.
document.querySelector("#refresh")!.addEventListener("click", async () => {
  render(await app.callServerTool({ name: "list_orders", arguments: { limit: 10 } }));
});

On the server, the extension package provides registerAppTool and registerAppResource helpers (imported from @modelcontextprotocol/ext-apps/server) that attach the _meta.ui metadata and serve the HTML with the right MIME type; follow the MCP Apps build guide for the current server wiring. Bundle the UI into a single HTML file (or declare its origins in _meta.ui.csp), because hosts render it under a deny-by-default CSP.

Authentication and consent for agent actions¶

Browser agents act with the user's session¶

When an agent operates your PWA in the user's browser, every request carries the user's cookies, and your server cannot reliably tell an agent click from a human click. That is by design, and it means the question is not "is this an agent?" but "is this action something the user actually intended?". Apply the same controls you would apply against a compromised or confused client:

1. Confirmation for consequential actions, rendered by your page and enforced by your server. A server-issued, single-use confirmation token bound to the user, action and parameters is created when the user presses Confirm on a review screen, and required by the endpoint that executes the action. WebMCP's consequentialHint and browsers' own confirmation prompts are welcome extra layers, not replacements.
2. Step-up authentication for the highest-risk actions (payment changes, account recovery settings, large transfers). A passkey assertion requires user presence at the authenticator, which an agent cannot supply on its own; see authentication and payments.
3. Idempotency keys on every state-changing endpoint, because agents retry.
4. Undo windows and notifications ("Order placed. Cancel within 30 minutes") so that mistakes are cheap.

/**
 * Requires that the session re-authenticated (for example with a passkey assertion)
 * within maxAgeSeconds. Returns a machine-readable error that the PWA turns into a
 * step-up prompt; an agent operating the page sees the same prompt the user does.
 */
export function requireRecentAuth(maxAgeSeconds = 300) {
  return (req, res, next) => {
    const authTime = req.session?.authTime ?? 0; // set when the user completes a WebAuthn ceremony
    if (Date.now() / 1000 - authTime <= maxAgeSeconds) return next();
    res.status(403).json({
      error: "step_up_required",
      method: "webauthn",
      message: "Confirm it is you to continue.",
    });
  };
}

// Usage: app.post("/api/payment-methods", requireSession, requireRecentAuth(), handler);

Delegated access for API-level agents¶

For MCP clients and other API-level agents, never ask users to hand over passwords or session cookies. Use OAuth delegation, as MCP's authorization model requires:

• Scoped, short-lived tokens per client (orders:read separate from orders:write), with refresh tokens the user can revoke per client from an account page ("Connected assistants").
• Record the client on every audited action (via: "mcp:<client id>"), so users and support can see what an assistant did.
• Consent screens that name the client and scopes in plain language, including write access.
• Rate limits per client and per user, because an agent loop can issue far more calls than a human.

Bot detection versus agents¶

Traditional bot defenses assume automation is hostile. With browser agents that assumption breaks in both directions: a legitimate user's agent can trip heuristics (fast form filling, synthetic events, automation-controlled browsers), while hostile automation can imitate an agent's user-agent string. Practical policy:

Web Bot Auth is the emerging standard for the second row. The IETF chartered the Web Bot Authentication working group to specify how automated clients, explicitly including "AI agents retrieving or interacting with content on behalf of end users", cryptographically authenticate to websites; its first working-group draft, draft-ietf-webbotauth-httpsig-protocol, profiles HTTP Message Signatures (RFC 9421) with a Signature-Agent header that identifies the operator's key directory, a web-bot-auth signature tag and JWK thumbprint key IDs. It is still an Internet-Draft. Google documents that its Google-Agent fetcher, used "by agents hosted on Google infrastructure to navigate the web and perform actions upon user request", supports the experimental Web Bot Auth protocol with the identity https://agent.bot.goog (Google user-triggered fetchers). Some CDNs and bot-management products verify these signatures for you at the edge.

A classification middleware keeps the policy explicit. Signature verification itself should come from a maintained implementation of RFC 9421 and the Web Bot Auth draft (or your CDN), not hand-rolled code:

import { verifyWebBotAuth } from "./web-bot-auth-verifier.js"; // wraps a maintained RFC 9421 library
import { isVerifiedCrawlerIp } from "./crawler-ip-ranges.js";   // refreshed from vendor-published lists

const KNOWN_AGENT_TOKENS = ["Google-Agent", "ChatGPT-User", "Claude-User", "Perplexity-User"];

export async function classifyAutomation(req, _res, next) {
  req.automation = { kind: "unknown" };

  if (req.get("Signature") && req.get("Signature-Input") && req.get("Signature-Agent")) {
    const result = await verifyWebBotAuth(req).catch(() => null);
    if (result?.valid) {
      // result.agent is the resolved Signature-Agent identity, e.g. "https://agent.bot.goog"
      req.automation = { kind: "signed-agent", operator: result.agent };
      return next();
    }
    // A present-but-invalid signature is a stronger negative signal than no signature.
    req.automation = { kind: "invalid-signature" };
    return next();
  }

  const ua = req.get("User-Agent") ?? "";
  const token = KNOWN_AGENT_TOKENS.find((t) => ua.includes(t));
  if (token) {
    // User-agent strings are trivially spoofed: only trust them together with IP verification.
    req.automation = (await isVerifiedCrawlerIp(req.ip, token))
      ? { kind: "declared-agent", operator: token }
      : { kind: "spoofed-agent-ua", claimed: token };
  }
  next();
}

Two further rules. Do not serve agents different content from users (cloaking); agents are increasingly used to verify claims on behalf of users, and mismatches erode trust and may violate search policies. And prefer rate limits and authentication over CAPTCHAs for protecting actions: a CAPTCHA blocks the agent the user deliberately asked to help, and at least one vendor (Anthropic) documents that its crawlers do not attempt to bypass CAPTCHAs.

robots.txt and AI crawler controls¶

robots.txt (RFC 9309) is advisory and applies to crawlers. Each vendor documents its own tokens and how they treat the file; the rows below reflect vendor documentation as of September 2026:

Sources: OpenAI crawlers, Anthropic crawlers, Google common crawlers, Google user-triggered fetchers, Perplexity crawlers.

A policy that allows AI search and user-initiated fetching, declines model training, and keeps private app routes out of every crawler:

# Private application routes: never useful to any crawler.
User-agent: *
Disallow: /app/
Disallow: /account/
Disallow: /api/
Allow: /

# Decline use of content for model training.
User-agent: GPTBot
Disallow: /

User-agent: ClaudeBot
Disallow: /

User-agent: Google-Extended
Disallow: /

# AI search crawlers: same rules as everyone else (inherit by listing explicitly).
User-agent: OAI-SearchBot
User-agent: Claude-SearchBot
User-agent: PerplexityBot
Disallow: /app/
Disallow: /account/
Disallow: /api/

Sitemap: https://shop.example/sitemap.xml

Note that a crawler that matches a specific User-agent group ignores the * group, which is why the search crawlers repeat the private-route rules. Keep the service worker script, manifest and static assets crawlable; blocking them can break rendering for search engines (SEO for PWAs has the PWA-specific rules). Remember what robots.txt cannot do: it does not stop user-triggered fetchers that ignore it, it does not stop browser agents (they are the user), and it is not access control. Private data belongs behind authentication.

llms.txt: a proposal, described honestly¶

/llms.txt is a proposal by Jeremy Howard, first published in September 2024 and revised as "v2" in August 2026, for a Markdown file that gives language models a curated entry point to a site: an H1 with the site name (the only required part), a blockquote summary, optional prose, and H2 sections containing lists of links, with a conventional ## Optional section for secondary material. It also proposes serving clean Markdown versions of pages at the same URL plus .md (for example /docs/page.html.md or /docs/page.md), discoverable with Link relations: rel="alternate"; type="text/markdown" for a page's Markdown version and rel="describedby" for the llms.txt that covers it.

What is and is not true about it as of September 2026:

• It is not a standard of any standards body; the proposal describes itself as a proposal.
• It is widely published, especially for developer documentation, where coding agents fetch it, and several AI vendors publish one for their own docs.
• Chrome's experimental Lighthouse Agentic Browsing category includes an llms.txt audit that fails only on a server error and marks a 404 as not applicable, "as providing the file is optional at the moment".
• No major search engine documents using it as a crawling or ranking signal. Do not expect it to change search visibility.

It costs little, and for a PWA with public documentation or a help center it is a genuinely useful map for agents. Keep it short and link to content that is itself clean:

# Example Shop

> Example Shop is an online store for kitchen and travel goods, available as a website and an
> installable PWA. Customers can browse products, place orders, track deliveries and start returns.

Prices are in EUR and include VAT. Orders ship to Germany, Austria and the Netherlands.
Agents acting for a signed-in customer can use the MCP server at https://shop.example/mcp
(OAuth 2.1; scopes orders:read and orders:write).

## Help

- [Shipping and delivery times](https://shop.example/help/shipping.md): Carriers, costs, cut-off times
- [Returns](https://shop.example/help/returns.md): 30-day return policy and how to start a return
- [Payment methods](https://shop.example/help/payments.md)

## Catalog

- [Product categories](https://shop.example/categories.md): Top-level categories with links

## Optional

- [About us](https://shop.example/about.md)
- [Press](https://shop.example/press.md)

Link: </help/shipping.md>; rel="alternate"; type="text/markdown", </llms.txt>; rel="describedby"

What agents mean for distribution¶

For a decade the distribution question for PWAs was "web or app store?" (PWA vs native, app stores). Agents add a third surface, and several of its properties favor the web:

• Agents operate URLs, not app binaries. Browser agents work on anything a browser can open; a native app's screens are reachable only through OS-level automation that vendors expose selectively. A PWA's public pages, deep links and forms are addressable by every agent today.
• The user's intent is expressed to the assistant, not to your home screen. When a user asks an assistant to "reorder the coffee filters", the winning service is the one the agent can complete the task with reliably. Clean semantics, WebMCP tools and an MCP server are what make that possible; an install prompt does not.
• Chat assistants are becoming app platforms. MCP Apps and vendor directories inside assistants (OpenAI accepts app submissions for ChatGPT, subject to its review) are, functionally, new app stores with review processes, policies and ranking. Your MCP server and a small web-built UI are the entry ticket; your PWA remains the full experience that users open, install and return to.
• Disintermediation is a real risk. If the assistant completes the task, the user may never see your UI, your brand or your cross-sell. Decide deliberately which tasks you expose as tools (status checks, reorders, support), which you keep in your UI (browsing, discovery, account management), and what you ask in return (sign-in with your account, deep links back into the PWA for anything rich).
• Install still matters for the relationship. Push notifications, badging, offline access and a home-screen icon are things an agent cannot provide on your behalf. The installation section remains the path to a direct relationship.

Checklist: an agent-ready PWA¶

• Every interactive control has a role and an accessible name; forms use <label>, autocomplete and associated error messages.
• CLS is low on key task pages; nothing important moves after load.
• Every meaningful state has a URL that loads directly, with correct HTTP status codes.
• Public pages are server-rendered with JSON-LD that matches visible content.
• The service worker never serves an offline fallback with status 200 for a real page while online.
• Consequential actions require a server-verified user confirmation; high-risk ones require step-up authentication.
• State-changing endpoints accept idempotency keys.
• A service layer with explicit actors backs both the PWA API and the MCP server; MCP uses OAuth with scoped, revocable tokens.
• WebMCP tools (if you run the origin trial) register per route, carry accurate annotations and are unregistered on navigation.
• robots.txt states your training and search choices; private routes are behind authentication, not just disallowed.
• Bot management distinguishes verified crawlers, signed agents and unknown automation, and does not rely on user-agent strings.
• Optionally, llms.txt points to clean Markdown help content.

Browser support¶

Support data as of September 2026. For live data see Chrome Status for WebMCP and MDN.

🧪: behind chrome://flags/#enable-webmcp-testing for local testing, or enabled per origin through the Chrome origin trial (Chrome 149 to 156 per Chrome Status) and the Microsoft Edge origin trial (expires November 17, 2026). Lighthouse's category requires Chrome 150 or later. Chrome Status lists "No signal" from Firefox and Safari for WebMCP.

Common pitfalls¶

• Treating agents as one thing. Crawlers, fetchers, browser agents and MCP clients need different policies; a single "block AI" rule blocks your users' assistants and does not stop the rest.
• Relying on robots.txt for privacy. User-triggered fetchers and browser agents are not governed by it. Authenticate private content.
• Trusting tool annotations as security. readOnlyHint, consequentialHint and MCP annotations inform hosts; they enforce nothing. Authorization and confirmation belong on the server.
• Stale WebMCP tools after SPA navigation. Tools must describe the current view; abort and re-register on route changes.
• Hard-coding navigator.modelContext. The API moved to document.modelContext; feature-detect.
• Origin-trial tokens that silently expire. Monitor expiry and keep the non-WebMCP path fully functional.
• Duplicated business logic in the MCP server. Put authorization and validation in a shared service layer.
• Password or cookie sharing for agents. Use OAuth delegation with scopes and per-client revocation.
• Returning untrusted user content from tools without marking it. Set untrustedContentHint (WebMCP) and delimit such content in MCP results.
• Blocking automation with CAPTCHAs on read-only pages. Rate-limit and authenticate instead.

Debugging¶

• See what agents see: Chrome DevTools' Accessibility pane and full-page accessibility tree view show roles and names; fix anything unnamed or generic on task paths.
• Run Lighthouse's Agentic Browsing category (Chrome 150 or later) on key templates and in CI; register imperative WebMCP tools early so the snapshot captures them.
• Inspect WebMCP tools with the Model Context Tool Inspector extension recommended in Chrome's documentation: list tools, call them with test input, check schemas and outputs.
• Test your MCP server with the SDK's client or a test host. curl -s -X POST https://shop.example/mcp -H 'Content-Type: application/json' -H 'Accept: application/json, text/event-stream' -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}' without a token should return 401 with a WWW-Authenticate header containing resource_metadata.
• Check crawler behavior in server logs by token and verified IP; Google Search Console's crawl stats cover Google's crawlers.
• Fetch your pages without JavaScript (curl, or DevTools with JavaScript disabled) to see what non-rendering agents receive.

Further reading¶

On this site

• AI in PWAs: overview
• Cloud AI in PWAs
• On-device AI in PWAs
• The future of AI and PWAs
• Accessibility for PWAs
• SEO for PWAs
• Authentication
• PWA vs native

External references

• WebMCP specification draft (W3C Web Machine Learning CG)
• WebMCP explainer repository
• Chrome for Developers: WebMCP
• Lighthouse: Agentic Browsing category
• Model Context Protocol specification
• MCP Apps overview
• MCP TypeScript SDK v2: serving over HTTP
• IETF Web Bot Auth working group
• RFC 9421: HTTP Message Signatures
• The /llms.txt proposal