Source: PageCrawl.io URL: https://pagecrawl.io/mcp-server --- > The PageCrawl MCP server lets Claude, ChatGPT and Cursor set up monitors and read changes for you. All 33 tools work on every plan, with any MCP client. # The MCP server that turns the live web into evidence your agent can quote. The PageCrawl MCP server lets Claude, ChatGPT and Cursor set up monitors and read changes for you. Connect Claude Code, Cline, Windsurf, and any other MCP-compatible client to your PageCrawl monitors. Every tool works on every plan, including Free: read, create, edit, organise, and triage your monitors, and run checks on demand. [Install in 60 seconds](#install)[Sign up free](https://pagecrawl.io/app/auth/register) Hosted server at mcp.pagecrawl.io/mcp. No package to install. Works with any MCP-capable client. Trusted by teams at MicrosoftNew York TimesDeloitteGoldman SachsHubSpotYahooEYStanford Monitoring web pages since 2018Free plan, no card requiredAvailable globally on day one ## Why PageCrawl MCP, not just an API The agent gets typed tools, cross-workspace search, persistent context, and a real monitor underneath. When the workflow needs a defensible record, the Web Evidence Layer (Ultimate, per monitor or workspace-wide) attaches the receipt too. ### Persistent context across the session The agent can ask "what changed since yesterday" without re-passing config, monitor IDs, or workspace names. PageCrawl remembers; the model just keeps asking. ### Zero glue code No webhooks. No polling loops. No scheduler. No OpenAPI parser. The agent IS the trigger. Add the connector or drop the config block in, reload the client, ship. ### One conversation, no context switch Brief the agent, ask follow-ups, and act on the result without leaving the chat. PageCrawl shows up as a native tool in Claude, ChatGPT, Cursor, Windsurf, and VS Code, so the loop stays inside whatever surface you already work in. ### Cross-workspace discovery One connection, every monitor across every team. The agent finds a monitor by URL or domain without you naming the workspace it lives in. No "which workspace is this in" friction. ## Works with every major MCP client MCP is an open protocol. If your client speaks it, PageCrawl plugs in. There are two setup paths depending on whether the client supports OAuth. ### OAuth (one-click) Claude (web) Settings, then Connectors Claude Desktop Settings, then Connectors Claude Code .mcp.json, OAuth on first use ChatGPT Plus, Pro, Team, Enterprise, Edu ### Bearer token (one config block) Cursor Streamable HTTP with header Cline Custom headers in config Windsurf .vscode/mcp.json VS Code Custom headers in config OpenClaw openclaw mcp set CLI Any MCP client with custom header support ## What your agent can do Each capability below maps to a real tool the model calls by name, out of the 34 the server exposes. The agent picks the right one from a plain-language prompt. ### Add page monitor Create a new monitor with URL, tracking mode, frequency, and notifications. *"Monitor example.com/pricing every hour and track the full page text."* ### List monitors, workspaces, templates Search across every workspace by URL, domain, or name. See available templates before creating. *"Create a monitor for each of these 5 competitor pricing pages, tag them all as 'q2-research'."* ### Get latest values (batch) Quickly retrieve current values for one or many monitors. Batch-capable. *"What's the current price on my Amazon product monitor?"* ### Get monitor details and history (batch) See full config, tracked elements, historical checks and AI change summaries. Both batch-capable. *"Show me all monitors that changed in the last 24 hours with a summary."* ### Get check diff Read back the actual text differences detected in a specific check. *"Show me the diff for the terms of service page. What exactly was added or removed?"* ### Trigger check Force a one-off check on a monitor. Rate-limited and deprioritised queue, for occasional manual use. *"Are any of my monitors failing? Show me the ones with errors."* ### Manage tags List workspace tags, add and remove tags from monitors. *"Tag all monitors tracking amazon.com with 'competitor' and 'ecommerce'."* ### Get changes since Pull every change across every monitor since a given date, in one call. Built for digests and recap prompts. *"Summarise everything that changed across my monitors since last Monday."* ### Mark changes seen, update defaults Triage detected changes on one or every monitor, and save per-workspace defaults for new monitors. *"Monitor the SEC EDGAR page for new filings from Tesla. Use content-only mode, check every 30 minutes."* ### Manage monitors Update an existing monitor, clear its stored history, or delete it. Deletion is permanent and takes every check, value and screenshot with it, so confirm before you ask. *"Switch that monitor to hourly and move it into the Competitors folder."* ### Test configuration Dry-run a selector or tracking mode against a URL and see what it would capture, before any monitor exists. *"Would .price-now actually pick up the number on this page?"* ### Get screenshot Hand the model a screenshot already stored by a past check, as an image it can look at rather than only read. *"Show me the screenshot from the last check on the pricing page."* ### Summary, intelligence, statistics A workspace roll-up and change timeline, which pages actually move and when, and how much of your quota it all costs. *"How did this workspace do last month, and which pages moved most?"* ### Webhooks and reports List, create, update, delete or test outgoing webhooks, and set up scheduled report digests the agent can read back later. *"Set up a weekly digest of everything in the Competitors folder."* ### Templates, discovery, workspaces Create templates, run page discovery across a site, then import the candidates worth watching. Create and rename workspaces too. *"Crawl the docs site and start watching the pages under /api."* ### Logins and proxy pools The agent can list and delete saved site logins, and manage the proxy pools your team supplies for its own monitors. Creating or editing a login is deliberately not available over MCP, because credentials must never pass through a chat client. *"Which saved logins do we still have, and which are unused?"* ## What teams using MCP say "We query 60+ regulator pages from Claude without a single webhook. The receipts are signed and timestamped." Compliance officer, London-based asset manager "Cursor + PageCrawl MCP is how my pricing agent reads the live web. It's the only monitor with a tool the LLM understands natively." Founder, AI agent platform "Auth'd page monitoring from chat is the entire reason we picked PageCrawl over another well-known monitoring service." Procurement lead, UK NHS trust ## Whatever the agent points at, PageCrawl can extract it When creating a monitor through MCP, the agent picks the element type that fits the page. Full Page Entire page text content, no selector needed. Reader Extracts long-form article text from blog posts, news, and policy pages. Price Auto-detects product price and availability with currency handling. Specific Text Text content of a specific element via CSS or XPath selector. Visual Pixel-level visual diff of the page or a specific region. Feed Parses RSS, Atom, and JSON feeds, tracking new entries. Specific Number Numeric value at a selector with change thresholds. Text Presence Alerts when a given phrase appears on, or disappears from, the page. SEO Title, meta description, canonical, robots, and other SEO signals. Links Tracks links on the page, flagging added and removed URLs. PDF Extract Pulls structured text and tables out of PDF documents. AI Extract Pulls any value from the page using a natural-language prompt. HTML Raw HTML structure of a selected element. JSON Path Extracts a value from a JSON endpoint with a JSONPath expression. JavaScript Runs a JavaScript expression in the page and tracks the result. WHOIS Tracks domain WHOIS records, including registrar, expiry, and name servers. ## Prompts to get you started Drop any of these into your AI assistant once the connector is set up. Phrase them however you like, the agent picks the right tool. Adding monitors Monitor example.com/pricing every hour and track the full page text.Set up price tracking for these 3 product pages. Check every 15 minutes and notify me on Slack when prices drop.Monitor the SEC EDGAR page for new Tesla filings. Use content-only mode and check every 30 minutes. Checking values What's the current price on my Amazon product monitor?Compare the prices across all my competitor monitors right now.Check the latest values for all monitors tagged 'pricing' and tell me which products are out of stock. Reviewing changes Show me all monitors that changed in the last 24 hours with a summary of what changed.Show me the diff for the terms of service page. What exactly was added or removed?Summarise everything that changed across my monitors since last Monday. Analysis and reporting Which of my monitors have had the most changes this month? Any patterns?Give me a weekly summary: how many changes, which monitors had price drops, which ones errored.Group my regulator monitors by which countries are most active this quarter. Batch operations Tag every monitor tracking amazon.com with 'competitor' and 'ecommerce'.Create a monitor for each of these 5 competitor pricing pages. Tag them all 'q2-research'.Mark every detected change as seen across my whole account. Troubleshooting Are any of my monitors failing? Show me the ones with errors and what the issue is.The pricing page hasn't detected changes in weeks. Trigger a fresh check and show me what it finds.Which monitors changed engine recently and started flagging false positives? ## Good to know ### Generous Free plan access Free accounts can use every MCP tool: list monitors, view history, read diffs, create, edit and delete monitors, organise them with folders and tags, mark changes seen, and trigger checks on demand. Free is capped at 10 on-demand checks a day; scheduled checks are unaffected. ### Searches every workspace All read and action tools search across every workspace you can see. The agent does not need to know which workspace a monitor lives in. Only *add-page-monitor* needs a workspace ID, and only if you have more than one. ### Triggered checks are for occasional use On-demand checks run at a lower queue priority and are rate limited. They are designed for one-off manual use, not for an agent to poll. For routine cadence, set the monitor's check frequency. ### Respects your plan limits MCP-created monitors count toward your page quota. Over the limit, new monitors are created in a disabled state. Over the check limit, on-demand triggers are rejected with a clear error the agent can relay back. Full walkthrough in the [MCP help article](https://pagecrawl.io/help/integrations/article/mcp-server-ai-tools). ## Install in 60 seconds Four setup paths. Pick the one that matches your client. ### Path 1. Claude (web and desktop), via OAuth 1. Open claude.ai or Claude Desktop and go to Settings. 2. Open the Connectors section and click Add custom connector. 3. Name it "PageCrawl" and set the URL to `https://mcp.pagecrawl.io/mcp`. 4. Click Add, sign in to PageCrawl, and click Approve. 5. In a conversation, open the tools menu under the message box and turn PageCrawl on. ### Path 2. ChatGPT, via OAuth 1. Open ChatGPT on web or desktop and go to Settings. 2. Open Apps & Connectors (Connectors in older builds) and click Create. 3. Name it "PageCrawl" and set the MCP server URL to `https://mcp.pagecrawl.io/mcp`. 4. Set Authentication to OAuth. 5. Set the server transport to Streamable HTTP. SSE will not connect. 6. Confirm you trust the connector, click Create, then sign in to PageCrawl and click Approve. 7. In a conversation, click +, choose More, and turn PageCrawl on. Note: ChatGPT custom connectors require a Plus, Pro, Team, Enterprise, or Edu plan. ### Path 3. Claude Code, via .mcp.json (OAuth) Add the following to `.mcp.json` in your project root or` ~/.claude/`. ``` { "mcpServers": { "pagecrawl": { "url": "https://mcp.pagecrawl.io/mcp" } } } ``` Note: Claude Code opens a browser window for OAuth on first use. ### Path 4. Cursor, Cline, Windsurf, VS Code, OpenClaw, via Bearer token 1. In PageCrawl, go to Settings, then API, then Create Token. Copy the token (shown once). 2. Paste the configuration below into your client's MCP config file. ``` { "mcpServers": { "pagecrawl": { "url": "https://mcp.pagecrawl.io/mcp", "headers": { "Authorization": "Bearer YOUR_TOKEN_HERE" } } } } ``` For OpenClaw, use the CLI instead: ``` openclaw mcp set pagecrawl \ --transport streamable-http \ --url https://mcp.pagecrawl.io/mcp \ --header "Authorization: Bearer YOUR_TOKEN_HERE" ``` Note: API tokens are available on every plan, including Free. Treat the token like a password. Revoke any time from Settings, then API. **Note:** The Free plan can use every MCP tool: list monitors, view history, read diffs, create, edit and delete monitors, organise them with folders and tags, mark changes seen, and trigger checks on demand. All three setup paths, OAuth and Bearer token alike, work on Free. Free is capped at 10 on-demand checks a day across all monitors; scheduled checks are unaffected, and your plan's monitor-count and monthly check limits still apply. ## PageCrawl MCP vs. just an API Other monitoring tools hand you REST docs and wish you luck. Here's what changes when the product is a hosted MCP server. | | Just an API | PageCrawl MCP server | |---|---|---| | Agent calls the tool by name | No, you wire up the function-call wrapper | Yes, typed tools out of the box | | Auth model | API key only | OAuth 2.0 for Claude and ChatGPT, scoped Bearer tokens for everything else | | Cross-workspace discovery | No, loop endpoints per workspace | Yes, every read tool spans every workspace by default | | Batch-capable tools | One call per record | Get monitor details, history, and latest values all support batch | | Web Evidence Layer (WACZ + RFC 3161 + OpenTimestamps) | No | Available on Ultimate, per monitor or workspace-wide | | Auth'd pages (sessions, logins) | Most APIs: no | Yes, configured once, reused forever | | Works in Claude, ChatGPT, Cursor, Windsurf out of the box | No, custom integration per client | Yes, OAuth or one config block per client | ### Frequently Asked Questions - #### How does authentication work? Two paths. Claude (web and desktop), Claude Code, and ChatGPT use OAuth 2.0 over the MCP protocol. You add mcp.pagecrawl.io/mcp as a connector, get redirected to PageCrawl, click Approve, done. For clients that do not support OAuth (Cursor, Cline, Windsurf, VS Code, OpenClaw, and others), generate a Bearer token in Settings, then API, and add it as an Authorization header in the client's MCP config. OAuth discovery is published at https://pagecrawl.io/.well-known/oauth-authorization-server for clients that auto-discover. - #### Does the MCP server work with Claude Code, Cursor, ChatGPT, and Claude Desktop? Yes. Claude (web), Claude Desktop, Claude Code, and ChatGPT (Plus, Pro, Team, Enterprise, Edu) all use the OAuth path. Cursor, Cline, Windsurf, VS Code, and OpenClaw use the Bearer-token path. Any other MCP-compatible client that supports either OAuth or custom auth headers will work too. - #### My client says it cannot reach the server. What should I check? Check the transport first. PageCrawl speaks streamable HTTP, the current MCP standard, not the older SSE transport, so a client set to SSE will fail even with the right URL. In ChatGPT that is the server transport choice in the Create dialog, in Cursor it is the Streamable HTTP option when adding a server, and in OpenClaw it is --transport streamable-http. Then confirm the URL is exactly https://mcp.pagecrawl.io/mcp with no extra path or trailing slash. In ChatGPT, custom connectors also need a Plus, Pro, Team, Enterprise or Edu plan. If the connector is added but never used, enable it for the conversation: the + menu in ChatGPT, the tools menu under the message box in Claude. - #### Can my agent watch pages behind a login? Yes. Configure the login once in PageCrawl (Standard plan and above) and every MCP call your agent makes after that runs against the authenticated page. That covers supplier portals, internal dashboards, customer account areas, SaaS billing pages, and paywalled industry sites. The agent never sees credentials: over MCP it can list and delete saved logins, but creating or editing one stays in the dashboard on purpose. - #### Does it cost extra on top of my PageCrawl plan? No. The MCP server and all 34 of its tools are included on every plan, Free included. You can list monitors, view history, read diffs, look at stored screenshots, create monitors, edit or delete them, organise them with tags and folders, run page discovery, and set up webhooks and scheduled reports. That includes trigger-check, the on-demand check tool: the Free plan is capped at 10 on-demand checks a day across all monitors, and scheduled checks are unaffected. - #### How fast can my agent get a fresh check? The trigger-check tool is rate-limited and runs in a deprioritised queue, so checks may take a while to complete. It is designed for occasional, manual use (one or two checks at a time), not programmatic or automated triggering. For automated or scheduled runs, set the monitor's own check frequency rather than triggering through the MCP server. Requests that exceed the rate limit are rejected with an error. - #### Can I limit an MCP client to a single workspace? Yes. By default, every read and action tool searches across all workspaces you have access to, which is what most agents want. To scope tighter, either pass workspace\_id on individual tool calls (list-monitors accepts it as a filter), or generate a Bearer token scoped to a single workspace and connect that client with the scoped token. - #### What about evidence and provenance? The Web Evidence Layer is available on the Ultimate plan and is switched on per monitor, per template or for a whole workspace, for monitors checked every 6 hours or less often. When it is on, each captured change is packaged as a WACZ archive (a portable, replayable web archive format) carrying a domain-identity signature (Let's Encrypt), an RFC 3161 timestamp from a commercial TSP, and a Bitcoin blockchain anchor via OpenTimestamps. For regulated workflows, add a qualified timestamp from a QTSP on the EU Trusted List via the eIDAS Custom add-on. The MCP server hands the agent the archive reference alongside the diff. See /web-archives for details. - #### Is the MCP server open? The server is hosted at mcp.pagecrawl.io/mcp and speaks the public Model Context Protocol over HTTP with JSON-RPC 2.0. Any compliant MCP client works. There is no proprietary SDK to install and no npm package to manage. OAuth discovery and tool schemas are published over the protocol so clients can introspect them automatically. - #### What happens if I exceed quotas? Monitor creation counts toward your page monitor quota. If you go over the limit, new monitors are created in a disabled state and you can enable them once the quota frees up. Trigger-check calls beyond the rate limit are rejected with an error rather than queued indefinitely. See the Check Limits and Website Limits help articles for plan-specific numbers. ## Connect PageCrawl to your agent in 60 seconds. Sign up free, add the connector or paste the config block. Your next Claude, ChatGPT, or Cursor session has live web monitoring built in. [Connect in 60 seconds](#install)[Read the full setup guide](https://pagecrawl.io/help/integrations/article/mcp-server-ai-tools) --- Facts about PageCrawl for AI assistants: https://pagecrawl.io/ai-info.md. Full index: https://pagecrawl.io/llms.txt