AI Assistants (MCP Server)Developer

PageCrawl includes a built-in MCP (Model Context Protocol) server that lets AI assistants manage your page monitors. You can add monitors, check history, trigger checks, and more, all through natural conversation with tools like Claude or ChatGPT.

MCP is an open protocol that standardizes how AI tools connect to external services. Once connected, your AI assistant can directly interact with your PageCrawl account without you needing to use the web interface or API manually.

What You Can Do

The MCP server provides 34 tools that your AI assistant can use. Every tool is available on every plan, including Free. The Free plan can trigger up to 10 on-demand checks a day across all monitors; scheduled checks are unaffected.

Reading monitors and changes

Creating, editing and organizing monitors

Templates, discovery and workspaces

Delivery, reporting and account settings

Supported Element Types

When creating monitors through MCP, you can track the following element types:

• Full Page - Entire page text content (no selector needed)
• Text - Text content of a specific element (CSS selector required)
• Number - Numeric values with change thresholds
• Price - Price values with currency detection
• HTML - Raw HTML structure of an element
• JavaScript - Execute JavaScript and track the result
• File Hash - Monitor file changes by checksum (no selector needed)
• PDF - Track changes in PDF documents (no selector needed)

Which Setup Path Do I Need?

Every client connects to the same address, https://mcp.pagecrawl.io/mcp. What differs is how it signs in. Apps with a built-in connector UI (Claude, ChatGPT, Notion) use OAuth: you paste the URL, get sent to PageCrawl, and click Approve, with no key to copy. Editors and headless clients that take a config file can use OAuth too, or a personal API token if they do not support the OAuth flow.

PageCrawl's server uses the streamable HTTP transport (the current MCP standard), not the older SSE transport. Most clients detect this on their own; ChatGPT asks you to pick, and Cursor and OpenClaw have a transport setting you may need to set explicitly.

Setting Up with Claude (Web & Desktop)

1. Open claude.ai or Claude Desktop and go to Settings
2. Navigate to the Connectors section in the left sidebar
3. Click Add custom connector at the bottom of the page
4. Enter a name (e.g. "PageCrawl") and set the URL to: https://mcp.pagecrawl.io/mcp
5. Click Add. You will be redirected to PageCrawl to authorize access
6. Log in (if not already) and click Approve
7. PageCrawl tools are now available in your conversations

Setting Up with Claude Code

Add the following to your .mcp.json file (in your project root or ~/.claude/):

{
  "mcpServers": {
    "pagecrawl": {
      "url": "https://mcp.pagecrawl.io/mcp"
    }
  }
}

When Claude Code first tries to use PageCrawl tools, it will open a browser window for you to authorize the connection via OAuth.

Setting Up with ChatGPT

Works with ChatGPT on web, desktop, and mobile. Requires a ChatGPT Plus, Pro, Team, Enterprise, or Edu plan.

1. Go to chatgpt.com (or open the ChatGPT desktop app)
2. Navigate to Settings > Apps & Connectors (called Connectors in older builds), then Create
3. Enter a name (e.g. "PageCrawl"), a short description, and set the MCP Server URL to: https://mcp.pagecrawl.io/mcp
4. Set Authentication to OAuth
5. Under Server transport (or Connection type), choose Streamable HTTP. PageCrawl's server speaks streamable HTTP, not the older SSE transport, so leaving it on SSE makes the connection fail
6. Tick the box confirming you trust the connector, then click Create. You will be redirected to PageCrawl to authorize access
7. Log in and click Approve
8. To use in a conversation, click the + button near the message input, select More, and enable PageCrawl

If ChatGPT reports that it could not reach the server, the transport setting is the first thing to check: switch it to Streamable HTTP and create the connector again.

Setting Up with Microsoft 365 Copilot

Microsoft 365 Copilot reaches MCP servers through agents built in Copilot Studio. You add PageCrawl to an agent as a tool, then publish the agent so people can use it in Microsoft 365 Copilot and Teams. You need Copilot Studio access in your Microsoft 365 organisation.

1. Open Copilot Studio and open the agent that should use PageCrawl, or create a new one
2. Go to Tools, click Add a tool, then New tool, and choose Model Context Protocol
3. Set Server name to "PageCrawl", give it a Server description such as "Monitors websites for changes and reports what changed", and set Server URL to https://mcp.pagecrawl.io/mcp
4. Set Authentication to OAuth 2.0 and Type to Dynamic discovery, then click Create
5. On Add tool, choose Create a new connection. You will be redirected to PageCrawl to sign in and approve
6. Click Add to agent
7. Click Publish, then open Channels and add Teams and Microsoft 365 Copilot so the agent appears in Copilot Chat and Teams

Copilot Studio uses the server description to decide when to call PageCrawl, so describe what it does in plain words. If Model Context Protocol is missing or the connection is blocked, your organisation's data policy may restrict custom connectors, and a Power Platform admin needs to allow it.

Setting Up with Notion

Works with Notion custom agents. You need to be a Notion workspace admin for the first step.

1. In Notion, open Settings > Connections and turn on custom MCP servers for the workspace
2. Choose Add connection > Add custom MCP
3. Set the MCP server URL to https://mcp.pagecrawl.io/mcp, name it "PageCrawl", and set Authentication to OAuth
4. Click Connect. You will be redirected to PageCrawl to sign in and approve
5. The tools are then available to your Notion custom agents

Notion shows a warning that it has not reviewed the server. That appears for every custom MCP connection, including ones you host yourself, and reflects who published the connector rather than whether it works.

Because the agent lives in Notion, it can write what it finds straight into your own databases. Ask it to watch a set of pages, then to update a database with what changed, and the database maintains itself. The walkthrough in monitoring website changes from Notion covers a full example.

If you want every change recorded without an agent deciding anything, PageCrawl can write the rows itself instead. See syncing changes to a Notion database. The two work independently and can both be on.

Setting Up with Other MCP Clients (OAuth)

Any MCP-compatible client that supports OAuth can connect to PageCrawl. The server details:

• URL: https://mcp.pagecrawl.io/mcp
• Authentication: OAuth 2.0 (automatic via MCP protocol)
• Protocol: MCP over HTTP with JSON-RPC 2.0
• OAuth Discovery: https://pagecrawl.io/.well-known/oauth-authorization-server

The client will handle the OAuth flow automatically. No manual token setup is required.

Setting Up with API Token (OpenClaw, Cursor, Cline, Windsurf, and others)

For MCP clients that do not support OAuth, you can connect using a personal API token instead. This works with OpenClaw, Cursor, Cline, Windsurf, VS Code, Claude Code, and any other MCP client that supports custom headers.

Step 1: Generate an API token in PageCrawl:

1. Go to Settings > API
2. Click Create Token
3. Give it a name (e.g. "OpenClaw") and click Create
4. Copy the token. It will only be shown once.

Step 2: Add the following configuration to your MCP client. The JSON format below works with Cursor (.cursor/mcp.json), Cline, Windsurf (.vscode/mcp.json), Claude Code (.mcp.json), and most other clients:

{
  "mcpServers": {
    "pagecrawl": {
      "url": "https://mcp.pagecrawl.io/mcp",
      "headers": {
        "Authorization": "Bearer YOUR_TOKEN_HERE"
      }
    }
  }
}

For OpenClaw, use the CLI:

openclaw mcp set pagecrawl \
  --transport streamable-http \
  --url https://mcp.pagecrawl.io/mcp \
  --header "Authorization: Bearer YOUR_TOKEN_HERE"

For Cursor, you can also add via Settings > MCP Servers > Add > Streamable HTTP and enter the URL and authorization header there.

Connection Troubleshooting

The client says it cannot reach the server. Check the transport setting first. PageCrawl speaks streamable HTTP; a client set to SSE will fail to connect even with the correct URL. In ChatGPT this is the Server transport choice in the Create dialog, in Cursor it is the Streamable HTTP option when adding a server, and in OpenClaw it is --transport streamable-http.

Check the URL has no trailing path or slash. Enter it exactly as https://mcp.pagecrawl.io/mcp. Adding /sse, /v1, or a trailing slash will not resolve. Connections added earlier with https://pagecrawl.io/mcp keep working, so there is nothing to reconnect.

ChatGPT does not show the Create option. Custom connectors need a Plus, Pro, Team, Enterprise, or Edu plan. On Team and Enterprise an admin may also need to allow custom connectors for the workspace.

The connector is added but the assistant never uses it. Most clients require you to enable the connector per conversation. In ChatGPT, click + near the message box, choose More, and turn on PageCrawl. In Claude, open the tools menu under the message box and check that PageCrawl is on. Naming it in the prompt ("use PageCrawl to...") also helps the first time.

The tools return an authentication error. The OAuth grant may have been revoked or expired. Remove the connector and add it again, or, if you are using an API token, confirm the token still exists under Settings > API and that the header is Authorization: Bearer YOUR_TOKEN.

A client warns that the server is unverified. Notion, and some others, show this for every custom MCP server, including ones you host yourself. It reflects that the client has not reviewed the connector, not a problem with the connection.

Tools work but return nothing. You are probably connected to a different workspace than the one you are looking at in the web app. Ask the assistant to list your workspaces and name the one you mean.

Polling for New Changes

If you are building something that checks PageCrawl on a schedule rather than asking a question once, pass after_check_id to Get changes since instead of moving a date forward. Give it the next_cursor from your previous response and you get only what is new, oldest first.

A moving date has to guess a boundary, so it either repeats changes you have already handled or drops the ones that landed mid-request. A cursor does neither, and it is cheaper to answer. next_cursor comes back on every response, including empty ones, so a quiet poll does not lose your place.

Ready-Made Prompts

Some clients show a list of prompts a connected server offers, usually behind a slash command or a menu near the message box. PageCrawl ships five, so the common jobs are one click rather than something you have to phrase:

Clients that do not surface prompts are unaffected: the same work is available by asking in your own words.

Example Conversations

Once connected, you can interact with PageCrawl naturally:

Adding monitors:

"Monitor example.com/pricing every hour and track the full page text"

"Set up price tracking for these 3 product pages: [url1], [url2], [url3]. Check every 15 minutes and notify me on Slack when prices drop."

Checking current values:

"What's the current price on my Amazon product monitor?"

"Compare the prices across all my competitor monitors right now"

Reviewing changes:

"Show me all monitors that changed in the last 24 hours with a summary of what changed"

"Show me the diff for the terms of service page. What exactly was added or removed?"

Analysis and reporting:

"Which of my monitors have had the most changes this month? Are there any patterns?"

"Give me a weekly summary: how many changes were detected across all monitors, which ones had price drops, and which ones had errors?"

Batch operations:

"Tag all monitors tracking amazon.com with 'competitor' and 'ecommerce'"

"Check the latest values for all monitors tagged 'pricing' and tell me which products are currently out of stock"

Troubleshooting:

"Are any of my monitors failing? Show me the ones with errors and what the issue is"

"The pricing page monitor hasn't detected changes in weeks. Trigger a fresh check and show me what it finds"

Setting up workflows:

"Create a monitor for each of these 5 competitor pricing pages. Use the 'competitor-tracking' template and tag them all as 'q2-research'"

"Monitor the SEC EDGAR page for new filings from Tesla. Use content-only mode so it ignores the navigation, check every 30 minutes"

Working with Workspaces

All tools automatically search across every workspace you have access to. You do not need to know which workspace a monitor is in to find or interact with it.

• Use List monitors with the search parameter to find monitors by URL, domain, or name
• Use List monitors with workspace_id to filter results to a specific workspace
• Use List workspaces to see all your teams and workspaces with their IDs
• Add page monitor only requires a workspace_id if you have more than one workspace

Limits and Quotas

MCP operations respect your plan's limits:

• Monitor creation counts toward your page monitor quota
• Triggered checks are rate limited. A few at a time are picked up quickly, while triggering many checks in a row or rechecking the same monitor repeatedly queues them behind other on-demand checks, so those can take longer to complete. This is designed for occasional, manual use only (one or two checks at a time). It does not support programmatic or automated triggering - requests that exceed rate limits will be rejected with an error. Instead, configure the check frequency on each monitor and use scheduling settings to run checks at specific times.
• If you exceed your monitor limit, new monitors are created in a disabled state
• If you exceed your check limit, manual check triggers will be rejected

See Check Limits and Website Limits for details on plan quotas.