You can find all your invoices here.
If you wish to receive invoices to your email each month/year, enter your email address in the billing details section:
If you would like to change or upgrade your plan, just go to your Subscription settings and choose a plan you want to switch to. Upgrades/downgrades are prorated, meaning, that the unused time will be applied as a credit for the next payment. e.g. you subscribed to $8/mo plan but you only used it for half-a-month and decided to upgrade to $30/mo plan. When upgrading, 4$ will be credited back and the remaining half-of-the-month of $30/mo plan will only cost you 11$.
You can cancel your subscription, by going to your Subscription settings and clicking on the red "Downgrade to Free" button. The subscription will be canceled immediately.
]]>Unfortunately, for security and to prevent service abuse, email addresses cannot be changed directly by users.
To change your email address please contact support at help_me@pagecrawl.io from your originally registered email address. We will verify the information and get back to you as soon as possible.
Email address for 'Free Forever' plan users cannot be changed to prevent service abuse.
]]>Unfortunately, it is not yet possible to pay via Paypal.
We only support subscription billing by credit/debit card for monthly and annual billing intervals.
]]>You can find all your invoices here.
If you wish to receive invoices to your email each month/year, enter your email address in the billing details section:
We accept all major credit and debit cards for subscriptions.
For Ultimate plans paid annually, we also support:
If you would like to arrange an alternative payment method, please contact support at support@pagecrawl.io.
]]>The most common reasons for a failed transaction include insufficient funds, incorrect card details, and suspicions of fraud.
In case of a transaction failure first, check if the card details you entered are correct and make sure that there are enough funds in your account to make a purchase.
If the transaction keeps getting declined try using another card or contact your card issuer. In most cases your card issuer will be able to remove the block and allow the transaction to go through.
Common reasons for a payment failure:
PageCrawl can monitor PDF files hosted online and notify you when the text content changes. It extracts text from the PDF, compares it against the previous version, and highlights exactly what was added, removed, or modified.
PDFs behind login authentication are also supported. Configure an authentication setup first, then select it when adding the PDF to monitor.
| Method | What It Detects | Diff Available |
|---|---|---|
| PDF text tracking | Text content changes (additions, deletions, edits) | Yes, line-by-line diff |
| File checksum | Any modification to the file (including metadata, images) | No, only detects that something changed |
Use PDF text tracking when you need to see exactly what text changed. Use file checksum monitoring when you need to detect any modification, including non-text changes.
While SMS messages can be useful for mission-critical applications, to avoid increasing the subscription costs, we do not include native SMS notifications in our subscription plans.
For personal use, we suggest using Telegram Messenger as an alternative of the SMS notifications. It is free of charge, and you only need Internet connection on your mobile phone, which you most likely already have and will need to review what has changed on your monitored page.
If you really need to receive change notifications by SMS, you can receive them by setting up Zapier integration to send SMS messages. Zapier allows integrating our application to over 2000 services easily (for an additional cost and there may be a limit for the number of SMS each month).
We have integrations with other notification channels, visit PageCrawl.io Integrations to learn more.
]]>We aim to respond to your inquiries promptly but sometimes due to an increased number of support requests Enterprise customer requests/emails are prioritized over the Standard customers. Therefore, the response time is faster, also you may expect a 'higher level' of support in case you are not able to set up the page the way you want.
For technical support our response times are prioritized according to your subscription plan:
No. We price our services based on the number of pages primarily and you can upgrade your plan if you need to track more pages.
]]>The Standard plan includes 15,000 checks, and the Enterprise plan allows for 100,000 checks each month. Both plans can be purchased in multiples if you require more pages checked or more frequent checks.
It all depends on how many pages you want to track and how frequently. Also, adjusting your schedule may reduce the number of checks needed. You may start with the Standard plan and upgrade if you notice that you need more.
A few rules of thumb:
If your estimated number of checks for this period will be over the limit, you will see an alert. You can check your usage statistics to find out your current estimate.
]]>Deletion of your account will result in loss of ALL data associated with it.
To delete your account go to the General Settings, scroll to the bottom of the page, press Permanently delete your account, and proceed with the instructions.
]]>
PageCrawl.io monitors websites for changes and sends instant notifications through your preferred channels. This guide walks you through connecting PageCrawl.io with Microsoft Teams to receive alerts directly in your Teams channels.
Before starting, ensure you have:
A PageCrawl.io account
→ Sign up here if you don't have one yet
Microsoft 365 For Business subscription
Basic Teams plans don't support external webhooks - you need a Business plan
1.1 In your Teams channel, click the Workflows menu
1.2 Select "Post to a channel when a webhook request is received"
1.3 Click Next and name your workflow Use a descriptive name like "PageCrawl Website Monitoring"
1.4 Copy the generated webhook URL
Choose your notification scope:
Option A: Monitor All Pages
→ Go to Workspace Settings
→ Paste the Teams webhook URL
→ Save changes
Option B: Monitor Specific Pages
→ Open settings for individual pages
→ Add the Teams webhook URL
→ Save changes
Tip: Set a default webhook for all pages, then override for specific ones that need special handling.
Not working? Check that:
We do have more supported notification channels to suit everyone's preferences.
PageCrawl allows you to track changes in websites and get notified instantly via your preferred method. In this article we will discuss how you can setup PageCrawl to receive notifications in Discord.
You need an PageCrawl.io account. This works in both Free and Paid accounts. If you don't already have one, go here to register an account.
Follow the steps below to retrieve a Discord Webhook URL
If you would like to receive notifications for all tracked pages, simply paste webhook URL in user notification preferences.
If you only want a single page to be notified about in Discord. Just set this Webhook URL in a specific page.
What if I can't edit the server? You should ensure you have permissions from the server owner to edit channel.
I didn't receive a notification Please wait for page to change. We will only send a notification when we detect a change.
I receive too many notifications? What can I do? You may setup notification rules to be notified only when e.g. text disappears, number increases, etc.
We do have more supported notification channels to suit everyone's preferences.
PageCrawl.io allows you to track changes in websites and get notified instantly via your preferred method. In this article we will discuss how you can setup PageCrawl to receive notifications in Telegram.
You need an PageCrawl.io account. This works in both Free and Paid accounts. If you don't already have one, go here to register an account.
Follow the steps below to retrieve a Telegram Chat ID. This is needed so you could receive notifications in a 1-to-1 chat, channel or a group conversation.
Simply begin a conversation with @PageCrawlBot and you will receive instructions how to configure it.
Instructions for Channels and Groups are identical. To include the bot in the Channel or Group you should invite @PageCrawlBot to the channel. You may likely also need to adjust bot permissions, so it could read and send messages. To get instructions what code you should put in PageCrawl.io settings, send a /start message to the bot: @PageCrawlBot /start
Keep in mind that Channels or Group conversations have a negative chat id! 1-to-1 conversations - always positive chat id.
If you would like to receive notifications for all tracked pages, enter the Chat ID you obtained in previously in user notification preferences.
If you only want a single page to be notified about in Telegram. Just set this Chat ID in a specific page.
What if I can't edit the server? You should ensure you have permissions from the server owner to edit channel.
I didn't receive a notification Please wait for page to change. We will only send a notification when we detect a change.
I receive too many notifications? What can I do? You may setup notification rules to be notified only when e.g. text disappears, number increases, etc.
We do have more supported notification channels to suit everyone's preferences.
If you're looking to track pages on websites that require login authentication, the answer is yes – it is possible. Please note that this feature is only available on paid plans.
Monitoring password-protected pages is a two-step process:
Before you can monitor password-protected pages, you need to set up an authentication configuration:
You can create multiple authentication configurations for different websites.
Once your authentication is configured:
The system automatically detects and shows only authentication configurations that match the domain of the URL you're monitoring. For example, if you're monitoring https://app.example.com/dashboard, it will show authentication configs set up for example.com.
If you want to track files such as PDFs, Excel spreadsheets, CSVs, or Word documents, you're in luck. These types of files can also be tracked, even if they are behind login authentication. Simply provide the link to the file and select the appropriate authentication configuration.
In case the website is using "HTTP Basic Authentication" (the browser popup that asks for credentials), you can enter the credentials under "Advanced Settings" when setting up your monitored page. This is different from form-based login authentication.
]]>PageCrawl provides three ways to integrate with external systems: a REST API, webhooks, and RSS feeds.
API and webhooks are available on paid plans.
The REST API lets you manage monitors programmatically, including creating pages, retrieving change history, and triggering checks. Find your API key in Settings > API.
See the API & Webhooks guide for endpoints and authentication details.
Webhooks send HTTP POST requests to your endpoint whenever a change is detected or an error occurs. Configure them in Settings > Workspace > Integrations > Webhooks.
See the Webhook Integration guide for setup, payload fields, and example payloads.
Access recent changes in Atom RSS format. Generate a public RSS URL for a single page or for all pages in the workspace.
See the RSS Feeds guide for setup instructions.
]]>PageCrawl can monitor CSV (comma-separated values) files hosted online and notify you when their content changes. It retrieves the file, compares the data against the previous version, and shows exactly what rows or values were added, removed, or modified.
CSV files behind login authentication are supported. Configure an authentication setup first, then select it when adding the file.
PageCrawl can monitor Excel files hosted online and notify you when their content changes. It extracts text and data from the spreadsheet, compares it against the previous version, and shows exactly what was added, removed, or modified.
xls, xlsx, ods
Excel files behind login authentication are supported. Configure an authentication setup first, then select it when adding the file.
PageCrawl can monitor PowerPoint presentations hosted online and notify you when their text content changes. It extracts text from the slides, compares it against the previous version, and shows exactly what was added, removed, or modified.
pptx
PowerPoint files behind login authentication are supported. Configure an authentication setup first, then select it when adding the file.
PageCrawl can monitor Word documents hosted online and notify you when their text content changes. It extracts text from the document, compares it against the previous version, and shows exactly what was added, removed, or modified.
doc, docx, odt
Word files behind login authentication are supported. Configure an authentication setup first, then select it when adding the file.
PageCrawl.io allows you to track changes in websites and get notified instantly via your preferred method. In this article we will discuss how you can setup PageCrawl to receive notifications in Slack.
Follow the steps below to create a new Incoming Webhook connector
Visit https://slack.com/apps/A0F7XDUAZ-incoming-webhooks to enable "Incoming WebHooks" for your workspace.
Please note, this is a legacy custom integration - an outdated way for teams to integrate with Slack. You may create Slack app instead, but the setup procedure of "Slack app" is significantly longer so we suggest using the legacy integration.
Simply click "Add to Slack" button. You may be prompted to sign in to your Slack account.
Here you will need to select a Slack channel where the messages from PageCrawl.io bot should be sent to and press "Add Incoming Webhook integration"
Finally you should receive URL address. Copy it and paste in the notification settings as indicated below.
If you would like to receive notifications for all tracked pages, simply paste webhook URL in user notification preferences.
If you only want a single page to be notified via Slack. Just set this Webhook URL for a specific page.
What if I can't the app? You should ensure you have permissions from the Slack workspace owner.
I didn't receive a notification Please wait for page to change. We will only send a notification when we detect a change.
I receive too many notifications? What can I do? You may setup notification rules to be notified only when e.g. text disappears, number increases, etc.
We do have more supported notification channels to suit everyone's preferences.
Be notified about website changes via Email
Be notified about website changes via Webhook
Be notified about website changes via Zapier
Monitoring tracked pages can sometimes result in frequent false-positive notifications, often stemming from pesky cookie popups. To address this issue and enhance your monitoring experience, we provide the "Blocking Cookies and Ads" action. This action effectively handles the majority of cookie windows and blocks ads, minimizing unnecessary notifications. Here are some considerations and alternatives to optimize your monitoring experience.
To mitigate false positives, we highly recommend implementing the "Blocking Cookies and Ads" action on all tracked pages. This action has proven to be remarkably effective, successfully handling approximately 99% of cookie popups and preventing ad content from triggering notifications.
In specific cases, if the tracked page is accessed from a location outside of Europe, cookie popups might not be displayed. As an alternative approach, you can opt to perform checks from a different country to avoid encountering cookie-related notifications.
Please be aware that a deprecated version of the "Block Cookies and Ads" action exists, which targets a narrower range of cookie popups. For optimal performance and to take advantage of the full feature set, we strongly advise updating to the current version. Keep in mind that automatic updates are not applied to prevent triggering unnecessary notifications.
]]>
Frequently, you encounter text like "updated 1 month ago" or "last changed 1 hour ago" that continually updates on your monitored pages. While this information might seem informative, it often leads to false-positive notifications.
To address this issue and improve your monitoring experience, we recommend applying the "Remove Dates" action to your tracked page. This action will intelligently detect and replace all date-related text with a standardized [DATE REMOVED] tag.
The "Remove Dates" action is designed to handle a wide range of common date formats, including:
Instead of replacing dates with [DATE REMOVED] placeholders you may completely ignore all changes in numbers by adding "Ignore numbers" filters to "Conditions/Filters" section. Only use this if you are not interested in numeric changes.
]]>
In certain situations, you may wish to exclude or remove a specific section on the page to prevent (false positive) notifications, especially when the content changes frequently. For instance, you might want to exclude a sidebar containing new blog posts or a Twitter feed at the bottom of the page.
When your tracked element type is "Full page" you may choose to track Everything on the page or Content only. If you choose Content only, text in header, sidebar, footer will not be tracked.
If you would like more control on what is removed, we recommend using the "Remove Element" action to exclude sections that do not interest you. You can either utilize the visual selector to remove the area or add the selector manually. Below you will find a few suggested selectors you can use.
Frequently, there are areas where tracking changes may not be of interest, including:
You can use the following selector (which you can paste into the "CSS/XPath selector") to exclude the mentioned elements: nav,aside,footer,.footer,header
Unfortunately, not all websites adhere to the content sectioning guidelines. In such cases, you may need to use the visual selector to identify the area or manually input the selector.
]]>PageCrawl provides built-in proxy locations and supports custom proxy servers for pages that require specific geographic access or have IP-based restrictions.
PageCrawl offers multiple proxy locations across North America, Europe, and the Middle East, plus a residential proxy option. Select a proxy location per page or apply one to multiple pages via Bulk Edit. You can also choose Random to rotate between locations automatically.
Use your own proxy servers when the built-in locations do not work for your use case.
Supported formats:
host:port
username:password@host:portConfiguration options:
| Method | How |
|---|---|
| Single page | Edit the page > Power User settings > Custom Proxy |
| Multiple pages | Select pages > Bulk Edit > Custom Proxies |
| Template | Add proxy settings to a template for reuse |
You can paste multiple proxy servers (one per line). PageCrawl will randomly select one for each check. If a proxy fails, the system automatically retries with a different proxy from the list.
When a page is blocked (timeout, 403, or 401), PageCrawl automatically switches to Stealth mode in addition to the proxy configuration. This combination resolves most access issues.
For pages that require residential proxies, PageCrawl offers Premium Residential Proxies with pay-as-you-go bandwidth starting at $10/GB. Purchase bandwidth in your account settings and select "Premium Residential" as the proxy location on your monitors. See the residential proxies guide for details on pricing, geo-targeting, and setup.
Most pages work fine without any proxy configuration. You only need a custom proxy if a website is actively blocking bots or restricting access by geographic location. Start without a proxy, and only set one up if you are seeing access errors (403, bot protection blocks, empty pages).
If the built-in proxy locations are not enough for your needs, you can use a third-party proxy provider. Here is what to look for and some popular options.
Understanding bandwidth usage:
Each page check downloads the full page without caching, so bandwidth adds up quickly. An average web page uses 2-3 MB per check. Heavier pages (news sites, e-commerce, image-heavy pages) can use 5-10 MB or more. For example, monitoring 50 pages every 30 minutes at 3 MB each would use roughly 7 GB per day, or around 216 GB per month. Because of this, avoid proxy providers that charge per GB of traffic. Those plans are designed for one-off scraping, not ongoing monitoring.
What to look for:
username:password@host:port authentication.Datacenter vs. residential proxies:
Datacenter proxies with unlimited bandwidth are the most cost-effective option for monitoring. They work well for most websites. Residential proxies (using real ISP addresses) are only needed for sites with strict bot detection that blocks datacenter IPs. If you need residential proxies, look for providers that offer them with unlimited bandwidth or per-IP pricing rather than per-GB billing.
Popular proxy providers that work with PageCrawl:
| Provider | Type | Pricing Model | Notes |
|---|---|---|---|
| Webshare | Datacenter, Residential | Per proxy, unlimited bandwidth | Free tier available, good for testing. Paid datacenter plans include unlimited bandwidth. |
| IPRoyal | Datacenter, Static residential | Per proxy (datacenter) | Datacenter proxies with unlimited traffic. Static residential proxies available per IP. |
| Proxy-Cheap | Datacenter, Static residential | Per proxy, unlimited bandwidth | Budget-friendly static residential and datacenter proxies with no traffic limits. |
| ProxyRack | Datacenter, Residential | Flat monthly rate | Unlimited bandwidth on most plans. Rotating and geo-targeted options. |
These are independent providers and not affiliated with PageCrawl. Prices and features may change.
Not every provider works for every website. A proxy that works perfectly for one site may get blocked on another. This depends on the website's bot detection, the proxy provider's IP reputation, and the type of proxies used. Always test a provider against your specific pages before committing to a long-term plan. Most providers offer short trial periods or small starter plans for this purpose.
Country-specific access: Some websites restrict content to visitors from a specific country (geo-blocking). Government portals, local news sites, and region-locked services often require an IP address from that country to load correctly. If you are monitoring pages like these, make sure the proxy provider offers proxies in the required country. Check the provider's location list before purchasing, as coverage varies significantly between providers, especially for smaller countries.
Note: Most providers give you a proxy endpoint in the username:password@host:port format. Paste it directly into the Custom Proxy field in PageCrawl. If the provider offers rotating proxies through a single gateway endpoint, you only need to add one line.
Free proxy servers are unreliable, slow, and frequently stop working. They should not be used for monitoring pages where uptime matters. Use the built-in proxy locations, your own paid proxy service, or contact us for residential proxy options.
Some websites use CAPTCHA challenges to block automated access. PageCrawl integrates with 2Captcha, a CAPTCHA-solving service, to handle these protections automatically.
Available on Enterprise and Ultimate plans.
2Captcha handles most common CAPTCHA types including reCAPTCHA v2, reCAPTCHA v3, hCaptcha, and image-based challenges.
CAPTCHA solving is billed by 2Captcha separately from your PageCrawl subscription. Typical costs are $1-3 per 1,000 CAPTCHAs solved. Check 2captcha.com/pricing for current rates.
There may be various reasons why a page fails to open. This guide describes the most common problems and suggests solutions to help you overcome these issues.
A timeout occurs when the page takes too long to respond. This may be a temporary issue with the page, or the page may be loading very slowly. Timeout limits vary depending on your plan:
To avoid timeouts please consider subscribing to a paid plan or upgrading your plan.
This error will be shown if the page has changed significantly and element with configured XPath/CSS selector could not be found. In this case, you should review the page and update selector if needed.
Some pages may use site protection features to block scrapers and website tracking tools like PageCrawl.io. Different pages may use different blocking mechanisms, but here are the most common ones:
Access Restricted to Specific Countries Page may be configured to only allow visitors from a specific country.
Proxy Location blocked The website may block the IP address of the proxy server PageCrawl is using.
Most often indicates that PageCrawl.io Bot was not allowed to access the website. Use "Residential proxy pool" to avoid being blocked.
In most cases this error indicates that page is no longer available to view. You should check and update the page URL.
500, 502, 503, 504 indicates that website server is not responsive, overloaded, currently in maintenance or experiencing server issues. If such error occurs, our bots will retry page check later.
The page can't be opened. In most cases website is down or the website in only reachable from a specific country
Pages may use CAPTCHA to protect the website from bots. To bypass this, you can use a service like 2Captcha which will use human workers to solve the captcha for you. PageCrawl.io has an integration with 2Captcha (you must be subscribed to Enterprise plan) you can sign up for and configure the API token generated from 2Captcha.
In some cases there could be an unexpected error that causes the PageCrawl bot to fail to check the page for changes. In case this error does not go away after a while, please contact support to notify us about the problem so we could prioritize the issue.
]]>
If you encounter a problem with PageCrawl's visual selector and are unable to open the page you are trying to access, there is another option you can try. You can manually copy the selector by opening the desired page in your preferred web browser. This manual method may be more time-consuming, but it can provide a reliable solution if the visual selector is not functioning properly. Additionally, by manually copying the selector, you can have greater control over the elements on the page and the data you want to extract.
This guide will show you how to do it quickly and easily for Chrome, Firefox and Safari browsers.
When it comes to web scraping, finding the right element on a webpage can be a challenge. This is where expression languages like XPath and CSS Selector come in handy. These two powerful tools help you locate elements on a webpage, and choosing between them can be difficult.
CSS Selectors are favored by many web developers as they are easy to learn if you already know CSS syntax. On the other hand, XPath Selectors offer greater power and flexibility, such as the ability to find elements that contain specific text. However, the learning curve for XPath can be steeper.
For those just starting out, CSS Selectors are the recommended choice due to their simplicity and versatility. Most advanced selectors can be written in CSS, making it a good option for web scraping beginners.
When it comes to CSS and XPath Selectors, there are two ways to generate them: relative and absolute.
Relative selectors are preferred in most cases, as they are less prone to break.
Absolute selectors, on the other hand, are useful when tracking a large number of pages, and you are only interested in specific elements. However, with even a slight change in page layout, the selector will break. If an element is added or removed from a page, the absolute XPath will need to be updated to continue tracking the page contents.
Relative selectors tend to be short, while absolute selectors can be lengthy. Here are some examples of relative and absolute selectors for both CSS and XPath:
There are multiple browser extensions available that can help you copy CSS or XPath Selectors. Two options that we tried and can recommend include "SelectorsHub" and "SelectorGadget".
If you prefer not to use a browser extension, you can also find CSS or XPath Selectors by inspecting an element. In most cases, you will get an absolute selector, and if the page content changes, you will need to update the selector.
In conclusion, choosing between XPath and CSS Selectors for web scraping comes down to your personal preference and level of experience. Both offer powerful tools for locating elements on a webpage, and with a little practice, you can become an expert in no time!
If you are reading this, you may have experienced the frustration of the language suddenly switching on your monitored page, causing false positive notifications. Unfortunately, the language behavior of a website is determined by the site developers, and there are several approaches they may use. Some websites base their language on the browser or system settings, which is the best option. Others guess the language based on the country information from the IP address, while others use a mixed approach. There are two approaches you can use to prevent the page language from changing.
To prevent language switching from occurring when monitoring a website for changes, there are a few things you can do. One option is to set the browser language to a specific language, such as "Danish", in "Advanced Settings" by editing the tracked page configuration in PageCrawl. However, keep in mind that some bot detection services can detect this, so use this option only if absolutely necessary.
If you are using "Stealth Mode", be aware that setting the browser language may cause issues. Overriding the browser language can be inconsistent with what bot detection services expect, which may trigger blocks.
Another option is to access the website from a fixed IP address by setting Proxy Location to "Fixed IP". This ensures that the same IP is used to check for changes on the page. However, if the proxy location gets blocked, PageCrawl may not be able to bypass the blocks and displays a crawl error.
]]>
Optimizing your website for search engines requires effective monitoring of SEO tags. PageCrawl makes it easy to track changes to title tags, meta descriptions, canonical URLs, robots directives, Open Graph tags, and headings.
The fastest way to monitor SEO tags is with the built-in SEO Tags tracking mode:
PageCrawl will automatically extract and track:
When any of these fields change, you will see exactly which tag was modified and what the previous and new values are.
If you plan to monitor SEO tags for multiple pages, we recommend creating a Template with the SEO Tags tracking type. This lets you reuse the configuration across many pages without repeating setup.
If you only need to monitor specific SEO tags (rather than all of them), you can create individual tracked elements using CSS or XPath selectors.
Use "Text" for the following tracked elements:
SEO
title/html/head/meta[@name="description"]/@content/html/head/meta[@name="keywords"]/@content/html/head/meta[@name="robots"]/@content/html/head/meta[@name="viewport"]/@contentSocial Media Tags
/html/head/meta[@property="og:title"]/@content/html/head/meta[@property="og:type"]/@content/html/head/meta[@property="og:image"]/@content/html/head/meta[@property="og:url"]/@contentUse "Text (all elements)" for the following tracked elements:
Headings
h1h2h3h4h5
You may also wish to track outgoing links that exist on the page. We suggest using "Text (all elements, sorted)" to capture links to other pages. You may use these selectors to track:
Use the following selector to track all links on a web page:
//a/@hrefTo track only external links (those not belonging to a specific website), use this selector:
//@href[not(contains(.,'not-this-website.com'))]
Note: You should substitute 'not-this-website.com' with the website URL.If you want to track links containing specific keywords in their URLs, use this selector as an example:
//a[contains(@href,'/download/oursoftware_')]/@hrefTo specifically track links leading to PDF documents, you can use this selector:
//a[contains(@href,'.pdf')]/@href//a[contains(text(),'Download')]/@href
Note: This selector is case-sensitive. e.g. if the text actually is "download", it will not be foundIf you want to track links with specific CSS classes, use this selector:
//a[contains(@class,'your-class-name')]/@href
Note: You should substitute 'your-class-name' with the class.To track links with specific attributes (other than href), use this selector and replace "attribute-name" with the name of the attribute you're interested in:
//a[@attribute-name='attribute-value']/@href
Note: You should substitute 'attribute-name' and 'attribute-value' with the relevant attribute values.Over 30% of websites now use bot protection services like Cloudflare, Akamai, and similar tools that block automated access. This means your monitored pages can stop returning data without warning.
PageCrawl provides multiple layers of protection to keep your monitors working, most of which happen automatically.
PageCrawl handles most bot protection automatically. When a check fails, PageCrawl detects the block and adjusts its approach on the next attempt. This includes automatic retries, switching to stealth mode, and rotating through different proxy locations.
For most pages, you do not need to configure anything. The steps below are only needed if automatic handling does not resolve the issue.
PageCrawl will show a warning on the page if it detects a block. You may also notice that the captured content is empty, shows an error code (403, 401), or looks different from what you see when you visit the page yourself.
Note: The settings below require Advanced mode. To enable it, click Edit on any page and toggle Advanced at the bottom of the form.
Follow these steps in order. After each step, wait for the check to complete before moving on.
This is the first thing to try and resolves most blocking issues.
If the content now loads correctly, you are done. Stealth mode will be used for all future checks on this page.
If Stealth mode alone does not work, the site may be blocking the specific IP address or region.
Random proxy rotation means each check comes from a different IP address, making IP-based blocking ineffective.
You can also try specific locations (London, New York, San Francisco, Toronto, Frankfurt) if you know the site serves content differently by region.
For sites with the strictest protections, residential proxies are the most effective option. These route requests through real consumer internet connections, making them virtually indistinguishable from regular visitors.
Residential proxy traffic is available as an add-on. You can purchase residential proxy traffic directly from your PageCrawl account.
Note: Residential proxies consume traffic from your purchased balance. Each check uses a small amount of traffic depending on the page size.
If none of the built-in options work, you can use your own proxy server from a third-party provider.
http://user:password@host:port)This is useful when you need a proxy from a specific country or provider, or when you already have a proxy subscription. See Custom Proxies for more details.
| Solution | How to Enable | When to Use |
|---|---|---|
| Stealth mode | Edit > Advanced > Engine: Stealth | First thing to try for any blocked page |
| Proxy rotation | Edit > Proxy: Random | When a specific IP is blocked |
| Residential proxy | Edit > Proxy: Residential | For the strictest access controls |
| Custom proxy | Edit > Advanced > Custom Proxy | When you need a specific provider or location |
If you have tried all the steps above and the page is still not loading:
<?xml version="1.0"?>
<catalog>
<book id="bk101">
<author>Gambardella, Matthew</author>
<title>XML Developer's Guide</title>
<genre>Computer</genre>
<price>44.95</price>
<publish_date>2000-10-01</publish_date>
<description>An in-depth look at creating applications with XML.</description>
</book>
</catalog>pagecrawl.io offers an efficient way to monitor and track changes in XML files. Instead of sifting through the whole XML file for changes, which can be overwhelming due to frequent updates, you can focus on specific things that matter. This helps you avoid getting flooded with unnecessary alerts for minor changes like 'updated at' dates.
This guide will walk you through the process of setting up and utilizing this feature to simplify your tracking experience.
To reduce the number of false positive you may want to monitor a specific attribute (or multiple attributes), whether it was added, removed or changed.
To begin tracking changes in XML files, follow these steps:
Access PageCrawl: Log in to your PageCrawl account or sign up if you're new to the platform.
Create a Monitored Page: Once logged in, navigate to the dashboard and click on the "New Page" button. This will initiate the setup process for monitoring pages for changes.
Instead of monitoring the entire XML file, you can narrow down your focus to specific attributes that are relevant to you. For example, you might want to track changes in book names within an XML catalog.
Consider the following example xml file structure:
<?xml version="1.0"?>
<catalog>
<catalog>
<book id="bk101">
<title>XML Developer's Guide</title>
<!-- Other book details... -->
</book>
<book id="bk102">
<title>Dummy XML Developer's Guide</title>
<!-- Other book details... -->
</book>
<!-- More book entries... -->
</catalog>
</catalog>
Follow these steps to configure tracking elements for your XML file:
Select Tracked Element: Within the PageCrawl setup interface, choose the "Text (all matches)" as tracking element type.
Specify Element to Track: In this step, you'll specify the exact element within the XML that you want to track. For instance, if you're interested in changes to book titles, you'll set the element as title.
In this case, by focusing on the title element, you'll receive notifications only when book titles change, new is added or removed, filtering out less significant updates.
If you would like to also keep the full history of what has changed in the XML document but only be notified when a specific attribute changes, you can also add "Full Page" as the Tracked Element and then add a condition to be notified when the monitored attribute changes.
]]>Note: This feature is available on paid plans only.
With pagecrawl.io, you can customize your notifications to receive updates either immediately as changes are detected or opt for summarized reports delivered on a daily, weekly, or monthly basis, all at times that align with your preferences.
By default, your account is configured to send email notifications immediately upon detecting changes. Here are scenarios where "Immediate" notifications might be the preferred choice:
Real-Time Monitoring: If you're monitoring critical web pages where even the slightest change holds significant importance, immediate notifications are ideal. For instance, if you're tracking a service status page or an e-commerce website with limited-stock items, getting notified immediately ensures you're always up-to-date.
Time-Sensitive Events: For pages related to events like ticket releases, flash sales, or limited-time offers, immediate alerts are essential. This option lets you act fast on opportunities and respond promptly to changing circumstances.
If real-time updates aren’t needed, summarized reports offer a more manageable approach:
Daily Reports: Best for pages with frequent updates where you’d rather not receive constant notifications. Daily reports summarize all changes in one email, making it easier to stay updated without distraction.
Weekly Reports: Ideal for a bigger-picture overview. Weekly reports work well for pages with moderate update frequency or for tracking trends on less time-sensitive content, like blogs or research articles.
You can choose from the following report options:
Control when PageCrawl runs checks on your monitored pages by setting active days, hours, and check frequency. This is configured per workspace.
Available on paid plans.
Set how often each page is checked. Available frequencies depend on your plan:
| Plan | Minimum Interval |
|---|---|
| Free | Every hour |
| Standard | Every 15 minutes |
| Enterprise | Every 5 minutes |
| Ultimate | Every 2 minutes |
Full frequency options: every 5 min, 15 min, 30 min, 45 min, hourly, every 2 hours, 3 hours, 6 hours, twice daily, daily, every 2 days, every 3 days, weekly, every 2 weeks, and monthly.
Limit checks to specific days and times for an entire workspace:
When outside the scheduled hours or on inactive days, PageCrawl pauses checks for all pages in the workspace. Checks resume automatically when the next active period begins.
Instead of receiving individual notifications for each change, you can configure a daily email digest:
The digest summarizes all changes detected since the last digest was sent.
The integration of PageCrawl.io with Zapier takes web monitoring to the next level by automating tasks and connecting your web monitoring data to countless other applications. In this guide, we'll explore how to set up this powerful integration and unlock a world of possibilities.
Zapier is an automation platform that connects your favorite apps and services, allowing them to work together seamlessly. By integrating PageCrawl.io with Zapier, you can:
Here's a step-by-step guide to help you integrate PageCrawl.io with Zapier and enhance your web monitoring capabilities:
If you're not already a PageCrawl.io user, sign up for an account.
Set up the monitoring settings for the web page you're interested in tracking. Customize the elements you want to monitor and your notification preferences.
Visit Integrations page and click on a button "Open on Zapier" and setup the Zapier + PageCrawl.io integration.
Define the actions you want to take when a trigger event occurs. This can include sending notifications, updating other apps, or performing custom actions.
Once you're satisfied with the setup, activate your Zap, and it will start automating tasks based on changes detected by PageCrawl.io.
Managing and tracking changes on websites is essential for various purposes, from monitoring competitors to ensuring your web services are running smoothly. PageCrawl.io simplifies this process by allowing you to effortlessly monitor web page changes and integrate the data directly into Google Sheets. In this guide, we'll explore how to set up this powerful integration to store website change history efficiently.
Google Sheets offers a versatile and collaborative platform for storing and analyzing data. By integrating PageCrawl.io with Google Sheets, you can keep all your web page change history in one place for easy access and analysis.
Here's a step-by-step guide to help you integrate PageCrawl.io with Google Sheets and start storing website change history effortlessly:
XPath selectors are powerful tools that help you identify and extract specific elements on a web page. In this guide, we'll explore common XPath selectors that you can use when monitoring websites for changes to make your web monitoring efforts more effective.
CSS Selectors are favored by many web developers as they are easy to learn if you already know CSS syntax. On the other hand, XPath Selectors offer greater power and flexibility, such as the ability to find elements that contain specific text. However, the learning curve for XPath can be steeper. If you already know CSS - that's good, you should be able to use it for most use cases. If you don't know any, we recommend starting with XPath, since it can be more flexible.
Here, you'll find a convenient 'cheat sheet' that comprehensively covers the most commonly used XPath selectors for your reference. We suggest taking a quick look through this list before proceeding to the Common XPath Selectors for Web Monitoring section below.
Before we start, you should familiarize yourself with some fundamental concepts to better understand the terminology and functionality. Here are a few key terms:
Attribute: An attribute provides additional information about an HTML element. It is always specified in the start tag of an element and usually comes in name/value pairs like name="value". For example, in <a href="https://example.com">, href is an attribute name and https://example.com is its value.
Element: An HTML element is an individual component of an HTML document or web page. It is written with a start tag, with an optional end tag, and content in between. For example, <p>This is a paragraph</p>; here, <p> is the start tag, </p> is the end tag, and This is a paragraph is the content.
ID: The id attribute is used to specify a unique id for an HTML element. You cannot have more than one element with the same id in an HTML document. It is used for identifying and targeting the element with CSS and JavaScript. For example, <div id="header"> defines a division with a unique id of header.
Class: The class attribute is used for specifying a class name for an HTML element. Unlike the id attribute, the same class can be used on multiple elements. This is useful for applying the same styling or behavior to different elements. For example, <span class="highlight"> assigns the highlight class to a span element, which can be targeted with CSS or JavaScript.
You might wonder where you can try the selector before pasting it in PageCrawl.io You should open browser console and use following commands to test your selector.
XPath
$x('//a')CSS
document.querySelectorAll('a')//: Selects all matching elements anywhere in the document./: Selects from the root element.element: Selects elements with the specified name.[@attribute]: Selects elements with the specified attribute.[@attribute='value']: Selects elements with a specific attribute value.[@attribute!='value']: Selects elements with an attribute value not equal to 'value'.[@attribute^='prefix']: Selects elements with an attribute starting with 'prefix'.[@attribute$='suffix']: Selects elements with an attribute ending with 'suffix'.[@attribute*='substring']: Selects elements with an attribute containing 'substring'.[@attribute1='value1' and @attribute2='value2']: Selects elements that meet multiple attribute conditions.[@attribute1='value1' or @attribute2='value2']: Selects elements that meet at least one of the attribute conditions.not(expression): Negates a condition.text(): Selects the text content of an element.contains(text(),'substring'): Selects elements containing specific text.starts-with(text(),'prefix'): Selects elements with text starting with 'prefix'.ends-with(text(),'suffix'): Selects elements with text ending with 'suffix'./parent::element: Selects the parent of the current element./child::element: Selects the children of the current element./ancestor::element: Selects ancestors of the current element./descendant::element: Selects descendants of the current element.[position()=1]: Selects the first matching element.[last()]: Selects the last matching element.[position()>2]: Selects elements after the first two.*: Selects all elements.element[*]: Selects elements with at least one child element.element[@*]: Selects elements with at least one attribute.element[contains(@attribute,'value')]: Selects elements with attributes containing 'value'.count(expression): Counts the number of matching elements.sum(expression): Sums numeric values within matching elements.concat(string1, string2): Combines two strings.substring(string, start, length): Extracts a substring.normalize-space(string): Removes leading/trailing spaces and collapses internal spaces.Here are some common XPath selectors that you can employ when monitoring websites for changes. Initially, basic XPath selectors will be covered, and we will then proceed to more advanced examples.
XPath allows you to target specific text elements on a webpage, which is useful for tracking changes in content, headlines, or paragraphs. For example:
//h1 // Selects all h1 headers on the page.
//p // Selects all paragraph elements.
//div[@class='content'] // Selects text within div elements with a specific class.XPath selectors help you monitor links, whether you want to track all links on a page, external links, or links with specific text. For instance:
//a[@href] // Selects all links with an href attribute.
//@href[not(contains(.,'example.com'))] // Selects external links (replace 'example.com' with the target domain).
//a[contains(text(),'Download')] // Selects links with specific anchor text, case-sensitive.To view more examples with links, visit Tracking links with text tutorial.
To monitor images on a webpage, you can use XPath selectors to identify images by their source (src) attribute or alt text. For example:
//img // Selects all image elements.
//img/@src // Selects the src attribute of all images.
//img[contains(@alt,'logo')] // Selects images with specific alt text.XPath selectors are particularly useful for extracting data from tables, which are commonly used on websites for displaying structured information. For example:
//table // Selects all tables on the page.
//table//tr // Selects all table rows.
//table//tr/td[2] // Selects the second column (td) in all rows.You can target elements with specific attributes or attributes containing certain values using XPath selectors. For instance:
//*[@id='specificId'] // Selects elements with a specific ID attribute.
//*[@class='highlight'] // Selects elements with a specific class attribute.
To monitor elements when their class or ID contains a part of text, you can use XPath selectors with the contains() function. For example:
//*[contains(@class, 'partial-text')] // Selects elements with a class containing 'partial-text'.
//*[contains(@id, 'partial-text')] // Selects elements with an ID containing 'partial-text'.
//input[starts-with(@name, 'user_')] // Selects input elements with names starting with 'user_'.
//input[contains(@id, 'search')] // Selects input elements with IDs containing 'search'.
//button[contains(@class, 'btn-')] // Selects buttons with class names containing 'btn-'.
This XPath selector is particularly valuable, especially when dealing with CSS classes that include unpredictable or random text fragments.
For instance, suppose you want to extract the text 'Quality Choice' from an image, as shown in the example above. However, the CSS class, such as productTile_urgencyMessaging__V5DTS includes a suffix like __V5DTS that is prone to change with each website update.
To avoid having to update the selector each time website updates, you may employ the XPath contains() function to select an element.
//*[contains(@class, 'productTile_urgencyMessaging')] // Retrieve 'Quality Choice' textXPath supports logical operators for combining conditions. This is particularly useful for complex selections. For example:
//a[@class='external' or @class='external-link'] // Selects links with class 'external' or 'external-link'.
//div[@class='important' and contains(text(),'Alert')] // Selects divs with class 'important' containing 'Alert'.
You can create complex XPath expressions by combining multiple conditions and functions. This provides immense flexibility in your selections. For example:
//div[@class='content' and (contains(text(),'Important') or contains(text(),'Alert'))]
//table[not(@class='hidden')]/tbody/tr[td[2]='Completed']/td[3]
To leverage these advanced XPath selectors effectively for website monitoring, you can integrate them with web monitoring tools such as PageCrawl.io:
Occasionally, you might encounter challenges when using the Visual Selector tool. This guide outlines some common problems and provides solutions to help you resolve them.
You can sometimes see a page loaded but missing some or all of their styles or elements on page.
Solution: To go around this issue you may try Enabling/Disabling JavaScript. If that does not help, you can always copy and paste the selector from your browser window.
In some instances, the Visual Selector tool may have difficulty loading certain pages. Our development team is continually working to enhance its compatibility. You may contact support to report a page that is not working.
Solution: If you encounter this issue, you can try pasting the selector directly from your web browser to work around the problem.
The Visual Selector tool may generate CSS selectors that become obsolete when a website updates. In certain cases, websites intentionally modify CSS selectors or add suffixes to thwart page monitoring tools like PageCrawl.
Solution: For example, a selector like .productTile_urgencyMessaging__V5DTS might include a suffix like __V5DTS that is prone to change. To avoid having to update the selector each time the website changes you may use a specialized XPath function to search if class name contains:
//*[contains(@class, 'productTile_urgencyMessaging')]Visit XPath tutorial for common selectors for more information how to create a XPath selector by yourself.
We offer three selector generation methods:
By default, you can use the CSS selector method. In some cases, generated CSS may be more effective on certain websites, while generated XPath works better on others. If you have expertise in writing CSS or XPath selectors, you have the flexibility to choose your preferred method and optimize it as necessary.
Looking to learn how to write a XPath selector yourself or explore common XPath selectors? Check out our XPath tutorial for common selectors. As a tip, you can also request ChatGPT to assist you in creating a CSS/XPath selector.
]]>False positive notifications can be frustrating when monitoring websites. These alerts signal changes that are either irrelevant or nonexistent, leading to wasted time and reduced efficiency.
When using PageCrawl to monitor website changes, the rate of false-positive alerts is typically low if pages are correctly configured. However, some detected changes may not be relevant to your specific monitoring needs. This comprehensive guide will show you how to effectively reduce unnecessary alerts and ensure you only receive notifications for meaningful changes.
Selecting the wrong type of element to monitor is one of the most common causes of false positives. With multiple monitoring options available, it's easy to get overwhelmed, especially if you're new to website monitoring.
Begin by tracking the text of the full page. This approach works best as a starting point for most monitoring scenarios, particularly when you need to monitor a large number of websites. If you notice frequent false positives, you can always revisit your setup and focus on specific page sections instead.
Monitoring Content Only is the first step to reduce false positives. This option filters out common page elements like headers, navigation menus, sidebars, and footers, focusing only on the main content area of the page. It's an effective way to eliminate noise from less relevant sections while still capturing most important content changes.
Reader mode takes content filtering a step further, similar to the reader mode you may have used on your phone. This mode monitors only the primary article text, using advanced algorithms to identify and extract the core content while filtering out everything else.
Reader mode is more restrictive than "Content Only" and works best for:
However, Reader mode may not work well on:
Note: If you find that important content changes are being missed, consider switching back to "Content Only" for broader coverage.
If tracking "Content Only" or "Reader mode" still results in unnecessary notifications, switch to the "Text" tracked element type and use our "Visual Selector" (click on the blue button) to pinpoint the exact area you want to monitor. Be aware that significant page redesigns can cause these selectors to stop working.
Advanced Tips:
Websites frequently undergo minor updates, such as date changes, without substantial alterations to their content. These small updates can create unnecessary alerts that distract from meaningful changes. Here's how to avoid them.
In Timeline, when reviewing detected changes, you can select irrelevant text and ignore any line that contains the selected text. For example, if a page has a section with a latest news headline like "Latest News: Bitcoin has reached a new all-time high," you can select "Latest News" and all lines containing this text will be ignored in future change detections. If you monitor multiple pages on the same website, this will be applied to all pages with the same domain name.
Alternatively, you can add an "Ignore Text" condition or create a global filter (update your team settings) to ignore it across all pages. Use % as a wildcard to indicate that any line containing a %specific word% or sentence should be ignored.
If a specific page area keeps triggering change detections, add a "Remove page element" action and select an area to suppress it completely.
Use the "Remove dates" action to replace dates with placeholders like [DATE REMOVED]. This prevents alerts for irrelevant updates like "updated 3 minutes ago" or publication timestamps such as "Updated at: 2025-02-25" that change frequently even when nothing was updated on the page.
You can configure a threshold to be alerted only when significant changes occur (e.g., when more than 1% of the page content changes). Before setting the threshold, review historic changes in Timeline to avoid setting it too high and missing important updates.
If numeric changes aren't relevant to you, you can add an action to ignore all numbers from triggering change detections. This is particularly useful for pages with counters, view counts, or other metrics that change frequently.
PageCrawl uses AI to analyze every detected change and help you focus on what matters most.
When a change is detected, our AI:
Use the feedback buttons to tell us which changes matter to you:
You can provide feedback:
Dynamic websites load or update parts of their content after the initial page load. For example, prices, stock availability, or user-specific recommendations might load dynamically, leading to unnecessary notifications. Here's how to handle these scenarios.
PageCrawl only captures text that is visible when in "Full-page text" mode. This can be problematic if the page contains collapsible sections (accordions, panels, etc.) that are only revealed when clicked.
To address this, add the "Reveal hidden text" action, which will automatically expand any collapsed sections on the page before capturing content.
PageCrawl waits until the page is fully loaded. However, in some situations, certain page elements only appear after additional time or after specific actions are executed (clicking, form submission, redirects, etc.).
You can add wait actions to ensure the page is completely ready before capturing content. Multiple "Wait" actions are available:
Note: Actions will wait up to 15 seconds before continuing. To avoid unnecessarily long wait times, different subscription tiers have varying timeout limits: Free (45 seconds), Standard (90 seconds), Enterprise (180 seconds). If loading takes longer than the timeout limit, the page will result in a timeout error.
Frequently updated areas like footers, headers, and sidebars can result in irrelevant notifications. These sections often include changing elements such as timestamps, menus, or recent updates that are unrelated to the main content.
header,nav,aside,footer to exclude them. This directly alters the page, and these areas will not be visible in screenshots. You may want to use this approach when using a Tracked Element other than "Full page text."main selector. If no such element exists (e.g., the website is not semantically structured), you will see a "No selector found" error.
Occasionally, a monitored page may fail to load properly, leading to blank content or error messages. While PageCrawl detects these situations in most cases, it can still trigger false positives. This often happens when a website doesn't report errors properly, relies on external data sources that fail to load, or when dynamic content is not displayed correctly.
Use the "Mark Check as Failed When" action to flag a page as failed without recording changes. For example:
Additionally, customize the "Report Errors" setting to trigger only after a certain number of consecutive failures (e.g., after 10 consecutive failed checks) to avoid being overwhelmed by temporary issues.
If you check pages frequently, ensure the "Delay when Failed" setting is deactivated (in Advanced preferences) to prevent page failures from reducing the page-checking frequency.
Websites may display varying content based on user sessions, location, or elements that frequently appear and disappear. This can lead to false positive notifications.
Once sufficient sample data is collected, PageCrawl will automatically suggest filters to reduce false triggers. Look for the "Frequently changing content detected" panel on your monitored page.
You can:
For changes that slip through, use the thumbs down button to mark them as noise.
By default, PageCrawl enables "Block cookie banners and ads" and "Hide website overlays and popups" actions to reduce unnecessary notifications. However, you can disable these settings if not needed.
Cookie banners often appear dynamically after the page loads, altering the content and triggering false positives.
Overlay popups, such as ads or newsletter subscription prompts, may appear sporadically and interfere with accurate monitoring.
These default settings simplify the monitoring process but can be adjusted based on your specific needs.
Sometimes pages use animations to reveal content sections that only appear as you scroll down the page.
By implementing these strategies, you can significantly reduce false positive notifications when monitoring websites with PageCrawl.
Quick wins for reducing false positives:
Remember:
With proper configuration and ongoing fine-tuning, you'll achieve efficient and reliable website change monitoring.
If you're still experiencing issues with false positives after trying these solutions, don't hesitate to contact our support team for personalized assistance with your specific monitoring setup.
]]>PageCrawl.io is a powerful website changes monitoring tool designed to help you keep track of all the pages within your website effortlessly. One of its standout features is the ability to crawl and automatically discover all pages within a website, much like Google's indexing process. This article will guide you through the process of utilizing PageCrawl.io to effectively track and manage all pages within your website.
Creating a template within PageCrawl.io is the initial step to enable auto-discovery for tracking all pages within a website.
Create a Template:
Provide Sample URL: Sample URL helps to automatically setup common parameters such as Base Discovery URL, filters and automatically detect sitemaps within the site.
Activate Automatic Page Discovery: Enable this feature to automatically uncover new pages as they're added to the site.
Choose Your Crawling Method:
Sitemap: Perfect if tracked site has a sitemap.xml file detailing all pages.
Scan Base URL: Start the crawl from your provided URL, letting the tool discover pages through internal links.
Deep Scan Website: Opt for an extensive exploration, ensuring maximum page coverage by following links extensively.
Automatic: Uses all available methods for page discovery.
Configuration: Fine-tune additional settings like tracked elements to monitor, update frequency, and specific directories for inclusion or exclusion.
Apply and Save: Save your template settings and apply them to the relevant projects within your PageCrawl.io account.
Wait for newly discovered pages to appear in your PageCrawl.io account.
Once your template is in place, PageCrawl.io systematically discovers and indexes all available pages within your website.
Real-Time Monitoring: Keep tabs on crawl progress through the dashboard, receiving live updates on discovered pages and any encountered issues.
Review Discovered Pages: Navigate through a detailed list of URLs sorted by categories or hierarchy within the dashboard.
Customized Monitoring: Set up tailored monitoring for specific pages or sections, configuring alerts to notify you of any modifications.
Analytical Insights: Utilize PageCrawl.io's analytical tools to gain deeper insights into page performance, traffic patterns, and content changes over time.
Optimization: Employ the insights gathered to optimize your website, refining user experience, enhancing SEO strategies, and rectifying any issues spotted during the crawl.
PageCrawl.io's automatic page discovery feature simplifies the process of monitoring all pages within a website. By following these steps, efficiently manage, monitor, and stay updated on your website's content, ensuring an informed approach to website management.
For further guidance or inquiries, consult PageCrawl.io's support resources or reach out to their customer service team.
Happy tracking!
]]>
When you visit a website for the first time, you may sometimes encounter an annoying ad or offer that overlays the content. While this is usually not a problem when monitoring websites for changes, it can still sometimes cause false-positive alerts if screenshots capture the content overlaid with the popup. These popups may only appear once, or for specific visitors or geographic locations.
To mitigate false positives, we highly recommend using the "Hide Overlays" action on affected pages. Keep in mind that this may not work on all pages.
If the "Hide Overlays" action did not work, or if all content on the page becomes invisible, you can manually target the overlay with the "Remove Elements" action to exclude it.
]]>
When monitoring web pages, you might find it useful to keep a historical HTML record for future reference. However, minor changes—such as dynamic updates to attributes, styles, or tags—can often trigger unnecessary alerts. These changes, while technically present in the HTML, might not affect the visual representation or the substantive content of the page.
Focus on Text Content: By monitoring the text content of a page rather than its HTML structure, you can significantly reduce the number of false alerts. Text content changes are more likely to represent meaningful updates to the page.
Use several Tracked Elements: In addition to text content, you can specify particular elements within the HTML that are of interest. This allows you to keep an eye on specific parts of a page without being overwhelmed by minor updates elsewhere.
To effectively manage your notifications and avoid being inundated with alerts for inconsequential changes, configure "Do not trigger notifications" filter. This filter is located within the "Conditions & Filters" section of your monitoring setup. Here's how to apply it:
By carefully adjusting your monitoring settings, you can ensure that you're alerted only to significant changes that impact the content's meaning or visual presentation. This approach helps maintain the effectiveness of your monitoring efforts without the distraction of frequent, unnecessary notifications.
]]>Yes, we support cryptocurrency payments for Ultimate plans paid annually.
To arrange payment, please contact support at support@pagecrawl.io.
]]>PageCrawl is designed to make website change monitoring and management seamless. The "Discover New Pages" feature takes your change monitoring to the next level by automatically identifying new links, tracking changes, and ensuring your online presence remains up-to-date. In this guide, we'll delve into the capabilities of this feature, including its scanning methods, automated monitoring, and filtering options.
This feature performs automated scans of your website, identifying new links that have been added. This proactive approach keeps you informed about any changes to your website's link structure and updates.
PageCrawl provides multiple scanning methods to suit your needs. All available discovery options are enabled by default (Mode: Automatic):
To start monitoring the website and automatically discover all new pages, configure a new Template which will serve as the basis for monitoring new pages.
You may choose to monitor all pages on the website or only those with a specific structure (e.g., if you only want to track product pages and not other pages).
File Checksum Monitoring detects when any online file has been modified by comparing its SHA-256 hash. Unlike text-based monitoring, this works with any file type, including zip archives, images, videos, and binary files. When a change is detected, the original file is stored so you can download and compare versions.
SHA-256 is a cryptographic hash function that produces a unique fingerprint for a file. If even a single byte changes, the hash changes completely, making it reliable for detecting modifications.
Any file accessible via URL, including: zip, rar, psd, video, audio, images, and more. Maximum file size is 15 MB. Contact support if you need to monitor larger files.
| Method | Best For | Shows Exact Changes |
|---|---|---|
| File checksum | Any file type (binary, images, archives) | No, only that the file changed |
| Text monitoring | PDF, Excel, Word, CSV, PowerPoint | Yes, line-by-line diff |
If you need to see exactly what text changed in a document, use the dedicated text monitoring for PDF, Excel, Word, CSV, or PowerPoint files instead.
PageCrawl can monitor publicly shared Google Sheets, Google Docs, and other Google Drive files for text changes. When content is added, edited, or removed, you receive a notification with a diff showing exactly what changed.
The Google file must be accessible via a shareable link. In Google Drive, set the sharing to "Anyone with the link can view" to allow PageCrawl to access the content.
| File Type | What Is Tracked |
|---|---|
| Google Sheets | Cell text content across all sheets |
| Google Docs | Full document text |
| Google Drive files | Text content (for supported formats like PDF, DOCX) |
PageCrawl can monitor Microsoft SharePoint pages and documents for text changes. When content is added, edited, or removed, you receive a notification showing what changed.
The SharePoint page or document must be accessible via a direct URL.
| Content Type | How It Works |
|---|---|
| SharePoint pages | Tracks text content changes on the page |
| Word documents | Extracts and compares text content |
| Excel files | Extracts and compares cell data |
| PDF files | Extracts and compares text content |
Monitoring text changes in PDF files can be essential for managing contracts, reports, or any important documents that may be frequently updated. Manually reviewing each document for changes can be time-consuming and prone to error. This is where PageCrawl.io comes in handy, offering an automated solution for tracking text changes in PDF files and notifying you whenever there’s an update.
PDFs are often used for official or finalized documents, which means any change can be significant. Whether it's contracts, legal documents, or product manuals, keeping an eye on text changes ensures that you're always aware of important updates. Monitoring PDF files helps with:
With PageCrawl.io, you can set up automated tracking for PDF files. It scans the text in your PDF files and alerts you whenever there’s a change, so you don’t have to sift through documents manually.
If the PDF you want to monitor does not contain readable text you can use File checksum monitoring instead to check if the PDF has been modified or changed. The downside of such approach is that you will not be able to quickly glance what exactly has changed but you will need to review page by page.
Setting up PDF monitoring is easy with PageCrawl.io. Here’s a quick guide:
Log in to your PageCrawl.io account or sign up if you’re new to the platform.
Navigate to the dashboard and click on the "Track New Page" button. Here, you can paste a link to the PDF file you want to monitor.
Customize how and when you receive notifications. You can choose to be notified immediately when text changes, or you can set up periodic checks if you want less frequent updates.
Some websites display PDF documents directly within a web page using iframes. This is common for contracts, terms of service, financial reports, and other documents that are embedded alongside regular page content.
PageCrawl automatically detects embedded iframes when you add a page for monitoring. When setting up fullpage monitoring on a page that contains iframes, you will see an "Include embedded content" checkbox. Enabling this option tells PageCrawl to extract and track text from the embedded PDF along with the rest of the page content.
This means you can monitor both the surrounding web page and the embedded PDF document in a single monitor, receiving notifications whenever either part changes.
]]>Select multiple monitored pages and change their settings in one operation. Bulk edit is available on paid plans.
| Operation | Description |
|---|---|
| Enable / Disable | Turn monitoring on or off for selected pages |
| Delete | Permanently delete selected pages and/or folders |
| Trigger check | Run an immediate check on all selected pages |
| Mark as seen | Clear the "changed" indicator on selected pages |
| Setting | Options |
|---|---|
| Check frequency | 5 min to monthly (depending on plan) |
| Engine | Default, Stealth, or Fast |
| Proxy location | London, New York, San Francisco, Toronto, Frankfurt, Residential, or Random |
| Custom proxies | Paste your own proxy list |
| Notifications | Email, Slack, Telegram, Discord, Teams, or disable |
| Notification emails | Choose which verified emails receive alerts |
| Labels | Add or remove labels |
| Folder | Move pages to a specific folder |
| Template | Apply a monitoring template |
| Screenshots | Enable or disable |
| Smart retries | Enable or disable automatic retry on failure |
| Device | Emulate a specific device viewport |
| Language | Set browser language |
| Ignored text | Add or replace text patterns to ignore |
| Full page selector | Choose between all content, main content, or article only |
| AI summaries | Enable or disable AI-powered change summaries |
| AI focus | Set custom AI instructions for what matters |
| AI tier | Basic or Pro (Pro requires Ultimate plan) |
| Cookie blocking | Add or remove cookie consent blocking |
| Overlay removal | Add or remove popup overlay hiding |
| Date exclusion | Add or remove date filtering |
| Number exclusion | Add or remove number filtering |
| Archive | Enable web archiving (Ultimate plan only) |
| Record always | Always save check results, even when no change detected |
Beyond editing, you can also add multiple pages at once:
| Method | Description |
|---|---|
| Paste URLs | Paste a list of URLs (one per line) to add them all at once |
| Upload file | Import URLs from a CSV or Excel file |
| Website scan | Scan an entire website to discover and add pages automatically |
Select pages and export their data to Excel, including current values, change history, and configuration.
PageCrawl provides three levels of organization for your monitored pages: labels for tagging, folders for grouping, and workspaces for separating entire environments.
Labels are color-coded tags you can attach to any monitored page. Each label has a name, optional description, and a color.
| Feature | Details |
|---|---|
| Colors | Each label has a hex color, auto-generated if not specified |
| Multiple labels per page | Attach as many labels as needed |
| Filtering | Filter your page list by one or more labels |
| Bulk tagging | Apply labels to multiple pages at once via Bulk Edit |
| Workspace-scoped | Labels belong to a workspace and are not shared across workspaces |
To manage labels, go to any page list and use the label filter, or manage them when editing a page.
Labels can also be applied automatically by AI. See AI Label Automation for details.
Folders let you group pages into a nested hierarchy with unlimited depth. Each folder belongs to a workspace and can contain both pages and sub-folders.
| Feature | Details |
|---|---|
| Nested hierarchy | Create sub-folders at any depth |
| Page counts | Each folder shows the total number of pages, including those in sub-folders |
| Bulk move | Move multiple pages to a folder via Bulk Edit |
| URL slugs | Each folder has a unique slug for direct navigation |
Workspaces are separate environments within your account. Each workspace has its own pages, folders, labels, notification settings, schedule, and integrations.
| Feature | Details |
|---|---|
| Separate everything | Pages, folders, labels, webhooks, and settings are workspace-scoped |
| Team access | Invite team members to specific workspaces |
| Independent settings | Each workspace has its own notification channels, schedule, AI configuration, and integrations |
| Quick switching | Switch between workspaces from the sidebar |
Use workspaces to separate monitoring by team, client, project, or environment (e.g., production vs staging).
| Action | Where |
|---|---|
| Create a folder | Click the folder icon in the page list sidebar |
| Create a label | When editing a page, or via the label filter |
| Create a workspace | Settings > Workspace > Manage Workspaces |
| Switch workspace | Sidebar workspace selector |
| Bulk assign labels/folders | Select pages > Bulk Edit |
PageCrawl renders web pages using a real browser, executing JavaScript and loading dynamic content exactly as a visitor would see it. You can choose between three engine modes depending on the page you are monitoring.
| Engine | Best For | How It Works |
|---|---|---|
| Default | Most websites | Full browser with JavaScript rendering |
| Stealth | Bot-protected pages | Enhanced mode for reliably accessing protected pages |
| Fast | Static pages, speed | Optimized for speed when JavaScript rendering is not needed |
The default engine loads pages using a real browser. It processes JavaScript, waits for dynamic content, handles cookies, and renders the page as a real user would see it. This works for the majority of websites.
Some websites use bot protection services that block automated access. Stealth mode is designed to reliably access these pages.
PageCrawl automatically switches to Stealth mode when a page is blocked (timeout, 403 Forbidden, or 401 Unauthorized). You can also enable it manually per page.
Fast mode is optimized for speed when JavaScript rendering is not needed, making it significantly faster and more resource-efficient. Use this for:
Fast mode supports Full Page, Text, HTML, Number, Price, Boolean, Availability, Links, and PDF element types. It does not support Visual comparison, screenshots, or actions (click, scroll, type).
| Scenario | Recommended Engine |
|---|---|
| Standard website | Default |
| JavaScript-heavy SPA | Default |
| Bot-protected page | Stealth |
| Page returning 403 or timeouts | Stealth |
| Static HTML page | Fast |
| API or JSON endpoint | Fast |
| Need screenshots or visual diff | Default or Stealth |
| High-frequency checks (every 5 min) | Fast (if page allows) |
Set the engine per page in the page editor under Power User settings, or apply it in bulk via Bulk Edit.
This bookmarklet is a quick tool for adding any webpage to your PageCrawl.io account in one click. By saving and clicking the bookmarklet while browsing, you’ll instantly open the PageCrawl.io "Track New Page" form with the URL and title of the current page already filled in for you.
If you often add new pages to PageCrawl.io, this bookmarklet can save you time by:
To save, simply drag the link above to your bookmarks bar, or right-click and select "Bookmark This Link."
Add to PageCrawl.io()%3B)
When you’re on a page you want to track in PageCrawl.io:
PageCrawl can generate RSS feeds for your monitored pages, allowing you to follow detected changes from any RSS reader or automation tool.
Each RSS feed has a unique URL with an access code. When a monitored page detects a change, the feed is updated with the new entry. Feeds follow the Atom format and can be consumed by any standard RSS reader.
You can create feeds for:
The feed URL contains a unique access code, so anyone with the link can view the feed without logging in. Keep feed URLs private if the monitored content is sensitive.
Add the feed URL to any RSS-compatible tool:
| Tool Type | Examples |
|---|---|
| RSS readers | Feedly, Inoreader, NewsBlur |
| Automation platforms | n8n, Zapier, Make |
| Dashboards | Custom widgets, internal portals |
| Browser extensions | RSS reader extensions for Chrome or Firefox |
| Action | How |
|---|---|
| List feeds | Go to Account Preferences > RSS Feeds |
| Create feed | Click Create Feed and select options |
| Delete feed | Click the delete button next to the feed |
PageCrawl provides a REST API and webhook system for integrating page monitoring into your own applications and workflows. Use the API to manage monitors programmatically and webhooks to receive real-time notifications when changes are detected.
Available on paid plans.
All API requests require a Bearer token. Find your API key in Settings > API.
Include it in the Authorization header:
Authorization: Bearer YOUR_API_KEY| Method | Endpoint | Description |
|---|---|---|
GET |
/api/pages |
List all monitored pages |
POST |
/api/pages |
Create a new monitored page |
GET |
/api/pages/{slug} |
Get page details and latest values |
PUT |
/api/pages/{id} |
Update page settings |
DELETE |
/api/pages/{id} |
Delete a monitored page |
PUT |
/api/pages/{id}/check |
Trigger an immediate check |
PUT |
/api/pages/{id}/status |
Enable or disable a page |
GET |
/api/pages/{id}/history |
Get check history for a page |
Additional endpoints are available for folders, tags, webhooks, and RSS feeds.
Webhooks send HTTP POST requests with a JSON body to your endpoint whenever a page change is detected or an error occurs. Configure webhooks in Settings > Workspace > Integrations > Webhooks.
| Setting | Description |
|---|---|
| Target URL | The HTTP endpoint that receives the POST request |
| Event triggers | Change detected, error, or both |
| Page filter | Limit to a specific page, or fire for all pages in the workspace |
| Payload fields | Select which fields to include (all by default) |
Available payload fields include page ID, title, change summary, diff data, screenshots, AI summary, AI priority score, and more. See the Webhook Integration guide for the full field reference and example payloads.
When monitoring pages that adjust their content based on the user's operating system, like those displaying OS-specific downloads or drivers, you might encounter challenges. Some sites perform OS detection and require interaction to display the desired information. Here's how you can effectively monitor such pages using PageCrawl.io.
There are two main ways to handle pages that require OS selection:
You can configure PageCrawl to use a specific User Agent string that mimics a Windows browser. This approach is simple and works for most basic OS detection scenarios.
How to set it up:
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.5735.199 Safari/537.36Advantages:
Limitations:
For pages with dropdown menus or forms where you need to select a specific OS version, you can use PageCrawl's Actions feature to automate the selection process.
How to set it up:
Example scenario: If a driver download page has a form with OS selection dropdown, you can:
Advantages:
Limitations:
Use the User Agent method if:
Use the Actions method if:
When monitoring changes on a webpage, the type of tracked element selected defines what kind of content will be tracked and how updates are detected. You may use multiple tracked elements for each monitored page to monitor different areas of the page. Below is a detailed breakdown of the different tracked element types:
<iframe> elements.Each tracked element type serves a unique purpose. Understanding these differences helps select the right type for specific monitoring needs, ensuring accuracy and reducing false positives. For more detailed guidance, refer to the tooltips within the interface or contact support for assistance.
]]>PageCrawl.io includes AI-powered analysis for all users. Every plan comes with monthly AI credits that work automatically with zero setup. When a page changes, AI summarizes what happened and scores how important the change is, so you only get notified about what matters.
For users who need more, you can also bring your own API key (BYOK) for unlimited AI usage and full model control.
Every plan includes monthly AI credits:
| Plan | Monthly Credits |
|---|---|
| Free | 10 |
| Standard | 100 (scales with quantity) |
| Enterprise | 1,000 (scales with quantity) |
| Ultimate | 5,000 (scales with quantity, includes Pro tier) |
Credits are based on page size. Each 4,000-token block costs 1 credit on Basic tier or 10 credits on Pro tier (Ultimate plan only). A typical blog post uses 1-2 credits. Credits reset monthly.
When credits run out, page monitoring continues normally, but AI summaries and importance filtering pause until the next billing cycle. You can also switch to BYOK at any time for unlimited usage.
No setup is required. AI features are enabled by default for all workspaces:
Workspace-specific: AI features are configured per workspace. You can have some workspaces with AI enabled and others without.
| Feature | Process |
|---|---|
| Summarization | Change detected > Content sent to AI > Human-readable summary generated > Included in notification |
| Importance Scoring | Change detected > AI analyzes content > Priority score assigned (0-100) > Low-priority changes filtered |
| Label Automation | Change detected > AI evaluates your label rules > Labels automatically added or removed |
| Setting | Description |
|---|---|
| Custom Instructions | Teach AI what matters for your monitoring (max 2,000 chars) |
| Summary Language | Generate summaries in 35+ languages |
| Notification Threshold | Set threshold (0-100) for Importance Scoring. Changes scoring below this still get tracked but do not trigger notifications. |
These settings are available when using your own API key:
| Setting | Description |
|---|---|
| Deep Analysis | Send full page content to AI for better context. Uses more tokens but provides more accurate analysis. When disabled, only the changed text (diff) is sent. |
| Run on First Check | Get AI analysis on the initial page check, before any changes are detected |
| AI Requests Per Month | Set a monthly cap to control costs. When the limit is reached, AI features pause until the next month. Leave empty for unlimited. |
| Per Page Per Day | Limit how many AI analyses a single page can trigger in 24 hours. Prevents noisy pages from consuming your entire budget. Default: 10. |
| Max Tokens | Limit content size per request. If content exceeds this limit, AI analysis is skipped for that change. |
A token is roughly 4 characters or about 3/4 of a word. With included credits, each 4,000-token block counts as 1 credit.
| Page Type | Typical Tokens |
|---|---|
| Simple (blog, article) | ~1,000-2,000 |
| Medium (product, news) | ~2,000-5,000 |
| Large (documentation) | ~5,000-10,000 |
If your included credits are not enough, or you want full control over model selection, you can connect your own API key from OpenAI, Google Gemini, Anthropic, or OpenRouter.
When using BYOK, AI credits are not consumed and you pay your AI provider directly. See the BYOK Setup Guide for detailed instructions.
| Scenario | Recommendation |
|---|---|
| Getting started | Use included credits (no setup needed) |
| High-volume pages | Enable Importance Scoring to filter noise |
| Technical pages | Enable Summarization for readable changes |
| Need unlimited AI | Connect your own API key (BYOK) |
| Critical pages | Use BYOK with premium models (GPT-4.1, Claude Sonnet) |
AI can automatically apply or remove labels on detected changes based on rules you define. Instead of manually categorizing changes, the AI reads each change and decides which labels to add or remove according to your instructions.
You can configure up to 10 label rules per workspace.
Each time a change is detected and AI analysis runs, the AI evaluates the change against your label rules and decides which labels to add or remove. The AI receives the current labels on the page, so it can remove labels that no longer apply (e.g., removing "Out of Stock" when a product is back in stock).
Labels are applied to the change record, making them available for filtering on the Review Board and in your page list.
| Label | Instruction |
|---|---|
| Breaking News | Apply when urgent or breaking news appears |
| Policy Update | Apply when terms, policies, or legal text changes |
| New Event | Apply when a new conference or event is announced |
| Job Posted | Apply when new job listings are added |
| Content Removed | Apply when significant content is deleted from the page |
| Aspect | Details |
|---|---|
| Included credits | Content is processed through PageCrawl's managed AI infrastructure |
| BYOK mode | Content is sent directly to your chosen AI provider |
| Storage | AI summaries stored in PageCrawl.io for your reference |
| Security | All transmission via HTTPS, API keys encrypted at rest |
| Provider policies | Review your AI provider's data usage and retention policies when using BYOK |
All PageCrawl.io plans include AI credits that work automatically with no setup required. This guide is for users who want to go beyond their included credits by connecting their own API key for unlimited AI usage, full model choice, and advanced features like Deep Analysis.
Most users won't need BYOK since all plans include AI credits. Consider BYOK if you:
You can switch between credits and BYOK at any time in your settings.
| Provider | Recommended Model | Best For | Get API Key |
|---|---|---|---|
| OpenAI | GPT-4o Mini | Best value for most users | platform.openai.com |
| Google Gemini | Gemini 2.5 Flash | Balance of quality and cost | ai.google.dev |
| Anthropic | Claude Haiku 4.5 | Fast and accurate | console.anthropic.com |
| OpenRouter | Any model | Access 200+ models via single API | openrouter.ai |
| Model | Use Case | Notes |
|---|---|---|
| GPT-4o Mini | Most users | Best balance of cost and quality |
| GPT-4.1 | Complex analysis | Most capable, higher cost |
| GPT-4.1 Nano | High volume | Fastest and cheapest |
| Model | Use Case | Notes |
|---|---|---|
| Gemini 2.5 Flash-Lite | Budget monitoring | Most affordable option |
| Gemini 2.5 Flash | General use | Good balance |
| Gemini 2.5 Pro | Complex tasks | Premium quality |
| Model | Use Case | Notes |
|---|---|---|
| Claude Haiku 4.5 | Most users | Fast and cost-effective |
| Claude Sonnet 4.5 | Complex tasks | Better quality, higher cost |
| Claude Opus 4.5 | Critical apps | Highest accuracy |
OpenRouter provides unified access to 200+ AI models from multiple providers through a single API key.
| Feature | Description |
|---|---|
| Unified billing | One account for all models |
| Automatic fallbacks | Switches models if one is unavailable |
| Free models | Access to Llama, Mistral, Qwen community models |
| Pricing | 5.5% platform fee on top of base model costs |
Recommended models: openai/gpt-4o-mini, anthropic/claude-haiku-4-5, google/gemini-2.5-flash
| Provider | Steps |
|---|---|
| OpenAI | Visit platform.openai.com > Create account > API Keys > Create new secret key > Add billing |
| Google Gemini | Visit ai.google.dev > Sign in with Google > Create project > Enable Gemini API > Generate API key |
| Anthropic | Visit console.anthropic.com > Create account > API Keys > Create new key > Add credits |
| OpenRouter | Visit openrouter.ai > Create account > Settings > API Key > Add credits |
Your workspace will automatically switch to BYOK mode and AI credits will no longer be consumed.
Toggle the features you want:
| Feature | Description |
|---|---|
| AI Summarization | Get intelligent summaries of page changes |
| Importance Scoring | AI scores each change from 0-100, filtering out low-priority noise |
| Custom Instructions | Add context for better analysis |
| Deep Analysis | Send full page content for better context (BYOK only) |
| Run on First Check | Get AI analysis on initial page check (BYOK only) |
If you want to stop using your own key and return to included credits:
You can customize AI settings at three levels:
| Level | Applies To | Best For |
|---|---|---|
| Workspace default | All pages | General settings |
| Template override | Pages using that template | Grouped pages (e.g., all product pages) |
| Page override | Individual pages | Critical or special pages |
Example strategy:
| Priority | Recommended Models |
|---|---|
| Cost optimization | Gemini 2.5 Flash-Lite, GPT-4o Mini |
| Accuracy | GPT-4.1, Claude Sonnet 4.5 |
| Speed | Claude Haiku 4.5, GPT-4o Mini |
| Complex content | Claude Sonnet 4.5, GPT-4.1 |
For most pages, a general-purpose model provides excellent results at a lower cost:
| Model | Provider | Best For |
|---|---|---|
| Gemini 2.5 Flash | General monitoring, good balance of speed and quality | |
| GPT-4o Mini | OpenAI | Reliable all-around performance |
For complex pages that require deeper analysis or more reasoning (e.g., dense legal documents, technical specifications, multi-section reports), choose a more powerful model:
| Model | Provider | Best For |
|---|---|---|
| Gemini 2.5 Pro | Complex documents requiring extended reasoning | |
| GPT-4.1 | OpenAI | Nuanced analysis and detailed comparisons |
| Claude Opus 4.5 | Anthropic | Critical documents requiring highest accuracy |
Tip: Start with a general-purpose model and upgrade to a more powerful one if you notice the AI missing important changes or providing superficial summaries.
| Content Type | Budget | Recommended | Premium |
|---|---|---|---|
| Blogs, News | Gemini Flash-Lite | GPT-4o Mini | - |
| E-commerce | Gemini Flash-Lite | GPT-4o Mini | Claude Haiku 4.5 |
| Legal, ToS | Claude Haiku 4.5 | Claude Sonnet 4.5 | Claude Sonnet 4.5 |
| API Docs | Gemini Flash-Lite | GPT-4o Mini | - |
Businesses rely on numerous third-party services, each with their own Terms of Service and Privacy Policy that can change at any time. These changes might affect your compliance status, operational procedures, or legal obligations. PageCrawl.io provides an automated way to track these critical documents, ensuring you're always informed when important updates occur.
This guide will show you how to set up automated monitoring for legal documents using PageCrawl.io's features.
When vendors update their terms without direct notification, it can impact your business in several ways. Payment processors might change their fee structures, cloud providers could modify data processing agreements, or analytics tools might update their data retention policies. Manual checking of these documents is time-consuming and prone to missing important updates.
Legal document monitoring typically focuses on tracking changes in Terms of Service, Privacy Policies, Data Processing Agreements, and Service Level Agreements from your vendors and partners.
The process of setting up monitoring for legal documents is straightforward and can be completed in a few minutes per page.
Start by monitoring your most critical vendor agreements first, then gradually expand to include other services. Use clear naming conventions for your monitoring tasks to easily identify which document changed when you receive an alert.
Create a structured approach to monitoring by categorizing your tracked pages. Group them into critical vendors (payment processors, infrastructure providers), data processors (analytics tools, CRM systems), and regulatory pages (government compliance guidelines).
Implement a tagging system from the start. Use tags like #vendor, #competitor, #gdpr, or #payment to quickly filter and manage your monitored pages. This becomes especially useful as your monitoring portfolio grows.
Not all changes are equal. Some updates might be minor formatting adjustments, while others could be significant legal modifications. PageCrawl.io helps you distinguish between these by highlighting exactly what changed, showing removed text in red and new text in green.
For each detected change, PageCrawl.io stores:
This comprehensive record ensures you have all the evidence needed for compliance audits and legal reviews.
If you're receiving too many alerts about minor changes, check that Reader mode is enabled to filter out navigation and footer updates. For more strategies on reducing false positives, see our guide on reducing false positives when monitoring websites.
If you're missing important changes, verify that the correct URL is being monitored and that the page is accessible.
Once you've set up basic monitoring, consider implementing advanced strategies such as keyword-based alerts for critical terms like "price increase" or "data breach", or comparison monitoring to track how your policies compare to competitors.
Begin with your most important vendor agreements. Setup takes just a minute or two per page, or you can save time by importing multiple URLs at once - simply copy and paste a list of URLs or upload an Excel file for bulk import.
PageCrawl.io handles the monitoring automatically once configured. You'll receive clear notifications when changes occur, allowing you to review and respond promptly to maintain compliance.
For businesses monitoring multiple vendors, check our pricing page - monitoring 500 URLs costs just $30/month, making enterprise-wide compliance monitoring affordable and efficient.
]]>
You can track numeric values on a page using the "Number" tracked element type. This extracts numbers from a selected area on the page and displays them in a chart so you can quickly see the history of values and spot trends. Instead of manually checking a number every day, PageCrawl monitors it for you and builds a visual record over time.
Common things to monitor with the Number tracker:
The visual selector is the easiest way - just point and click on the number you want to track. PageCrawl will figure out the selector for you automatically.
If you prefer to manually write selectors by analyzing HTML source, here are some examples:
For a price like this:
<span class="price">$49.99</span>Use: //span[@class="price"] or .price
For an inventory count:
<div class="stock">150 items available</div>Use: //div[@class="stock"] or div.stock
For a specific ID:
<p id="total-views">2,543 views</p>Use: //p[@id="total-views"] or #total-views
For a rating or score:
<span class="rating">4.5</span>Use: //span[@class="rating"] or .rating
Once you've set up your number tracker, PageCrawl will:
The chart displays your complete history, making it easy to spot patterns and see how values change over different time periods. You can see exactly when changes happened and track the progression of any number over days, weeks, or months.
Your number tracking chart shows:
This gives you a clear visual picture of what's happening with the metric you're tracking.
PageCrawl displays comprehensive statistics about your tracked number:
These statistics are color-coded:
This helps you quickly understand the overall behavior of your metric without manually analyzing the chart.
The chart visualization includes powerful interactive features to help you analyze your data:
Date Range Filters:
Chart Controls:
Visual Annotations:
Legend:
Tooltips: When you hover over any point on the chart, you'll see:
Performance Optimizations: For long tracking periods with thousands of data points:
These features make it easy to spot trends, identify when significant changes occurred, and understand your data at a glance.
If you need to monitor the same numeric value across multiple pages on a website, you can:
This saves you time and makes it easy to track metrics across your entire site.
If you're tracking the same type of number across different pages (for example, the price of a product on multiple retailers), you can overlay them all on a single chart to compare side by side.
How to set it up:
PageCrawl will suggest relevant monitors automatically, prioritizing monitors in the same folder, on the same domain, or tracking similar products.
What the combined chart shows:
Reading the comparison:
This is especially useful for:
Your comparison selections are saved, so the next time you open the monitor the same comparison lines will appear on the chart.
E-commerce Store: Track product prices across listings. When prices drop or go on sale, you'll see it immediately in the chart. Compare pricing across multiple product pages to spot trends.
Real Estate Pricing: Track property prices on listing sites. Monitor how prices change over time, identify when properties go on sale, or track pricing trends in your area of interest.
Competitor Pricing: Monitor competitor product prices, discount percentages, or pricing changes. The chart gives you a clear view of when they adjust their prices.
Job Postings: Track how many open positions a company has posted. The chart shows when they're actively hiring and when positions get filled.
Education Programs: Monitor tuition costs, enrollment numbers for programs, or available spots in courses. Track how these metrics change throughout the year.
Government Fees & Services: Monitor permit costs, license fees, visa application prices, or other government service charges that may be subject to change.
Stock Price Monitoring: Monitor the current price of a stock or cryptocurrency. The chart shows you exactly when the price changed and by how much.
]]>This guide covers the PageCrawl side of SSO setup: importing your identity provider's metadata, enabling SSO, configuring enforcement and user provisioning. For step-by-step instructions on configuring your identity provider (Azure AD, Google Workspace, Okta, etc.), see the Identity Provider Setup Guide.
Single Sign-On (SSO) allows your team members to securely access PageCrawl using your organization's identity provider, such as Azure AD, Google Workspace, Okta, or OneLogin.
To use SAML SSO, your team must meet the following requirements:
Navigate to Settings → Team → Auth & SSO in your PageCrawl account. You must be a team administrator to access these settings.
When you first access the SSO settings page, PageCrawl automatically generates a unique identifier (UUID) and creates an initial SSO configuration for your team. This UUID is immediately available and used to create your Entity ID and Metadata URL.
Before configuring your Identity Provider, copy the Metadata URL displayed in the blue information box at the top of the SSO settings page.
The URL will look like: https://pagecrawl.io/sso/saml/abc-123-def-456/metadata
Important: Copy the actual URL shown in PageCrawl, not this example.
Most Identity Providers can automatically import all necessary configuration (Entity ID, ACS URL, Logout URL, etc.) from this metadata URL.
Note: If your IdP requires manual entry, the individual URLs are also displayed in the same box:
Follow the instructions in our Identity Provider Setup Guide for your specific IdP (Azure AD, Google Workspace, Okta, etc.).
You'll need to create a SAML application in your IdP and provide the ACS URL and Entity ID from step 2.
You have three options to configure your IdP:
Option A: Metadata URL (Recommended)
Option B: Metadata XML
Option C: Manual Entry
Configure the following settings based on your needs:
Turn on SAML authentication for your domain.
When enabled, password login will be disabled for users with your email domain. Users must authenticate via your identity provider.
Enable Automatic Account Creation
When JIT provisioning is enabled, you can configure:
Default Role for New SSO Users
Default Workspaces
Auto-Create Personal Workspace
Personal workspace creation depends on your subscription plan:
If you enable "Auto-Create Personal Workspace" and have reached your limit, new SSO users will be assigned to default workspaces instead of creating personal workspaces.
Once configured, users with your email domain will:
If JIT provisioning is enabled and they're a new user, an account will be created automatically with the configured role and workspace assignments.
Error: "Unable to provision SSO user: Team has reached its member limit."
Solution:
Error: "Automatic account creation is disabled. Please ask your team administrator to enable JIT provisioning."
Solution:
Symptoms: User gets error after authenticating at IdP.
Solution:
Symptoms: "Invalid signature" or authentication fails at final step.
Solution:
Common Issues:
EntityDescriptor format, not EntitiesDescriptor<?xml declarationCause: Team has reached workspace limit for subscription plan.
Solution:
Q: Can I have multiple identity providers? A: No, PageCrawl supports one identity provider per team.
Q: What happens to existing users when I enable SSO? A: Existing users can continue using password login unless you enable "Enforce SSO". With JIT provisioning enabled, their accounts will be automatically linked to SSO on first SSO login.
Q: Can I disable SSO after enabling it? A: Yes, you can disable SSO anytime in the settings. Users will revert to password-based login.
Q: What if my IdP certificate expires? A: Users won't be able to log in until you update the certificate. Update metadata in PageCrawl SSO settings as soon as your IdP rotates certificates.
Q: Why can't I use Gmail or other free email providers? A: SSO requires corporate email domains for security. Free email providers don't provide the organizational control needed for enterprise SSO.
Q: How do I migrate all users to SSO? A: Enable SSO with JIT provisioning first. Test with a few users. Once confirmed working, enable "Enforce SSO" to require all users to use SSO.
Q: What happens if we reach our member or workspace limit? A: New SSO users won't be able to log in if member limit is reached. If workspace limit is reached, personal workspaces won't be created, but users will still be assigned to default workspaces.
For assistance with SSO configuration or to request early access, contact support@pagecrawl.io.
]]>This guide covers the identity provider (IdP) side of SSO setup with step-by-step instructions for Azure AD, Google Workspace, Okta, OneLogin, and custom SAML providers. For PageCrawl-side settings (enabling SSO, enforcement, JIT provisioning), see the SSO Configuration Guide.
Before you begin, ensure you have:
IMPORTANT: Complete this step first before configuring your Identity Provider
Navigate to Settings → Team → Auth & SSO in PageCrawl
Copy the Metadata URL shown in the blue Service Provider information box
https://pagecrawl.io/sso/saml/abc-123-def-456/metadataKeep this URL handy - most Identity Providers can automatically import all configuration from this metadata URL
Note: If your IdP doesn't support metadata import, copy the individual URLs from PageCrawl (they will also be shown in the same box):
Additional information for reference:
urn:oasis:names:tc:SAML:1.1:nameid-format:emailAddress)https://pagecrawl.io/sso/saml/abc-123.../metadata)https://pagecrawl.io/sso/saml/abc-123.../acs)The default Name ID (user.mail) is sufficient. No additional changes needed.
https://pagecrawl.io/sso/saml/abc-123.../acs)https://pagecrawl.io/sso/saml/abc-123.../metadata)https://pagecrawl.io/sso/saml/abc-123.../acs)https://pagecrawl.io/sso/saml/abc-123.../metadata)https://pagecrawl.io/sso/saml/abc-123.../metadata)https://pagecrawl.io/sso/saml/abc-123.../acs)https://pagecrawl\.io/sso/saml/[^/]+/acshttps://pagecrawl.io/sso/saml/abc-123.../acs)If your identity provider isn't listed above but supports SAML 2.0, you can configure it manually:
In your IdP, create a new SAML application with these settings:
https://pagecrawl.io/sso/saml/abc-123.../metadata)https://pagecrawl.io/sso/saml/abc-123.../acs)From your identity provider, collect:
After configuration, test your SSO:
If you encounter issues, check:
EntitiesDescriptor element. Use EntityDescriptor format.For assistance with your specific identity provider, contact support@pagecrawl.io.
]]>Every PageCrawl.io plan includes AI credits that work automatically with no setup. For most users, the included credits are all you need. This guide is primarily for users who want to bring their own API key (BYOK) and choose a specific model, covering free options to premium models with cost comparisons based on 2026 pricing.
Using included credits? You don't need to choose a model. PageCrawl automatically uses optimized models on your behalf. Each 4,000-token block costs 1 credit (Basic tier) or 10 credits (Pro tier, Ultimate plan only). See AI-Powered Change Detection for details on how credits work.
Pricing updates frequently. Verify current rates at: OpenAI, Gemini, Anthropic, OpenRouter
AI models enhance website monitoring by automatically summarizing changes, assigning priority scores, and distinguishing meaningful updates from noise.
PageCrawl.io supports four AI providers:
A token is roughly 4 characters or about 3/4 of a word. AI providers charge based on tokens processed:
PageCrawl's AI costs are dominated by input tokens (the page content sent to AI). Output tokens are minimal because summaries are typically just 1-2 paragraphs (~100-200 tokens).
Typical token usage per check:
Example cost calculation (Gemini 2.5 Flash at $0.30/M input):
Example cost calculation (Claude Opus 4.5 at $5.00/M input):
Since output is just a short summary (~150 tokens), output costs add less than 10% to the total. Additionally, AI only runs when a meaningful change is detected on the page. PageCrawl's advanced change detection infrastructure filters out tiny, insignificant changes before they ever reach AI, so you only spend tokens on changes that actually matter.
Prices per million tokens. Most of your cost will be input tokens, as output (summaries) is minimal.
| Model | Provider | Input | Output | Best For |
|---|---|---|---|---|
| Gemini 2.5 Flash-Lite | $0.10/M | $0.40/M | High volume, budget option | |
| GPT-4o Mini | OpenAI | $0.15/M | $0.60/M | Budget OpenAI option |
| Gemini 2.5 Flash | $0.30/M | $2.50/M | Good balance of cost/quality |
| Model | Provider | Input | Output | Best For |
|---|---|---|---|---|
| Claude Haiku 4.5 | Anthropic | $1.00/M | $5.00/M | Quality analysis |
| Gemini 2.5 Pro | $1.25/M | $5.00/M | Google's best | |
| GPT-4.1 | OpenAI | $2.00/M | $8.00/M | Complex pages |
| Claude Sonnet 4.5 | Anthropic | $3.00/M | $15.00/M | Premium accuracy |
| Claude Opus 4.5 | Anthropic | $5.00/M | $25.00/M | Maximum capability |
Privacy Note: Some providers may use data for training. Review each provider's data usage policy for sensitive content. See Privacy section for details.
PageCrawl.io only calls AI when a page actually changes. If you monitor 1,000 pages and only 150 change, you pay for 150 AI requests, not 1,000.
Pricing: $0.15/$0.60 per million tokens (input/output) Recommended For: Most BYOK users
GPT-4o Mini offers the best balance of cost and performance. It's reliable, fast, and very affordable.
Pricing: $0.10/$0.40 per million tokens Recommended For: High-volume, cost-sensitive monitoring
Note: Google Gemini's free tier was reduced significantly in 2026 (from 250-1000 requests/day to only 5-20 requests/day). For practical use, billing is recommended.
GPT-4.1: $2.00/$8.00 per million tokens Claude Sonnet 4.5: $3.00/$15.00 per million tokens Recommended For: Legal documents, terms of service, compliance monitoring, and other critical content where accuracy matters most
| Content Type | Budget Option | Recommended | Premium |
|---|---|---|---|
| Blogs, News, Docs | Gemini Flash-Lite | GPT-4o Mini | - |
| E-commerce, Pricing | Gemini Flash-Lite | GPT-4o Mini | Claude Haiku 4.5 |
| Legal, ToS, Compliance | Claude Haiku 4.5 | Claude Sonnet 4.5 | Claude Sonnet 4.5 |
| Competitor Monitoring | Gemini Flash-Lite | GPT-4o Mini | Claude Haiku 4.5 |
| API Docs, Changelogs | Gemini Flash-Lite | GPT-4o Mini | - |
Costs can vary significantly. These are estimates only. Your actual costs depend on:
Token usage by page type:
Recommendation: Start with budget-friendly models like Gemini Flash-Lite and set strict monthly limits to avoid unexpected bills.
| Model | ~500 tok | ~2K tok | ~5K tok | ~10K tok |
|---|---|---|---|---|
| Gemini Flash-Lite | $0.05 | $0.21 | $0.53 | $1.06 |
| GPT-4o Mini | $0.08 | $0.32 | $0.80 | $1.59 |
| Gemini Flash | $0.15 | $0.62 | $1.55 | $3.10 |
| Claude Haiku 4.5 | $0.50 | $2.00 | $5.00 | $10.00 |
| Gemini Pro | $0.63 | $2.50 | $6.25 | $12.50 |
| Model | Light (~500) | Average (~2K) | Heavy (~5K) | Very Heavy (~10K) |
|---|---|---|---|---|
| Gemini Flash-Lite | $0.12 | $0.47 | $1.19 | $2.39 |
| GPT-4o Mini | $0.18 | $0.72 | $1.80 | $3.58 |
| Gemini Flash | $0.34 | $1.40 | $3.49 | $6.98 |
| Claude Haiku 4.5 | $1.13 | $4.50 | $11.25 | $22.50 |
You can reduce token usage in PageCrawl.io settings:
Tip: Check your actual token usage in PageCrawl.io's AI statistics to estimate your costs accurately.
OpenRouter provides unified access to AI models from multiple providers through a single API key.
Benefits: Unified billing, automatic fallbacks, access to free models (Llama, Mistral, Qwen)
Pricing: 5.5% platform fee on top of base model costs
Best for: Experimenting with multiple models, accessing free community models
Recommended models: openai/gpt-4o-mini, anthropic/claude-haiku-4-5, google/gemini-2.5-flash
| Provider | Get Key At |
|---|---|
| OpenAI | platform.openai.com > API Keys |
| Google Gemini | ai.google.dev > Get API Key |
| Anthropic | console.anthropic.com > API Keys |
| OpenRouter | openrouter.ai > Settings > API Key |
You can customize AI models at three levels:
Example strategy:
| Provider | Data Usage | Best For |
|---|---|---|
| OpenAI/Anthropic | API data not used for training | Confidential content, legal docs |
| Google Gemini | Review Google's data policies | General monitoring |
| OpenRouter | Varies by underlying model | Check each model's policy |
When using included AI credits, content is processed through PageCrawl's managed AI infrastructure. When using BYOK, content is sent directly to your chosen provider.
Data protection policies: OpenAI, Anthropic, Google
Privacy note: Free tier models (including some OpenRouter models) may use your data for training. Use paid tiers for sensitive content.
Do I need BYOK to use AI? No. All plans include AI credits that work automatically. BYOK is optional for users who want unlimited usage or specific model control.
What happens when my credits run out? Page monitoring continues normally, but AI summaries pause until credits reset next month. You can also switch to BYOK for unlimited usage.
Can I switch between credits and BYOK? Yes, at any time in Settings > Workspace > Integrations > AI.
Can I switch models after starting? Yes. Changes apply immediately to new checks. Historical data remains intact.
Do I pay for checks that don't find changes? No. AI only runs when pages actually change.
Can I use different models for different pages? Yes, via workspace defaults, template overrides, and page-level overrides.
The PageCrawl browser extension lets you instantly add any webpage to your monitoring list with just a few clicks. View recent changes, switch between workspaces, and start monitoring new pages - all without leaving your current tab.
The PageCrawl extension is available for:
Note: Click the pin icon next to PageCrawl to keep it visible in your browser toolbar for quick access.
After installing the extension, click the PageCrawl icon in your browser toolbar. You'll see a welcome screen prompting you to log in.
If you don't have an account yet, click "Don't have an account? Sign up" to create one.
Once connected, the extension opens to your Recent Changes timeline. This shows the latest detected changes across all your monitored pages:
Click any change to open it directly in your dashboard and see the full details.
To add a new page to your monitoring:
Best for: Blog posts, news articles, documentation pages
Monitors text content on the page. Choose your tracking level:
Keyword Monitoring: Optionally enter keywords (comma-separated) to only be notified when specific words appear or disappear. Leave empty to be notified of all changes.
Best for: Prices, stock status, specific data points
You can also manually enter a CSS selector if you prefer.
Track as Number: Enable this to extract numeric values from the element. This allows you to track trends and percentage changes over time.
Keyword Monitoring: Same as Full Page - enter keywords to filter notifications.
Best for: Charts, images, layouts, design changes
The extension will capture screenshots of this area and compare them for changes.
Change Threshold: Set how much the area must change before you're notified:
Best for: Product pages, e-commerce sites
PageCrawl will automatically detect and track the main price on the page. This is optimized for common e-commerce platforms and product pages.
Choose how often PageCrawl should check for changes:
You can quickly access PageCrawl from any webpage using the right-click context menu:
What happens next depends on whether the page is already monitored:
If the page is already monitored: You'll be taken directly to the page's dashboard where you can view change history, adjust settings, or check the current status.
If the page is not monitored: You'll be taken to the page creation form with the URL pre-filled, ready to set up monitoring.
The extension header provides quick access to:
For advanced configuration (notifications, proxies, actions, etc.), click "More options →" below the Start Monitoring button. This opens the full page creation form on PageCrawl with your current settings pre-filled.
]]>Add any webpage to PageCrawl.io monitoring directly from Safari's Share Sheet on your iPhone or iPad. Just tap Share, tap the shortcut, and you're done.
Tap the button below on your iPhone or iPad to install the "Add to PageCrawl" shortcut:
When prompted, tap Get Shortcut to install it.
This shortcut also works on macOS! In Safari on your Mac:
Alternatively, for desktop browsers you can use our bookmarklet — just drag it to your bookmarks bar for one-click access.
Android doesn't have a Shortcuts app, but you can use our bookmarklet in any mobile browser. Add it to your bookmarks, then tap it when viewing a page you want to monitor.
We aim to respond to your inquiries promptly but sometimes due to increased number of support requests, Enterprise and Ultimate customer requests/emails are prioritized over Standard customers. Therefore, the response time is faster, and you may expect a 'higher level' of support in case you are not able to set up the page the way you want.
For technical support our response times are prioritized according to your subscription plan:
PageCrawl.io provides dedicated n8n community nodes that integrate directly into your n8n instance. With the PageCrawl Trigger and PageCrawl nodes, you can trigger workflows when changes are detected and interact with the PageCrawl.io API to manage pages, retrieve diffs, and download screenshots, all from within n8n's visual workflow editor.
n8n is a workflow automation tool that you can self-host or run in the cloud. By connecting PageCrawl.io to n8n, you can:
PageCrawl.io provides two n8n nodes:
The trigger node starts your workflow automatically when something happens on a monitored page. Supported events:
You can filter triggers by workspace and by specific page, or listen for changes across all pages in a workspace. The node automatically registers and cleans up webhooks with the PageCrawl.io API.
The action node lets you interact with the PageCrawl.io API within your workflows. Available resources and operations:
Page operations
Check operations
Screenshot operations
@pagecrawl/n8n-nodes-pagecrawl as the package name.With the trigger in place, add any n8n nodes to define what happens when a change is detected. Some examples:
You can also add the PageCrawl action node mid-workflow to fetch additional data, such as downloading a diff image to attach to a notification or retrieving the full page details.
Once your workflow is tested and working, activate it so it runs automatically whenever changes are detected.
PageCrawl uses role-based access control to manage what each team member can do. There are four roles, each with different permission levels.
| Role | Manage Team | Manage Workspaces | Edit Pages | View Pages |
|---|---|---|---|---|
| Owner | Yes | Yes | Yes | Yes |
| Administrator | Yes | Yes | Yes | Yes |
| Standard User | No | No | Yes | Yes |
| Viewer | No | No | No | Yes |
Each team has exactly one Owner (the account creator). The Owner has full control over all team settings, billing, and member management. Ownership cannot be transferred or removed.
Administrators can manage the team on behalf of the Owner:
Standard Users can work within their assigned workspaces:
Standard Users cannot invite members, change roles, or access workspaces they haven't been assigned to.
Viewers have read-only access to their assigned workspaces:
To manage roles and access:
Members only see workspaces they've been assigned to. Administrators can assign workspace access per user. If all workspace access is removed from a user, they are removed from the team entirely.
This means you can have team members who only see specific projects, clients, or departments without exposure to other workspaces.
]]>PageCrawl offers advanced configuration options for users who need fine-grained control over their monitoring setup. This guide covers the key power-user features.
When editing a monitored page, you can enable Power User mode using the toggle in the page settings. This reveals additional settings that are hidden by default to keep the interface clean for everyday use.
With Power User mode enabled, you get access to:
Power User settings are marked with a special icon throughout the edit form so you can easily identify them.
PageCrawl offers two ways to add and edit monitored pages:
Simple Mode (default) guides you through setup step by step. It auto-detects the best settings, shows a live preview, and covers the most common use cases. Best for getting started quickly.
Advanced Mode gives you full control over every setting in a single form. Use it when you need to:
You can switch to Advanced Mode from the Simple Mode page by clicking the "Advanced setup" link at the bottom. If you prefer to always use Advanced Mode, check the "Always show Advanced Setup" option.
Each monitored page can track multiple elements simultaneously, each with its own comparison method:
| Type | What It Tracks |
|---|---|
| Full Page | Entire page text content |
| Text | Text content of a specific element (by CSS/XPath selector) |
| Number | Numeric values with configurable change thresholds |
| Price | Price values with currency detection |
| Availability | In-stock/out-of-stock status |
| Links | All outgoing links on the page |
| Visual | Visual screenshot comparison with diff percentage |
| HTML | Raw HTML structure of an element |
| Boolean | Presence or absence of an element |
| JSON | JSON response content with path extraction |
Each tracked element can have its own set of actions and comparison settings.
Templates let you save a monitoring configuration and apply it to multiple pages automatically. This is especially useful when combined with Page Discovery for auto-monitoring newly discovered pages.
To create a template:
Templates can also define URL filters for page discovery, so new pages matching your criteria are automatically monitored with the template's settings.
Edit settings across multiple pages at once:
Available on paid plans.
Configure AI-powered change analysis per workspace:
Each plan includes monthly AI credits. You can also bring your own API key (BYOK) for unlimited usage. See AI BYOK Setup for details.
Control exactly when PageCrawl checks your pages:
This helps reduce unnecessary checks during off-hours and keeps your check quota focused on the times that matter.
Apply text filters across all pages in a workspace:
Useful for filtering out dynamic content like timestamps, ad copy, or session IDs that appear across many pages.
Choose where PageCrawl checks your pages from:
Configure per page or apply via bulk edit.
]]>PageCrawl lets you use JavaScript in two powerful ways: as a tracked element to extract and monitor computed values, and as a custom action to manipulate the page before monitoring. Both run JavaScript directly in the browser context with full access to the DOM.
A JavaScript tracked element lets you execute JavaScript code on a page and monitor the return value for changes. This is useful when the data you want to track is not directly accessible via CSS or XPath selectors, for example computed values, data attributes, or content that requires logic to extract.
How to set it up:
How it works: Your JavaScript code runs directly in the browser, giving it full access to the page's DOM, window object, and all standard browser APIs. The return value is captured and compared against the previous check to detect changes.
Examples:
Extract the page title:
document.titleGet text from a specific element:
document.querySelector('.status-badge').innerTextCount the number of items in a list:
document.querySelectorAll('.job-listing').lengthExtract a data attribute:
document.querySelector('[data-version]').getAttribute('data-version')Combine multiple values into one:
Array.from(document.querySelectorAll('.feature-list li')).map(el => el.textContent.trim()).join(', ')Extract JSON-LD structured data:
JSON.parse(document.querySelector('script[type="application/ld+json"]').textContent).nameCount words on a page:
document.body.innerText.split(/\s+/).filter(w => w.length > 0).lengthFor multi-line logic, wrap your code in an immediately invoked function:
Extract a software version number from a release page:
(() => {
const text = document.querySelector('.release-header, [class*="version"]')?.textContent || '';
const match = text.match(/v?(\d+\.\d+\.\d+)/);
return match ? match[1] : 'Version not found';
})()Build a summary from a table:
(() => {
const rows = document.querySelectorAll('table tbody tr');
return Array.from(rows).map(row => {
const cells = row.querySelectorAll('td');
return Array.from(cells).map(c => c.textContent.trim()).join(' | ');
}).join('\n');
})()Count job listings by department:
(() => {
const jobs = document.querySelectorAll('.job-listing');
const departments = {};
jobs.forEach(job => {
const dept = job.querySelector('.department')?.textContent.trim() || 'Other';
departments[dept] = (departments[dept] || 0) + 1;
});
return Object.entries(departments).map(([k, v]) => `${k}: ${v}`).join('\n');
})()Extract all outbound links from a page:
(() => {
const host = window.location.hostname;
const links = Array.from(document.querySelectorAll('a[href]'))
.map(a => a.href)
.filter(href => href.startsWith('http') && !href.includes(host));
return [...new Set(links)].join('\n');
})()Monitor the number of open issues or pull requests:
(() => {
const text = document.querySelector('[data-tab-item="issues"] .Counter, .issues-count')?.textContent.trim();
return text ? parseInt(text.replace(/,/g, ''), 10) : 'Not found';
})()Extract and format event dates from a schedule page:
(() => {
const events = document.querySelectorAll('.event-item, .schedule-row');
return Array.from(events).map(ev => {
const date = ev.querySelector('.date, time')?.textContent.trim();
const title = ev.querySelector('.title, .event-name')?.textContent.trim();
return `${date}: ${title}`;
}).join('\n');
})()Important notes:
null or undefined, an empty string is storedCustom JavaScript actions let you run JavaScript code on the page as part of the action sequence, before the tracked elements are extracted. Use them for complex interactions that other action types (click, type, wait) cannot handle.
How to set it up:
How it works: The JavaScript runs in the browser context, similar to tracked elements. The key difference is that the return value is ignored. JavaScript actions are used for their side effects: modifying the DOM, triggering events, or setting up the page state needed for accurate monitoring.
When to use JavaScript actions: PageCrawl has built-in actions for common tasks like clicking elements, typing text, scrolling, waiting, removing elements, and selecting dropdown options. Use JavaScript actions when you need to do something the built-in actions cannot handle, such as setting browser storage, dispatching custom events, modifying element properties, or running multi-step DOM manipulation.
Examples:
Set localStorage or sessionStorage to change page behavior:
localStorage.setItem('region', 'us-east')Set a cookie to bypass a language selector or A/B test:
document.cookie = 'lang=en; path=/; max-age=86400'Replace dynamic content (session IDs, timestamps, random tokens) with static text to reduce false positives:
document.querySelectorAll('[data-session-id], .csrf-token, .nonce').forEach(el => el.textContent = '[REDACTED]')Trigger a framework event that a regular click action does not fire (e.g., React, Vue, Angular):
(() => {
const input = document.querySelector('#search-input');
const nativeInputValueSetter = Object.getOwnPropertyDescriptor(window.HTMLInputElement.prototype, 'value').set;
nativeInputValueSetter.call(input, 'monitoring keywords');
input.dispatchEvent(new Event('input', { bubbles: true }));
})()Toggle a checkbox and dispatch both change and click events to satisfy form validation:
(() => {
const checkbox = document.querySelector('#agree-terms');
checkbox.checked = true;
checkbox.dispatchEvent(new Event('change', { bubbles: true }));
checkbox.dispatchEvent(new Event('click', { bubbles: true }));
})()Switch a page to a specific view mode by modifying URL parameters without a full reload:
(() => {
const url = new URL(window.location);
url.searchParams.set('view', 'list');
url.searchParams.set('per_page', '100');
window.history.replaceState({}, '', url);
window.dispatchEvent(new PopStateEvent('popstate'));
})()Expand all collapsed sections at once on a FAQ or documentation page:
document.querySelectorAll('details:not([open])').forEach(el => el.setAttribute('open', ''))Remove inline styles that hide content behind a paywall or login wall:
(() => {
document.querySelectorAll('.article-body, .content-area').forEach(el => {
el.style.maxHeight = 'none';
el.style.overflow = 'visible';
el.classList.remove('truncated', 'blurred', 'paywall');
});
document.querySelectorAll('.paywall-overlay, .signup-gate').forEach(el => el.remove());
})()Important notes:
| JavaScript Tracked Element | Custom JavaScript Action | |
|---|---|---|
| Purpose | Extract and monitor a value | Manipulate the page before extraction |
| Return value | Captured and tracked for changes | Ignored |
| Error handling | Check fails if code errors | Errors silently ignored, check continues |
| When it runs | During element extraction | Before element extraction (in action sequence) |
| Use case | "Get me this computed value" | "Set up the page so I can monitor it correctly" |
Extract then monitor: Use a JavaScript action to set up the page (e.g., click "Load more"), then use a regular Text or Full Page tracked element to capture the content. This is often simpler than writing a JavaScript tracked element.
Normalize before compare: Use a JavaScript action to replace dynamic content (timestamps, session IDs, random values) with static placeholders, then track the normalized page content. This reduces false positives without needing global filters.
Complex extraction: When the value you want to monitor requires logic (math, filtering, combining multiple elements), use a JavaScript tracked element instead of trying to target it with CSS selectors.
Your code runs in the browser page context with full access to:
document.querySelector(), document.body, document.title, etc.window.location, window.innerWidth, window.scrollTo(), etc.localStorage, sessionStorage, fetch(), etc.Your code does not have access to Node.js APIs or the file system.
]]>PageCrawl lets you track multiple parts of the same page independently. Each tracked element gets its own comparison method, selector, label, and threshold, so you can monitor different sections of a page with the settings that make the most sense for each one.
Different parts of a page often change in different ways. For example, on a product page you might want to:
Each of these uses a dedicated element type designed for that kind of data, giving you more precise alerts and fewer false positives than tracking the entire page as a single unit.
Each tracked element can use one of these comparison types:
| Type | Description |
|---|---|
| Full Page | Tracks the entire visible page content |
| Text | Extracts and compares text content from a CSS/XPath selector |
| Number | Extracts a numeric value for threshold-based comparison |
| Price | Specialized number extraction that handles currency symbols and formatting |
| Availability | Detects in-stock/out-of-stock status from common patterns |
| Visual | Compares screenshots of a specific element for visual changes |
| HTML | Compares the raw HTML of a selected element |
| Boolean | Checks whether an element exists or is visible on the page |
| Links | Extracts and compares all links within a selected area |
| JavaScript | Evaluates a custom JavaScript expression and tracks the return value |
| Text (All Matches) | Extracts text from all elements matching a selector |
| Text (All Matches Sorted) | Same as above, but sorted alphabetically for order-independent comparison |
| HTML (All Matches) | Extracts HTML from all elements matching a selector |
Once you add more than one tracked element, the monitor stays in Advanced Mode. To return to Simple Mode, remove the extra elements first so only one remains.
Each tracked element has its own independent settings:
You do not need to write CSS selectors or XPath expressions manually. Use the visual selector tool to click on elements directly on the page. PageCrawl generates the appropriate selector for you automatically.
Product page monitoring - Use the Price element type for the product price, the Availability element type for stock status, and a Text element for the product description. Each triggers its own alert so you know exactly what changed.
Content sections and sidebar tracking - Monitor the main article content as text and the sidebar navigation as HTML. Catch content updates without being distracted by layout changes.
Multi-section compliance monitoring - Track terms of service, privacy policy sections, and legal disclaimers as separate elements on the same page. Each section triggers its own alert when updated.
Actions are tasks that PageCrawl executes in the browser before taking a page snapshot. They let you automate interactions like dismissing cookie banners, clicking tabs, logging in, scrolling to load content, or waiting for dynamic elements to appear.
Actions are configured per tracked element and execute in order from top to bottom.
Open any monitored page and click Edit. In the page configuration form, find the Actions section. Click Add Action to add a new action, then select the action type from the dropdown.
| Action | What It Does |
|---|---|
| Mark as failed | Mark the check as failed when conditions are met (page inaccessible, contains specific text, etc.) |
| Action | What It Does |
|---|---|
| Remove cookies | Automatically hide cookie consent banners |
| Remove overlays | Hide website overlays and popups |
| Remove dates | Replace dates with "[DATE REMOVED]" to prevent false positives |
| Remove element | Remove a specific element by CSS or XPath selector |
| Remove text | Remove elements containing specific text |
| Action | What It Does |
|---|---|
| Wait for text | Wait up to 15 seconds for specific text to appear on the page |
| Wait for text to disappear | Wait up to 15 seconds for specific text to disappear |
| Wait for element | Wait for an element (by XPath or CSS selector) to appear |
| Wait for redirect | Wait for the page to redirect to a new URL |
| Wait | Pause for a specified number of seconds |
| Action | What It Does |
|---|---|
| Click button | Click a button element |
| Click element | Click any element by selector |
| Click at coordinates | Click at specific X/Y pixel coordinates |
| Hover | Hover over an element |
| Type text | Type text into an input field |
| Select option | Select an option from a dropdown |
| Submit form | Submit a form |
| Scroll to bottom | Scroll the page to the bottom (useful for lazy-loaded content) |
| Go back | Navigate back in browser history |
| Show hidden elements | Force hidden elements to be visible |
| Action | What It Does |
|---|---|
| Disable JavaScript | Disable JavaScript before the page loads |
| Set cookie | Set or manage browser cookies |
| Execute JavaScript | Run custom JavaScript code on the page |
| Handle CAPTCHA | Interact with CAPTCHA challenges |
Dismiss cookie banners: Add a "Remove cookies" action to automatically hide consent popups that can trigger false change notifications.
Load lazy content: Add "Scroll to bottom" followed by "Wait" (2-3 seconds) to load content that only appears when scrolling.
Navigate to a tab or section: Add a "Click element" action with the CSS selector of the tab you want to monitor.
Login to a page: Add "Type text" actions for username and password fields, followed by "Click button" to submit the login form.
Wait for dynamic content: Add "Wait for text" with the text that appears after the page finishes loading (e.g., "Showing results").
]]>The Review Board is a Kanban-style board that helps you organize and track detected changes across your monitored pages. Instead of reviewing changes one by one, you can drag and drop change cards between customizable lanes to manage your review workflow.
Navigate to the Review tab in the main sidebar to open the board.
Each time PageCrawl detects a change on one of your monitored pages, a card appears on the board. Cards show:
Click any card to view the full change details, timeline, and AI summary.
By default, the board includes three lanes: To Review, Reviewed, and Flagged. You can customize these to match your workflow:
Common lane setups:
Use the toolbar at the top of the board to filter changes:
When enabled, giving thumbs-up or thumbs-down feedback on a change notification automatically moves the card to your "Reviewed" lane. Enable this from the gear icon menu on the board.
You can configure which lane cards move to after positive or negative feedback.
]]>Most websites maintain an XML sitemap listing every page on the site. They do this for SEO: a sitemap tells Google, Bing, and other search engines exactly which URLs exist, when each one was last modified, and how often it changes. Without a sitemap, search engines have to discover pages by crawling links one by one, which is slow and often misses freshly published or deeply nested content. Because Google rewards indexable content, almost every CMS (WordPress, Shopify, Squarespace, Wix, etc.) generates and publishes a sitemap automatically.
For change monitoring, that same sitemap is a goldmine - it is the website's own up-to-date list of every page that matters, maintained by the site itself. PageCrawl can monitor these sitemaps to detect new pages, removed URLs, and structural changes automatically.
PageCrawl supports two distinct ways to monitor a sitemap, and you should pick the one that fits your goal:
Most teams pick one or the other for a given site depending on whether they need deep per-page tracking or just new-URL alerts.
This is the heavy-duty approach. Each new URL discovered in the sitemap becomes its own tracked page in your workspace, with full change history, screenshots, content alerts, and AI summaries.
competitor.com)Large websites may add many pages between checks. Filters help you focus on what matters:
/products/, /blog/2026/*)/products/accessories/)Exclude filters always take priority over include filters. You can combine multiple filter types.
When auto-monitoring is enabled, pages matching your filters are automatically added to your monitoring workspace. For example:
No manual setup required. Combined with templates, auto-monitored pages inherit your preferred check frequency, notification channels, and tracking settings.
Not all websites have complete sitemaps. PageCrawl supplements sitemap monitoring with additional discovery methods:
See Page Discovery for full details on all discovery methods.
Sitemap monitoring via Page Discovery is available on all plans:
| Plan | Pages per Website |
|---|---|
| Free | Up to 2,000 |
| Standard | Up to 20,000 |
| Enterprise | Up to 100,000 |
All plans include filters, notifications, and auto-monitoring.
This is the lightweight approach. Instead of creating one tracked page per URL, the entire sitemap becomes a single tracked element. You get an alert when URLs are added or removed, but PageCrawl does not fetch or track the content of each page.
<url> entry<loc> URL (the stable key)There is only one Change record in your workspace - the sitemap monitor itself - regardless of how many URLs the sitemap contains.
competitor.com/sitemap.xml)Feeds are capped at a per-plan number of items so a 50,000-URL sitemap does not produce 50,000-item JSON blobs on every check:
| Plan | Maximum Items Per Feed |
|---|---|
| Free | 10 |
| Standard | 100 |
| Enterprise | 1,000 |
| Ultimate | 10,000 |
Items are returned in document order. For RSS and Atom feeds this is fine because the newest items are conventionally at the top, but sitemaps do not guarantee that. If your sitemap has more URLs than your plan cap, the UI shows a notice and suggests either raising the cap or using Page Discovery instead, which has no per-feed cap (it uses your monitor quota).
For sites with both a sitemap and an RSS or Atom feed, the RSS/Atom feed is usually a better choice for Feed mode because new content is guaranteed to appear at the top. Try /feed, /rss, or /atom.xml on the site.
For fully-fledged monitoring with per-page change history, screenshots, content alerts, AI summaries, and proper handling of large sitemaps, use Page Discovery (Scan a Website) instead. Feed mode is intentionally minimal - it is a fast way to get new-URL notifications without the overhead of tracking each page, but it cannot replace Page Discovery for serious change monitoring.
If you are choosing between monitoring a site's sitemap and its RSS or Atom feed, the two are not equivalent:
sitemap.xml. New posts appear there as soon as the CMS regenerates the sitemap.For tracking new content, both work - the RSS feed is usually more reliable because new posts are guaranteed to appear at the top, but you cannot use the RSS feed to discover the site's full back catalog. Use the sitemap when you need complete URL coverage and the RSS feed when you only care about new content.
PageCrawl can automatically create a full web archive of your monitored pages every time a change is detected. Archives capture the complete page (HTML, CSS, images, scripts) so you can replay it exactly as it appeared at that moment.
Archives are saved in the WACZ (Web Archive Collection Zipped) format, an open standard for web archiving used by libraries, governments, and legal teams worldwide.
Available on Ultimate plan.
If WACZ generation fails (e.g., due to complex page structure), PageCrawl falls back to creating a self-contained HTML archive instead.
PageCrawl offers both screenshots and web archives, but they serve different purposes:
| Screenshot | Web Archive (WACZ) | |
|---|---|---|
| What it captures | A flat image of the visible page | The complete page: HTML, CSS, JavaScript, images, fonts |
| Interactivity | None (static image) | Fully interactive: scroll, click links, hover over elements |
| Content below the fold | Only if full-page screenshot is enabled | Always included, the entire page is preserved |
| Dynamic content | Shows one visual state | Preserves interactive elements, dropdowns, tabs |
| File size | Small (typically under 1 MB) | Larger (includes all page assets) |
| Best for | Quick visual reference, visual diff comparison | Legal evidence, compliance records, full preservation |
Screenshots are great for a quick visual snapshot and for visual change detection (highlighting pixel differences). Web archives go further by preserving the entire page so you can interact with it later exactly as it appeared.
The Internet Archive (archive.org) and PageCrawl both preserve web pages, but they work very differently:
Archive.org (Wayback Machine):
PageCrawl Web Archiving:
In short, archive.org is best for general public web preservation. PageCrawl archiving is designed for active monitoring where you need precise, private, frequent snapshots tied to detected changes.
To view an archived page:
The viewer uses ReplayWeb.page to render WACZ archives interactively in your browser. You can scroll, click links, and interact with the page as if you were browsing it live at that point in time.
You can download any archive file directly:
Downloaded archives can be used for legal evidence, compliance records, or offline browsing.
Archives are enabled at the workspace level. Contact support or check your workspace settings to enable archiving for your monitored pages.
]]>Workspaces let you organize your monitored pages into separate environments, each with its own settings, notifications, and team member access. Use workspaces to separate monitoring by project, client, department, or any other grouping that makes sense for your workflow.
Every workspace has independent settings for:
Use the workspace selector dropdown in the sidebar to switch between your workspaces. Each workspace shows its own set of pages, changes, and settings.
Administrators can control which team members have access to each workspace:
Members only see workspaces they've been assigned to. This lets you give client-facing teams access to client workspaces without exposing internal monitoring.
See User Roles & Permissions for details on what each role can do.
By client: One workspace per client, each with its own notification channels and team access.
By department: Marketing monitors competitor pages, Legal monitors compliance pages, Product monitors feature pages, each in their own workspace.
By priority: A "Critical" workspace with immediate notifications and frequent checks, and a "Background" workspace with weekly reports and less frequent checks.
By region: Separate workspaces for different geographic regions, each with region-specific proxy settings and timezones.
]]>PageCrawl can automatically save page screenshots to your Dropbox whenever a change is detected. This gives you a visual archive of every change, stored in your own cloud storage for easy access and sharing.
When a change is detected on a monitored page and screenshots are enabled, PageCrawl uploads the screenshot to your chosen Dropbox folder. Files are organized by page name and timestamp:
{your-folder}/{page-name}/{datetime}.jpgThis makes it easy to browse through the history of visual changes for any monitored page.
Once connected, screenshots will be uploaded automatically whenever a change is detected on any of your monitored pages that have screenshots enabled.
After connecting your Dropbox account, you can:
If your Dropbox access token expires, the connection is automatically disabled and you will receive a notification. Simply reconnect your Dropbox account at Settings > Workspace > Integrations to restore screenshot syncing.
Dropbox screenshot sync is available on all plans.
]]>PageCrawl includes a built-in MCP (Model Context Protocol) server that lets AI assistants manage your page monitors. You can add monitors, check history, trigger checks, and more, all through natural conversation with tools like Claude or ChatGPT.
MCP is an open protocol that standardizes how AI tools connect to external services. Once connected, your AI assistant can directly interact with your PageCrawl account without you needing to use the web interface or API manually.
Available on all plans. Free plan users have read-only access (list monitors, view history, check diffs). Paid plans (Standard and above) have full access including creating monitors, triggering checks, and managing tags.
The MCP server provides the following tools that your AI assistant can use:
| Tool | What It Does |
|---|---|
| Add page monitor | Create a new monitor with URL, tracking mode, frequency, and notifications |
| List monitors | Search and view monitors across all workspaces by URL, domain, or name |
| Get monitor details | See full configuration of a specific monitor including tracked elements and latest values. Supports batch requests |
| Get monitor history | Retrieve historical checks and detected changes with AI summaries. Supports batch requests |
| Get latest values | Quickly retrieve just the current values for one or more monitors (e.g., current price). Supports batch requests |
| Get check diff | View the actual text differences detected in a specific check |
| Trigger check | Trigger a one-off check on a monitor |
| Manage tags | List workspace tags, or add and remove tags from monitors |
| Mark changes seen | Mark detected changes as reviewed on one or all monitors |
| List templates | View available templates that can be applied when creating monitors |
| List workspaces | View all your teams and workspaces with their IDs |
| Update monitor defaults | View or update default settings for new monitors created via MCP |
When creating monitors through MCP, you can track the following element types:
https://pagecrawl.io/mcp
Add the following to your .mcp.json file (in your project root or ~/.claude/):
{
"mcpServers": {
"pagecrawl": {
"url": "https://pagecrawl.io/mcp"
}
}
}When Claude Code first tries to use PageCrawl tools, it will open a browser window for you to authorize the connection via OAuth.
Works with ChatGPT on web, desktop, and mobile. Requires a ChatGPT Plus, Pro, Team, Enterprise, or Edu plan.
https://pagecrawl.io/mcpAny MCP-compatible client that supports OAuth can connect to PageCrawl. The server details:
https://pagecrawl.io/mcphttps://pagecrawl.io/.well-known/oauth-authorization-serverThe client will handle the OAuth flow automatically. No manual token setup is required.
For MCP clients that do not support OAuth, you can connect using a personal API token instead. This works with OpenClaw, Cursor, Cline, Windsurf, VS Code, Claude Code, and any other MCP client that supports custom headers.
Step 1: Generate an API token in PageCrawl:
Step 2: Add the following configuration to your MCP client. The JSON format below works with Cursor (.cursor/mcp.json), Cline, Windsurf (.vscode/mcp.json), Claude Code (.mcp.json), and most other clients:
{
"mcpServers": {
"pagecrawl": {
"url": "https://pagecrawl.io/mcp",
"headers": {
"Authorization": "Bearer YOUR_TOKEN_HERE"
}
}
}
}For OpenClaw, use the CLI:
openclaw mcp set pagecrawl \
--transport streamable-http \
--url https://pagecrawl.io/mcp \
--header "Authorization: Bearer YOUR_TOKEN_HERE"For Cursor, you can also add via Settings > MCP Servers > Add > Streamable HTTP and enter the URL and authorization header there.
Note: API tokens require a paid plan. Treat your token like a password. You can revoke tokens at any time from Settings > API.
Once connected, you can interact with PageCrawl naturally:
Adding monitors:
"Monitor example.com/pricing every hour and track the full page text"
"Set up price tracking for these 3 product pages: [url1], [url2], [url3]. Check every 15 minutes and notify me on Slack when prices drop."
Checking current values:
"What's the current price on my Amazon product monitor?"
"Compare the prices across all my competitor monitors right now"
Reviewing changes:
"Show me all monitors that changed in the last 24 hours with a summary of what changed"
"Show me the diff for the terms of service page. What exactly was added or removed?"
Analysis and reporting:
"Which of my monitors have had the most changes this month? Are there any patterns?"
"Give me a weekly summary: how many changes were detected across all monitors, which ones had price drops, and which ones had errors?"
Batch operations:
"Tag all monitors tracking amazon.com with 'competitor' and 'ecommerce'"
"Check the latest values for all monitors tagged 'pricing' and tell me which products are currently out of stock"
Troubleshooting:
"Are any of my monitors failing? Show me the ones with errors and what the issue is"
"The pricing page monitor hasn't detected changes in weeks. Trigger a fresh check and show me what it finds"
Setting up workflows:
"Create a monitor for each of these 5 competitor pricing pages. Use the 'competitor-tracking' template and tag them all as 'q2-research'"
"Monitor the SEC EDGAR page for new filings from Tesla. Use content-only mode so it ignores the navigation, check every 30 minutes"
All tools automatically search across every workspace you have access to. You do not need to know which workspace a monitor is in to find or interact with it.
search parameter to find monitors by URL, domain, or nameworkspace_id to filter results to a specific workspaceworkspace_id if you have more than one workspaceMCP operations respect your plan's limits:
See Check Limits and Website Limits for details on plan quotas.
]]>Webhooks allow PageCrawl to send HTTP POST requests to any external URL whenever a page change is detected or an error occurs. Use webhooks to connect PageCrawl with custom applications, automation platforms, databases, or any service that accepts HTTP requests.
Target URL: The HTTP endpoint that will receive the POST request.
Event Triggers: Choose which events fire the webhook:
Page Filter: Optionally limit the webhook to a specific monitored page. If not set, the webhook fires for all pages in the workspace.
Active/Inactive Toggle: Disable a webhook without deleting it.
By default, webhooks send all available fields. You can customize the payload by selecting only the fields you need:
| Category | Fields |
|---|---|
| Basic | id, title, status, changed_at, visual_diff, difference, human_difference, short_summary |
| Differences | markdown_difference, html_difference |
| Images | text_difference_image, page_screenshot_image |
| Page Info | page metadata, page_elements array |
| Content | contents, original (for extracted values) |
| Comparison | previous_check data |
| JSON | json, json_patch |
| AI | ai_summary, ai_priority_score |
After saving a webhook, click the Test button to send a sample payload to your endpoint. This verifies the connection works before relying on it for real notifications.
{
"id": 12345,
"title": "Product Page - Example.com",
"status": "change_detected",
"changed_at": "2026-01-15T10:30:00Z",
"visual_diff": 12.5,
"difference": 3,
"human_difference": "3 lines changed",
"short_summary": "Price updated from $99 to $89",
"ai_summary": "The product price was reduced by 10%.",
"ai_priority_score": 85
}Email is the default notification channel in PageCrawl. It is enabled on all plans and requires no additional setup. As soon as you add a page to monitor, you will receive email notifications whenever changes are detected.
Every change notification email includes:
Email notifications can include several attachments to give you a complete picture of the change:
You can configure which attachments are included at Settings > Workspace > Notifications.
On paid plans, you can add additional recipients to your change notifications:
This is useful for keeping team members, clients, or stakeholders informed without requiring them to have a PageCrawl account.
You can choose how often you receive email notifications:
| Frequency | Description |
|---|---|
| Immediate | Get an email as soon as a change is detected |
| Daily | Receive a daily digest summarizing all changes from the past 24 hours |
| Daily (weekdays) | Same as daily, but only Monday through Friday |
| Weekly | Receive a weekly digest summarizing all changes from the past 7 days |
| Monthly | Receive a monthly digest summarizing all changes from the past 30 days |
Digest reports include AI summaries for each change, making it easy to review multiple updates without feeling overwhelmed.
You can customize how text differences are displayed in your email notifications:
These options let you focus on the type of changes that matter most to you.
When you are monitoring 5 or more pages on the same domain, PageCrawl automatically groups notifications by domain. This keeps your inbox organized and makes it easier to review related changes together.
Each email notification includes feedback links that let you mark a change as Important or Noise. PageCrawl's AI learns from your feedback and uses it to improve future importance scoring, so over time you receive fewer irrelevant notifications.
PageCrawl supports several other notification channels to suit your preferences:
]]>Conditions and Filters let you control which changes trigger notifications on a per-page basis. Instead of receiving a notification for every detected change, you can define rules so that only meaningful changes are reported.
When adding a page, the simple setup mode offers common conditions directly for price, number, and selected area tracking (such as price thresholds, percentage change alerts, and keyword monitoring). For the full set of conditions described below, click "More options" to switch to Advanced Mode. When editing an existing page, toggle Advanced Mode on. In both cases, you will find the Conditions & Filters section.
In the page editor, switch to Advanced Mode (click "More options" when adding a new page, or toggle "Advanced Mode" when editing an existing page). Look for the Conditions & Filters section with the description: "Looking for specific changes or alerts for certain keywords? Customize conditions to minimize unnecessary change alerts."
Toggle the switch on to enable conditions. Once enabled, you can add one or more conditions by clicking the Add Condition button.
When you have multiple conditions, you can choose how they are evaluated using the Match all conditions toggle:
This lets you build precise rules. For example, with AND logic you could require that a specific keyword appeared AND a price dropped below a threshold.
By default, when conditions are not met, the change detection is not recorded and no notification is sent. This means the next check compares against the last version that did meet conditions.
Enable Always record change detections to record every change regardless of whether conditions are met, but only send notifications when conditions match. This is particularly useful with one-directional conditions like "Keyword appeared" or "Keyword disappeared", where skipping unmatched detections could cause the condition to never trigger again.
The most commonly used condition. It triggers a notification only when a specific keyword is added to or removed from the page.
Enter one or more keywords (each keyword is a separate tag). The condition is met when any of the specified keywords appear in newly added text or disappear from removed text.
Match mode options control how keywords are compared against the page text:
| Match Mode | Case Sensitive | Whole Word | Example: keyword "assist" |
|---|---|---|---|
| Match any text (default) | No | No | Matches "assist", "Assist", "assistance", "ASSISTANT" |
| Match any text (case sensitive) | Yes | No | Matches "assist", "assistance" but not "Assist" |
| Match exact words only | No | Yes | Matches "assist", "ASSIST" but not "assistance" |
| Match exact words (case sensitive) | Yes | Yes | Matches only "assist" exactly |
Filters remove noise by excluding certain types of changes from triggering notifications.
Exclude specific words, sentences, or patterns from change detection. Place each entry on a separate line. This is useful for text that changes frequently but is not relevant, like timestamps, cookie banners, or dynamic counters.
Supported patterns:
This website uses cookies)% to match any text within a line. For example, %Published at% will ignore any line containing "Published at", such as "Published at: 2024-12-24 by John"/custom-regex-pattern-\d+/). Requires a paid plan.Note: If the ignored text line is replaced with a new line that is not in the filter, the change detection will still trigger.
Prevents any numeric changes on the page from triggering change detections. Useful when pages contain counters, view counts, or other dynamic numbers that are not relevant to you.
These conditions let you control notifications based on specific text content. They are available for text-based tracked elements (not visual elements).
Triggers when a keyword is added to the page. Unlike "Keyword appeared or disappeared", this will not notify you when a keyword is removed.
Important: If "Always record change detections" is not enabled, using this condition alone can cause missed detections. When the keyword is not found, no change is recorded, so the comparison baseline never updates. We recommend using "Keyword appeared or disappeared" instead, or enabling "Always record change detections".
Triggers when a keyword is removed from the page. The condition compares the current check with the previous one and fires if the keyword was present before but is now gone.
The same warning about "Always record change detections" applies here.
Available for individual tracked elements (not full page monitors). The condition is met when the element's text matches the specified value exactly.
Available for individual tracked elements (not full page monitors). The condition is met when the element's text does not match the specified value exactly.
The condition is met when the tracked element's text contains any of the specified keywords. Best used in combination with other conditions, for example: "the page must always contain the text 'Welcome' AND a keyword appeared." If you only need to know when text is added or removed, use "Keyword appeared or disappeared" instead.
The condition is met when the tracked element's text does not contain any of the specified keywords. Useful for combined conditions like "the page does not contain 'Website failed to load' AND a change was detected." If you only need to know when text is added or removed, use "Keyword appeared or disappeared" instead.
These conditions are only available for "Number" and "Price detect" tracked elements. They allow you to set thresholds and track numeric changes with precision.
| Condition | Description | Example |
|---|---|---|
| Greater than | Triggers when the number exceeds the specified value | Value is 150, triggers when number > 150 |
| Greater than or equals | Triggers when the number is at or above the specified value | Value is 150, triggers when number >= 150 |
| Less than | Triggers when the number drops below the specified value | Value is 50, triggers when number < 50 |
| Less than or equals | Triggers when the number is at or below the specified value | Value is 50, triggers when number <= 50 |
These conditions compare the current value against the previous value to detect significant changes.
| Condition | Description | Example |
|---|---|---|
| Increased or Decreased by at least x percent | Triggers when the number changes in either direction by at least x%. | Value is 10, x is 20%. Triggers when value becomes 12+ or 8 or less. |
| Increased or Decreased by at least x | Triggers when the number changes in either direction by at least x (absolute). | Value is 10, x is 5. Triggers when value becomes 15+ or 5 or less. |
| Increased by at least x percent | Triggers only when the number goes up by at least x%. | Value is 10, x is 20%. Triggers when value becomes 12 or more. |
| Increased by at least x | Triggers only when the number goes up by at least x (absolute). | Value is 10, x is 5. Triggers when value becomes 15 or more. |
| Decreased by at least x percent | Triggers only when the number goes down by at least x%. | Value is 10, x is 20%. Triggers when value becomes 8 or less. |
| Decreased by at least x | Triggers only when the number goes down by at least x (absolute). | Value is 10, x is 5. Triggers when value becomes 5 or less. |
Price drop alert: Monitor a product price with a "Number" tracked element. Add a "Less than" condition with your target price. You will only be notified when the price falls below your threshold.
Stock availability: Monitor an "In Stock" label with a "Keyword appeared or disappeared" condition. Set the keyword to "Out of Stock" to get notified the moment availability changes.
Ignore cookie banners: Add an "Ignore text" filter with entries like This website uses cookies and Accept all cookies to prevent cookie consent changes from triggering notifications.
Significant price changes only: Use "Increased or Decreased by at least x percent" with a value of 10 to only be notified when a price changes by 10% or more, filtering out minor fluctuations.
Combined conditions: Monitor a product page with AND logic: "Keyword appeared" for "Sale" combined with "Less than" 100 on the price element. You will only be notified when the product goes on sale AND the price drops below 100.
]]>Web push notifications deliver instant alerts directly to your browser when PageCrawl detects a change on your monitored pages. No extra apps, no browser extensions, and no webhook configuration needed.
When a monitored page changes, PageCrawl sends a native browser notification to all your subscribed devices. You'll see the notification even when PageCrawl.io isn't open in your browser.
If AI summarization is enabled for the page, the notification includes a brief summary explaining what changed, so you can decide at a glance whether to investigate.
That's it. Notifications start immediately.
You can subscribe on multiple devices (desktop, laptop, phone, tablet). Each device receives notifications independently. To manage your subscribed devices:
| Browser | Desktop | Mobile |
|---|---|---|
| Chrome | Yes | Yes (Android) |
| Firefox | Yes | Yes (Android) |
| Edge | Yes | - |
| Safari 16+ | Yes (macOS) | Yes (iOS) |
| Channel | Setup | Speed | Best For |
|---|---|---|---|
| Web Push | None | Instant | Personal monitoring, time-sensitive changes |
| None | Minutes | Searchable archive, batch review | |
| Slack | Webhook URL | Instant | Team collaboration |
| Discord | Webhook URL | Instant | Community monitoring |
| Teams | Webhook URL | Instant | Enterprise environments |
| Telegram | Bot token | Instant | Mobile-first users |
You can use push notifications alongside other channels. A common setup:
Configure different notification channels per page in the page settings.
]]>PageCrawl can automatically group monitors that track the same product on different websites, giving you a real-time view of how prices compare across retailers. When the competitive landscape shifts, you can get alerts and export comparison spreadsheets.
| Capability | Description |
|---|---|
| Side-by-side pricing | See all retailer prices for a product in one place via the Matched Pages panel |
| Comparison alerts | Get notified when a price becomes the cheapest, most expensive, or when the spread exceeds a threshold |
| Cross-retailer export | Download a spreadsheet with one row per product and columns per retailer |
| Smart suggestions | When linking monitors, PageCrawl suggests the most relevant candidates |
| Automatic grouping | Monitors are grouped automatically when product identifiers match |
| Reference labels | Manually group monitors using labels with a shared prefix |
| Google Sheets integration | Include comparison data and label-based columns in automated Google Sheets exports |
PageCrawl uses multiple signals to determine whether two monitors on different websites track the same product. When a match is found, the monitors are placed into a comparison group automatically.
Matching happens after each page check and when labels are updated. If the same product is listed on five different retailer websites and each monitor is set up with price tracking, PageCrawl will link all five into a single group.
You can also group monitors manually from the comparison panel on any monitor's detail page, or by applying reference labels (covered below).
Each comparison group can contain up to 20 monitors.
When a monitor belongs to a comparison group, its detail page shows a Matched Pages panel. This panel displays:
From this panel you can:
When adding monitors to a comparison group, PageCrawl ranks candidates by relevance. Suggestions consider multiple factors including product identifiers, reference labels, folder grouping, domain similarity, and name overlap.
If the product comparison feature is enabled, suggestions are enhanced with stronger signals from product identifiers and reference labels. Without the feature enabled, suggestions still work but rely on name and structural similarity only.
You can also type in the search box to filter across all monitors in your workspace.
Comparison alerts notify you when a monitor's price changes its competitive position within the group. There are three alert types:
| Alert Type | When It Fires | Configuration |
|---|---|---|
| Cheapest | This monitor's price is the lowest in the group | No additional configuration needed |
| Most Expensive | This monitor's price is the highest in the group | No additional configuration needed |
| Price Spread | The gap between the lowest and highest price in the group exceeds a percentage | Set the spread threshold percentage |
Alerts are transition-based. You receive a notification when the state changes (e.g., a monitor becomes the cheapest), but not on every subsequent check where it remains the cheapest. When the condition clears, the alert resets and can fire again later.
For example, if Monitor A is tracking a laptop at $999 and becomes the cheapest in a group of five retailers:
Price Spread alerts work similarly. If you set a 20% threshold and the spread increases from 15% to 25%, you receive a notification. The alert clears when the spread drops below 20%.
Comparison alerts are evaluated after every page check, using the most recent values from all group members. Alerts are delivered through your configured notification channels (email, Slack, Discord, Teams, Telegram, webhooks).
Export a comparison spreadsheet to analyze all your grouped products and their prices in a single file.
| Column | Description |
|---|---|
| Product | Product name from page metadata, or monitor name as fallback |
| GTIN | Global Trade Item Number if detected |
| SKU | Stock Keeping Unit if detected |
| Brand | Product brand if detected |
| [retailer domain] | One column per unique retailer domain, containing the current tracked value |
Each row represents one comparison group. If a group has members on amazon.com, bestbuy.com, and walmart.com, the spreadsheet will have three retailer columns.
If the same retailer domain appears more than once in a group (e.g., two product variants on the same site), the column headers are disambiguated with the monitor name.
Only monitors that belong to a comparison group are included in the export. Ungrouped monitors are excluded.
Reference labels provide a way to manually group monitors using a label prefix. This is useful when automatic matching is not sufficient, or when you want to define your own product identifiers.
Apply a label with a specific prefix to monitors that track the same product. For example:
| Monitor | Label |
|---|---|
| Laptop X on Amazon | ref:LAPTOP-X-2024 |
| Laptop X on Best Buy | ref:LAPTOP-X-2024 |
| Laptop X on Walmart | ref:LAPTOP-X-2024 |
All three monitors share the label ref:LAPTOP-X-2024, so PageCrawl groups them together.
The default prefix is ref, but you can change it in your workspace settings.
You can apply reference labels in several ways:
prefix:valueWhen a reference label is added or changed, PageCrawl automatically re-evaluates comparison groups.
Tag prefix columns turn label prefixes into structured data columns available in exports and Google Sheets integrations.
sku, brand, ref)| Setting | Description |
|---|---|
| Prefix Columns | List of prefixes to expose as export/Google Sheets columns (max 10) |
| Comparison Prefix | The prefix used for product comparison grouping (default: ref) |
Once configured, tag prefix columns appear as available columns in your Excel and Google Sheets export settings alongside the built-in columns (name, URL, current value, etc.).
For example, if you configure prefixes sku and brand:
sku:WGT-500 and brand:Acme will show WGT-500 in the SKU column and Acme in the Brand columntag_sku and tag_brand in column configurationWhen you change the comparison prefix (e.g., from ref to group), PageCrawl automatically re-evaluates groups for monitors that have labels with the new prefix. Existing groups built from product identifiers are not affected.
Note: Prefix names must be lowercase alphanumeric characters or underscores, with a maximum length of 50 characters.
When Page Discovery finds new pages and product comparison is enabled, PageCrawl checks whether the discovered page matches an existing monitored product. If a match is found, the discovered page shows the matched product's name and domain, helping you decide whether to add it to monitoring.
This is particularly useful for automatically finding the same product on newly discovered retailer pages.
Product comparison works best with monitors using price or number tracking modes, since these produce numeric values that can be compared. Full-page text monitors will appear in groups but cannot generate comparison alerts.
If you manage a large catalog, establish a naming convention for reference labels. Using the same internal product ID across all retailers (e.g., ref:INTERNAL-SKU-001) ensures consistent grouping.
Let automatic matching handle the initial grouping, then review and adjust using reference labels for any products that were not matched correctly. Automatic and manual matching work together and complement each other.
Rather than adding comparison alerts to every monitor, focus on the products where competitive pricing matters most. This keeps your notifications actionable and avoids alert fatigue.
Schedule regular exports to track pricing trends over time. Combined with Google Sheets integration, you can build dashboards that update automatically.
| Limit | Value |
|---|---|
| Max group size | 20 monitors per comparison group |
| Max prefix columns | 10 per workspace |
| Prefix name length | 50 characters |
Product comparison is available as a team-level add-on. Contact support or your account manager to enable it for your account.
Premium residential proxies let you monitor websites that block standard datacenter IP addresses. They use real residential internet connections from 200+ countries, making your monitoring checks appear as regular user traffic.
Most websites work fine with the datacenter proxies already included in every PageCrawl plan. You only need residential proxies if:
Before purchasing, try these free alternatives:
Residential proxies are priced at $10/GB of data transferred. Every page check consumes bandwidth based on the page size:
| Page Type | Approximate Size Per Check |
|---|---|
| Simple text page (blog, news article) | ~0.5 MB |
| Standard e-commerce or listing page | ~2 MB |
| Heavy page with images and scripts | ~5 MB |
Bandwidth never expires. You can purchase 1 GB today and use it over months.
Check frequency has a large impact on bandwidth consumption. The same 10 pages can cost very different amounts depending on how often you check:
| Frequency | 10 Pages Monthly Cost |
|---|---|
| Daily | ~$10 (0.6 GB) |
| Hourly | ~$150 (14.4 GB) |
| Every 15 minutes | ~$570 (57.6 GB) |
For most monitoring use cases, daily or hourly checks are sufficient. Only use high-frequency residential proxy checks when near real-time monitoring is essential.
When using residential proxies, you must select a target country from 200+ supported countries. This is useful for monitoring localized pricing, regional content, or geo-restricted pages.
Premium residential proxy bandwidth is available on Enterprise and Ultimate plans. Contact us if you have questions about upgrading.
Feed tracking mode treats an RSS feed, Atom feed, or XML sitemap as a list of individual items rather than a single blob of text. Instead of "the page changed", you get "2 new posts added: [titles and links]". This matches how you actually want to consume a feed: item by item.
Pick Feed mode when the URL you are monitoring is a structured list that updates over time:
/feed, /rss.xml, /atom.xml, /feeds/posts/default, /index.xml)/sitemap.xml, /sitemap_index.xml)github.com/owner/repo/releases.atom)reddit.com/r/subreddit/.rss)PageCrawl auto-detects the feed format when you paste the URL and switches to Feed mode automatically. You can also pick it manually from the tracking mode selector.
Compared to Full Page text tracking, Feed mode gives you:
| Feature | Full Page Text | Feed Mode |
|---|---|---|
| Compares raw content | Yes | No (parses items) |
| Reports which items changed | No | Yes, with titles and links |
| Ignores reordering | No (false alerts) | Yes |
| Deduplicates by stable key | No | Yes (guid, id, link) |
| Caps item count | No | Yes (configurable limit) |
| Runs without a browser | Only if page is plain text | Yes, for XML feeds |
| Handles "No exact matches" fallbacks | No | Yes |
The end result: fewer false alerts, clearer notifications, and lower monitoring cost per check.
Feed tracking mode parses:
<guid>, <enclosure>, <media:content>, and <content:encoded>rdf:about identifiers<link rel="alternate"> and <media:thumbnail><urlset>) and sitemap index (<sitemapindex>)jsonfeed.org/version/1)For HTML pages like product grids, inventory lists, or news listings, Feed mode falls back to DOM pattern detection, which identifies repeated card-like elements on the page and tracks them as items.
When you paste a URL into Track New Page, PageCrawl performs a content-based check:
If the detection cannot classify the URL as an XML feed, the tracking mode stays at Full Page and you can switch to Feed manually if you want to use DOM pattern detection on an HTML page.
Every feed tracking element has a Track first N items cap. The default is 10 for new monitors. You can raise it up to your plan's maximum.
The limit exists for three reasons:
For RSS and Atom feeds, "first N" means the first N items in document order, which is the convention these formats use to put the newest items at the top. Reading position 0 through N-1 gives you the N most recent posts.
XML sitemaps are different. There is no convention requiring sitemaps to list new URLs first. New pages can appear anywhere in the file, including appended at the bottom. To handle this, PageCrawl sorts sitemap entries by their <lastmod> date (newest first) before applying the cap, so the most recently modified URLs always win.
For sitemaps that do not include <lastmod> on every URL, the dated entries are sorted first and the dateless entries fall to the bottom of the sort in their original document order. If you need to track every page on a very large sitemap regardless of modification date, use Page Discovery instead - it auto-monitors new pages as they appear without depending on the position-based cap.
| Plan | Maximum Items Per Feed |
|---|---|
| Free | 10 |
| Standard | 100 |
| Enterprise | 1,000 |
| Ultimate | 10,000 |
The default is 10 across all plans. You can raise it from the tracking mode panel any time after the monitor is created.
By default, Feed mode notifies you when items are added to the feed. You can also opt into:
Each item is identified by a stable key in this order: GUID → link → title. That means content changes on the same item are correctly recognized as updates, not as a new item.
Feed mode runs via a lightweight HTTP fetch without a browser, so you can check feeds frequently without burning through plan limits:
| Feed Type | Recommended Frequency |
|---|---|
| Security advisories | Every 15 minutes |
| News and competitor blogs | Every 30 to 60 minutes |
| GitHub release feeds | Every 1 to 2 hours |
| Podcast feeds | Every 6 to 12 hours |
| Sitemaps for large sites | Every 1 to 4 hours |
| Low-volume blogs | Daily |
Note: if you raise the frequency below 30 minutes on a browser-only feed (an HTML inventory page rather than an XML feed), PageCrawl will use the browser engine for reliability.
GitHub release feed:
https://github.com/owner/repo/releases.atomWordPress blog:
https://example.com/feed/Reddit subreddit:
https://www.reddit.com/r/webdev/.rssSite sitemap:
https://example.com/sitemap.xmlFor each of these, paste the URL into Track New Page. PageCrawl detects the format, switches to Feed mode, and shows the first 10 items as a preview before you save.
Every time PageCrawl detects a change, you can give quick feedback with the thumbs up and thumbs down buttons. This feedback helps you organize your review workflow and tells PageCrawl which changes are useful and which ones are noise.
The feedback buttons appear in several places:
You can give feedback directly from any of the notification channels above, no login required. You are taken to a short confirmation page that records the feedback, then returned to the change (or a simple confirmation screen if you are not signed in).
Pressing thumbs up flags the change as important or useful. This tells PageCrawl:
If your workspace has feedback auto-review enabled, the change card also moves from "To Review" to your chosen destination lane on the Review Board (for example, a "Reviewed" or "Important" lane). You can configure which lane thumbs-up feedback moves cards to from the Review Board settings.
Pressing thumbs down flags the change as noise or irrelevant. This does several things:
You can accept or dismiss the suggested action. Accepting it applies the filter so similar changes are automatically filtered on future checks.
If you press thumbs down on a change involving state-toggle text (for example, "in stock" to "out of stock", "available" to "unavailable", "open" to "closed"), PageCrawl shows a warning. This is because telling the system to ignore a change in one direction could cause it to also ignore the reverse change, which is often something you actually want to be alerted about. Read the warning carefully before confirming.
Press thumbs up when:
Examples:
Press thumbs down when:
Examples:
Press thumbs down even if the change is minor. Over time, consistent feedback makes your monitors much quieter and more precise.
If a change is neutral (neither clearly useful nor clearly noise), you can leave it without feedback and simply mark it as reviewed. Feedback is not mandatory. Only use it when you have a clear opinion, because consistent signals produce better filtering than mixed ones.
If you change your mind, reopen the change and press the same button again to clear the flag, or press the opposite button to overwrite the previous feedback. Clearing feedback does not automatically remove any filters that were added as a result of it. Those filters are managed separately under the page's actions and ignore rules.