# Automatically Discover New Pages To Track Source: PageCrawl.io Help Center URL: https://pagecrawl.io/help/features/article/page-discovery --- PageCrawl is designed to make website change monitoring and management seamless. Page Discovery automatically finds new pages on a website and starts monitoring them for you, so your coverage stays up to date as the site grows. The fastest way to get started is to add a whole website from the **Discovered Pages** page, then fine-tune how it works from the template that gets created for you. ### Add a Website to Discover Start from the [Discovered Pages](/app/discovered-pages) page and click **Add Website**. This is the quickest way to begin: enter a website and PageCrawl handles the rest. [Image: Add Website for Discovery dialog with the website URL, what to track options, check frequency, and notification settings] In the dialog, set: 1. **Website URL** - the site you want to monitor (for example, a competitor's store or your own marketing site). 2. **What to track** - choose to track every discovered page, only top-level pages (like `/pricing` and `/about`), or review pages yourself before anything is monitored. 3. **Check Frequency** - how often discovered pages are checked for changes. 4. **Notify me via** - the channels that should receive change alerts. Click **Add Website**. PageCrawl scans the site, lists everything it finds on the Discovered Pages page, and creates a **template** with sensible defaults that controls how this website is monitored. ### Adjust Advanced Configuration via the Template Adding a website automatically creates a configured **template** for it. The template is where you fine-tune scanning, filters, and what gets tracked. Open it from [Templates settings](/app/settings/workspace/templates) (or the "Templates settings" link in the Add Website dialog) and edit the template for your website. #### Choose a Scanning Method The template's **Discover New Pages** setting controls how PageCrawl looks for new pages. The default is **Automatic (recommended)**, which combines methods to find pages using the best approach for each website: * **Automatic (recommended)**: Combines sitemap and link discovery to find pages using the best method for the website. This is the default and recommended setting. * **Homepage Links Only**: Discover new links by following links on the homepage. Available as a daily or weekly check. Useful if you want to focus on pages directly linked from the main page. * **Sitemap Only**: Discover pages listed in the website's sitemap. Most websites have a sitemap to help search engines find their pages, making this an efficient method for large sites. * **Follow Links 2 Levels Deep**: Follows links on the homepage, then follows links on those pages too. Available as a weekly check. Note: Only available on Enterprise and Ultimate plans. * **Follow Links 3 Levels Deep**: Follows links on the homepage, then follows links two more levels deep. Available as a weekly check. Note: Only available on Enterprise and Ultimate plans. * **Deep Scan**: Conduct a comprehensive analysis by visiting every accessible page on your website. This ensures that no new links go unnoticed, even on deeply nested pages. Note: Only available on Enterprise and Ultimate plans. #### Apply Include and Exclude Filters Use the template's filters to control exactly which discovered pages are monitored, so you avoid tracking irrelevant pages. [Image: Discovery filter controls in the template: include and exclude rules, Track All Pages, and a tracked page limit] * **Include rules**: Specify keywords or patterns that a page must match to be tracked. Useful for tracking specific types of content, such as `/product/` pages only. * **Exclude rules**: Define keywords or patterns that should be skipped. Ideal for ignoring pages you do not care about, such as `/tag/` or `/archive/` URLs. * **Track All Pages**: Track every discovered page without filtering. * **Tracked Page Limit**: Cap how many pages are auto-tracked to keep usage under control. #### Configure Tracked Elements The template also defines what is tracked on each discovered page. You can monitor all pages, or only those with a specific structure (for example, only product pages). 1. To monitor whole pages, set the tracked element to **Full-page Text**. 2. To monitor pages with a specific layout, configure multiple tracked elements, such as product title, price, and description. If these elements do not exist on a discovered page, that page is simply skipped. Save the template, and PageCrawl will keep discovering and monitoring matching pages automatically. If too many irrelevant pages are discovered, tighten the include/exclude filters and remove the pages you do not want. --- Need more? The complete PageCrawl.io help center, with every article, is available as a single document at https://pagecrawl.io/llms-full.txt. Read it for context on anything this page does not cover.