# How to Monitor Websites Behind a Login: Multi-Step Form and Click Flows Source: PageCrawl.io Blog URL: https://pagecrawl.io/blog/monitor-website-behind-login-form-steps-automation --- HTTP Basic Auth is the easy case. You drop a username and password into the monitor settings, the browser sends an `Authorization` header, and the page loads. Done. The hard case is everything else: the login form rendered by JavaScript, the email-first flow that asks for your password on a second screen, the dashboard that only shows real numbers after you click a tab, the cookie banner that covers the submit button. Most of the pages worth monitoring live in that second category. A vendor's pricing portal, an internal admin dashboard, a partner program console, a SaaS billing page that only renders after authentication. None of these respond to a Basic Auth popup. They want a real login: fill the email field, click Next, fill the password, click Sign In, wait for the redirect, then maybe click into the right section before the content you care about even appears. This guide is about automating that interaction. It covers the action primitives (fill, click, wait, navigate), how to chain them for multi-step and click-gated flows, how to find selectors that survive a redeploy, and how to handle the things that break automated logins. If you only need a simple credentials box, start with the [guide to monitoring password-protected websites](/blog/monitor-password-protected-websites), which covers Basic Auth and single-form logins in detail. This post is the form-driven, multi-step sequel. ### Why form logins are harder than Basic Auth Basic Auth is a transport-level mechanism. The credentials travel in a header before the page renders, so there is nothing to interact with. Form logins are application-level. The page renders first, then you fill inputs, fire events, and submit. Several things make this fragile: - **The form is rendered by JavaScript.** The input fields do not exist in the initial HTML, so a monitor has to wait for them before it can type into them. - **The flow is split across steps.** Email on screen one, password on screen two, sometimes an organization or workspace picker in between. - **The real content is click-gated.** Many dashboards load a default view, and the data you want only appears after you click a tab, expand a panel, or select a date range. - **Overlays get in the way.** Cookie banners, "install our app" interstitials, and onboarding tooltips can sit on top of the form and swallow your click. A monitoring tool that only fetches raw HTML cannot do any of this. You need one that drives a real browser and lets you script the interaction. PageCrawl handles this through **Actions**: an ordered list of steps that run before every check. ### The four actions you actually need Almost every authenticated flow is built from four primitives. Learn these and you can express nearly any login. #### Navigate Go to a URL. Use it to land on the login page first, and again at the end to reach the page you actually want to monitor after the session is established. #### Fill Type a value into an input identified by a CSS selector. This is how you enter an email, a password, a workspace name, or a search term. Fill triggers the input and change events the page expects, so client-side validation and "Next" buttons that stay disabled until a field is valid will behave correctly. #### Click Click an element identified by a selector: a submit button, a "Next" button, a tab, a cookie-accept button, a checkbox. Click is the workhorse of multi-step and click-gated flows. #### Wait Pause before the next step. Use it to let a redirect finish, a second form step render, or an asynchronous data panel load. You can wait a fixed number of seconds, or wait for a specific selector to appear, which is more robust than guessing a duration. A complete sequence is just these four in the right order. Here is a single-step form: ``` Navigate -> https://app.example.com/login Wait -> for selector input[name="email"] Fill -> input[name="email"] = monitor@yourcompany.com Fill -> input[name="password"] = •••••••••• Click -> button[type="submit"] Wait -> for selector .dashboard-header Navigate -> https://app.example.com/billing ``` The final Navigate is important. After login, the app drops you on a default landing page. You usually want to monitor a specific deeper page, so navigate there once the session cookie is set. ### Multi-step (email-first) login flows Google, Microsoft, Okta, and a growing number of SaaS apps split login into two screens. You enter your email, click Next, and only then does the password field appear. Treating this as a single form fails because the password input does not exist yet when the page first loads. The fix is to insert a click and a wait between the two fills: ``` Navigate -> https://login.example.com Wait -> for selector input[type="email"] Fill -> input[type="email"] = monitor@yourcompany.com Click -> button#next Wait -> for selector input[type="password"] Fill -> input[type="password"] = •••••••••• Click -> button#signin Wait -> for selector .app-shell ``` The two **Wait for selector** steps are doing the heavy lifting. Rather than guessing "3 seconds should be enough," you wait until the exact element you need is present. This survives a slow network and a slow render, and it fails fast and clearly when a selector is wrong. Some flows add a third screen, like a workspace or organization picker: ``` ... after password submit ... Wait -> for selector .workspace-list Click -> a[data-workspace="acme-prod"] Wait -> for selector .dashboard-header ``` Chain as many screens as the flow has. The pattern never changes: fill what is on screen, click to advance, wait for the next screen's anchor element, repeat. ### Click-gated content: the part people forget Logging in is half the job. The other half is getting the page into the state where the value you care about is actually visible. A lot of authenticated content is hidden behind a click even after you are signed in. Common cases: - **Tabbed dashboards.** The default tab shows an overview; your metric lives under "Usage" or "Billing." - **Collapsed panels.** A "Show details" or "Advanced" toggle hides the real numbers. - **Lazy tables.** A grid renders empty until you click "Load data" or change a filter. - **Date-range views.** The current value only appears after you pick "This month" or "Last 30 days." Add the clicks after navigation, then wait for the content to settle: ``` Navigate -> https://app.example.com/dashboard Wait -> for selector nav.tabs Click -> nav.tabs a[href="#usage"] Wait -> for selector .usage-meter ``` After this sequence, the usage panel is open and rendered, and the monitor can compare it on every check. If you only watch one number on the page (a quota, a balance, a count), point a [tracked element](/blog/css-selector-guide-target-elements-monitoring) at it so you get alerted on that value specifically instead of on every cosmetic change elsewhere. ### Setting it up in PageCrawl, step by step **Step 1: Create the monitor with the final URL.** Enter the deep page you ultimately want to watch (for example `https://app.example.com/billing`). This is what PageCrawl checks for changes after the actions run. **Step 2: Open the Actions panel.** This is where you build the ordered sequence. Each action has a type (Navigate, Fill, Click, Wait) and the relevant inputs (a URL, a selector, a value, a duration). **Step 3: Build the login sequence.** Add a Navigate to the login URL, then the Fill / Click / Wait steps that match your flow from the sections above. Use **Wait for selector** rather than fixed durations wherever you can. **Step 4: Add any click-gated steps.** After the post-login Navigate to your target page, add the clicks that open the tab or panel holding the content you care about, followed by a Wait for that content. **Step 5: Turn on screenshots.** New PageCrawl monitors capture a screenshot on every check by default, and you should leave it on here. The screenshot is how you verify the login actually worked. If a check ever shows you the login page instead of the dashboard, you know a selector broke or the session failed, and you can fix the sequence instead of trusting a silent green checkmark. **Step 6: Run a manual check and inspect the result.** Trigger one check and look at the captured screenshot and content. If you see the authenticated page, you are done. If you see the login form, a cookie wall, or an error, adjust the offending action. Common culprits: a selector that changed, a missing wait before a step, or an overlay covering the button. **Step 7: Choose notification channels.** Route alerts to email, Slack, Discord, Teams, Telegram, or a [webhook](/blog/webhook-automation-website-changes) so the right people hear about changes. For pipelines, send the change into [n8n](/blog/n8n-website-monitoring-automate-change-detection) or [Zapier](/blog/zapier-website-monitoring), or build your own logic with [AI agents that react to website changes](/blog/ai-agents-react-website-changes-webhooks). ### Finding selectors that do not break Your action sequence is only as stable as its selectors. A login that works today and silently fails next month is worse than no monitoring, because you stop checking the page yourself. Pick selectors that survive a redeploy. Open the login page, right-click the field, and Inspect. Then prefer attributes in this order: | Selector basis | Example | Stability | |----------------|---------|-----------| | `id` | `#email` | High, rarely changes | | `name` | `input[name="password"]` | High, tied to form submission | | `type` | `input[type="email"]` | High for login forms (usually one of each) | | `data-*` test hooks | `button[data-testid="signin"]` | High, added for the app's own tests | | `aria-label` | `button[aria-label="Sign in"]` | Medium, accessibility-driven | | Stable class | `.login-submit` | Medium | | Generated class | `.css-1a2b3c` | Low, changes every build | | Positional | `form > div:nth-child(3) input` | Low, breaks on layout changes | Avoid the bottom two rows. Auto-generated class names from CSS-in-JS libraries change on every build, and positional selectors break the moment someone adds a field. For a deeper treatment of choosing resilient targets, see the [CSS selector guide](/blog/css-selector-guide-target-elements-monitoring) and the comparison of [XPath and CSS selectors for monitoring](/blog/xpath-css-selectors-web-monitoring). ### Handling the things that break automated logins #### Cookie banners and overlays A consent banner or interstitial can sit on top of the form and absorb your click, so the submit silently does nothing. Add a click to dismiss it before the login steps: ``` Navigate -> https://app.example.com/login Click -> button#accept-cookies Wait -> 1 second Fill -> input[name="email"] = ... ``` PageCrawl also ships **Remove Cookies** and **Remove Overlays** as default actions on new monitors, which clear most banners automatically without you naming a selector. Keep them on. If a stubborn banner remains, add the explicit click. #### Two-factor authentication 2FA is the hardest blocker because it requires a code you do not have at check time. Practical options, roughly in order of preference: - **Create a dedicated monitoring account with 2FA disabled**, scoped to read-only access. This is the simplest reliable approach when the security policy allows it. - **Use an app-specific password** where the provider offers one (some Google and Microsoft setups). - **Monitor an API instead of the web page.** If the service exposes an authenticated API, point your monitor at a JSON endpoint with a long-lived token. See the [API monitoring guide](/blog/api-monitoring-track-changes-alerts) for how to track structured responses. #### OAuth and SSO redirects "Sign in with Google" and enterprise SSO (Okta, Azure AD, Auth0) bounce you through a third-party identity provider and back. The action sequence has to follow the redirect: ``` Navigate -> https://app.example.com/login Click -> button.sso-login Wait -> for selector input[type="email"] Fill -> input[type="email"] = monitor@yourcompany.com Click -> button#next Wait -> for selector input[type="password"] Fill -> input[type="password"] = •••••••••• Click -> button#signin Wait -> for selector .app-shell ``` Add generous waits between steps to absorb redirect latency. Where you have the choice, a direct username/password login is more reliable than OAuth, because identity providers sometimes challenge automated sign-ins with extra verification. If you control the app, a read-only API token avoids the interactive flow entirely. #### Sessions expire, and that is fine PageCrawl logs in fresh on every check rather than storing a session between runs. This is by design: sessions expire, and a fresh login is more reliable than a stale cookie. The tradeoff is that your login sequence runs on every check, so keep it lean and make sure the credentials account stays valid. ### Real-world flows #### Authenticated billing dashboard A finance team watches their cloud provider's billing console for unexpected cost spikes. The flow is email-first login, then a click into the "Cost breakdown" tab, then a tracked element on the month-to-date total. When the number jumps, they get a Slack alert with the before and after values, days before the invoice lands. #### Partner program console A partnerships lead monitors three vendor partner portals for changes to commission tiers and program terms. Each portal has its own login sequence and a click into the "Program details" section. Weekly checks surface changes ahead of quarterly reviews, with an AI summary of what actually moved so nobody reads the whole page. #### Internal admin dashboard An ops team monitors their own staging admin panel for unauthorized configuration changes. The monitor logs in with a read-only account, navigates to the settings page, and compares it daily. Any unexpected diff is a signal to investigate. This pairs well with watching [SaaS changelogs and product updates](/blog/changelog-monitoring-saas-tools-updates) for the third-party tools the same team depends on. ### Security practices for stored credentials Putting a password into any tool deserves a moment of thought. A few rules keep the risk low: - **Use a dedicated monitoring account**, not your personal admin login. You can revoke it independently and trace its activity in audit logs. - **Scope it to read-only.** The monitor never needs write, delete, or admin access to compare a page. - **Use a unique, strong password** generated for this account alone, never reused elsewhere. - **Watch the watcher.** Glance at the monitoring account's audit-log activity occasionally; unusual patterns mean something is off. - **Read the terms of service.** Some platforms restrict automated access, especially competitor-owned ones. Know the rules before you point a monitor at a portal you do not own. ### Choosing your PageCrawl plan PageCrawl's **Free plan** lets you monitor **6 pages** with **220 checks per month**, which is enough to validate the approach on your most critical pages. Most teams graduate to a paid plan once they see the value. | Plan | Price | Pages | Checks / month | Frequency | |------|-------|-------|----------------|-----------| | Free | $0 | 6 | 220 | every 60 min | | Standard | $8/mo or $80/yr | 100 | 15,000 | every 15 min | | Enterprise | $30/mo or $300/yr | 500 | 100,000 | every 5 min | | Ultimate | $99/mo or $999/yr | 1,000 | 100,000 | every 2 min | Annual billing saves two months across every paid tier. Enterprise and Ultimate scale up to 100x if you need thousands of pages or multi-team access. At an engineering hourly rate, Standard at $80/year pays for itself the first time you catch a breaking API change, a deprecated endpoint, or a silent config change before it takes down production. 100 monitored pages is enough to cover the changelogs and docs of every third-party API your stack depends on. Enterprise at $300/year adds higher check frequency, 500 pages, and full API access. All plans include the **PageCrawl MCP Server**, which plugs directly into Claude, Cursor, and other MCP-compatible tools. Developers can ask "what changed in the Stripe API docs this month?" and get a summary pulled from your own monitoring history. AI assistants can create monitors through conversation on every plan, including Free, and paid plans add on-demand checks and monitor management, turning your tracked pages into a living knowledge base instead of a pile of alert emails. ### Getting Started Start with one authenticated page that you currently check by hand. Build the login sequence using the four actions, lean on **Wait for selector** instead of fixed pauses, and add any clicks needed to expose the content you care about. Run a manual check, confirm the screenshot shows the real page and not the login form, then let it run for a week. Once that first monitor proves out, expand to the rest of your portals one login flow at a time. Reuse the patterns here for multi-step and click-gated pages, and route alerts to wherever your team already works. The free tier covers 6 monitored pages with screenshots on by default, which is enough to wrap your most important authenticated dashboards before you decide whether to scale up. --- Need more? The complete PageCrawl.io help center, with every article, is available as a single document at https://pagecrawl.io/llms-full.txt. Read it for context on anything this page does not cover.