Catching Security Regressions Across a Portfolio of Client Sites

Managing one website's security is straightforward. Managing thirty is a different problem entirely. A staging deploy strips a CSP header, a marketing team changes DNS to a new email provider and breaks DMARC, a developer rotates a TLS cert and accidentally drops the intermediate. Each issue is small, but multiplied across a client portfolio it becomes a constant, low-grade fire.

This post walks through a realistic workflow for monitoring multiple client websites for security regressions — what to track, how to baseline it, how to detect drift, and how to actually fix things without burning out your team.

What counts as a security regression?

Before you can monitor for regressions, you need a shared definition. A regression is any measurable drop in security posture compared to a known good baseline. In practice that means:

• TLS/SSL: certificate expiry, weakened cipher suites, missing intermediate certs, dropped HSTS
• HTTP headers: removed or weakened CSP, missing X-Content-Type-Options, X-Frame-Options, Referrer-Policy
• Cookies: missing Secure, HttpOnly, or SameSite attributes after a framework upgrade
• DNS: missing or broken SPF, DKIM, DMARC, CAA records
• CORS: wildcard origins appearing on authenticated endpoints
• Mixed content: HTTP assets sneaking back in via a new third-party script
• Software exposure: server banners revealing versions, debug endpoints left enabled

Most regressions are not caused by attackers. They're caused by deploys, plugin updates, DNS changes, and CDN configuration drift.

Building a portfolio baseline

You can't detect drift without a baseline. For each client site, capture a snapshot of the current state across every category above. This becomes the reference point for every future scan.

What to record per site

1. Primary domain plus any subdomains in active use (www, app, api, staging if public)
2. Current security grade or score from your scanner of choice
3. Full header set served on the homepage and at least one authenticated route
4. TLS certificate details: issuer, expiry, SANs, chain
5. DNS records relevant to email and certificate authority (SPF, DKIM, DMARC, CAA, MX)
6. Known third-party scripts and their domains

Store this in something queryable — a Notion database, an Airtable, or a simple Postgres table. The format matters less than the discipline of updating it when intentional changes happen.

Scanning on a schedule, not just on deploy

Regressions happen between deploys too. A CDN config push, a DNS change made by a client's marketing team, an expiring cert — none of these trigger your CI pipeline. You need recurring external scans.

A reasonable cadence for an agency portfolio:

• Daily: TLS expiry checks, header presence checks, DNS record diffs
• Weekly: full security scan with grading across all categories
• On-demand: triggered after any production deploy via webhook

This is where a tool like WebSentry earns its place — running A–F graded scans across SSL, headers, CSP, cookies, DNS, and CORS, and surfacing the diff between scans rather than just the absolute score. The diff is what matters when you're managing twenty clients; an absolute grade tells you nothing if you don't know whether it dropped from an A yesterday.

Centralising alerts without drowning in noise

If every header change pages someone at 2am, your team will mute the channel by week two. Tier your alerts.

Tier 1: page someone now

• TLS certificate expiring in under 7 days
• TLS certificate invalid or chain broken
• Site returning 5xx or unreachable
• DMARC policy dropped to p=none from a stricter setting

Tier 2: ticket within 24 hours

• CSP removed or significantly weakened
• New wildcard CORS on a previously-restricted endpoint
• Cookie attributes regressed (Secure or HttpOnly removed)
• New mixed content warnings

Tier 3: weekly review

• Minor header changes (Referrer-Policy variant, Permissions-Policy tweaks)
• New third-party script domains
• Server banner changes

Route Tier 1 to PagerDuty or an on-call Slack channel. Route Tier 2 to your ticketing system with the client tag pre-applied. Tier 3 goes into a weekly digest one person reviews on Monday morning.

A real example: the CSP that vanished

An agency we spoke with had a client running a Next.js site behind Vercel. A developer added a new analytics tag and, to make it work quickly, edited next.config.js to remove the script-src directive entirely rather than adding a nonce or hash. The deploy went green. The site worked.

Three weeks later, a scheduled scan flagged the CSP regression. The diff showed the previous policy versus the current one, the developer remembered the change, and a proper fix went in within a day. Without external monitoring, that policy would likely have stayed broken for months until someone ran a manual audit.

Reporting that clients actually read

Internal monitoring is half the job. The other half is communicating posture to clients in a way that justifies your retainer and gives them confidence.

A monthly report per client should include:

• Current grade and the trend over the last six months
• Any regressions detected and how they were resolved
• Outstanding issues with a recommended timeline
• Comparison against industry baseline where useful

Keep it to one page. Clients don't read ten-page security PDFs. They do read a single dashboard screenshot with three bullet points underneath.

Automating the boring parts

The work that kills agency margins is the manual checking. Automate:

1. Onboarding scans: when a new client signs, run a full scan and store the baseline automatically
2. Deploy-triggered scans: webhook from your CI to your scanner so every production deploy generates a fresh snapshot
3. Diff generation: don't read full scan reports, read the diff against the previous scan
4. Client reports: generate the monthly PDF or HTML report from the same data, don't write it by hand

WebSentry's API and scheduled scans are designed for exactly this pattern — point it at a list of domains, get back graded results you can pipe into your own dashboard or client reports.

Try it on your own portfolio

If you want to see what your current portfolio looks like, run a free scan on any of your client domains at websentry.dev and see where the regressions are hiding before someone else finds them.