Catching Security Regressions Before Deploy: CI/CD Security Tests

Most security regressions don't come from sophisticated attacks. They come from a developer removing a header to debug something, a framework upgrade that silently relaxes the CSP, or a TLS config that drifts after an infrastructure change. By the time someone runs a scanner against production, the bad config has been live for weeks.

The fix is to treat website security like you treat tests: run it automatically on every pull request and every deploy. Here's how to wire that into a CI/CD pipeline without turning your build times into a coffee break.

What to actually test in CI

Not every security check belongs in your pipeline. Pen tests, fuzzing, and DAST scans against production are valuable but slow. For CI, focus on fast, deterministic checks that catch regressions:

• HTTP response headers — CSP, HSTS, X-Content-Type-Options, Referrer-Policy, Permissions-Policy
• TLS configuration — protocol versions, cipher suites, certificate validity
• Cookie flags — Secure, HttpOnly, SameSite on session cookies
• CORS policy — no wildcard origins on authenticated endpoints
• Dependency vulnerabilities — npm audit, pip-audit, or equivalent
• Secret scanning — catch leaked tokens before they hit main
• DNS hygiene — SPF, DMARC, CAA records on staging deploys

Where each check runs

Layer your checks based on cost:

1. Pre-commit / PR: secret scanning, dependency audit, static config linting
2. Post-deploy to staging: full header scan, TLS check, cookie inspection, CORS probe
3. Post-deploy to production: smoke test of headers and TLS, alert on regressions

GitHub Actions: a working example

Here's a workflow that runs security checks against a staging deployment after every merge:

name: Security Checks

on:
  deployment_status:

jobs:
  scan-staging:
    if: github.event.deployment_status.state == 'success' && github.event.deployment.environment == 'staging'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Check security headers
        run: |
          URL="https://staging.example.com"
          HEADERS=$(curl -sI "$URL")
          echo "$HEADERS" | grep -i "strict-transport-security" || exit 1
          echo "$HEADERS" | grep -i "content-security-policy" || exit 1
          echo "$HEADERS" | grep -i "x-content-type-options: nosniff" || exit 1

      - name: Check TLS with testssl.sh
        run: |
          docker run --rm drwetter/testssl.sh             --severity HIGH --quiet             https://staging.example.com

      - name: Dependency audit
        run: npm audit --audit-level=high

This is intentionally minimal. Each grep is a regression gate — if someone removes HSTS, the build fails before the change reaches production.

Going further with a graded scan

Grepping headers catches the obvious stuff but misses subtleties: a CSP with unsafe-inline, a cookie missing SameSite, a misconfigured Permissions-Policy. For a complete picture you want a scanner that grades the full surface.

WebSentry exposes an API that returns a JSON report with an A–F grade per category (SSL, headers, CSP, cookies, DNS, CORS). You can fail the build if the grade drops below a threshold:

      - name: WebSentry scan
        run: |
          RESULT=$(curl -s "https://websentry.dev/api/scan?url=https://staging.example.com")
          GRADE=$(echo "$RESULT" | jq -r '.overall_grade')
          echo "Grade: $GRADE"
          case "$GRADE" in
            A|B) exit 0 ;;
            *) echo "Security grade dropped to $GRADE"; exit 1 ;;
          esac

GitLab CI equivalent

The same pattern works in .gitlab-ci.yml:

security-scan:
  stage: post-deploy
  image: alpine:latest
  before_script:
    - apk add --no-cache curl jq
  script:
    - |
      curl -sI "$STAGING_URL" | tee headers.txt
      grep -i "strict-transport-security" headers.txt
      grep -i "content-security-policy" headers.txt
  only:
    - main

Common mistakes that make CI security checks useless

Scanning production from CI on every commit

You'll rate-limit yourself and add noise. Scan staging on every merge; scan production on a schedule (daily or after explicit deploys).

Failing on warnings you can't fix today

If your CSP legitimately needs unsafe-inline for a legacy widget, encoding that as a hard fail means everyone learns to ignore the alert. Set a baseline grade and fail only on regressions from it.

Not pinning the scanner version

If you use a Docker image like drwetter/testssl.sh, pin a tag. Otherwise a scanner update can break builds with no code change.

Ignoring the report

The output of a security scan is only useful if someone reads it. Pipe the JSON into your CI artifacts, post a summary comment on the PR, or forward findings to Slack. A check that silently passes is worse than no check.

A checklist for rolling this out

1. Run a baseline scan of staging and production today. Record the current grade.
2. Add header and TLS checks to your post-deploy job. Start in warn-only mode.
3. After a week of clean runs, flip them to hard fails.
4. Add dependency audit to PR builds with --audit-level=high to avoid noise.
5. Schedule a nightly production scan and route alerts to whoever owns the site.
6. Document the baseline grade and what each check is protecting against, so the next engineer doesn't delete a check they don't understand.

What a healthy pipeline looks like

When this is working, a developer who accidentally weakens the CSP gets a red PR check within minutes. A framework upgrade that drops a header is caught on the staging deploy. A certificate that's about to expire triggers an alert with a week to spare. Security stops being something you remember to check and becomes a property of the deployment process.

If you want to see where your site stands right now before wiring any of this in, run a free scan at websentry.dev — you'll get an A–F grade across every category mentioned above, which makes a good baseline for your first CI check.