How ShieldWave scans a live site
ShieldWave checks a website from the outside, the way any visitor sees it. This page says exactly how much traffic a scan sends to your live site, how it keeps that traffic within a set budget, and when it slows down or stops so a scan never puts your site under strain. The technical details of the scanner itself, and how to keep it out, are on the ShieldWave-Scanner page.
- Traffic budget
- At most 5 requests per second for the free check and 15 for a full scan. One request every 2 seconds is the slowest it ever goes.
- One at a time
- Only one scan of a site runs at a time, even across our servers, so parallel scans never add up.
- It watches your server
- It measures your response times and errors, and slows down, pauses or stops when your server is under strain.
- Opt out
- A robots.txt rule for ShieldWave-Scanner, or an exclusion request, keeps it off your site. See the details.
- Contact
- support@shieldwave.io
The traffic budget for each plan
Every scan runs under one profile. The profile sets the most requests per second the scan sends to your site, and a hard time limit for the whole scan. The scan never goes above the requests per second in this table.
| Profile | Used for | Requests per second, at most | Time limit |
|---|---|---|---|
| basic | The free check on shieldwave.io, and any check of a site whose control is not confirmed | 5 | 90 seconds |
| standard | The full scan of a confirmed site: paid plans and the one-off scan | 15 | 3 minutes |
| quick | The API only, with the quick profile, on a confirmed site | 10 | 60 seconds |
| deep | The API only, with the deep profile, on a confirmed site | 3 | 6 minutes |
| stealth | The API only, with the stealth profile, a slow check for sites with strict limits on the pace of requests | 1 | 4 minutes |
All the requests of one scan to your site and its subdomains go through a single pacer made for that scan. When several requests are ready at the same moment, they are spread out one after another at the profile's rate, so they never arrive in a burst.
One scan at a time per site
Before a full scan sends anything, it takes a short lock on your site, held in our database so it counts across all of our servers. A second full scan of the same site waits for that lock. If the first scan finishes, the waiting one starts. If it waits about 90 seconds and the site is still busy, the second scan stops and reports that another scan of the site is already running, rather than sending a second set of requests on top of the first.
The lock keeps a heartbeat while the scan runs and expires on its own if the server holding it stops, so a crash never leaves your site locked. The free check and the free tools read only a little and are not held back by this lock.
What ShieldWave measures on your server
Before the active checks begin, the scanner sends a few light reads of your home page to measure a baseline: how quickly your server answers when it is not under load (its p95 response time). During the scan it keeps watching the answers to its own requests: the rolling p95 response time and the share of answers that are server errors (5xx).
When it slows down, pauses and stops
The scan keeps its traffic within the budget above, and pulls back further when your server shows strain. It only ever slows down, never speeds past the profile rate.
- When your server answers HTTP 429 or 503, the rate drops to 40 percent at once.
- When your server starts answering slowly (its p95 goes past twice the baseline and at least 500 milliseconds slower than the baseline), or when more than one in five answers is a 5xx error, the rate is halved.
- If it stays bad after that, the scan pauses for about 30 seconds to let your server recover, then measures again with a few light reads.
- If it is still bad after the pause, the scan stops its remaining active checks and marks the scan, so it never keeps pushing a server that is struggling.
- The rate never goes below one request every 2 seconds.
A single HTTP 403 does not slow the scan. A firewall turning away one request says nothing about the load on your server, so a 403 is treated as an ordinary answer. Real strain is judged by the response times and errors above.
Template checks share the same budget
The standard and deep profiles also run a set of template checks (Nuclei) for known exposures and misconfigurations. These run as a separate tool, so they take a fixed share out of the same budget: 6 of the 15 requests per second in the standard profile and 1 of the 3 in the deep profile. The rest of the scan slows by the same amount, so the total never goes above the profile rate. When your robots.txt has any Disallow rule for ShieldWave-Scanner, the template checks are skipped, because they cannot follow path rules.
Turning it off or slowing it down
You are always in control of whether ShieldWave scans your site. A robots.txt group named for the scanner, with Disallow: /, stops a scan before it sends anything. You can also ask us to exclude your site from every check. The full instructions, and how to recognise the scanner in your logs, are on the ShieldWave-Scanner page. Your server can also push back at any time by answering HTTP 429 or 503, and the scan slows down at once.
What the scan reports back
Every scan records what it sent and shows it in plain words in the report and on the scan result page: how many requests it sent, the highest rate it reached, whether it slowed down, paused or stopped and why, and whether a robots.txt opt-out was followed. So you can always see exactly how a scan behaved on your site.
Contact
If a scan of your site was not expected, or you want to ask a question or report misuse, write to support@shieldwave.io. Include the site and, if you can, a line from your access log with the time of the request.