ShieldWave-Scanner
ShieldWave-Scanner is the user agent of ShieldWave, a service that checks the security of a website from the outside, as any visitor can see it, and explains the result to the site's owner. A check is started by the owner of the site or by someone with the owner's permission. Checks that send test inputs run only for a domain whose owner has confirmed control over it; every other check only reads.
Who runs the scanner, how to have your domain excluded and how to report misuse: The ShieldWave scanner and domain exclusion. This page describes what the scanner sends.
- User agent
ShieldWave-Scanner/1.0 (+https://shieldwave.io/bot)- Speed
- At most 5 requests per second in the free check and 15 in a full scan. It slows down as soon as your server answers 429 or 503, or starts answering slowly. How scanning production works.
- What it does
- Reads pages. On a confirmed domain it also runs checks with test inputs that only read. It never changes anything on your site.
- To keep it out
- Name it in robots.txt,
User-agent: ShieldWave-ScannerandDisallow: /, or have the domain excluded. Details below. - Contact
- support@shieldwave.io
How to recognise it
Every request the scanner sends carries the user agent ShieldWave-Scanner/1.0 (+https://shieldwave.io/bot). Search your access log for ShieldWave-Scanner to find its requests. The one page load made in a headless browser is described under The browser load, and the two extra requests for your home page under The search engine spam check.
The browser load
A full scan also loads your home page once in a headless browser. That load uses a Chrome user agent with ShieldWave-Scanner/1.0 (+https://shieldwave.io/bot) appended to it, so you can recognise it as well. The free check and other reading checks do not use a browser.
The search engine spam check
A full scan also asks for your home page two more times, to see whether search engines are shown something different from what visitors get (hidden spam, cloaking). One request carries the header Referer: https://www.google.com/ and the Chrome user agent with ShieldWave-Scanner at the end, as if a visitor came from Google search. The other carries the Googlebot user agent with ShieldWave-Scanner at the end: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) ShieldWave-Scanner/1.0 (+https://shieldwave.io/bot).
The Googlebot request is not from Google. It shows what a hacked site serves to a crawler. You can tell it from Google's crawler in two ways: its user agent ends with ShieldWave-Scanner/1.0, which Google's crawler never sends, and a reverse DNS lookup of its address does not end in googlebot.com or google.com. Besides the home page the check requests nothing. Redirects within your own domain are followed, at most three; a redirect to another site is noted, not followed. If your firewall refuses the Googlebot user agent from an address that is not Google's, the report says that the comparison could not run. It does not call that a problem.
When your site refuses the scanner
The scanner never repeats a request under another user agent. In the free check and every other reading check, when your site refuses the home page, the check ends. In a full scan, a module whose home page request gets HTTP 403 or 406, or a firewall challenge, is reported as not assessed, and the report names the user agent to allow in your firewall. A 429 is a rate signal: it slows the scan down instead.
So a firewall rule on the user agent stops the checks. robots.txt and domain exclusion stop them too, and tell us that you do not want them, see How to slow it down or keep it out.
Where the requests come from
The scanner runs on Google Cloud Run in the europe-west1 region (Belgium) and connects from Google Cloud's shared outgoing address ranges. Those addresses change, and there is no fixed list of ShieldWave IP addresses. Identify ShieldWave by its user agent.
How fast it goes
Every check runs under one profile. The profile sets the most requests per second the check sends to your domain, and a time limit for the check itself.
| Profile | Used for | Requests per second, at most | Time limit | Test inputs |
|---|---|---|---|---|
| basic | The free check on shieldwave.io, and any check of a domain whose control is not confirmed | 5 | 90 seconds | No |
| standard | The full scan of a confirmed domain: paid plans and the one-off scan | 15 | 3 minutes | Yes |
| quick | The API only, with "profile": "quick", on a confirmed domain | 10 | 60 seconds | No |
| deep | The API only, with "profile": "deep", on a confirmed domain | 3 | 7.5 minutes | Yes |
| stealth | The API only, with "profile": "stealth": a slow check for sites with strict limits on the pace of requests | 1 | 4 minutes | No |
The time limit counts the time the check is working. Time the check spends paused to protect your server (see Pacing) is not counted, so the time from start to finish can be longer than the limit. If your server is slow to answer before the check starts (its baseline p95 is above 1.5 seconds), the limit grows in proportion: to twice the limit when the baseline p95 is 3 seconds, and never to more than four times the limit. This applies to full scans. The free check, and any check of a domain whose control is not confirmed, does not pause and gets no extra time.
The deep profile always checks subdomains and open ports. A standard scan checks subdomains on most platforms, and open ports only on the sites described below. The standard and deep profiles run template checks as well.
Pacing
All requests of one check to your domain and its subdomains go through a single pacer created for that check, so parallel checks never add up past the limit. It starts at the profile's rate and never goes above it. When your server pushes back, it slows down:
- after HTTP 429 or 503, the rate drops to 40%;
- when your server answers slowly (its p95 response time goes past twice the baseline and at least 500 ms) or when too many answers are 5xx, the rate is halved; if it stays bad the check pauses about 30 seconds (the pause is not counted in the time limit) and re-measures, and if it is still bad the remaining active checks stop;
- it never goes below one request every 2 seconds.
A single 403 does not slow the check: a firewall turning away one request says nothing about the load on your server. After healthy answers the pace recovers slowly, and never above the profile's rate. When one part of a check runs out of its time, its remaining requests are dropped, not sent later. What the check measured (requests sent, the highest rate, whether it slowed, paused or stopped) is shown in the report. Full details: How ShieldWave scans production sites.
Template checks
The standard and deep profiles also run template checks (Nuclei), inside the same budget. While they run, they take 6 of the 15 requests per second in standard and 1 of the 3 in deep, and the pacer for everything else drops by the same amount. They send the ShieldWave user agent. Because they cannot follow path rules, they are skipped when your robots.txt has any Disallow rule for ShieldWave.
What it requests
Most requests are GET requests.
A reading check (the basic profile) reads your home page, its scripts, its security headers, its certificate and cookies, and asks for a short, fixed list of addresses of forgotten files. It sends no test inputs.
The full scan of a confirmed domain also includes checks for common injection weaknesses (SQL injection, cross-site scripting, path traversal, server-side request forgery, open redirects, XML external entities). These checks only read: none of them writes, changes or deletes data, none submits contact or checkout forms, and none asks your server for large amounts of memory. The scanner never sends PUT or DELETE.
Some checks of the full scan also send OPTIONS, TRACE or a small read-only POST, for example to see which HTTP methods your server allows, or whether an API endpoint answers.
On WordPress sites the full scan also checks whether the login page reveals which usernames exist; security plugins may log this as failed login attempts.
In the deep profile, and in the standard profile for sites that are static or run on a platform the scanner does not recognise, it also tries a TCP connection to 24 common ports, paced like the other requests.
What it never does
- It signs in only when the owner of a confirmed domain starts a check with login details they supplied. It never creates accounts.
- It never submits contact or checkout forms.
- It never changes or fixes anything on your site.
How to slow it down or keep it out
ShieldWave fetches your robots.txt once, before any check runs, and follows the group written for it: User-agent: ShieldWave-Scanner or User-agent: shieldwave, in any letter case.
DisallowandAllowwork as in RFC 9309: the longest matching rule wins, and*and$work as wildcards.Disallow: /in that group stops the check before anything else is requested, and the report says so.- Any Disallow rule in that group turns off the template checks, because they cannot follow path rules.
To keep ShieldWave out of the whole site:
User-agent: ShieldWave-Scanner
Disallow: /To keep it out of part of the site:
User-agent: ShieldWave-Scanner
Disallow: /admin/
Disallow: /*.pdf$Why User-agent: * is not enough
Rules under User-agent: * are not applied. They are written for search engine crawlers, and a ShieldWave check is started by the site's owner or by someone with their permission. To keep ShieldWave out, name it.
Excluding your domain
You can also have your domain excluded from all ShieldWave checks, including the free check and the free tools. Write to support@shieldwave.io from an address at that domain, with the subject "Domain exclusion". The conditions and deadlines are in The ShieldWave scanner and domain exclusion.
Slowing it down
Answer with HTTP 429 or 503 and the check slows down at once, as described under Pacing. It also slows down on its own when your server starts answering slowly. A 429 is never retried with another user agent. A firewall rule that answers 403 does not slow the check: the refused requests are not retried under another user agent, and the report shows the refused checks as not assessed. robots.txt or exclusion tells ShieldWave that you do not want the checks.
Contact
If a check of your site was not expected, or you want to report misuse or ask a question, write to support@shieldwave.io. Include the domain and a line from your access log with the time of the request.