Somewhere between 20% and 40% of raw web traffic is automated. If you count all of it as impressions, you are over-reporting to sponsors — and the discovery of that is much more damaging than the smaller number would have been.
What the traffic actually is
Search crawlers. Googlebot and friends. Well behaved, clearly identified, easy to exclude.
SEO and monitoring tools. Ahrefs, Semrush, uptime checkers. Identifiable, and they can be a surprising share of a small site's requests.
Scrapers. Content thieves and AI training crawlers. Varying honesty about who they are.
Malicious automation. Credential stuffing, spam bots. Usually disguised.
For a small site, the first two categories are the bulk of it.
Why it matters more for you than for a big publisher
A large site's numbers are audited and their sponsors expect a margin of error. Your sponsor is a small business owner who will, at some point, compare your impression figure against their own analytics.
If your number is 40% higher than anything they can see, the conversation that follows is not about bot traffic. It is about whether they trust you.
What to filter
Always:
- Requests with a known crawler user-agent
- Requests that never execute JavaScript — most ad servers get this for free, since the tag has to run to make a request at all
Usually:
- Known tool and monitoring user agents
- Requests from data-centre IP ranges, if your ad server can identify them
Be careful with:
- Aggressive heuristics that catch privacy-conscious humans. Someone with an unusual browser configuration is a reader, not a bot.
The JavaScript advantage
Serving ads through a client-side tag filters out most crawlers automatically, because a crawler that does not run JavaScript never makes the ad request. Server-side insertion has no such protection — every crawler hit counts as an impression.
That is a real accuracy argument for client-side ad serving, separate from the caching and rotation reasons.
What to tell sponsors
One line on your reports:
Impressions exclude known bots and crawlers, and count only ads that actually rendered in a browser.
That sentence does more for your credibility than a bigger number ever will. It also pre-empts the comparison with their analytics, which will show different figures for entirely legitimate reasons.
Clicks deserve more scrutiny than impressions
An inflated impression count is embarrassing. An inflated click count is worse, because clicks are what sponsors judge you on and what they may be paying for.
Watch for repeated clicks from the same visitor in quick succession, clicks with no corresponding impression, and click rates that jump implausibly. A sudden move from 0.6% to 4% is not good news; it is something to investigate before you report it.
The principle
Under-report rather than over-report. A sponsor who finds your figures conservative trusts everything else you tell them. One who finds them inflated re-examines every number you ever sent.