What publishers need to know about detecting bot traffic

As the ad tech industry grows, so does the fraudulent efforts of cybercriminals. Invalid traffic—also known as bot traffic—isn’t really new, but it has become a drain on publishers’ efforts and their revenue as it’s costing the industry roughly $65 billion each year.

If you’ve been in the ad publishing game for a while, you’ve probably had at least one payment withheld or one account put on hold by a monetization partner due to bot traffic. This ultimately leads to ad quality concerns.

The programmatic advertising world provides an “open” ecosystem for all ad tech players to move through. As such, there’s no real filtering system on the behalf of any programmatic party, which allows “bad actors” (cybercriminals) to enter into the ecosystem and negatively interfere with publishers and advertisers.

As a publisher going up against bot traffic, offense is your best defense. That means you need to commit to preventing ad fraud through bot traffic at all costs. But what exactly is bot traffic and how does one detect it?

What is bot traffic and how does it affect publishers?

Bot traffic is characterized as any non-human traffic on a website or app. Not all bot traffic is bad, as bots only act on what they’re programmed to do and nothing else.

Generally speaking, a bot is a piece of code programmed to perform certain tasks that run within an algorithm. The algorithm is what turns these tasks into repetitive functions, such as collecting large amounts of data in a fraction of a second. Therefore, if a bot is programmed to crawl websites for the purpose of data scraping or launching DDoS (Denial of Service) attacks, then it’s a bad bot.

It was initially thought that bot traffic only affected advertisers. As it turns out, the fake ad interactions that bot traffic generates are just as problematic for publishers. Here’s how:

  • It contributes to revenue theft. There are certain ad fraud techniques, such as domain spoofing, that steal revenue from your site traffic. When a cybercriminal spoofs your domain, it tricks users into thinking they’re on your website and takes advantage of your revenue each time there’s any ad engagement.
  • Deflation. Some bots generate fake ad inventory, which increases the amount of available inventory on the market. In turn, this drives up the supply which drives down the prices for all ad inventory.
  • You could end up blocklisted. SSPs and ad exchanges don’t like seeing invalid activity, such as fake clicks or impressions coming from a publisher’s website. When this happens, regardless of your involvement, they may blocklist your website to protect themselves. Once you’re blocklisted, you won’t be able to sell any remnant impressions to anyone.

What are the different types of bot traffic?

When we talk about bot traffic, we’re talking about one of the many types of ad fraud. For example, there’s domain spoofing, cookie stuffing, ad injections, etc. However, bot traffic itself can be broken down into several types, including the following:

Denial of Service (DDoS) Network Bots

DDoS bots are some of the oldest and most malicious bots out there. These bots are a type of software that gets implemented on an unknowing individual’s computer with the purpose of taking down a specific website or server.

Essentially, DDoS bots are launched to cause downtime by sending a large amount of random traffic to a certain network. While these bots may not be stealing information, the sheer surge in fake traffic causes the websites to crash.

Web Scraper Bots

Web scraper bots “scrape” webpages for private data, including contact information. They will also steal text and images from websites to re-use them elsewhere without permission. This is in order to generate their own ad revenue.

Click Fraud Bots

Click fraud bots are especially malicious as they target paid advertisements. Unlike the bots that generate unwanted traffic, these bots specifically engage in ad fraud.

This is the type of non-human traffic that drives up the clicks on paid ads, costing advertisers billions each year—which ends up costing publishers billions while also putting them at risk of being blocklisted.

Vulnerability Scanner Bots

Vulnerability bots exist to crawl as many websites as possible for vulnerabilities to report them back to their creators. However, in the instance of ad fraud, the information they pick up gets sent to a third-party vendor that either sells the information or uses it to hack the website in question.

Spam Bots

Spam bots work to leave messages on a website’s comment section. To avoid being outed, they mimic human behavior, i.e., using emojis. However, they’re not just commenting—they’re spreading spam across websites that can scrape contact information, create fake user accounts, and even operate primary accounts.

Keep in mind that bots are programmed to do what their creator wants them to do.

In ad fraud instances, they’re typically programmed to act like any other website visitor to generate page views, ad clicks, and even ad impressions. Cybercriminals use this to their advantage and do everything from creating their own website to redirect user traffic to interfering with a publisher’s website to steal their revenue.

How to detect bot traffic

Now that we understand what bot traffic is and the various ways it can be implemented, let’s look at how you can detect bot traffic on your site.

Here are the most common signs of bot traffic problems:

  • Unusually high pageviews. If you notice a sudden high spike in page views, it’s a telltale sign that you’ve got a bot “clicking” through your pages.
  • Unusually high bounce rate. Bounce rates fluctuate as they indicate the number of users who leave a specific page before viewing or clicking anything. A sharp increase in your bounce rate likely means that bots are targeting a single page on your site.
  • Very high or low session durations. The amount of time users spend on a page is a metric that stays pretty steady. However, an unexpected increase or decrease in this metric could mean that you’ve got bots perusing your site very slowly or they’re clicking through your pages at a much more rapid pace than a human user could or would.
  • Suspicious conversions. If you notice an increase in certain conversions—like new accounts—but see senseless names or email addresses, you’ve got form-filling bots and/or spam bots on your hands.
  • Sudden traffic spike from an unexpected referral source. If you see a sudden spike in users coming from a particular region or country that’s unlikely to have that many people fluent in your native language, then you’ve got a bot problem.

Now that you know where to look, learn 7 different ways to prevent bot traffic from affecting your website. Don’t forget to also look at our preventive measures against invalid traffic (IVT) to protect your content from ad fraud.

In summary

Bot traffic is not only detrimental to advertisers, but to publishers as well. These fake ad interactions can lead to ad revenue theft, price deflation, and result in you being blocklisted by your SSPs. We’ve identified 5 types of bots that you should be wary of: DDoS, Web Scraper, Click Fraud, Vulnerability, and Spam bots.

All of which can negatively impact your ad revenue. The good news is that there are a few ways for you to immediately spot any instances of bot traffic, such as:

  • An unusually high pageviews
  • An unusually high bounce rate
  • Very high or low session durations
  • Suspicious conversions
  • A sudden traffic spike from an unexpected referral source

Bots will always find new ways to commit fraudulent activity across the Web. That’s why it’s imperative that you know how to detect bot traffic so you can prevent it accordingly.

While you're here...

Did you know that the average publisher loses 10-40% of their revenue to adblocking? What you may not know is that adblocking has largely shifted to ad-filtering, with over 250M users allowing a safer, less interruptive ad experience to be served to them—in turn supporting their favorite sites and creators.

Blockthrough's award-winning technology plugs into publishers' header bidding wrapper and ad server to scan ad creatives for compliance with the Acceptable Ads Standard to activate this "hidden" audience and generate incremental revenue, while respecting the choice and experience of ad-filtering users.

Want to learn more?