How to monitor and prevent invalid traffic (IVT)
According to the Advertiser Perceptions DSP Report for Q3 2018, 37% of surveyed agencies and marketers picked ad fraud (viewability fraud + invalid traffic) as the worst aspect of programmatic ad buying, ahead of brand safety concerns, poor inventory quality, and hidden costs.
Invalid traffic (IVT) is one of the oldest problems associated with digital advertising. Detection technology has evolved too though. Publishers and advertisers can access sophisticated tools when it comes to monitoring, analyzing, and curbing invalid traffic and activity.
Nevertheless, large scale ad fraud operations like Methbot, DrainerBot, SilentFade, and CTV fraud recently uncovered by Oracle still continue to pop up on a fairly regular basis.
For publishers, most challenges related to audience and ad serving have a limited revenue impact. With invalid traffic, there is always a risk of irrevocable account suspensions. In this post, we’ll look at what invalid traffic means, the two main types of IVT, some telltale signs of IVT to keep an eye on, and preventive measures you can take to protect your site.
What is invalid traffic?
The Trustworthy Accountability Group (TAG), a 600+ member industry association dedicated to fighting ad fraud and promoting brand safety, defines invalid traffic as:
Traffic that does not meet certain ad serving quality or completeness criteria, or otherwise does not represent legitimate ad traffic that should be included in measurement. Among the reasons why ad traffic may be deemed invalid is it is a result of non-human traffic (spiders, bots, etc.), or activity designed to produce fraudulent traffic.
In other words, IVT is any activity that doesn’t come from a real user with genuine interest. This can include accidental clicks, fraudulent clicks, incentivized clicks, advertising botnets, and more.
Invalid traffic hurts advertisers directly because it reduces their return on ad spend. According to a report by IAB and Ernst & Young, invalid traffic cost the US digital advertising industry $4.6 billion in ad spend waste and efforts made in the fight against ad fraud in 2015. A more recent study by Juniper Research projects that the industry is on track to lose $100 billion by 2023.
Since the ad-supported model of the Web is funded by advertisers, ad platforms strive to deliver better service for them. Making invalid traffic a problem for publishers as well.
How does invalid traffic hurt publishers?
There are two ways in which invalid traffic hurts publishers.
First, IVT hurts inventory reputation. Advertisers have access to sophisticated traffic quality detection tools at their disposal via DSPs and ad quality vendors. If an advertiser notices that a website they are buying inventory from demonstrates a consistent pattern of high invalid traffic, they might start bidding lower or even blacklist the publisher.
The second way IVT hurts publishers is by causing account suspensions. Google’s support forums are filled with accounts of AdSense and AdX publishers stating that their account was suspended due to invalid activity. Google uses a variety of techniques to track and prevent IVT across its network. Including automated filtering, real-time and near real-time monitoring, manual detection and reviews, botnet hunting, and finally as a last resort, account suspensions. Moreover, once deactivated, it is notoriously difficult to get an account reinstated.
We’re using Google as an example here because its ad network (AdSense), ad exchange (AdX), and ad server (GAM) are the most widely used sell-side monetization tools. Most other independent ad servers, SSPs, and DSPs also have in-house IVT detection techniques and frequently suspend accounts to keep their inventory clean and attractive to buyers.
The two types of invalid traffic
Invalid traffic is broadly classified into two categories: General Invalid Traffic (GIVT), which includes search crawlers, traffic from known data centers linked to invalid activity, and irregular patterns such as duplicate clicks. And Sophisticated Invalid Traffic (SVIT), where fraudsters make an extra effort to mask their behavior as legitimate, some examples include incentivized clicks, misleading user interface, and fraudulent browser automation.
Here’s a table that summarizes the key differences between them.
| General Invalid Traffic (GIVT) | Sophisticated invalid traffic (SIVT) | |
|---|---|---|
| Intent | Benign and generally harmless | Intentionally fraudulent |
| Detection | Easily identified using routine means of filtration, parameter checks, or industry standards | Requires advanced analytics, multi-point collaboration/coordination, and human intervention to detect and analyze |
| Examples | – Known data center traffic – Bots, spiders, and other crawlers – Activity-based filtration – Non-browser user-agent headers or unknown browsers – Prefetch or browser-prerendered traffic (unless never counted as a gross impression) | – Bots and crawlers pretending to be legitimate users – Hijacked devices and user sessions – Invalid proxy traffic – Adware and malware – Incentivized manipulation of measurements – Falsely represented sites and ads – Cookie stuffing – Manipulation or falsification of location data |
Using Google Analytics for IVT detection
While most larger publishers use third-party traffic verification vendors for curbing IVT and ensuring quality control, anyone can start monitoring their website for invalid traffic for free using some telltale signs in Google Analytics. Here are some commons signs associated with IVT:
Sudden traffic spikes
Traffic increases that happen due to a publisher’s efforts on improving content quality, search optimization, and better UX are typically gradual in nature. Let’s say your daily traffic hovers around 80,000 users. One day, it suddenly spikes to 300,000 for no clear reason. Most likely, IVT caused the spike.
Bounce rate is close to 100%
Bounce rates measure the percentage of users who exit a website without engaging in any meaningful interaction. In other words, the lower your bounce rate, the more engaged your users. A particularly high bounce rate that edges close to 100% may indicate that a significant part of your traffic during a particular period of time was invalid.
Session duration is close to 0
This is another way to identify anomalous traffic. The average session duration is calculated by dividing the total duration of all sessions (in seconds) by the number of sessions. The only way that number can be zero is if all the users exited the website under 1 second during a particular period of time. Therefore, this is only possible if a large percentage of traffic is invalid.
These metrics are found in GA under Audience > Overview.
Until recently, Google Analytics used to provide information about the ISP sending traffic to tracked websites using the dimensions ‘network domain’ and ‘service provider’. This was an easy way for publishers to filter out suspicious networks using their network ID. As of last year, Google has deprecated those dimensions with no viable replacements offered.
Preventive measures against IVT
Understand your traffic and site visitors
Tracking suspicious traffic patterns is the first and most important step to take when trying to ward off IVT. Based on what you see, you may want to remove ad codes on high-risk traffic segments, channels, and geos. Smaller publishers can use Cloudflare’s bot blocking. Larger publishers might want to try a dedicated ad fraud vendor such as IAS or Pixelate.
Don’t click ads on your own site
Sounds like a no-brainer but still worth repeating. Some ad providers have ways of detecting and eliminating self-clicks, but doing so might just as easily lead to account suspensions. If you absolutely must click your own site, whether it is for testing or if you’re just interested in the destination URL, do it in a controlled environment using Google Publisher Console.
Avoid low-quality and untrustworthy partners
Some publishers notice an increase in IVT and activity after partnering with low-quality ad networks, search engines, or directory listings. Do your due diligence when selecting new partners for traffic acquisition by reading online reviews and speaking to other publishers who have used their service. Read your ad provider’s policies about what types of traffic are allowed.
Properly configure your ads
Custom implementations can sometimes have unintended consequences on ad requests. Make sure that ad placements adhere to your ad provider’s policies and are free from programming errors. Using ad refresh? Ensure the refreshed inventory is declared as such. Set a healthy time interval for subsequent refreshes.
Know your ad provider’s policies
While there are general guidelines for preventing IVT and activity, ad networks, ad exchanges, and ad servers also enforce their own set of rules. When signing up to work with a new ad vendor, make sure you’ve read and understood all their network policies. Specifically in relation to ad placements, traffic acquisition, and traffic quality, so as to avoid account suspensions.
Conclusion
The old adage of “prevention is better than cure” explains how publishers should treat IVT better than anything else. By watching for signs of IVT, working to improve traffic quality, and avoiding ad configuration mistakes, it’s possible to keep IVT at bay before it becomes a problem.
A good place to start is pulling up your audience report in Google Analytics for the last one year and checking the graph for any abrupt traffic spikes. If you spot any anomalies—bring it up with your ad provider and see if they have any recommendations.
Traffic anomalies can provide insight into the presence of IVT, but it’s not a foolproof detection strategy. According to Statista, around 10% of global traffic is invalid. So, what you really want to ensure is that your site is not receiving IVT that is significantly more than that benchmark. Speak to some trusted ad fraud vendors. See if they are willing to let you measure the level of invalid traffic on your site during evaluation. Before you decide to use their solution.
Additional reading
- TAG Invalid Traffic Taxonomy
- AdSense Traffic Provider checklist
- Invalid Traffic Detection and Filtration Guidelines Addendum