4 Tips for Limiting Your Site's Bad Bot Traffic
If the aim of any website is to attract humans to become customers, then stopping any request that's not a human from accessing your site seems like a logical step.
Data from Distil Networks’ 2018 Bad Bot Report: The Year Bad Bots Went Mainstream states that only 57.8 percent of traffic on websites is human. The rest are bots. And those bots are either good (20.4 percent) or bad (21.8 percent). But how would you know who is a real human and who is a bot? The truth is, most online retailers can’t tell the difference and simply ignore the problem.
If bots can manipulate the U.S. presidential election, ultimately forcing social media tech giants to testify in front of congress, then causing havoc on a retail e-commerce site seems like easy money. And yes, those bad bots are on your website for a reason. They're not benign shoppers “just browsing” your site filling time before they go somewhere else.
So what are they doing? As an industry, e-commerce or retail websites see the full gamut of bad bot attacks. These include price and content scraping, account takeovers, credit card fraud, and gift card abuse. Whether carried out by aggressive competitors or nefarious fraudsters, the real problems they cause could harm the success — or even the continuance — of your business.
What’s worse is that e-commerce websites have the highest proportion of sophisticated bad bot traffic (22.9 percent) than any other industry.
Where are the bad bots coming from? The answer is that almost half (45.2 percent) come from the United States. Furthermore, 82.7 percent of bad bots come from data centers. You would be correct in thinking that human customers don’t typically connect to websites from a data center.
So what should you do about it? Here are a few simple recommendations to help you reduce the amount of bad bot traffic on your website:
- Block or CAPTCHA outdated user agents. Most humans use browsers that auto-update, so blocking requests from old versions is a simple way to reduce bad bot traffic.
- Block data center traffic. Humans don't access websites from data centers, so blocking known data center traffic also reduces the amount of bad bots. (At a minimum, block data centers that have a very high percentage of bad bot traffic, including Digital Ocean, OVH SAS, OVH Hosting, Choopa, GigaNET.)
- Protect every bad bot access point. Be sure to apply the same security across your website, APIs and mobile apps. Protecting only one is like locking your front door but leaving your windows and backdoor wide open.
- Investigate traffic spikes. Any anomalous increase in traffic on any page is indicative of bad bot behavior. For example, unexplained spikes of failed login attempts or gift card balance requests probably means your website is under attack.
The bad bot problem affects every industry, but every company has a unique bad bot problem. Ignore it at your peril. The success of your business just might depend on it.
Edward Roberts is director of product marketing at Distil Networks, a cybersecurity company that specializes in bot detection and mitigation, and has over 20 years of experience in technology marketing.
Edward Roberts is Director of Product Marketing at Distil Networks, a cybersecurity company that specializes in bot detection and mitigation, and has over 20 years of experience in technology marketing. Previously, he led product marketing for the Counter Security team at Juniper Networks. Before that, he ran marketing for Mykonos Software, a web security company.
