beebright -

Battling bad bots – how to escape the great data scrape

The battle has been going on for a while, but there are signs that there is no let-up in the need to fight bad bots, says Spencer Young, EMEA RVP at Imperva

The battle over data-scraping bad bots has been rumbling on for some time now, but with LinkedIn dealing with the fallout from a third major incident in four months, there are signs that the situation is rapidly worsening for companies.

Advances in bad bot technology mean operators are now able to get a wealth of personal information on customers – including their home addresses, phone numbers and email addresses – purely from publicly available pages. While these incidents don’t constitute a data breach or hack, for customers whose data ends up being abused in scams, this may be a distinction without a difference.

Data scraping doesn’t just affect social media companies. Bad bots are ripping through virtually every part of the internet – in particular, industries such as telcos, sports and news. In fact, bad bots now account for nearly one-third of all web traffic. For the channel, this is an opportunity to talk to customers about how to deal with this digital scourge – in particular about bad bot threats evolving, what steps need to be taken, and how to achieve this without creating disruption for legitimate customers.

Today, bad bots are operating ceaselessly around the clock, with operators using them for everything from sensitive data scraping to account takeover attacks. Even simple bots can choke server bandwidth and create unnecessary costs for internet service providers (ISPs).

Unfortunately, this breed of bad bots is in the minority, with 57% of bad bot traffic coming from advanced persistent bots (APBs), which are more sophisticated, mimic human behaviour and use a range of methods to evade detection. The latest research indicates that 34% of all account takeover attempts now come from malicious bots, and the average website is under attack 16% of the time.

When it comes to data scraping, although bad bots aren’t generally getting data such as passwords or credit card numbers used in credential stuffing attacks, it is important to highlight that they are still harvesting a goldmine of personal information that can be used for all sorts of scamming and phishing attacks. This can include anything from tricking customers into fraudulent transactions to loss of IP and other sensitive data – not to mention huge reputational damage and regulatory fines.

And data scraping is just the tip of the iceberg. Because bad bots excel at repetitive tasks that would take humans a significantly long time to complete, they can be used for any number of malicious purposes, including competitive data mining, personal and financial data harvesting, brute-force login, digital ad fraud, denial of service, denial of inventory, spam, transaction fraud, and more.

Holding back the flood

The UK is one of the most attractive targets for bot operators, with nearly one in 10 attacks last year aimed at British companies. Dealing with this flood of inbound bad bot traffic every day, without affecting legitimate customers, is a huge challenge and requires a comprehensive, cohesive solution. The days when a few small website tweaks would do the trick are long gone.

Today, in order to successfully identify and filter out bad bot traffic, businesses need to secure all access points – including websites, mobile apps and application programming interfaces (APIs) – with tools that can analyse traffic in real time and offer a range of response options depending on the type of bad bot detected.  

Beyond that, because bad bot activity is evolving so quickly, companies need help to keep up with the latest threats and defence requirements, which is where the channel can act as a partner and expert consultant to help guide business strategy and investment.  

Bad bot activity is growing at a record-breaking pace and with massive data dumps from scraping bots hitting the headlines constantly, it’s an issue that is only going to become more salient for businesses.

But for many organisations, understanding how swiftly bad bots are evolving and the damage they are doing is hard enough, never mind finding effective ways to mitigate and prevent them. Right now, they need trusted partners who can help illustrate the threats they face and find comprehensive, cohesive solutions that provide proper protection.

Read more on Threat Management Solutions and Services