■ Block file download abusers by IP, simply and efficiently (shared hosting)

Almost 2 years ago, I received an angry email from my webhost company, saying that my daily bandwidth skyrocketed from 2 to 300 GB (!) in what it looked like a DDOS attack against zabkat website. Back then I sorted out the problem blacklisting the IP addresses responsible for the majority of downloads.

Believe it or not, 2 years down the road, I still have a residual problem, approximately 10GB/day downloads that are clearly malicious. I have tried many tricks, moved my download files round, changed .htaccess permissions, but these guys won't give up. They must be very sore with xplorer² or just evil for the sake of it — I don't know what to surmise. The fact is that somebody out there makes a considerable effort every now and then to change their IP addresses and circumvent the defences I put on my downloadable installers. What's their motive? I don't know, but I'm fed up with it.

The problem is thus: A block of IP addresses like 111.206.170.xxx (they are actually reported as abusers) request a download every 15 seconds, all day long. Going through the access logs, I block the worst IP offenders from accessing the download directories adding them to .htaccess DENY FROM 111.206.170.xxx. This solves the problem for a week or so, till our friends catch wind of the change and switch to a different set of IPs. And the cat and mouse game continues ad infinitum!

How many IP blocks do they have? They must have ran out by now. But they don't look like p0wned PCs; they come in batches of the same IP range.

My new strategy is to block direct downloads, "hiding" them in a directory that is protected by DENY FROM ALL, and only offer access through PHP, where I check the IP of the caller, and block them immediately if they are in the dodgy list. All incoming download requests are checked continuously for signs of abuse. Of course this is easier said than done!

If you google for this problem, you will see people talking about custom Apache modules like mod_evasive (abandonware), iptables, fail2ban, mod_bandwidth etc, which I don't know what they are — or can hardly afford the time to study them. "Luckily" my website is on shared hosting so they wouldn't be allowed even if I was a web expert. For poor-mans-hosting you read about database solutions, where each and every IP is stored and checked for overuse. This can be done "simply" using PHP's mySQL functions.

We like things simple here at zabkat, and a database that stores all download requests doesn't sound simple. For each bad actor, you also keep 999 genuine customer downloads. After a few months, the entire website will be filled with a huge IP database and bitcoin-mining-levels of resources would be spent searching in it. So the bad guys win again. No, we need something much simpler.

Inside xplorer² I use a compact class that automatically figures out the most frequently browsed folders. It is a very simple algorithm called FREQUENT(K), which can be implemented in about 10 lines of PHP code and takes less than 1KB of disk to store it as an array. My download check PHP file first checks for a blacklisted IP address, then it adds the new IP in the frequent table, which automatically rebalances itself using the aforementioned simple algorithm.

This method is forgiving the occasional download burst by some user who may have problems downloading. 10 downloads in a day won't score high in the frequency table, but 1000 will most certainly do! The idea is to lock out the abusers, not legitimate customers.

At present the solution is semi-automatic, since every now and then I must examine the IP frequency table and manually extract the offender IPs, and add them in the abuse array in the download PHP file. I suppose you could have a separate file for all abusive IPs, that is updated automatically each week. It is work in progress, but already much easier than downloading huge access log files, processing them with Excel and so on.

So come on you persistent hacker people, whatever will you think of next? Don't you have something better to do with your lives? <g>