Forum

Since a few days there is a spambot visiting my website. It generates a lot of network traffic and it makes the website overall very very slow and bad responsive.

Is there an easy way to block such visits via Hiawatha?

Hope someone can help me.

It best way to do that is to block the IP of the spambot in your firewall.

A webserver has no knowledge about the content of the HTTP traffic. So, it cannot see whether a HTTP request contains spam or not. You should detect that in your webapplication.

It's not really spamming because that's impossible for the bot. to leave a message. However it keeps crawling and accessing a lot of pages. I'll see if I can find out if it's a single IP address or not. But I thought Hiawatha had also functionality to block such visits.

If the bot is just visiting page, it's hard to automaticly detect that the client is a spambot. Serving pages is what the webserver is there for.... However, if it's visiting pages at a much faster rate than a normal person would do, that's something Hiawatha can detect and block. Take a look at the BanOnFlooding setting in the manual pages. Be careful with that setting, because if set incorrectly you'll block fast browsing normal visitors. Can you post a single accesslog line from the spambot? Maybe there is something in it we can use to detect and block the spambot.

I ran some extra logging and I've found some hosts to block. What is the best way of doing this? By firewall, /etc/hosts.allow, PHP-script or as a rewrite-condition?

If you want to block IP addresses, you want to spend as less time processing their network traffic as possible. The best way to do this is via the firewall.