Today it is „dex Bot Search“ and „Mozilla/4.76 [ru] (X11; U; SunOS 5.7 sun4u)“ bringing my server almost to crash.
Update July 14, 2010 please read to the end of the post..
Yesterday it was „Casper Bot Search“, I blocked it succussfully with an entry on top of the .htaccess:
BrowserMatchNoCase Casper bad_bot
Order Deny,Allow
Deny from env=bad_bot
It looks like „dex Bot Search“ tries to POST to contact.php, a file or link not available on my domain, so getting lot’s of 404 errors.
In access.log I find now:
193.39.66.14 - - [01/Jul/2010:12:36:34 +0200] "POST /34//contact.php HTTP/1.1" 403 1022 "-" "dex Bot Search" 193.39.66.14 - - [01/Jul/2010:12:36:34 +0200] "POST //contact.php HTTP/1.1" 403 1022 "-" "dex Bot Search" 193.39.66.14 - - [01/Jul/2010:12:36:35 +0200] "POST /34/cms-auf-1blu-unlimited//contact.php HTTP/1.1" 403 1022 "-" "dex Bot Search" 193.39.66.14 - - [01/Jul/2010:12:36:35 +0200] "POST //contact.php HTTP/1.1" 403 1022 "-" "dex Bot Search" 193.39.66.14 - - [01/Jul/2010:12:36:35 +0200] "POST //contact.php HTTP/1.1" 403 1022 "-" "dex Bot Search" 193.39.66.14 - - [01/Jul/2010:12:36:35 +0200] "POST /34//contact.php HTTP/1.1" 403 1022 "-" "dex Bot Search" 81.176.226.100 - - [01/Jul/2010:12:46:46 +0200] "POST /?s=blog%20%20//contact.php HTTP/1.1" 403 1036 "-" "dex Bot Search" 81.176.226.100 - - [01/Jul/2010:12:46:46 +0200] "POST /?s=blog%20%20//contact.php HTTP/1.1" 403 1036 "-" "dex Bot Search" 81.176.226.100 - - [01/Jul/2010:12:46:46 +0200] "POST /?s=blog%20%20//contact.php HTTP/1.1" 403 1036 "-" "dex Bot Search" 81.176.226.100 - - [01/Jul/2010:12:46:46 +0200] "POST /?s=blog%20%20//contact.php HTTP/1.1" 403 1036 "-" "dex Bot Search"
Having the above script in .htaccess already, it was easy to block and deny this bot too, I just extended the entry like this:
BrowserMatchNoCase Casper bad_bot
BrowserMatchNoCase dex bad_bot
Order Deny,Allow
Deny from env=bad_bot
This will also block YandexBot, getting a 403 error, I do not care.
Do not ask me what is the purpose of these POST attacks, for me they just slowed down my vserver..
UPDATE:
getting POST now from „Mozilla/4.76 [ru] (X11; U; SunOS 5.7 sun4u)“
added to .htaccess like this:
BrowserMatchNoCase Casper bad_bot
BrowserMatchNoCase dex bad_bot
BrowserMatchNoCase sun4u bad_bot
Order Deny,Allow
Deny from env=bad_bot
UPDATE 02.07.2010
extended bot signatures by ’sledink‘ and ‚kangen‘ due to POST to not existing file contact.php
access.log looks like this:
92.48.70.83 - - [02/Jul/2010:19:27:06 +0200] "POST /contac.php HTTP/1.1" 404 22568 "-" "rk q kangen" 66.210.175.75 - - [02/Jul/2010:19:27:17 +0200] "POST /contact.php HTTP/1.1" 403 1022 "-" "Mozilla/4.76 [ru] (X11; U; SunOS 5.7 sun4u)" 208.115.217.194 - - [02/Jul/2010:19:27:17 +0200] "POST /?s=forum/contact.php HTTP/1.1" 403 1036 "-" "Casper Bot Search" 205.237.100.89 - - [02/Jul/2010:23:05:34 +0200] "POST /?s=blog%20%20//index.php HTTP/1.1" 403 1036 "-" "sledink Bot Search" 205.237.100.89 - - [02/Jul/2010:23:05:34 +0200] "POST /?s=blog%20%20//index.php HTTP/1.1" 403 1036 "-" "sledink Bot Search" 205.237.100.89 - - [02/Jul/2010:23:05:35 +0200] "POST /?s=blog%20%20//index.php HTTP/1.1" 403 1036 "-" "sledink Bot Search"
Update July 14, 2010
the bots try to POST now to different files, as there are:
contact.php, config.php, view.php and ‚powered‘
So by now I block and deny them in .htaccess in the following way:
RewriteEngine On
RewriteCond %{REQUEST_URI} contact\.php [NC,OR]
RewriteCond %{REQUEST_URI} config\.php [NC,OR]
RewriteCond %{REQUEST_URI} view\.php [NC,OR]
RewriteCond %{REQUEST_URI} powered [NC,OR]
RewriteCond %{QUERY_STRING} ^.*(contact\.php).*$ [NC,OR]
RewriteCond %{QUERY_STRING} ^.*(config\.php).*$ [NC,OR]
RewriteCond %{QUERY_STRING} ^.*(view\.php).*$ [NC,OR]
RewriteCond %{QUERY_STRING} ^.*(powered).*$ [NC]
RewriteRule . nouse.php [L]
all POST to the files above are redirected to a file in root folder of the attacked site.
This file nouse.php contains just the error header in php:
header("HTTP/1.1 403 Forbidden");
block search bot
5. Jul. 2010 um 20:29 Uhr
It’s much easier to just:
RewriteCond %{THE_REQUEST} contact.php [NC]
RewriteRule (.*) – [F]
This takes out all the bots at once, rejecting with a 403.
RT
6. Jul. 2010 um 00:13 Uhr
thank you, RT
after extending the deny rules for new bot signatures like:
BrowserMatchNoCase Casper bad_bot
BrowserMatchNoCase dex bad_bot
BrowserMatchNoCase sun4u bad_bot
BrowserMatchNoCase sledink bad_bot
BrowserMatchNoCase kangen bad_bot
BrowserMatchNoCase kmccrew bad_bot
I switched to:
RewriteEngine On
RewriteCond %{REQUEST_URI} contact\.php [NC,OR]
RewriteCond %{QUERY_STRING} ^.*(contact\.php).*$ [NC]
RewriteRule . abuse.php [L]
No idea if {THE_REQUEST} considers the query string following a ‚?‘ ??
I still got some 200’s like
77.79.245.90 – – [05/Jul/2010:16:56:45 +0200] „POST /?s=robots%20%20/contact.php HTTP/1.1“ 200 27261 „-“ „Casper Bot Search“
with contact.php after a /?s= search query
that’s why I added the condition
RewriteCond %{QUERY_STRING} ^.*(contact\.php).*$
I am rewriting to a ‚custom‘ 403-error file, here abuse.php in the root folder.
This file contains only:
might be a very simple and fast error message..
27. Jun. 2011 um 11:42 Uhr
I don’t see why you want to block the search bot?? isn’t it all about optimization for the search bot?
27. Jun. 2011 um 12:56 Uhr
this bot tries to POST to contact.php, means it wants to write to your site.
A search bot uses GET and not POST.