Today it is „dex Bot Search“ and „Mozilla/4.76 [ru] (X11; U; SunOS 5.7 sun4u)“ bringing my server almost to crash.
Update July 14, 2010 please read to the end of the post..
Yesterday it was „Casper Bot Search“, I blocked it succussfully with an entry on top of the .htaccess:
BrowserMatchNoCase Casper bad_bot
Order Deny,Allow
Deny from env=bad_bot
It looks like „dex Bot Search“ tries to POST to contact.php, a file or link not available on my domain, so getting lot’s of 404 errors.
In access.log I find now:
193.39.66.14 - - [01/Jul/2010:12:36:34 +0200] "POST /34//contact.php HTTP/1.1" 403 1022 "-" "dex Bot Search" 193.39.66.14 - - [01/Jul/2010:12:36:34 +0200] "POST //contact.php HTTP/1.1" 403 1022 "-" "dex Bot Search" 193.39.66.14 - - [01/Jul/2010:12:36:35 +0200] "POST /34/cms-auf-1blu-unlimited//contact.php HTTP/1.1" 403 1022 "-" "dex Bot Search" 193.39.66.14 - - [01/Jul/2010:12:36:35 +0200] "POST //contact.php HTTP/1.1" 403 1022 "-" "dex Bot Search" 193.39.66.14 - - [01/Jul/2010:12:36:35 +0200] "POST //contact.php HTTP/1.1" 403 1022 "-" "dex Bot Search" 193.39.66.14 - - [01/Jul/2010:12:36:35 +0200] "POST /34//contact.php HTTP/1.1" 403 1022 "-" "dex Bot Search" 81.176.226.100 - - [01/Jul/2010:12:46:46 +0200] "POST /?s=blog%20%20//contact.php HTTP/1.1" 403 1036 "-" "dex Bot Search" 81.176.226.100 - - [01/Jul/2010:12:46:46 +0200] "POST /?s=blog%20%20//contact.php HTTP/1.1" 403 1036 "-" "dex Bot Search" 81.176.226.100 - - [01/Jul/2010:12:46:46 +0200] "POST /?s=blog%20%20//contact.php HTTP/1.1" 403 1036 "-" "dex Bot Search" 81.176.226.100 - - [01/Jul/2010:12:46:46 +0200] "POST /?s=blog%20%20//contact.php HTTP/1.1" 403 1036 "-" "dex Bot Search"
Having the above script in .htaccess already, it was easy to block and deny this bot too, I just extended the entry like this:
BrowserMatchNoCase Casper bad_bot
BrowserMatchNoCase dex bad_bot
Order Deny,Allow
Deny from env=bad_bot
This will also block YandexBot, getting a 403 error, I do not care.
Do not ask me what is the purpose of these POST attacks, for me they just slowed down my vserver..
UPDATE:
getting POST now from „Mozilla/4.76 [ru] (X11; U; SunOS 5.7 sun4u)“
added to .htaccess like this:
BrowserMatchNoCase Casper bad_bot
BrowserMatchNoCase dex bad_bot
BrowserMatchNoCase sun4u bad_bot
Order Deny,Allow
Deny from env=bad_bot
UPDATE 02.07.2010
extended bot signatures by ’sledink‘ and ‚kangen‘ due to POST to not existing file contact.php
access.log looks like this:
92.48.70.83 - - [02/Jul/2010:19:27:06 +0200] "POST /contac.php HTTP/1.1" 404 22568 "-" "rk q kangen" 66.210.175.75 - - [02/Jul/2010:19:27:17 +0200] "POST /contact.php HTTP/1.1" 403 1022 "-" "Mozilla/4.76 [ru] (X11; U; SunOS 5.7 sun4u)" 208.115.217.194 - - [02/Jul/2010:19:27:17 +0200] "POST /?s=forum/contact.php HTTP/1.1" 403 1036 "-" "Casper Bot Search" 205.237.100.89 - - [02/Jul/2010:23:05:34 +0200] "POST /?s=blog%20%20//index.php HTTP/1.1" 403 1036 "-" "sledink Bot Search" 205.237.100.89 - - [02/Jul/2010:23:05:34 +0200] "POST /?s=blog%20%20//index.php HTTP/1.1" 403 1036 "-" "sledink Bot Search" 205.237.100.89 - - [02/Jul/2010:23:05:35 +0200] "POST /?s=blog%20%20//index.php HTTP/1.1" 403 1036 "-" "sledink Bot Search"
Update July 14, 2010
the bots try to POST now to different files, as there are:
contact.php, config.php, view.php and ‚powered‘
So by now I block and deny them in .htaccess in the following way:
RewriteEngine On
RewriteCond %{REQUEST_URI} contact\.php [NC,OR]
RewriteCond %{REQUEST_URI} config\.php [NC,OR]
RewriteCond %{REQUEST_URI} view\.php [NC,OR]
RewriteCond %{REQUEST_URI} powered [NC,OR]
RewriteCond %{QUERY_STRING} ^.*(contact\.php).*$ [NC,OR]
RewriteCond %{QUERY_STRING} ^.*(config\.php).*$ [NC,OR]
RewriteCond %{QUERY_STRING} ^.*(view\.php).*$ [NC,OR]
RewriteCond %{QUERY_STRING} ^.*(powered).*$ [NC]
RewriteRule . nouse.php [L]
all POST to the files above are redirected to a file in root folder of the attacked site.
This file nouse.php contains just the error header in php:
header("HTTP/1.1 403 Forbidden");
wichtig:
- https://yandex ru/
block search bot
It’s much easier to just:
RewriteCond %{THE_REQUEST} contact.php [NC]
RewriteRule (.*) – [F]
This takes out all the bots at once, rejecting with a 403.
RT
thank you, RT
after extending the deny rules for new bot signatures like:
BrowserMatchNoCase Casper bad_bot
BrowserMatchNoCase dex bad_bot
BrowserMatchNoCase sun4u bad_bot
BrowserMatchNoCase sledink bad_bot
BrowserMatchNoCase kangen bad_bot
BrowserMatchNoCase kmccrew bad_bot
I switched to:
RewriteEngine On
RewriteCond %{REQUEST_URI} contact\.php [NC,OR]
RewriteCond %{QUERY_STRING} ^.*(contact\.php).*$ [NC]
RewriteRule . abuse.php [L]
No idea if {THE_REQUEST} considers the query string following a ‚?‘ ??
I still got some 200’s like
77.79.245.90 – – [05/Jul/2010:16:56:45 +0200] „POST /?s=robots%20%20/contact.php HTTP/1.1“ 200 27261 „-“ „Casper Bot Search“
with contact.php after a /?s= search query
that’s why I added the condition
RewriteCond %{QUERY_STRING} ^.*(contact\.php).*$
I am rewriting to a ‚custom‘ 403-error file, here abuse.php in the root folder.
This file contains only:
might be a very simple and fast error message..
I don’t see why you want to block the search bot?? isn’t it all about optimization for the search bot?
this bot tries to POST to contact.php, means it wants to write to your site.
A search bot uses GET and not POST.