Today it is “dex Bot Search” and “Mozilla/4.76 [ru] (X11; U; SunOS 5.7 sun4u)” bringing my server almost to crash.
Update July 14, 2010 please read to the end of the post..
Yesterday it was “Casper Bot Search”, I blocked it succussfully with an entry on top of the .htaccess:
BrowserMatchNoCase Casper bad_bot
Order Deny,Allow
Deny from env=bad_bot
It looks like “dex Bot Search” tries to POST to contact.php, a file or link not available on my domain, so getting lot’s of 404 errors.
In access.log I find now:
193.39.66.14 - - [01/Jul/2010:12:36:34 +0200] "POST /34//contact.php HTTP/1.1" 403 1022 "-" "dex Bot Search" 193.39.66.14 - - [01/Jul/2010:12:36:34 +0200] "POST //contact.php HTTP/1.1" 403 1022 "-" "dex Bot Search" 193.39.66.14 - - [01/Jul/2010:12:36:35 +0200] "POST /34/cms-auf-1blu-unlimited//contact.php HTTP/1.1" 403 1022 "-" "dex Bot Search" 193.39.66.14 - - [01/Jul/2010:12:36:35 +0200] "POST //contact.php HTTP/1.1" 403 1022 "-" "dex Bot Search" 193.39.66.14 - - [01/Jul/2010:12:36:35 +0200] "POST //contact.php HTTP/1.1" 403 1022 "-" "dex Bot Search" 193.39.66.14 - - [01/Jul/2010:12:36:35 +0200] "POST /34//contact.php HTTP/1.1" 403 1022 "-" "dex Bot Search" 81.176.226.100 - - [01/Jul/2010:12:46:46 +0200] "POST /?s=blog%20%20//contact.php HTTP/1.1" 403 1036 "-" "dex Bot Search" 81.176.226.100 - - [01/Jul/2010:12:46:46 +0200] "POST /?s=blog%20%20//contact.php HTTP/1.1" 403 1036 "-" "dex Bot Search" 81.176.226.100 - - [01/Jul/2010:12:46:46 +0200] "POST /?s=blog%20%20//contact.php HTTP/1.1" 403 1036 "-" "dex Bot Search" 81.176.226.100 - - [01/Jul/2010:12:46:46 +0200] "POST /?s=blog%20%20//contact.php HTTP/1.1" 403 1036 "-" "dex Bot Search"
Having the above script in .htaccess already, it was easy to block and deny this bot too, I just extended the entry like this:
BrowserMatchNoCase Casper bad_bot
BrowserMatchNoCase dex bad_bot
Order Deny,Allow
Deny from env=bad_bot
This will also block YandexBot, getting a 403 error, I do not care.
Do not ask me what is the purpose of these POST attacks, for me they just slowed down my vserver..
UPDATE:
getting POST now from “Mozilla/4.76 [ru] (X11; U; SunOS 5.7 sun4u)”
added to .htaccess like this:
BrowserMatchNoCase Casper bad_bot
BrowserMatchNoCase dex bad_bot
BrowserMatchNoCase sun4u bad_bot
Order Deny,Allow
Deny from env=bad_bot
UPDATE 02.07.2010
extended bot signatures by ‘sledink’ and ‘kangen’ due to POST to not existing file contact.php
access.log looks like this:
92.48.70.83 - - [02/Jul/2010:19:27:06 +0200] "POST /contac.php HTTP/1.1" 404 22568 "-" "rk q kangen" 66.210.175.75 - - [02/Jul/2010:19:27:17 +0200] "POST /contact.php HTTP/1.1" 403 1022 "-" "Mozilla/4.76 [ru] (X11; U; SunOS 5.7 sun4u)" 208.115.217.194 - - [02/Jul/2010:19:27:17 +0200] "POST /?s=forum/contact.php HTTP/1.1" 403 1036 "-" "Casper Bot Search" 205.237.100.89 - - [02/Jul/2010:23:05:34 +0200] "POST /?s=blog%20%20//index.php HTTP/1.1" 403 1036 "-" "sledink Bot Search" 205.237.100.89 - - [02/Jul/2010:23:05:34 +0200] "POST /?s=blog%20%20//index.php HTTP/1.1" 403 1036 "-" "sledink Bot Search" 205.237.100.89 - - [02/Jul/2010:23:05:35 +0200] "POST /?s=blog%20%20//index.php HTTP/1.1" 403 1036 "-" "sledink Bot Search"
Update July 14, 2010
the bots try to POST now to different files, as there are:
contact.php, config.php, view.php and ‘powered’
So by now I block and deny them in .htaccess in the following way:
RewriteEngine On
RewriteCond %{REQUEST_URI} contact\.php [NC,OR]
RewriteCond %{REQUEST_URI} config\.php [NC,OR]
RewriteCond %{REQUEST_URI} view\.php [NC,OR]
RewriteCond %{REQUEST_URI} powered [NC,OR]
RewriteCond %{QUERY_STRING} ^.*(contact\.php).*$ [NC,OR]
RewriteCond %{QUERY_STRING} ^.*(config\.php).*$ [NC,OR]
RewriteCond %{QUERY_STRING} ^.*(view\.php).*$ [NC,OR]
RewriteCond %{QUERY_STRING} ^.*(powered).*$ [NC]
RewriteRule . nouse.php [L]
all POST to the files above are redirected to a file in root folder of the attacked site.
This file nouse.php contains just the error header in php:
header("HTTP/1.1 403 Forbidden");
block search bot How to Block a Bot User Agent. Many search engines use a technique called spidering which means they send a user agent to visit your website and follow links to various pages. There are many cases where Webmasters do not want to allow search engine bots to visit their websites. In this case they can block the Search Engine bots to How can i use ros.txt to all s Technology Internet Talk about backend web development techniques and technologies. Where they are now and where they are A (Japanese: Burobo) is a type of ro seen in LaRousse City in Destiny Tags: block, bulbapedia, community, driven, Learn how to s from accessing your site. Find more useful articles in the SiteGround Knowledge Base a great resource for many Web Hosting related topics.
5. July 2010 um 20:29 Uhr
It’s much easier to just:
RewriteCond %{THE_REQUEST} contact.php [NC]
RewriteRule (.*) – [F]
This takes out all the bots at once, rejecting with a 403.
RT
6. July 2010 um 00:13 Uhr
thank you, RT
after extending the deny rules for new bot signatures like:
BrowserMatchNoCase Casper bad_bot
BrowserMatchNoCase dex bad_bot
BrowserMatchNoCase sun4u bad_bot
BrowserMatchNoCase sledink bad_bot
BrowserMatchNoCase kangen bad_bot
BrowserMatchNoCase kmccrew bad_bot
I switched to:
RewriteEngine On
RewriteCond %{REQUEST_URI} contact\.php [NC,OR]
RewriteCond %{QUERY_STRING} ^.*(contact\.php).*$ [NC]
RewriteRule . abuse.php [L]
No idea if {THE_REQUEST} considers the query string following a ‘?’ ??
I still got some 200′s like
77.79.245.90 – - [05/Jul/2010:16:56:45 +0200] “POST /?s=robots%20%20/contact.php HTTP/1.1″ 200 27261 “-” “Casper Bot Search”
with contact.php after a /?s= search query
that’s why I added the condition
RewriteCond %{QUERY_STRING} ^.*(contact\.php).*$
I am rewriting to a ‘custom’ 403-error file, here abuse.php in the root folder.
This file contains only:
< ?php
header("HTTP/1.1 403 Forbidden");
?>
might be a very simple and fast error message..
27. June 2011 um 11:42 Uhr
I don’t see why you want to block the search bot?? isn’t it all about optimization for the search bot?
27. June 2011 um 12:56 Uhr
this bot tries to POST to contact.php, means it wants to write to your site.
A search bot uses GET and not POST.