go41

Deny dex Bot Search POST contact.php HTTP 1.1 in htaccess

von Joern am 1. Jul. 2010 | 4 Kommentare

Today it is „dex Bot Search“ and „Mozilla/4.76 [ru] (X11; U; SunOS 5.7 sun4u)“ bringing my server almost to crash.

Update July 14, 2010 please read to the end of the post..

Yesterday it was „Casper Bot Search“, I blocked it succussfully with an entry on top of the .htaccess:

BrowserMatchNoCase Casper bad_bot
Order Deny,Allow
Deny from env=bad_bot

It looks like „dex Bot Search“ tries to POST to contact.php, a file or link not available on my domain, so getting lot’s of 404 errors.

In access.log I find now:

193.39.66.14 - - [01/Jul/2010:12:36:34 +0200]
"POST /34//contact.php HTTP/1.1" 403 1022 "-" "dex Bot Search"
193.39.66.14 - - [01/Jul/2010:12:36:34 +0200]
"POST //contact.php HTTP/1.1" 403 1022 "-" "dex Bot Search"
193.39.66.14 - - [01/Jul/2010:12:36:35 +0200]
"POST /34/cms-auf-1blu-unlimited//contact.php HTTP/1.1" 403 1022 "-" "dex Bot Search"
193.39.66.14 - - [01/Jul/2010:12:36:35 +0200]
"POST //contact.php HTTP/1.1" 403 1022 "-" "dex Bot Search"
193.39.66.14 - - [01/Jul/2010:12:36:35 +0200]
"POST //contact.php HTTP/1.1" 403 1022 "-" "dex Bot Search"
193.39.66.14 - - [01/Jul/2010:12:36:35 +0200]
"POST /34//contact.php HTTP/1.1" 403 1022 "-" "dex Bot Search"
81.176.226.100 - - [01/Jul/2010:12:46:46 +0200]
"POST /?s=blog%20%20//contact.php HTTP/1.1" 403 1036 "-" "dex Bot Search"
81.176.226.100 - - [01/Jul/2010:12:46:46 +0200]
"POST /?s=blog%20%20//contact.php HTTP/1.1" 403 1036 "-" "dex Bot Search"
81.176.226.100 - - [01/Jul/2010:12:46:46 +0200]
"POST /?s=blog%20%20//contact.php HTTP/1.1" 403 1036 "-" "dex Bot Search"
81.176.226.100 - - [01/Jul/2010:12:46:46 +0200]
"POST /?s=blog%20%20//contact.php HTTP/1.1" 403 1036 "-" "dex Bot Search"

Having the above script in .htaccess already, it was easy to block and deny this bot too, I just extended the entry like this:

BrowserMatchNoCase Casper bad_bot
BrowserMatchNoCase dex bad_bot
Order Deny,Allow
Deny from env=bad_bot

This will also block YandexBot, getting a 403 error, I do not care.

Do not ask me what is the purpose of these POST attacks, for me they just slowed down my vserver..

UPDATE:
getting POST now from „Mozilla/4.76 [ru] (X11; U; SunOS 5.7 sun4u)“

added to .htaccess like this:

BrowserMatchNoCase Casper bad_bot
BrowserMatchNoCase dex bad_bot
BrowserMatchNoCase sun4u bad_bot
Order Deny,Allow
Deny from env=bad_bot

UPDATE 02.07.2010
extended bot signatures by ’sledink‘ and ‚kangen‘ due to POST to not existing file contact.php

access.log looks like this:

92.48.70.83 - - [02/Jul/2010:19:27:06 +0200]
"POST /contac.php HTTP/1.1" 404 22568 "-" "rk q kangen"
66.210.175.75 - - [02/Jul/2010:19:27:17 +0200]
"POST /contact.php HTTP/1.1" 403 1022 "-" "Mozilla/4.76 [ru] (X11; U; SunOS 5.7 sun4u)"
208.115.217.194 - - [02/Jul/2010:19:27:17 +0200]
"POST /?s=forum/contact.php HTTP/1.1" 403 1036 "-" "Casper Bot Search"
205.237.100.89 - - [02/Jul/2010:23:05:34 +0200]
"POST /?s=blog%20%20//index.php HTTP/1.1" 403 1036 "-" "sledink Bot Search"
205.237.100.89 - - [02/Jul/2010:23:05:34 +0200]
"POST /?s=blog%20%20//index.php HTTP/1.1" 403 1036 "-" "sledink Bot Search"
205.237.100.89 - - [02/Jul/2010:23:05:35 +0200]
"POST /?s=blog%20%20//index.php HTTP/1.1" 403 1036 "-" "sledink Bot Search"

Update July 14, 2010
the bots try to POST now to different files, as there are:
contact.php, config.php, view.php and ‚powered‘

So by now I block and deny them in .htaccess in the following way:

RewriteEngine On
RewriteCond %{REQUEST_URI} contact\.php [NC,OR]
RewriteCond %{REQUEST_URI} config\.php [NC,OR]
RewriteCond %{REQUEST_URI} view\.php [NC,OR]
RewriteCond %{REQUEST_URI} powered [NC,OR]
RewriteCond %{QUERY_STRING} ^.*(contact\.php).*$ [NC,OR]
RewriteCond %{QUERY_STRING} ^.*(config\.php).*$ [NC,OR]
RewriteCond %{QUERY_STRING} ^.*(view\.php).*$ [NC,OR]
RewriteCond %{QUERY_STRING} ^.*(powered).*$ [NC]
RewriteRule . nouse.php [L]

all POST to the files above are redirected to a file in root folder of the attacked site.
This file nouse.php contains just the error header in php:

header("HTTP/1.1 403 Forbidden");

block search bot In order to block all bots from accessing your site, you should create a robots.txt file with the following content: User-agent: * Disallow: / To block all Googlebot is Google's web crawling bot including using robots.txt to block access to Googlebot and all respectable search engine bots will To prevent all s from indexing a page: To prevent just Google: If you have control over it, you probably want to add nofollow to links that point Block search indexing with 'noindex' You can prevent a page from appearing in Google by including a noindex meta tag in the page's HTML code, or by returning a 'noindex' header in the Tags: block, search, indexing, I'm trying to all s/crawlers/spiders for a special directory. How can I do that with htaccess? I ed a bit and found a solution by

Autor:

Du findest mich auch auf Google+, Twitter und Facebook!

4 Kommentare

  1. It’s much easier to just:

    RewriteCond %{THE_REQUEST} contact.php [NC]
    RewriteRule (.*) – [F]

    This takes out all the bots at once, rejecting with a 403.

    RT

  2. thank you, RT
    after extending the deny rules for new bot signatures like:
    BrowserMatchNoCase Casper bad_bot
    BrowserMatchNoCase dex bad_bot
    BrowserMatchNoCase sun4u bad_bot
    BrowserMatchNoCase sledink bad_bot
    BrowserMatchNoCase kangen bad_bot
    BrowserMatchNoCase kmccrew bad_bot

    I switched to:


    RewriteEngine On
    RewriteCond %{REQUEST_URI} contact\.php [NC,OR]
    RewriteCond %{QUERY_STRING} ^.*(contact\.php).*$ [NC]
    RewriteRule . abuse.php [L]

    No idea if {THE_REQUEST} considers the query string following a ‚?‘ ??
    I still got some 200’s like
    77.79.245.90 – – [05/Jul/2010:16:56:45 +0200] „POST /?s=robots%20%20/contact.php HTTP/1.1“ 200 27261 „-“ „Casper Bot Search“
    with contact.php after a /?s= search query

    that’s why I added the condition
    RewriteCond %{QUERY_STRING} ^.*(contact\.php).*$

    I am rewriting to a ‚custom‘ 403-error file, here abuse.php in the root folder.
    This file contains only:
    < ?php header("HTTP/1.1 403 Forbidden"); ?>

    might be a very simple and fast error message..

  3. I don’t see why you want to block the search bot?? isn’t it all about optimization for the search bot?

    • this bot tries to POST to contact.php, means it wants to write to your site.
      A search bot uses GET and not POST.

Schreibe einen Kommentar

Pflichtfelder sind mit * markiert.