In light of religious terrorist attacks in Paris, please see Category:blasphemy!

Freedom Porn:free technology/BotTrap

From Freedom Porn
Jump to: navigation, search

A lot of robots are out there on the internets, most of them are quite friendly, and most will listen to you if you talk to them via robots.txt, but there are some which are just rude. The rude ones are often ones who want to do something bad to you, like scan your site for e-mail addresses and then send you information about penis enlargement, the rest are Nigerian princes.

So what you can do is to make a trap for them. You make a link that no human will ever see, and you put a rule in robots.txt saying that no robot should go there. This way anybody who goes to that link is not wanted on the site.

Licence

Available under the terms of GNU General Public License 3.0

Produced in the year 2010.
GPLv3
For more information please see Freedom Porn:copyrights.


Warning

Do not go and try to find where the trap on this site is, you will be blocked and a very rude letter will be written to your ISP.

Features

Installation

robots.txt

Add the following to your robots.txt.

User-agent:  *
Disallow: /hidden.php
Disallow: hidden.php

It is probably a good idea to check that good robots understand what you mean, use a robots checker for that.

Also you may want to wait a few days before doing other steps, in case a good robot doesn't check robots.txt every day.

hidden.php

You can change the name of this file, in fact that is good if you do, so that robots don't know what you are doing. Make sure that you change example.com to your domain name and WebMaster to your email. Possibly will need to change $blacklist, and $visits to include the full path to the files.
<html>
<head><title>You have been banned</title></head>
<body>
<h1>You have been banned!</h1>
<p>You have entered a <a href='http://www.kloth.net/internet/bottrap.php'>bot trap</a>, you will be unable to browse this site now. If you think that this was done in error, please e-mail &ldquo;WebMaster (a) example.com&rdquo; (with no spaces or quotes) and explain your situation.</p>
<?php
  /* whitelist: end processing end exit */
//  if (preg_match("/10\.22\.33\.44/",$_SERVER['REMOTE_ADDR'])) { exit; }
//  if (preg_match("Super Tool",$_SERVER['HTTP_USER_AGENT'])) { exit; }
  /* end of whitelist */
  /* scan the blacklist.dat file for addresses of SPAM robots
 to prevent filling it up with duplicates */
  $blacklist = "./blacklist.dat";
  $visits = './blacklist-returns.dat';
 
  $fp = fopen($blacklist, "r") or die ("Error opening file ... <br>\n");
  $seen=false;
  while ($line = fgets($fp,255)) {
    $u = explode(" ",$line);
    $u0 = $u[0];
    if (preg_match("/{$u0}/",$_SERVER['REMOTE_ADDR'])) $seen=true;
  }
  fclose($fp);
 
  $tmestamp = time();
  $datum = date("Y-m-d (D) H:i:s",$tmestamp);
 
  if (!$seen)
  { /* we just see a new bad bot not yet listed ! */
      // append bad bot address data to blacklist log file:
    $fp = fopen($blacklist,'a+');
    fwrite($fp,"{$_SERVER['REMOTE_ADDR']} - - [{$datum}] \"{$_SERVER['REQUEST_METHOD']} {$_SERVER['REQUEST_URI']} {$_SERVER['SERVER_PROTOCOL']}\" {$_SERVER['HTTP_REFERER']} {$_SERVER['HTTP_USER_AGENT']}\n");
    fclose($fp);
 
      // send a mail to hostmaster
    $from = "badbot-watch@example.com";
    $to = "WebMaster@example.com";
    $subject = "alert: bad robot";
    $msg = "A bad robot hit {$_SERVER['REQUEST_URI']} {$datum}\n";
    $msg .= "address is {$_SERVER['REMOTE_ADDR']} (host is {$_SERVER['REMOTE_HOST']}), agent is {$_SERVER['HTTP_USER_AGENT']}, it claimed to have come from {$_SERVER['HTTP_REFERER']}\n";
    $msg .= "http://dnstools.com/?count=1&lookup=on&wwwhois=on&arin=on&checkp=on&portNum=80&ping=on&all=on&submit=Go!&target={$_SERVER['REMOTE_ADDR']}\n";
    $msg .= "\n\n";
    $msg .= "Potential abuse e-mail:\n";
    $msg .= "Hello,\n\n";
    $msg .= "I am writing to report a potentially misconfigured or malicious web bot that has hit a bot-trap on my site at $datum running on {$_SERVER['REMOTE_ADDR']} (reported agent was {$_SERVER['HTTP_USER_AGENT']}).\n\n";
    $msg .= "If you would like to check what that bot did on the site you can check http://www.example.com/capturedbots.php?ip={$_SERVER['REMOTE_ADDR']}\n\n";
    $msg .= "\t\t-Volodya\n";
    mail($to, $subject, $msg, "From: $from");
  }
 
/* a new visit, log it */
$fp = fopen($visits,'a+');
fwrite($fp,"{$_SERVER['REMOTE_ADDR']} - - [{$datum}] \"{$_SERVER['REQUEST_METHOD']} {$_SERVER['REQUEST_URI']} {$_SERVER['SERVER_PROTOCOL']}\" {$_SERVER['HTTP_REFERER']} {$_SERVER['HTTP_USER_AGENT']}\n");
fclose($fp);
?>
</body>
</html>

blacklist.php

Also please make sure to change your e-mail address, so that some unfortunate real person may request to be unblocked.
<?php
$badbot = 0;
/* look for the IP address in the blacklist file */
$blacklist = "./blacklist.dat";
$visits = './blacklist-returns.dat';
$fp = fopen($blacklist, "r") or die ("Error opening file ... <br>\n");
$ip_addr = $_SERVER['REMOTE_ADDR'];
while ($line = fgets($fp,255))
{
  $u = explode(" ",$line);
  $u0 = $u[0];
  if ($u[0]===$ip_addr) {
    $badbot++;
  }
}
fclose($fp);
if ($badbot > 0) { /* this is a bad bot, reject it */
  $tmestamp = time();
  $datum = date("Y-m-d (D) H:i:s",$tmestamp);
 
  sleep(12);
 
/* send email about the bot returning */
$to = "WebMaster@example.com";
$subject = "alert: return of a bad bot";
$from = "badbot-watch@examle.com";
$msg = "{$_SERVER['REMOTE_ADDR']} just couldn't get enough, it returned again. It went to {$_SERVER['HTTP_HOST']}{$_SERVER['REQUEST_URI']}\n";
mail($to, $subject, $msg, "From: {$from}");
 
/* write the log of the visit */
$fp = fopen($visits,'a+');
fwrite($fp,"{$_SERVER['REMOTE_ADDR']} - - [{$datum}] \"{$_SERVER['REQUEST_METHOD']} {$_SERVER['REQUEST_URI']} {$_SERVER['SERVER_PROTOCOL']}\" {$_SERVER['HTTP_REFERER']} {$_SERVER['HTTP_USER_AGENT']}\n");
fclose($fp);
 
 
?>
<html><head>
<title>You have been banned</title>
</head><body>
<h1>You have been banned!</h1>
<p>Unfortunately, due to abuse (or a possible mistake on your part), your ip address (<?=$ip?>) has been banned from this site. You will need to contact the administrator of this site to get your self unblocked, to get the the e-mail is &ldquo;WebMaster&rdquo; then the &ldquo;at&rdquo; sign, and after than example.com.</p>
</body></html>
<?php
  exit;
}

capturedbots.php

<pre><?php
$ip=$_GET['ip'];
 
$blocks = './blacklist.dat';
$visits = './blacklist-returns.dat';
$fp = fopen($blocks, "r") or die ("Error opening file ... \n");
$blocked=false;
while ($line = fgets($fp,255))
{
  $u = explode(" ",$line);
  if ($ip=='' or strpos($u[0], $ip)!==false)
  {
    $blocked=true;
  }
}
fclose($fp);
$fp = fopen($visits, "r") or die ("Error opening file ... \n");
while ($blocked && $line = fgets($fp,255))
{
  $u = explode(" ",$line);
  if ($ip=='' or strpos($u[0], $ip)!==false)
  {
    echo $line;
  }
}
fclose($fp);
?></pre>

LocalSettings.php

Or any other file that is guaranteed to run, maybe even index.php
require_once("blacklist.php");

data files

There are two data files that you will need to create, just create an empty web-writable blacklist.dat and blacklist-returns.dat.

Link

Link can look something like this:

  <span style='dislay: none;'><a href='/hidden.php' title='Do not click on this link'>&zwnj;</a></span>

A single-pixel images are also used for this purpose sometimes.

External link