|
I've posted about OmniExplorer_Bot before (and even falsely accused it of spamming) and a lot of other people also posted and complained about the bot and its tendency to cause huge amounts of traffic. In the end, it seemed like the folks who run OmniExplorer_Bot finally got a clue. They even posted an apology on their website and promised that the bot would behave now.
Unfortunately, it doesn't.
After the traffic caused by OmniExplorer_Bot was down to an acceptable level, I decided to test if it was really behaving now. So I unblocked it, but added it to the robots.txt. Here's what happened: 65.19.150.243 - - [13/Jul/2005:00:51:33 -0400] "GET /robots.txt HTTP/1.1" 200 686 "-" "OmniExplorer_Bot/2.35 (+http://www.omni-explorer.com) WorldIndexer"
65.19.150.243 - - [13/Jul/2005:00:51:35 -0400] "GET /submit.php?type=story HTTP/1.1" 200 18297 "-" "OmniExplorer_Bot/2.35 (+http://www.omni-explorer.com) WorldIndexer"
So it's checking the robots.txt now (which it didn't do before). In that robots.txt, however, are rules to
- disallow OmniExplorer_Bot access to the site and
- disallow all bots access to submit.php
And OmniExplorer_Bot promptly chose to ignore both.
For the record: The bot was unblocked more than 12 hours before the above two requests and it had indeed accessed robots.txt several times since then. So it had more than enough time to digest it.
Sorry guys, that was your last chance. I now offically declare this bot (and its parent company) to be evil, stupid, and not playing nice. Block it, forever.
|