|
Here's an odd one that caught my eye: Requests that include a username field in the HTTP request (in Apache's "combined" and "common" logfile formats, that's usually the second '-' right in front of the timestamp):
70.50.246.167 - pensa [02/Jun/2005:00:13:10 -0400] "GET /search.php HTTP/1.0" 200 58392 "-" "Mozilla/3.0 (compatible)"
70.50.246.167 - bob [02/Jun/2005:00:44:26 -0400] "GET /index.php?page=4 HTTP/1.0" 200 36886 "-" "Mozilla/3.0 (compatible)"
70.50.246.167 - bgkot [02/Jun/2005:00:49:32 -0400] "GET /portal.php/link/20050525064325213 HTTP/1.0" 301 0 "-" "Mozilla/3.0 (compatible)"
70.50.246.167 - dcyber [02/Jun/2005:00:53:24 -0400] "GET /article.php/20050314062205630 HTTP/1.0" 200 20716 "-" "Mozilla/3.0 (compatible)"
70.50.246.167 - beege [02/Jun/2005:04:38:45 -0400] "GET /portal.php/link/20050518184817770 HTTP/1.0" 301 0 "-" "Mozilla/3.0 (compatible)"
70.50.246.167 - www.portodeisanti [02/Jun/2005:05:17:52 -0400] "GET /staticpages/index.php/20011217123134458 HTTP/1.0" 200 23692 "-" "Mozilla/3.0 (compatible)"
70.50.246.167 - www.lgibson [02/Jun/2005:07:05:29 -0400] "GET /article.php/20050508150404304/print HTTP/1.0" 200 1735 "-" "Mozilla/3.0 (compatible)"
Other requests from the same IP did not have a username. I found more of the same in the logfiles for the past couple of days. The IP address belongs to Sympatico, a Canadian ISP.
That user agent string also looks odd (not because of the "Mozilla/3.0" but because of the complete lack of further information, e.g. about the operating system). So I searched for that - and found even more of these requests ... Not sure what to make of this, though. For the month of May, I see many requests from Canadian ISPs, but also from all over the world. Some include a username, some don't. The only thing they have in common is that they seem to arbitrarily request URLs from all over the place. And they never load any images or stylesheets.
I found a discussion mentioning that generic browser strings like this may be used by ISP's caches but that doesn't seem to fit the pattern I see in this case. And, as I mentioned, it doesn't access any images.
I don't think it's a single bot that uses this exact user agent string. It's more likely that several tools use this as some sort of generic identification. The requests with the username probably all come from the same software, though. But I have no idea why it would do this.
Ann Elisabeth has a case where this user agent string was used by a spamming bot.
When blocking this (exact!) user agent, you may risk blocking the occasional legit visitor. But in the vast majority of the cases, I think it would block harvesters and other bots of dubious nature.
Classification: Block at your own risk ...
|