[Dshield] Test

George A. Theall theall at tifaware.com
Tue Feb 21 12:22:34 GMT 2006


On Mon, Feb 20, 2006 at 12:25:39PM +0000, Ed Truitt wrote:

> Maybe the idea was to read robots.txt to find out what the site owner
> didn't want indexed - then treat *that* as the highest priority stuff to
> grab... 

That's an interesting idea, just not the behaviour I've observed with 
this particular 'bot.

George
-- 
theall at tifaware.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 191 bytes
Desc: not available
Url : http://www.dshield.org/pipermail/list/attachments/20060221/e61a0e5b/attachment.bin


More information about the list mailing list