[Dshield] What is this?

Ed Truitt ed.truitt at etee2k.net
Tue Jul 2 23:27:38 GMT 2002

AltaVista is a web search engine.  They use the spider to map / index the
web sites.  To deter them, you should create a "robots.txt" file and put in
what you want the search engines to ignore.  There is a document on how to
create such a file available at http://www.robotstxt.org/wc/norobots.html.

You may not have the files on your server, but it is quite possible that the
URLs are referenced in the pages you do have.

Ed Truitt
PGP fingerprint:  5368 D25E 468C A250 9833  CCD6 DBAE 9C25 02F9 0AB9

"Note to spammers:  my 'delete' key is connected to YOUR ISP.
 Also, if you send me UCE, I reserve the right to post your spew
on my Web site, with the appropriate color commentary, so that
others may have a good laugh at your expense."

----- Original Message -----
From: "Antti Tolamo" <usenet at linux.tola.org>
To: <list at dshield.org>
Sent: Tuesday, July 02, 2002 1:38 PM
Subject: RE: [Dshield] What is this?

> At 08:44 2.7.2002 -0700, you wrote:
> >"Scooter-3.2.EX" shows as a Spider from AltaVista.  An NSLOOKUP shows
> >
> >Name:    buildrack16.sv.av.com
> >Address:
> >
> >which is an AltaVista name and IP address.
> I used nslookup and traceroute, however
> they only told me above and that it's DNS
> _may_ have something to do with Altavista.
> Why it however also tries to access _lot_ of  Apache files that
> I don't have? What's the purpose? Is it result of
> some search or what? And why atleast two days a row?
> Do they evaluate webservers or what?
> Regards,
> Antti

More information about the list mailing list