[unisog] Getting things deleted from Google's cache

David Lundy dlundy at pacific.edu
Thu Apr 6 16:25:49 GMT 2006


Has anyone considered the issue of caching on other search engines?  I
did a quick check and found that Ask Jeeves and Yahoo cache.  Others
that I checked did not appear to.   I suspect there are other caching
servers.

All the Web - no
Alta Vista -no
Ask Jeeves - yes
Excite - no
LookSmart - no
Lycos -no
Open Directory Project - no
Yahoo - yes
Wisenut - no

Clearly the problem is bigger than Google.

Dave Lundy
Acting IT Security Officer
University of the Pacific

>>> r.fulton at auckland.ac.nz 04/05/06 5:09 PM >>>
Hi,
	We had an incident where a file was exposed to the Internet on a
web
server that should not have been visible outside. The problem was
first
discovered back in December and the firewall settings changed so the
the
server was no longer accessible from outside.  Everyone heaved a sight
of relief and went back to sleep.  Last week we found out that the
file
was still available via Google!  Ouch!!

It turns out that Google will delete pages from the cache if and only
if
it gets a 404 from the url (I assume there must be some timeout ???).
By simply removing the server from the net we did not get the file out
of Google's cache and it sat there for four months.

We are now 'resurrecting' a skeleton server to expire all material
from
Google's cache.

One other thing I found ( thanks to "es0" on #unisog) is the url to
request Google to remove items from the cache urgently.  This still
requires that Google gets a 404 for the url and takes "3 to 5 days".

http://services.google.com:8882/urlconsole/controller?cmd=reload&lastcmd=login


This site also has information on how to use meta tags to control what
Google indexes.  I wish Google had prominent links to this url on
their
contact and security pages, it would have saved me quite some time.

This information was published in the spring issue of 2600.

Cheers, Russell
_______________________________________________
unisog mailing list
unisog at lists.sans.org 
http://www.dshield.org/mailman/listinfo/unisog 



More information about the unisog mailing list