[Dshield] One of our virtual servers is getting hit by a DDoSattacking one domain searching for the robots.txt file

Maxime Ducharme mducharme at cybergeneration.com
Tue Jan 4 15:41:50 GMT 2005


Usually robots.txt is used by search engines.

The IPs here all looks like dyn IPs, which means it
is home users.

I never seen this kind of activity on our servers.

Timestamps indicate you get alot of requests at the same
time, which is anormal.

These IPs are unlikely to be spoofed since HTTP use TCP,
which is alot harder to spoof (depending if you got an old OS).
see http://www.hackinthebox.org/article.php?sid=6394

I suggest to extract list of attacking IPs and block them
on your firewall temporarly.

# grep robots.txt access.log |awk '{print $2}' |sort |uniq >tmp_ips.log
# while read ip
# do
#   <add drop rule to your packet filter>
# done<tmp_ips.log

Keep an eye on this and unblock them if you see less activity.

Hope that helps, good luck !

Maxime Ducharme
Programmeur / Spécialiste en sécurité réseau

----- Original Message ----- 
From: "David McCall" <david at atgi.net>
To: <list at lists.dshield.org>
Sent: Monday, January 03, 2005 11:32 AM
Subject: [Dshield] One of our virtual servers is getting hit by a
DDoSattacking one domain searching for the robots.txt file


> I've pointed the DNS to 127.0.0.1 which hasn't made much of a dent if at
all.  I'm thinking that the IP's attacking are spoofed
> but I don't know how to trouble shoot it:
>
> www.mpecllc.com 66.130.71.170 - - [03/Jan/2005:08:24:16 -0800] "GET
/robots.txt HTTP/1.1" 302 225 "-" "Mozilla/4.0 (compatible; MSIE
> 6.0; Windows NT 5.1)"
> www.mpecllc.com 69.40.200.9 - - [03/Jan/2005:08:24:16 -0800] "GET
/robots.txt HTTP/1.1" 200 26 "-" "Mozilla/4.0 (compatible; MSIE
> 6.0; Windows NT 5.1)"
> www.mpecllc.com 70.16.82.63 - - [03/Jan/2005:08:24:16 -0800] "GET
/robots.txt HTTP/1.1" 302 225 "-" "Mozilla/4.0 (compatible; MSIE
> 6.0; Windows NT 5.1)"
> www.mpecllc.com 24.242.195.213 - - [03/Jan/2005:08:24:16 -0800] "GET
/robots.txt HTTP/1.1" 200 26 "-" "Mozilla/4.0 (compatible; MSIE
> 6.0; Windows NT 5.1)"
> www.mpecllc.com 68.62.192.89 - - [03/Jan/2005:08:24:16 -0800] "GET
/robots.txt HTTP/1.1" 302 225 "-" "Mozilla/4.0 (compatible; MSIE
> 6.0; Windows NT 5.1)"
> www.mpecllc.com 70.17.131.139 - - [03/Jan/2005:08:24:16 -0800] "GET
/robots.txt HTTP/1.1" 200 26 "-" "Mozilla/4.0 (compatible; MSIE
> 6.0; Windows NT 5.1)"
> www.mpecllc.com 68.238.43.121 - - [03/Jan/2005:08:24:16 -0800] "GET
/robots.txt HTTP/1.1" 200 26 "-" "Mozilla/4.0 (compatible; MSIE
> 6.0; Windows NT 5.1)"
> www.mpecllc.com 216.164.60.3 - - [03/Jan/2005:08:24:16 -0800] "GET
/robots.txt HTTP/1.1" 200 26 "-" "Mozilla/4.0 (compatible; MSIE
> 6.0; Windows NT 5.1)"
> www.mpecllc.com 198.140.195.22 - - [03/Jan/2005:08:24:16 -0800] "GET
/robots.txt HTTP/1.1" 302 225 "-" "Mozilla/4.0 (compatible;
> MSIE 6.0; Windows NT 5.1)"
> www.mpecllc.com 70.16.82.63 - - [03/Jan/2005:08:24:16 -0800] "GET
/robots.txt HTTP/1.1" 200 26 "-" "Mozilla/4.0 (compatible; MSIE
> 6.0; Windows NT 5.1)"
> www.mpecllc.com 68.103.153.70 - - [03/Jan/2005:08:24:16 -0800] "GET
/robots.txt HTTP/1.0" 302 213 "-" "Mozilla/4.0 (compatible; MSIE
> 6.0; Windows NT 5.1)"
> www.mpecllc.com 70.16.82.63 - - [03/Jan/2005:08:24:16 -0800] "GET
/robots.txt HTTP/1.1" 302 225 "-" "Mozilla/4.0 (compatible; MSIE
> 6.0; Windows NT 5.1)"
> www.mpecllc.com 68.237.17.252 - - [03/Jan/2005:08:24:16 -0800] "GET
/robots.txt HTTP/1.1" 200 26 "-" "Mozilla/4.0 (compatible; MSIE
> 6.0; Windows NT 5.1)"
> www.mpecllc.com 24.129.70.50 - - [03/Jan/2005:08:24:16 -0800] "GET
/robots.txt HTTP/1.1" 302 225 "-" "Mozilla/4.0 (compatible; MSIE
> 6.0; Windows NT 5.1)"
> www.mpecllc.com 24.110.97.52 - - [03/Jan/2005:08:24:16 -0800] "GET
/robots.txt HTTP/1.1" 200 26 "-" "Mozilla/4.0 (compatible; MSIE
> 6.0; Windows NT 5.1)"
> www.mpecllc.com 24.241.116.141 - - [03/Jan/2005:08:24:16 -0800] "GET
/robots.txt HTTP/1.1" 200 26 "-" "Mozilla/4.0 (compatible; MSIE
> 6.0; Windows NT 5.1)"
> www.mpecllc.com 65.25.137.52 - - [03/Jan/2005:08:24:16 -0800] "GET
/robots.txt HTTP/1.1" 200 26 "-" "Mozilla/4.0 (compatible; MSIE
> 6.0; Windows NT 5.1)"
> www.mpecllc.com 24.14.191.240 - - [03/Jan/2005:08:24:16 -0800] "GET
/robots.txt HTTP/1.1" 200 26 "-" "Mozilla/4.0 (compatible; MSIE
> 6.0; Windows NT 5.1)"
> www.mpecllc.com 69.165.152.38 - - [03/Jan/2005:08:24:16 -0800] "GET
/robots.txt HTTP/1.1" 302 225 "-" "Mozilla/4.0 (compatible; MSIE
> 6.0; Windows NT 5.1)"
> www.mpecllc.com 68.62.65.177 - - [03/Jan/2005:08:24:16 -0800] "GET
/robots.txt HTTP/1.1" 302 225 "-" "Mozilla/4.0 (compatible; MSIE
> 6.0; Windows NT 5.1)"
> www.mpecllc.com 66.190.95.172 - - [03/Jan/2005:08:24:16 -0800] "GET
/robots.txt HTTP/1.1" 200 26 "-" "Mozilla/4.0 (compatible; MSIE
> 6.0; Windows NT 5.1)"
> www.mpecllc.com 68.190.49.156 - - [03/Jan/2005:08:24:16 -0800] "GET
/robots.txt HTTP/1.1" 200 26 "-" "Mozilla/4.0 (compatible; MSIE
> 6.0; Windows NT 5.1)"
> www.mpecllc.com 170.215.99.165 - - [03/Jan/2005:08:24:16 -0800] "GET
/robots.txt HTTP/1.1" 302 225 "-" "Mozilla/4.0 (compatible;
> MSIE 6.0; Windows NT 5.1)"
> www.mpecllc.com 68.205.47.120 - - [03/Jan/2005:08:24:16 -0800] "GET
/robots.txt HTTP/1.1" 302 225 "-" "Mozilla/4.0 (compatible; MSIE
> 6.0; Windows NT 5.1)"
> www.mpecllc.com 68.205.47.120 - - [03/Jan/2005:08:24:16 -0800] "GET
/robots.txt HTTP/1.1" 200 26 "-" "Mozilla/4.0 (compatible; MSIE
> 6.0; Windows NT 5.1)"
> www.mpecllc.com 141.156.33.112 - - [03/Jan/2005:08:24:16 -0800] "GET
/robots.txt HTTP/1.1" 200 26 "-" "Mozilla/4.0 (compatible; MSIE
> 6.0; Windows NT 5.1)"
>
>
> Anyone with any ideas on how to stop this would be appreciated.
>
> thanks ahead of time:
>
> D.C.McCall
> UNIX Administrator
> ===================
> AdvancedTelcomInc
> admin at atgi.net
>
> -------------- Sponsor Message ------------------------------------
> SANS Intrusion Immersion Training: Orlando, FL, February 3-9th
> http://www.sans.org/orlando05
>
> _______________________________________________
> send all posts to list at lists.dshield.org
> To change your subscription options (or unsubscribe), see:
http://www.dshield.org/mailman/listinfo/list
>




More information about the list mailing list