[Intrusions] We are experiencing a DDoS attacking one of our domains- mpecllc.com
David McCall
david at atgi.net
Wed Jan 19 16:16:27 GMT 2005
Hi Ryan,
Before I answer, I've chatted with NYPD Cyber Crimes div. Anthony R. and it looks like a zombie worm, Symantec may have already
reverse engineered it, and, it also looks like the perp. has been arrested:
>From SANS today: DDoS Suspect Arrested in Scotland
http://news.bbc.co.uk/1/hi/scotland/4175801.stm
>1) What is causing you more of a problem - the DNS load on your authoritative servers or the HTTP traffic to your web server?
the dns load is min. it is the actual hits on the web server that brought the systems down.
>2) You may mow down some innocent children here, you could reconfigure Apache (with some Mod_Rewrite rules) to drop requests with
>the "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)" User-Agent Field. All of the DDoS clients in your logs had this exact
>user-agent listed.
Yes I noticed the signature, but just about everything we tried with Apache didn't change the floods. Eventually, sometimes even
after 36hours, the server just couldn't handle the load.
>3) Do you have any NIDS with flexible response capability (such as Snort's connection resets)? You may be able to reset HTTP
>requests for robots.txt and entry.php. Due to the amount of traffic, the session sniping effectiveness may be reduced however
>this would take some of the load off of your web server.
That's a great idea, however we have moved the affected/attacked domain into a single Raq4 sandboxed server all by itself, and
even tried a SQUID front end filter w/o any success. And after my chat with NYPD Anthony, he says that until the worm is completely
wiped from all the infected zombie computers on the planet, the domain name is toast. Not worth the effort to even
get it running again.
>4) What are your plans for the new firewall? Are you hoping to throttle DNS queries?
yesterday we failed with SQUID and I believe we are just going to byte the bullet on this domain and leave it off. Eventually
(??) the dns queries should go away, but I'll keep monitoring weekly.
>5) As mentioned below, it looks like most of the DDoS clients are home users -
Yes, we noticed this too, kinda follows the MO described in the arrest news article.
Name: ca-crlsbd-cuda1-c7b-b-25.crlsca.adelphia.net
Name: 24.247.2.226.gha.mi.chartermi.net
Name: adsl-68-253-255-21.dsl.emhril.ameritech.net
Name: pcp01349395pcs.lowmrn01.pa.comcast.net
Name: n128-227-58-20.xlate.ufl.edu
Name: cpe-024-211-249-043.nc.rr.com
Name: ip70-178-8-21.ma.dl.cox.net
Name: ar60.lsanca1-4.29.92.66.lsanca1.dsl-verizon.net
Name: d207-81-81-85.bchsia.telus.net
Name: pcp09791096pcs.summit01.nj.comcast.net
Name: adsl-63-198-19-106.dsl.chic01.pacbell.net
Name: nat.trusecureonline.com
Name: adsl-69-211-75-13.dsl.milwwi.ameritech.net
Name: fl-65-40-1-55.sta.sprint-hsd.net
Name: pool-141-157-196-180.ny325.east.verizon.net
Name: h216-170-177-114.216-170.unk.tds.net
Name: ip24-250-111-104.dc.dc.cox.net
Name: cs6669186-184.houston.rr.com
I ran a quick check of DNS cache entries on a bunch of ISPs using one of the tools as dnsstuff.com -
http://www.dnsstuff.com/tools/ispdns.ch?name=mpecllc.com&type=A It looks like most of the ISPs do not have cache info for your IP so
they would forward their clients to your NS. Sure would be nice if the ISPs cached some of this info.
thanks again to everyone that contributed to this, I just felt so alone for about 10 days...... and well, you all are great!!!
David C.McCall
UNIX Administrator
===================
EschelonTelecom
admin at atgi.net
david at atgi.net
Most Respectfully,
Ryan C. Barnett
SANS: GCIA, GCFA, GCIH, GCUX, GSEC
Department of Justice - ATF
Information Services Division
Operations Security Team Lead
> -----Original Message-----
> From: intrusions-bounces at lists.sans.org
> [mailto:intrusions-bounces at lists.sans.org]On Behalf Of David McCall
> Sent: Monday, January 17, 2005 3:59 PM
> To: Intrusions List (GCIA Practicals)
> Subject: RE: [Intrusions] We are experiencing a DDoS attacking
> oneofourdomains - mpecllc.com
>
>
> Thanks Ryan,
>
> We've got BigIP's here also, but since the attacked site is
> only one domain, we're hopeing
> to keep this guy in a sand box somewhere so that our other 3K
> + customers won't get to involved.
>
> Yea, I've tuned Apache, and have been using ipchains, which
> also brought the server down when the
> kernel table reached a little over 21K ip addresses......
>
> upstream from us is mci.com and verio.net....
>
> they've been contacted but I've not been given anything back
> except a form-email...
>
> We're going to try another type of firewall this afternoon at
> 3pm eastern....but...I'm not going
> to hold my breath.
>
> What really scares me is this could happen to any number of
> domain names....and at multiple sites...
>
> and there doesn' seem to be anything to do about it because
> it's rather low level in the emergency
> chain at present.
>
> regards
>
> David C.McCall
> UNIX Administrator
> ===================
> EschelonTelecom
> admin at atgi.net
> david at atgi.net
>
> -----Original Message-----
> From: intrusions-bounces at lists.sans.org
> [mailto:intrusions-bounces at lists.sans.org]On Behalf Of
> Barnett, Ryan C.
> (EDS)
> Sent: Monday, January 17, 2005 9:02 AM
> To: Intrusions List (GCIA Practicals)
> Subject: RE: [Intrusions] We are experiencing a DDoS attacking one
> ofourdomains - mpecllc.com
>
>
> David,
> These types of DDoS attacks suck... A few comments/recommendations -
>
> 1) These HTTP requests are not spoofed. Spoofed TCP packets
> only work with Syn flood attacks. In order for the network connection
> to make its way up the OSI stack to layer 7 (your web
> server), the 3-way handshake must be completed - thus meaning
> that the IPs
> listed in your logs are indeed the ones sending the requests.
> Your response may be - "So what?" Well, this means that you could
> potentially implement some form on IP block here, although
> the sheer numbers of clients participating would make this hard.
>
> 2) Looking at the hostnames of the IPs in your web logs, it
> looks like all of these are home users with high-speed
> connections. For
> instance -
>
> IP address: 66.69.186.184
> Host name: cs6669186-184.houston.rr.com
>
> These computers probably have malware installed and are
> un-knowingly participating.
>
> 3) Have you tuned your OS and web server for performance?
> There are many steps to take which will help in speeding up your web
> server - number of listening sockets, timeouts, etc...
> http://httpd.apache.org/docs-2.0/misc/perf-tuning.html
>
> 4) If you are using Apache, you could implement
> mod_dosevasive - http://www.nuclearelephant.com/projects/dosevasive/
>
> 5) Your ISP/Upstream provied may be able to assist if they
> can throttle incoming connections (with BipIP or something).
>
> Hope this helps.
>
> Most Respectfully,
> Ryan C. Barnett
> SANS: GCIA, GCFA, GCIH, GCUX, GSEC
> Department of Justice - ATF
> Information Services Division
> Operations Security Team Lead
>
>
> > -----Original Message-----
> > From: intrusions-bounces at lists.sans.org
> > [mailto:intrusions-bounces at lists.sans.org]On Behalf Of David McCall
> > Sent: Wednesday, January 12, 2005 3:03 PM
> > To: intrusions at lists.sans.org
> > Subject: [Intrusions] We are experiencing a DDoS attacking one of
> > ourdomains - mpecllc.com
> >
> >
> > At last count I have isolated 21,000 unique IP addresses that
> > are pounding our dns and web servers
> > for robots.txt and entry.php.......
> >
> > This domain is effectively down because of this issue. We
> > had to move it to a sand box server and
> > this was ineffective after 24 hours. If I enable the domain
> > and turn on the web site the initial
> > attack is 200-400 attacks per second, consisting of a GET for
> > robots.txt or entry.php, neither of
> > which exist on the site.
> >
> > I've notified mci.com and verio.net abuse depts.
> >
> > I've filed a report with FBI and wanted to make you all aware
> > of what is going on:
> >
> > IFCC COMPLAINT REFERRAL REPORT
> > Complaint Number: I05011113017305
> > The following information was provided by the victim and will
> > be forwarded to the appropriate law
> > enforcement or regulatory agency.
> > Computer Intrusion/Hacking
> > Date of Complaint: 1/11/2005 1:01:35 PM
> > Victim Information
> > Business Name: Eschelon Telecom
> > Name: David Chester McCall
> > DOB: 11/21/1954
> > Gender: M
> > Phone #: 707-284-5695
> > Email: david at atgi.net
> > Address: 19 Old Courthouse Square
> > Santa Rosa, CA 95404
> > Live in city limits: No
> > County: Sonoma
> > Country: USA
> > Do you have pertinent documents in paper form? No
> > Please indicate who your local law enforcement agency is:
> > http://ci.santa-rosa.ca.us/default.aspx?PageId=119
> > Please List the easiest way and most convenient time to contact you:
> > david at atgi.net
> > 707-477-7466 cell phone
> > 707-792-0482 home
> > Information about the Business that victimized you.
> > Name:
> > Gender: U
> > Phone #:
> > Current Email:
> > Address:
> > Country: USA
> >
> > Contact between you and the Person/company that victimized you.
> > Type of Contact: Web Page
> > Date of Contact: 01/05/2005
> > Contact Information:
> > DDoS attack on one of our hosted domains: www.mpecllc.com
> > Brief log exerpt below:
> > www.mpecllc.com 68.70.227.25 - - [10/Jan/2005:11:10:49 -
> > 0800] "GET /robots.txt HTTP/1.1" 302 225 "-" "Mozilla/4.0
> > (compatible;
> > MSIE 6.0; Windows NT 5.1)"
> > www.mpecllc.com 24.247.2.226 - - [10/Jan/2005:11:10:49 -
> > 0800] "GET /robots.txt HTTP/1.1" 302 225 "-" "Mozilla/4.0
> > (compatible;
> > MSIE 6.0; Windows NT 5.1)"
> > www.mpecllc.com 68.253.255.21 - - [10/Jan/2005:11:10:49 -
> > 0800] "GET /robots.txt HTTP/1.1" 302 225 "-" "Mozilla/4.0
> > (compatible;
> > MSIE 6.0; Windows NT 5.1)"
> > www.mpecllc.com 68.80.225.249 - - [10/Jan/2005:11:10:49 -
> > 0800] "GET /robots.txt HTTP/1.1" 302 225 "-" "Mozilla/4.0
> > (compatible;
> > MSIE 6.0; Windows NT 5.1)"
> > www.mpecllc.com 128.227.58.20 - - [10/Jan/2005:11:10:50 -
> > 0800] "GET /robots.txt HTTP/1.1" 302 225 "-" "Mozilla/4.0
> > (compatible;
> > MSIE 6.0; Windows NT 5.1)"
> > www.mpecllc.com 24.211.249.43 - - [10/Jan/2005:11:10:50 -
> > 0800] "GET /robots.txt HTTP/1.1" 302 225 "-" "Mozilla/4.0
> > (compatible;
> > MSIE 6.0; Windows NT 5.1)"
> > www.mpecllc.com 70.178.8.21 - - [10/Jan/2005:11:10:50 -
> > 0800] "GET /robots.txt HTTP/1.1" 302 225 "-" "Mozilla/4.0
> > (compatible;
> > MSIE 6.0; Windows NT 5.1)"
> > www.mpecllc.com 4.29.92.66 - - [10/Jan/2005:11:10:50 -
> > 0800] "GET /robots.txt HTTP/1.1" 302 225 "-" "Mozilla/4.0
> > (compatible;
> > MSIE 6.0; Windows NT 5.1)"
> > www.mpecllc.com 207.81.81.85 - - [10/Jan/2005:11:10:50 -
> > 0800] "GET /robots.txt HTTP/1.1" 302 225 "-" "Mozilla/4.0
> > (compatible;
> > MSIE 6.0; Windows NT 5.1)"
> > www.mpecllc.com 68.36.53.42 - - [10/Jan/2005:11:10:50 -
> > 0800] "GET /robots.txt HTTP/1.1" 302 225 "-" "Mozilla/4.0
> > (compatible;
> > MSIE 6.0; Windows NT 5.1)"
> > www.mpecllc.com 63.198.19.106 - - [10/Jan/2005:11:10:50 -
> > 0800] "GET /robots.txt HTTP/1.1" 302 225 "-" "Mozilla/4.0
> > (compatible;
> > MSIE 6.0; Windows NT 5.1)"
> > www.mpecllc.com 65.196.186.6 - - [10/Jan/2005:11:10:50 -
> > 0800] "GET /robots.txt HTTP/1.1" 302 225 "-" "Mozilla/4.0
> > (compatible;
> > MSIE 6.0; Windows NT 5.1)"
> >
> > www.mpecllc.com 69.211.75.13 - - [10/Jan/2005:11:10:50 -
> > 0800] "GET /robots.txt HTTP/1.1" 302 225 "-" "Mozilla/4.0
> > (compatible;
> > MSIE 6.0; Windows NT 5.1)"
> > www.mpecllc.com 65.40.1.55 - - [10/Jan/2005:11:10:50 -
> > 0800] "GET /robots.txt HTTP/1.1" 302 225 "-" "Mozilla/4.0
> > (compatible;
> > MSIE 6.0; Windows NT 5.1)"
> > www.mpecllc.com 141.157.196.180 - - [10/Jan/2005:11:10:50 -
> > 0800] "GET /robots.txt HTTP/1.1" 302 225 "-" "Mozilla/4.0
> > (compatible;
> > MSIE 6.0; Windows NT 5.1)"
> > www.mpecllc.com 216.170.177.114 - - [10/Jan/2005:11:10:50 -
> > 0800] "GET /robots.txt HTTP/1.1" 302 225 "-" "Mozilla/4.0
> > (compatible;
> > MSIE 6.0; Windows NT 5.1)"
> > www.mpecllc.com 24.250.111.104 - - [10/Jan/2005:11:10:50 -
> > 0800] "GET /robots.txt HTTP/1.1" 302 225 "-" "Mozilla/4.0
> > (compatible;
> > MSIE 6.0; Windows NT 5.1)"
> > www.mpecllc.com 66.69.186.184 - - [10/Jan/2005:11:10:50 -
> > 0800] "GET /robots.txt HTTP/1.1" 302 225 "-" "Mozilla/4.0
> > (compatible;
> > MSIE 6.0; Windows NT 5.1)"
> > Additional Information:
> > We have had to disable this customer site as the attack eventually
> > brings down the server. However this attack continues and can be
> > viewed by the DNS queries from our name servers (brief
> > listing below):
> > Jan 11 09:27:03 e4500a named[280]: XX /24.25.195.1/mpecllc.com/A/IN
> > Jan 11 09:27:09 e4500a named[280]: XX
> > /66.186.224.158/mpecllc.com/A/IN
> > Jan 11 09:27:12 e4500a named[280]: XX /68.1.208.23/mpecllc.com/A/IN
> > Jan 11 09:27:20 e4500a named[280]: XX /66.129.37.38/mpecllc.com/A/IN
> > Jan 11 09:27:23 e4500a named[280]: XX /68.73.225.60/mpecllc.com/A/IN
> > Jan 11 09:27:26 e4500a named[280]: XX
> /167.206.3.247/mpecllc.com/A/IN
> > Jan 11 09:27:43 e4500a named[280]: XX /209.204.64.3/mpecllc.com/A/IN
> > Jan 11 09:27:25 e4500B named[20019]: XX
> /68.35.192.6/mpecllc.com/A/IN
> > Jan 11 09:27:26 e4500B named[20019]:
> > XX /167.206.3.248/mpecllc.com/A/IN
> > Jan 11 09:27:27 e4500B named[20019]: XX
> > /68.73.225.60/mpecllc.com/A/IN
> > Jan 11 09:27:28 e4500B named[20019]:
> > XX /213.129.10.130/mpecllc.com/A/IN
> > Jan 11 09:27:30 e4500B named[20019]: XX /65.32.1.79/mpecllc.com/A/IN
> > Jan 11 09:27:34 e4500B named[20019]:
> > XX /209.244.4.189/mpecllc.com/A/IN
> > Jan 11 09:27:40 e4500B named[20019]:
> > XX /167.206.3.184/mpecllc.com/A/IN
> > Jan 11 09:27:45 e4500B named[20019]: XX
> > /24.140.1.132/mpecllc.com/A/IN
> > Jan 11 09:27:58 e4500B named[20019]:
> > XX /204.127.202.35/mpecllc.com/A/IN
> >
> > Jan 11 09:28:10 e4500B named[20019]:
> > XX /206.135.241.66/mpecllc.com/A/IN
> > Jan 11 09:27:30 queue named[26183]: [ID 295310 daemon.info]
> > XX /137.39.110.165/mpecllc.com/A/IN
> > Jan 11 09:27:32 queue named[26183]: [ID 295310 daemon.info]
> > XX /208.204.150.212/mpecllc.com/A/IN
> > Jan 11 09:27:33 queue named[26183]: [ID 295310 daemon.info]
> > XX+/152.1.1.206/mpecllc.com/A/IN
> > Jan 11 09:27:37 queue named[26183]: [ID 295310 daemon.info]
> > XX /152.38.30.122/mpecllc.com/A/IN
> > Jan 11 09:27:44 queue named[26183]: [ID 295310 daemon.info]
> > XX /63.64.9.19/mpecllc.com/A/IN
> > Jan 11 09:27:47 queue named[26183]: [ID 295310 daemon.info]
> > XX /207.65.122.221/mpecllc.com/A/IN
> > Jan 11 09:27:52 queue named[26183]: [ID 295310 daemon.info]
> > XX /24.92.32.23/mpecllc.com/AAAA/IN
> > Jan 11 09:27:55 queue named[26183]: [ID 295310 daemon.info]
> > XX /209.86.63.205/mpecllc.com/A/IN
> > Jan 11 09:28:03 queue named[26183]: [ID 295310 daemon.info]
> > XX /216.162.16.130/mpecllc.com/A/IN
> > Jan 11 09:28:05 queue named[26183]: [ID 295310 daemon.info]
> > XX /206.64.117.231/mpecllc.com/A/IN
> > Jan 11 09:28:05 queue named[26183]: [ID 295310 daemon.info]
> > XX /216.162.16.131/mpecllc.com/A/IN
> > Jan 11 09:28:07 queue named[26183]: [ID 295310 daemon.info]
> > XX /68.57.192.6/mpecllc.com/A/IN
> > Jan 11 09:28:09 queue named[26183]: [ID 295310 daemon.info]
> > XX /68.168.192.5/mpecllc.com/A/IN
> > Jan 11 09:28:17 queue named[26183]: [ID 295310 daemon.info]
> > XX /152.3.250.1/mpecllc.com/A/IN
> > Jan 11 09:28:21 queue named[26183]: [ID 295310 daemon.info]
> > XX /24.28.99.62/mpecllc.com/A/IN
> > Jan 11 09:28:25 queue named[26183]: [ID 295310 daemon.info]
> > XX /137.159.198.137/mpecllc.com/A/IN
> > Jan 11 09:28:31 queue2 named[29380]: [ID 295310 daemon.info]
> > XX /12.38.46.250/mpecllc.com/A/IN
> > Jan 11 09:28:31 queue2 named[29380]: [ID 295310 daemon.info]
> > XX /209.244.4.51/mpecllc.com/A/IN
> > Jan 11 09:28:31 queue2 named[29380]: [ID 295310 daemon.info]
> > XX /63.243.88.30/mpecllc.com/A/IN
> > Jan 11 09:28:32 queue2 named[29380]: [ID 295310 daemon.info]
> > XX /69.152.0.5/mpecllc.com/A/IN
> > Jan 11 09:28:36 queue2 named[29380]: [ID 295310 daemon.info]
> > XX /64.78.119.1/mpecllc.com/A/IN
> >
> > Jan 11 09:28:37 queue2 named[29380]: [ID 295310 daemon.info]
> > XX /192.216.106.50/mpecllc.com/A/IN
> > Jan 11 09:28:37 queue2 named[29380]: [ID 295310 daemon.info]
> > XX /32.97.118.68/mpecllc.com/A/IN
> > Jan 11 09:28:39 queue2 named[29380]: [ID 295310 daemon.info]
> > XX /206.230.181.2/mpecllc.com/A/IN
> > Jan 11 09:28:40 queue2 named[29380]: [ID 295310 daemon.info]
> > XX /205.188.118.22/mpecllc.com/A/IN
> > Jan 11 09:28:43 queue2 named[29380]: [ID 295310 daemon.info]
> > XX /64.7.232.10/mpecllc.com/A/IN
> > Jan 11 09:28:46 queue2 named[29380]: [ID 295310 daemon.info]
> > XX /216.226.178.11/mpecllc.com/A/IN
> > Jan 11 09:28:51 queue2 named[29380]: [ID 295310 daemon.info]
> > XX /129.219.13.81/mpecllc.com/A/IN
> > Jan 11 09:28:31 webserv named[24989]: [ID 295310 daemon.info]
> > XX /24.247.24.41/mpecllc.com/A/IN
> > Jan 11 09:28:35 webserv named[24989]: [ID 295310 daemon.info]
> > XX /151.164.1.3/mpecllc.com/A/IN
> > Jan 11 09:28:37 webserv named[24989]: [ID 295310 daemon.info]
> > XX /216.144.187.199/mpecllc.com/A/IN
> > Jan 11 09:28:38 webserv named[24989]: [ID 295310 daemon.info]
> > XX /12.34.129.27/mpecllc.com/A/IN
> > Jan 11 09:28:38 webserv named[24989]: [ID 295310 daemon.info]
> > XX /205.152.132.23/mpecllc.com/A/IN
> > Jan 11 09:28:42 webserv named[24989]: [ID 295310 daemon.info]
> > XX /66.133.128.138/mpecllc.com/A/IN
> > Jan 11 09:28:42 webserv named[24989]: [ID 295310 daemon.info]
> > XX /204.127.198.60/mpecllc.com/A/IN
> > Jan 11 09:28:56 webserv named[24989]: [ID 295310 daemon.info]
> > XX /24.29.99.16/mpecllc.com/A/IN
> > Jan 11 09:28:56 webserv named[24989]: [ID 295310 daemon.info]
> > XX /167.206.3.249/mpecllc.com/A/IN
> > Jan 11 09:28:56 webserv named[24989]: [ID 295310 daemon.info]
> > XX /216.110.87.10/mpecllc.com/A/IN
> > Jan 11 09:28:58 webserv named[24989]: [ID 295310 daemon.info]
> > XX+/66.189.130.21/mpecllc.com/A/IN
> > Jan 11 09:28:59 webserv named[24989]: [ID 295310 daemon.info]
> > XX /216.47.193.14/mpecllc.com/A/IN
> > Jan 11 09:29:03 webserv named[24989]: [ID 295310 daemon.info]
> > XX /66.189.130.5/mpecllc.com/A/IN
> > Jan 11 09:29:07 webserv named[24989]: [ID 295310 daemon.info]
> > XX /129.22.4.3/mpecllc.com/A/IN
> > Jan 11 09:29:07 webserv named[24989]: [ID 295310 daemon.info]
> > XX+/206.47.244.102/mpecllc.com/A/IN
> > Jan 11 09:29:07 webserv named[24989]: [ID 295310 daemon.info]
> > XX /199.18.39.5/mpecllc.com/A/IN
> >
> > I have no immediate power to isolate where this is coming
> > from but I can make you all aware of
> > what it is doing exactly. mpecllc.com is now pointed to a
> > 127. IP address but this has not
> > stopped the queries on our name servers, as you can see above.
> >
> > thanks ahead of time for any help you can offer..
> >
> > regards
> >
> > Domain Name Services - AdvancedTelcomInc
> > David C. McCall - david at atgi.net
> > _______________________________________________
> > Intrusions mailing list
> > Intrusions at lists.sans.org
> > http://www.dshield.org/mailman/listinfo/intrusions
> >
> _______________________________________________
> Intrusions mailing list
> Intrusions at lists.sans.org
> http://www.dshield.org/mailman/listinfo/intrusions
>
>
>
>
>
> _______________________________________________
> Intrusions mailing list
> Intrusions at lists.sans.org
> http://www.dshield.org/mailman/listinfo/intrusions
>
_______________________________________________
Intrusions mailing list
Intrusions at lists.sans.org
http://www.dshield.org/mailman/listinfo/intrusions
!DSPAM:41ed941e152996990113254!
More information about the Intrusions
mailing list