[unisog] logistics of campus wide scanning and passive traffic analysis

Russell Fulton r.fulton at auckland.ac.nz
Sun May 9 02:07:54 GMT 2004

On Sun, 2004-05-09 at 11:25, Peter Van Epp wrote:
> 	As to scanning the entire net, should we have a serious problem I expect
> that such scanning will get approved (whether the manpower is available to do 
> it or deal with the results is another question however :-)).

I can and do scan and analyse our /16 and report results to the people
who need to know in about 15 minutes of my time.  Typically this will be
a couple of minutes to create a new directory and kick off the scan then
10-15 minutes to do the analysis after the scan is complete, eyeball the
results and send them out to the support staff.

I have a script which reads a 'contacts' file consisting of a list of
     1. subnets and department affiliations 
     2. sub domains and department affiliations
     3. regexp which match dn 
     4. contacts for departments
[aside: I really want to get all this information into our network
management database where it belongs...]

and then reads a scan results file deciding who to notify for each
machine.  The script can read several standard formats such as nmap (I
have not done one for xml yet, just the generic -oM) and IP or IP and
DN, etc.

It has evolved over the last 18 month and will no doubt continue to do
so (adding Nessus and Nmap xml output is on my list of things to do).

My point is that scanning and reporting need not be burdensome provided
you don't get too ambitious.  Running a full nessus scan is a total
waste of time, to be useful scans need to be tightly focused to avoid
false positives.

> 	I say semi acceptable cost because my boss firmly believes there is a
> "commercial AI driven IDS device" out there that will do everything at much
> less staff cost (and sometimes he has the vendor foils and the overhead 
> projector that is their preferred hardware platform to prove it :-)), and I'm 
> not looking for it hard enough. I expect I'm going to have another round in 
> this battle sometime fairly soon as another commercial miracle gets installed 
> beside argus and we see what it finds (although lately that wouldn't be much 
> of anything, there isn't much exciting going on). The last several times we've 
> tried this argus happily detected a compromise, and given the IPs, time and 
> port numbers of the compromise the commercial stuff either couldn't see it 
> (because detection depended on knowing how much traffic a host had done in the 
> last 24 hours, versus what it did in the 24 hours before that), or detected it 
> just fine along with 40 other false positives but with no indication which one 
> you should pay attention to. Its possible someone will manage to suprise me, 
> but they haven't yet :-). 

There is a commercial product that claims to do this, it was developed
here in Auckland.  It started life as a solution to the "ddos problem"
and I was invited to be part of the group who were developing it -- I
declined because, at the time, I thought they were peddling snake oil. 
The original people have moved on and the focus of the project shifted
to more realistic goals of classifying 'normal' traffic and thus
detecting and possibly dealing with abnormal traffic.  I've seen demos
of the system and now almost wish I had not been so picky ;)  The system
comes in two parts a monitor and an active response module which can
talk to routers and firewall to divert or block traffic.

I'm on leave at home at the moment and can't remember the url for the
product otherwise I'd post it.

We are about to do a trial installation of the monitor and if it lives
up to its promise I will report here with all the gory details :)

BTW I don't see this as a replacement for argus -- it won't help me find
out how a machine was compromised yesterday (let alone two weeks ago),
but it may be more effective at spotting generic changes in behaviour
that signal a compromise or misuse.  It remains to be seen whether or
not the false +ve rate is acceptable in our environment where change is
constant and normal.  Also argus (or scripts using argus data) can
always be used to find specific behaviour that generic tools like this
might miss once one knows what to look for.

I agree with Peter that the immediate of future Network Intrusion
Detection lies in the analysis of traffic patterns not traffic content
and I also agree with John that host based measures are going to become
more important too. I see these approaches as complementary, not as

Aside:  Marty's talk at CanSecWest was not about Snort per se but about
passive traffic analysis, Sourcefire are moving into this area now and
have a product that works along side snort. 


More information about the unisog mailing list