[unisog] IDS vs. Privacy

Peter Van Epp vanepp at sfu.ca
Tue Feb 3 02:27:55 GMT 2004

On Mon, Feb 02, 2004 at 02:33:10PM -0600, E. Larry Lidz wrote:
> Hello,
> I was asked, as moderator, to pose this question to the group from
> someone at an institution who wished to remain anonymous. They fear that
> if this message was public their institution might be the target of
> unwanted attention from the underground.
> The institution has about 25,000 machines on their network, and had been
> running an IDS system which received a copy of all traffic across the
> network's gateway to the Internet/I2. The IDS system had a track record
> of being successful -- it detected most of the viruses, worms, port
> scans, spam relays, proxies, rogue FTP sites, rogue IRC bots, and so
> forth.
> IT management then changed. The IDS system was shut off with no advance
> notice over the concern that it might lead to a compromise of privacy
> policies. The new management believes that people having access raw
> packets is an unacceptable risk. They felt that technologies that
> summarize information (Cisco Flows from a router/switch, mirroring
> traffic to an IDS system that has no ability to sniff, etc.) about the
> traffic is acceptable, however.

	I suspect someone is unclear on the concept here. If you mirror (or
put in an ether or fibre tap which is my preference) link traffic you can't 
modify (even by accident or compromise of your IDS/monitor which you should be
considering the possibility of) the production traffic, but you can certainly 
still sniff it with the privacy risk of data exposure that entails. At some 
point you need to trust your staff to not sniff traffic if they aren't supposed 
to (or more likely trust your staff to not disclose confidential information
that they have been exposed to) because if they have physical access, there 
isn't any way I know of that you can in practical terms stop them  except to 
be able to trust them to obey the policy.
	Moreover the IDS doesn't affect (other than it may prevent it by 
detecting the host compromise) an attacker from compromising a host or piece
of network gear and sniffing the traffic. This is a far more likely privacy
risk than your IDS I would think. The IDS is probably a lot better secured than
most of your hosts.

> They would like to know: has anyone been in a similar situation? If so,

	No, although we considered this potential objection from the user 
community when we originally decided what to do in this regard and now I'm
fighting for our current solution because I think its better than any of the
more intrusive IDS solutions I've seen :-).

> were you able to bring back your IDS? What arguments were compelling to
> management?

	Plausable deniablity :-)? It amounts to a balance between privacy on 
the one hand and security/cost/contractual obligations associated with the 
external network connection on the other hand. This is something that your
managment and possibly your user community has to work out for themselves.
Around here we tend to copy the Telephone company model: in order to insure
the proper operation of systems and networks our technical staff can do and
look at anything necessary to achieve that aim. What they can't do without 
written authorization from the VP level is disclose anything that they see 
while keeping the systems and network running. The analogy is to a Telco line
technician monitoring your phone line to maintain line quality. If he or she
hears you plotting a crime while doing that they can't disclose that 
information to anyone else. There may be a court approved wiretap that detects
that same information in an authorized manner, but the technician that discovers
it as part of maintance isn't allowed to disclose it to a third party.

>	      Are other institutions similarly concerned about the privacy
> issues involved? Why or why not?

	For many years now there has been an argus server (via a network tap
so that no matter what may happen to it, it can't affect the outgoing traffic)
that collects the header information (but totally ignores user data in the 
packets) and records it in compact form. Note that the latest argus version can
also collect the first couple of hundred bytes of user data if enabled, but we
have not done so at this point (for one thing it doubles the already large 
amount of data generated, but the privacy issue is more important). I expect
I could probably get approval on the same basis as above i.e. nothing captured
will be disclosed and the data will be only used to protect the network and
machines from attack if I thought it was worthwhile, but so far I don't see the
	It may be profitable to explore the option that the IDS will look at
the data, but what it finds won't be disclosed other than in the form "your
machine appears to have virus X" or "your machine looks to have been 
compromised  by exploit Y" according to the IDS. I.e. the machine has seen the
private data in order to do its job (and the machine may include an analyist
to help it over the hard bits :-)) but neither of them will disclose the 
details of what was found. 
	Having initially deployed argus at least somewhat to avoid the privacy
issue, I now resist my boss's offer to buy me the latest super duper AI enabled
all bells and whistles IDS that will make my life wonderful because I haven't
found one yet that works as well as argus does for us. The down side is that
argus takes a reasonable amount of skilled staff time to run. The attraction
(or so the salesman keeps demonstrating on the overhead projecter if not yet
in real life :-)) of IDS is that it will free up that manpower. So far I 
haven't seen that or heard of a real site that agrees with that.
	It does however depend on what you are trying to achieve. We are trying
to protect the institution from being cut off the net and/or sued (not always
successfully :-)) for attacks originating on our site and heading outwards.
In theory we are also concerned about blowing the bank on network bandwith 
charges, but I don't seem to have found the fatal pain level on that just
yet ...  We tend to be less concerned in the general case (business systems 
are an exception) about internal attacks, partly because we don't have the 
staff or funding to do anything about it. In this senario argus works well. 
Most any attack or compromise (including ones such as an open anon ftp server
that has been exploited that an IDS without a notion of how much traffic a host
has done lately have trouble detecting without false positives) will show up 
as a change in the normal data patterns, usually outbound network traffic in 
either volume or number of offsite addresses accessed (in the case of viri and 
port scans) or both accessed by an internal host. This is especially striking 
when you can look at yesterday, last week or last year for the same host 
(although I rarely have to do that, the previous 24 hours is usually enough).
	For instance the 30 or so hosts that got hit by mydoom last week got 
detected sometimes by the mail server detecting them attempting to send a 
virus (at the cost of having to rush and additional loadbalanced virus checker 
in to production to handle the sudden load) but mostly by argus detecting that 
an internal host that isn't one of our mail servers is sending a large number 
of sizable messages via SMTP to hundreds of its closest new friends (new in 
the last hour or so in fact). There may have been a false positive or two, but 
by and large the machines flagged were infected when their support persons 
checked them out once the virus sigs were out. They were quickly supressed and
while we are still seeing ones and twoses (and are still checking) the crisis
was dead by around Tuesday (except for the poor support folks that had to clean
the machines up of course). 
	The thing to remember in this is that all this is being done with only 
the network headers (cflow data would do as well as argus I expect, there are 
conversion programs for one to the other these days) which mostly avoids the 
privacy issues that started this discussion. So its possible that you want to
change your methods rather than argue for your previous implementation 
(although it will almost certainly take less resources to be allowed to 
reinstate a previously implemented and understood solution). 
	Hope this helps somewhat and good luck!

Peter Van Epp / Operations and Technical Support 
Simon Fraser University, Burnaby, B.C. Canada

More information about the unisog mailing list