[unisog] Quota system based on netflows

Peter John Hill pjhill at u.washington.edu
Tue Jan 30 18:03:00 GMT 2007


Just getting caught up in this thread. I was part of the team at  
Carnegie Mellon that solved our bandwidth crisis that peaked at the  
release of Kazaa. One of my coworkers and I gave a presentation at an  
Internet 2 Joint Techs conference on the subject:

http://www.net.cmu.edu/pres/jt0803/

The story is also a case study in:
http://bwmo.net/

The subject of campus bandwidth management is near and dear to my  
heart. Some comments, hopefully helpful

Jonathan Glass jonathan.glass at oit.gatech.edu wrote on Tue Jan 30  
16:03:38 GMT 2007

> Just as an aside, we capture an average of 18GB a day of Netflow.  I'd
> hate to have to process all that and make policy decisions in real  
> time.

Why do all that processing all at once? The solution at CMU was to  
have your collectors suck down your traffic from span ports on your  
routers (we collected full core and border traffic with separate  
boxes). All the probes do is create the flow data. They don't need  
big disks, only fast processors and NICs. There are some good choices  
in cards now.. Before, Endace was the best game in town.. Now I am  
evaluating mryi.com products to see how they do.

Then you have collectors. They do not need ten gig nics, as the flow  
data compresses very nicely. They do need lots of disks and a good  
amount of cpu. Multiple processors do nicely, as this host will be  
running ra or radium and collecting data from the probes. They  
connect to a sasl protected tcp port on the probes and pull in the  
argus data and write it to disk. Then, every so often, say 5 minutes,  
you run the argusarchive process that copies that data into a logical  
directory structure based on date and gzips the file (all argus  
clients can read gzipped files without needing to zcat or un gzipping  
it).

Now, instead of once per day analyzing 18 GBs of data, you take that  
5 minute collection of data and do your analysis and summarize it..  
In this case, you would want to correlate mac addresses with flow  
data. You would want to tie your dhcp lease information in with the  
flow data so that you have a better idea on which machine is actually  
sourcing the data. Then, at the end of the day, you take all those  
summaries and aggregate them into a daily top talkers list.

CMU has a database with a web front end that anyone on campus can  
look up a machine registered to them and figure out how much  
bandwidth it used..

IMPORTANT NOTES:
This system allows CMU to place policy limits on only commodity  
Internet traffic. They currently do not restrict any bandwidth across  
Internet2/Research Networks.

The system at CMU applies to everyone on campus. It allows exemptions  
to machines that have been blessed by whichever department controls  
that machine (someone has to own the machine)

They have a system in place that allows users to register mac  
addresses and track ownership.. If you do not have this information,  
it is hard to proceed with anything more refined that the hard whack  
of unintelligent rate limits.

If you are someone that is looking at this problem and want to find a  
solution, check out the two links above. If the band-aid you are  
using now is working... fantastic. CMU's band-aid (an Allot  
Netenforcer) worked for about a year.

good luck!
Peter



More information about the unisog mailing list