[unisog] HTTP Session Reconstruction and Monitoring

Jay Giyanani jgiyanani at niksun.com
Fri Nov 10 17:30:15 GMT 2006

Hello Peter,
please review the message that was posted in 2004. We have this solution
available that can handle all the requirements. It is called NETDETECTOR.
Attached please find the brochure. If you are interested or need more
information, please feel free to contact me.
Best Regards,
Jay G.
On Fri, Dec 10, 2004 at 02:23:20PM -0700, Jacob Roberts wrote:

> We would like to improve our ability to monitor inappropriate web

> surfing activity on our public access workstations.


> We have 3 basic requirements. The system can:

> 1. Handle our large amount of traffic

> 2. Reconstruct HTTP sessions (e.g. an analyst can retrieve a view of the

> visited web site based on captured packet data)

> 3. Configure rules for specific traffic matching.



> Does anyone know of any enterprise level applications that can do these

> things.  We'd prefer an Open-source solution.


> We are currently testing Steel-Cloud by Computer Associates.  It meets

> reqs 2 and 3 but fails to meet our needs for req 1.


> Thanks,

> Jake Roberts

> Brigham Young University

> _______________________________________________

> unisog mailing list

> unisog at lists.sans.org <http://www.dshield.org/mailman/listinfo/unisog> 

> http://www.dshield.org/mailman/listinfo/unisog

	While I don't know of an application that will do what you want out

the box, assuming that you can get the required traffic to disk via tcpdump

(and appropriate filters if required), which may of course not be a trivial

task depending how fast your link is :-), then an open source application 

called tcpflow will reconstruct session data from tcpdump input (the

for 3) would need to be tcpdump filters and you would probably need some

or such like to mung the tcpstreams in to appropriate html to ship at a

again to get it to display for your analyst. The etherial network sniffing 

package is also reputed to be able to recover tcpstreams as well, although I

expect for what you want tcpflows file output will be more useful.

	Since it looked like item 1 was a problem for the solution that you 

found, then, assuming you aren't Linux adverse, there is a kernel mod 

(ringbuffer) from www.ntop.org that may help. It basically short circuits

entire tcp stack and mmaps the input buffer from the adapter in to a

copy of libpcap. Assuming that solution can read from a disk file of the 

transaction that may mean problem solved with a tcpdump that can get the

to disk at wire speed and then feeding it to the CA application.

	On my argus sensor box (which is only using the first 128 bytes of

packet though, and you will need all of it presumably) I can keep up with a 

jumbo frame transfer at ~950 megabits per second on a gig link without

packet loss (before this the same machine lost %50 of the traffic even at

128 bit slice length). Unfortunatly the jumbo frames are the best case

and the more typical web mix of smaller packets may still induce packet loss

from the author's testing (as may trying to go to disk without a good 

multispindle raid controller!) depending on your link speed. 

	There is a fair bit of tinkering required here (and possibly a lot

high speed data capture knowhow required :-)) but it may be possible and 

possibility increases as link speed reduces and thus the strain of the

reduces as well.

	I'm also presuming that you have considered the privacy and legal 

implications of doing this in your juristiction. Depending on whether the

has a reasonable expectation of privacy (which may be negated by signon 

banners or messages on the wall, but you likely need legal advise by a

familiar with your juristiction's laws) this may be considered wiretapping

thus illegal.

Peter Van Epp / Operations and Technical Support 

Simon Fraser University, Burnaby, B.C. Canada"


Jay Giyanani 
Manager : International Business Development 
Niksun, Inc. 
1100 Cornwall Road 
Monmouth Junction, NJ 08852, USA 
Tel: 732-821-5000 x 3374 
Fax: 732-821-6000 
Cell: 609-774-3229 
Email: jgiyanani at niksun.com 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.sans.org/pipermail/unisog/attachments/20061110/5e130171/attachment-0001.htm 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: NIKSUN_NetDetector_Datasheet_v4.pdf
Type: application/pdf
Size: 1675352 bytes
Desc: not available
Url : http://lists.sans.org/pipermail/unisog/attachments/20061110/5e130171/attachment-0001.pdf 

More information about the unisog mailing list