[Dshield] reporting on firewall logs

shawn reed shwn_rd at yahoo.com
Wed Oct 27 13:23:10 GMT 2004

Sorry, I didn't quite give all the details. There are 3 checkpoint firewalls that send their exported log files daily by ftp to a machine running Redhat 9. On the linux box, the logfiles are sorted using an awk script.
I wanted to get some feedback on the whether the method I use to sort traffic into incoming, outgoing and internal is accurate.
Thanks for the suggestion to send to mysql db. I expect this will speed up the processing time.
ULOG is not in the patch-o-matic. It's available from gnumonks.org. However, I'm not sure if the stock kernel has ULOG support. You may have to custom compile.
On Tue, 2004-10-26 at 13:07, shawn reed wrote:
> Redhat 9. Using gawk to extract and sort the logs.
>From Netfilter Patch-o-matic, install ulog (which substitutes for the
LOG target). Follow the instructions to input the ULOG output to a MySQL
db. The DB is updated in real time. Make sure that you set ULOG to batch
- print and store - 25 to 50 records at a time. This will save lots of
Once you have the data in a db you can do whatever you want with them.
MySql has the necessary built-ins to convert unix to human time and
binary IPs to octets.
> On Tue, 2004-10-26 at 10:57, shawn reed wrote:
> > I'd like to get some feedback/expertise on the process that I am using for reporting on firewall logs. Would this acurately reflect the profile of traffic through the firewall? 
> OS? DShield Client?
> >  
> > 1. Extract the following fields: date, time, action, protocol, src IP, dst IP, src port, src address.
> >  
> > 2. Divide log into three categories based on the direction of the traffic: incoming, outgoing, internal. 
> > (This is done by using a list of internal IP blocks and pulling out the lines of the log based on the src and dst IP. 
> > i.e. 
> > src, dst external_ip >outgoing.log
> > src external_ip, dst >incoming.log
> > src, dst >internal.log
> >  
> > 3. Create reports from the three categories of log files: incoming, outgoing, internal.

Do you Yahoo!?
Yahoo! Mail - You care about security. So do we.

More information about the list mailing list