[Dshield] Shorewall log parser?

Bogdan Stancescu mgv at fx.ro
Thu Nov 14 21:59:00 GMT 2002


Thank you for all the info!

Here are all my conclusions, for future reference (searched for 
"shorewall" in the mail archives and found no reference):

- both the iptables framework and dshield.py parse shorewall log entries 
correctly. I chose dshield because it's way faster and it also produces 
dshield logs as opposed to iptables logs (which I understand is more 
efficient at your end);
- I use Mandrake and I get duplicate log entries - both in syslog and in 
messages. I chose to use messages because it seems more standard, but 
there's little other reason to pick any of the two in particular;
- Mandrake is set up to compress the rotated logs, so that may be a 
problem with dshield.py because it parses the whole file. I set up 
logrotate to rotate messages daily, and used its prerotate section to 
perform the parsing and send the message as documented in the framework 
documentation (trivial edit to run dshield.py);
- haven't checked out the database thing dshield.py does because I'm too 
lazy to - but I think it's worth a look if you have more time and 
disposition than myself to tweak it.

What I still don't understand is if (and if possible, how) I could 
report http attack attempts which are logged in my Apache log with a 
completely different format. Although I fully understand that's not the 
kind of data you people expect (there's no way to tell if an http 
request was a potential attack or not without looking at the actual http 
request, and you don't store that kind of data), I see such a request 
not in the least conceptually different from any other kind of attack 
over the net. IMHO a couple of well thought regexps could match most 
such attempts and log them as well...

Anyway, hope this message helps some future newbies like myself :)

Bogdan

Wayne Larmon wrote:

>>Errata: I just saw what I wrote below - I have a natural gift for
>>writing wrong paths - it's obviously not /etc/messages but
>>/var/log/messages.
>>
>>BTW (and this is more or less off-topic), I get such log entries both in
>>messages and in syslog. Should I expect them to be different? Is one
>>recommended for parsing over the other? Or should I somehow merge them
>>prior to parsing?
>>    
>>
>
>I'm not sure what Shorewall is doing.  In the versions of Linux that I'm
>familiar with, everything goes to /var/log/messages.  There is no "syslog"
>file.  It is up to you to add extra critera to /etc/syslog.conf to direct
>certain log lines to go to other log files.  Did Shorewall do something like
>this?  To direct the iptables log lines to a different log file, in addition
>to going to /var/log messages?
>
>If they did do this, and the log lines are going to both /var/log/messages
>and to a syslog file, then you only want to process from one of them, so
>that you aren't sending us duplicate log lines.
>
>  
>
>>Anyway, dshieldpy seems to be working, so I'll use that if you're going
>>to tell me that line looks right (didn't want to start sending logs
>>before making sure they're properly formatted). The last "2" on the
>>second line in my previous line shouldn't be there - I accidentally
>>copied the first character on the next line.
>>    
>>
>
>This looks correct
>
>2002-11-13 11:26:20 +02:00	59254230	12	200.31.37.65	2686	217.156.116.130	21
>TCP	S
>
>Send them in.
>
>Wayne Larmon
>DShield.org
>
>
>  
>
>>Wayne Larmon wrote:
>>
>>    
>>
>>>>Yes, that's the first thing I tried - didn't work. The problem is I
>>>>don't know how pure iptables logs look like. Here's how a Shorewall
>>>>entry looks like in my /etc/messages (told you I'm a newbie - please let
>>>>me know if I'm looking in the wrong place):
>>>>
>>>>Nov 10 02:34:39 bogdan kernel: Shorewall:net2all:DROP:IN=eth0 OUT=
>>>>MAC=52:54:05:e5:67:42:00:02:44:4f:2b:9b:08:00 SRC=213.224.93.36
>>>>DST=217.156.116.130 LEN=48 TOS=0x00 PREC=0x80 TTL=112 ID=34579 DF
>>>>PROTO=TCP SPT=1334 DPT=3042 WINDOW=8192 RES=0x00 SYN URGP=0
>>>>
>>>>
>>>>        
>>>>
>>>This is correct.  This is an iptables log line.  I just tried
>>>      
>>>
>>our Framework
>>    
>>
>>>iptables script on this line, and it converted it OK.
>>>
>>>http://www.dshield.org/framework.html
>>>
>>>If you are having configuration problems with this client, contact me off
>>>list.
>>>
>>>Wayne Larmon
>>>wlarmon at dshield.org
>>>DShield.org
>>>
>>>
>>>
>>>      
>>>
>>>>Wayne Larmon wrote:
>>>>
>>>>
>>>>
>>>>        
>>>>
>>>>>>Hello everybody!
>>>>>>
>>>>>>I'm a newbie, so please don't flame if I happen to ask the wrong
>>>>>>question - looked for an answer both on dshield's site and on
>>>>>>            
>>>>>>
>>Google and
>>    
>>
>>>>>>haven't found it...
>>>>>>
>>>>>>...And the question obviously is: is there any readily available
>>>>>>Shorewall log parser for dshield?
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>            
>>>>>>
>>>>>http://www.shorewall.net/ says that it uses iptables.  Have you
>>>>>
>>>>>
>>>>>          
>>>>>
>>>>tried one of
>>>>
>>>>
>>>>        
>>>>
>>>>>our iptables scripts?
>>>>>
>>>>>http://www.dshield.org/framework.html
>>>>>
>>>>>or http://www.dshield.org/linux_clients.html#dshieldpy
>>>>>
>>>>>
>>>>>Wayne Larmon
>>>>>DShield.org
>>>>>
>>>>>
>>>>>_______________________________________________
>>>>>Dshield mailing list
>>>>>Dshield at dshield.org
>>>>>To change your subscription options (or unsubscribe), see:
>>>>>
>>>>>
>>>>>          
>>>>>
>>>>http://www.dshield.org/mailman/listinfo/list
>>>>
>>>>
>>>>        
>>>>
>>>>>
>>>>>          
>>>>>
>>>>_______________________________________________
>>>>Dshield mailing list
>>>>Dshield at dshield.org
>>>>To change your subscription options (or unsubscribe), see:
>>>>http://www.dshield.org/mailman/listinfo/list
>>>>
>>>>
>>>>
>>>>        
>>>>
>>>_______________________________________________
>>>Dshield mailing list
>>>Dshield at dshield.org
>>>To change your subscription options (or unsubscribe), see:
>>>      
>>>
>>http://www.dshield.org/mailman/listinfo/list
>>    
>>
>>>
>>>      
>>>
>>_______________________________________________
>>Dshield mailing list
>>Dshield at dshield.org
>>To change your subscription options (or unsubscribe), see:
>>http://www.dshield.org/mailman/listinfo/list
>>
>>    
>>
>
>
>_______________________________________________
>Dshield mailing list
>Dshield at dshield.org
>To change your subscription options (or unsubscribe), see: http://www.dshield.org/mailman/listinfo/list
>
>  
>




More information about the list mailing list