[Dshield] DShield.py

Pieter-Bas IJdens pbijdens at emea.mi4.org.uk
Wed Nov 7 11:34:26 GMT 2001

> > You could try hard linking the file, instead of copying it.
> Isn't 'cp' supposed to be atomic? Or is only 'mv' safe?

The problem here is that there are updates taking place between the 'cp' of
the old log file, and the rotation. In these cases, using hard linking
allows you to fill the time-gap between the copy and the unlink action of
the old log file. Atomicity of cp plays no role here.

Hint: This is also very useful when backing up database files that get
replaced after a checkpoint has been sucessfully made to a shadow file. If
you hard link the original first, then even if the file gets unlinked and
replaced during the backup, you still consistently back up the old data by
backing up the linked file instead.

All you do know with cp is that you never get trash (random data). But
that's just because the order of the cached write and the file length update
is done correctly in the filesystem implementations. Atomicity is not
specified in the manual, nor in the info pages, so I would never assume it
is atomic in any way.

The 'mv', 'rm' and 'ln' commands are safe because they don't have any effect
on the file data. I tested a cross-filesystem move on linux 2.4.12 while
writing a 32MB file and the md5sum says mv is not atomic for the persisent

On the local fileysstem the results were always identical. Note that the
readback by the writing process detectes no anomalies ever. This is because
while the file is open it always operates on the local data, regardless of
what you do with the file [even if you unlink it]. You should maybe play
with this yourself. Please find attached the code I used to perform these
quick experiments. That and two shell windows suffice to re-create them.


-------------- next part --------------
A non-text attachment was scrubbed...
Name: atomic.c
Type: application/octet-stream
Size: 770 bytes
Desc: not available
Url : http://www.dshield.org/pipermail/list/attachments/20011107/0addd335/atomic.obj

More information about the list mailing list