Re: shred: bug or feature? think you should know about it

[Anton Chuvakin, Ph.D.]

Bob,

Thanks for the message.

>If you would be so kind as to educate this poor hacker.  What is
>"slack" space?
ext2 uses blocks (here: minimum addressable parts of disk) of whatever 
size, depending upon the partition size. In my case, block size was 4KB. 
mke2fs chooses from 1,2,4 KB when creating the ext2 fs on the disk.
If you store a small file (less than block size) or a file with size 
with not a factor of 1,2,4 KB, some space (up to a block size per file) is 
wasted and is NOT accessible from the normal filesystem interface.

Some more details are 
http://www.linuxdoc.org/HOWTO/mini/Partition/formating.html 
Quote: "Files come in any size. They don't end on block boundaries. So
with every file a part of the last block of every file is wasted. Assuming
that file sizes are random, there is approximately a half block of waste
for each file on your disk."

>> 2. # dd if=/dev/zero of=/mnt/floppy/oooo ; sync ; /bin/rm /mnt/floppy/oooo ; 
>> sync
>This confuses me.  What is /mnt/floppy?  Is it a local directory or
>has it been previously mounted?  I assume you droppd a mount command
>after having run mkfs on a floppy.  I assume you mounted it between
>steps 1 and 2?
Yeah, my bad ;-)

1a. mount /mnt/floppy

>I assume the dd stops when the output device fills up and returns an
>error?  Thereby completely filling up the filesystem upon which
>/mnt/floppy is mounted?  Just to zero everything, of course.  Now into
>the core of the discussion.
Yes, that was just to show that there is nothing on the disk, only zeros. 
I zero the /dev/fd0 to show that the string is not a leftover from 
yesterday experiments ;-)

>A file on the filesytem, presumably from the 'mkfs -t ext2 /dev/fd0'
>above but without knowing the contents of your /etc/fstab this is
>uncertain.  [And in fact the mount command would depend upon the OS
>you are running on.  Likely this is linux since other systems would
>actually require you to say what you were mounting where.]
Yeah, I just created a small file with those two words on a floppy. No 
magic here. You are right about fstab, but it indeed associates 
/mnt/floppy with /dev/fd0.

>> 5. # echo "evil data" | bmap --mode putslack /mnt/floppy/TEST
>Now the real questions.  What is 'bmap'?  What is slackspace?  What
>does bmap do to file TEST?
Ah! That is where it gets interesting. bmap is a curious little sucker of
a program that uses its own interface to ext2 to get to files. Available
from ftp://ftp.scyld.com/pub/forensic_computing/bmap/ Full story is in the
ps file included with RPM package (but not with source, for whatever
reason). bmap can operate (read, write, clean) with slack space, i.e. with
wasted space remaining in the block.

>If I were to 'grep data /mnt/floppy/TEST' what would I see?
# cat /mnt/floppy/TEST
good data

And it is the only file on the /dev/fd0 filesystem.

>> 6. # strings /dev/fd0 | grep data
>> good data
>> evil data
>So both strings are 'on the floppy' somewhere.  But not necessarily in
>the file TEST.
Well, technically "evil data" is not IN the file TEST, but it is stored in 
the same disk block as file TEST.

>> 7. # shred -u -v -z /mnt/floppy/TEST
>> 8. # strings /dev/fd0 | grep data
>> evil data

>on a journalling filesystem.  
>[If I had been polled as to whether
>shred should be included in utils I would have voted not since it is
>too hard to get right.  Better to not include it at all.]
Yeah, I totally agree here. At least, after shredding one should always do 
smth to the effect of:  "grep whatever_was_shredded /dev/hdaX" (which is 
prohibitive for big disks, hard for binary files shredded, etc, etc). It 
is hard to promise unrecoverable deletion, no matter how many random 
overwrite passes one does, if filesystem just does smth weird with all 
this data.

>Since shred is only run on file TEST any data that is not stored in
>file TEST could not reasonably be expected to be overwritten.  The
>data here convinces me that bmap stores 'evil data' someplace other
>than in the file TEST.  Since shred only works on the file TEST it
>does not remove the evil data since it is elsewhere. 
Well, OK. I agree. Actually, knowing this made me write with "bug or
feature", rather then "bug" in the subject of my email. I think this
reasoning is OK, and thus you can consider it all a curious factlet, as
oposed to gaping hole in shred ;-)

>For that you would have to shred wherever bmap stored that data.  Where
>is that? What am I missing?
Ha-ha, you can't shred that. Well, if you shred ALL BLOCKS that file was
occupying, including a free space in the end, than you will kill the "evil
data" as well. And whether it is a good idea - I do not know, since one
will be shredding beyond the end of the file, however with no chance to
destroy anything but the file. 

Best,
-- 
Anton Chuvakin, Ph.D.
Senior Security Analyst 
netForensics -  http://www.netforensics.com
732-393-6071