Re: removing an ext2fs file forces disk activity

[Marcus Brinkmann]

On Mon, Mar 25, 2002 at 08:47:44PM +0100, Niels Möller wrote:
> How hard would it be to create a new store type that basically
> implements only a write-cache: It would have store_write put the
> modified block into a queue, from which blocks are written to the
> underlying store later by a separate syncing thread. store_read would
> have to check this queue for blocks that are modified but not yet
> written to disk, but would otherwise just pass the read on to the
> underlying store.

Well, if this is possible we could just avoid syncing the blocks back to the
store in the filesystem in the first place.  Thomas' point is exactly that
to make the expected guarantees you can not cache the writes in any
simplistic fashion:  You have dependencies between some writes, where you
have to assert a special order of doing the write back (imagine the task
crashes in the middle of such a store synchronization run).

> I'm a little confused about the interface between the filesystem and
> its store,

For each file, a memory object is created.  This is then vm_map'ed into the
server task.  See ext2fs/pager.c, diskfs_get_filemap, and libpager.
This is the kernel's external pager interface.  Whenever you access the
vm_map'ed region and fault, the kernel will send a message to the pager (in
libpager, linked into the servers address space).  This will then lead to
pager_read_page being called, which is a callback function for libpager.
If you look into ext2fs/pager.c, this will eventually call store_read.

> If in fact store_read and store_write, not store_map, is the interface
> that is used, that should make caching a little easier.

Well, this is the interface used by the pager to read/write the underlying
store data.  Note that the actual caching is done in the kernel
(gnumach/vm/*).

> Where are the rpc:s defined that are used by stores and pagers? I
> would have expected som hurd/store.defs and hurd/pager.defs, but I'm
> not very familiar with the hurd sources.

For stores, only the fsys_get_storage_info is interesting at startup,
usually filesystems open the underlying store themselves and store_read and
store_write are really library functions.

For the pager, you also have the libpager library with its callback
functions pager_read_page etc, which are called whenever libpager receives
an RPC from the kernel to page in or out pages from the store.  There is
also a synchronization interface (and the amount of synchronization done by
the filesystem here is what started the discussion.  For the kernel<->pager
communication, you need to look at the external memory object interface in
Mach.

If you want to get better caching by dropping synchronization you don't need to 
add a cache
under the existing cache, you can just drop the synchronization.  However,
if you add caching you will have to face the synchronization issues either
way if you don't drop synchronization issues.

Thanks,
Marcus


-- 
`Rhubarb is no Egyptian god.' Debian http://www.debian.org brinkmd@debian.org
Marcus Brinkmann              GNU    http://www.gnu.org    marcus@gnu.org
Marcus.Brinkmann@ruhr-uni-bochum.de
http://www.marcus-brinkmann.de