Re: Review of Thomas's >2GB ext2fs proposal

[Thomas Bushnell BSG]

"Neal H. Walfield" <neal@cs.uml.edu> writes:

> De-mapping is not an eviction strategy.  How do you choose which
> mappings to get rid of once the cache is filled?  If the operations
> are really of trivial cost, then we needn't cache the mappings; we
> just vm_map and vm_unmap the pages each time they are requested and
> thereby eliminate any reference counting system.

I don't *have* an eviction strategy, because I don't *need* one,
because I'm never evicting anything.

The mappings are of small cost, but not zero cost, so it's worth a
reference count to keep them around.

> What is the time stamp you will use?  I don't need to save any extra
> data in my proposal.  This is extra memory you are using.

No time stamp necessary, you just maintain an LRU list which has all
the currently unused mappings.  When a ref count goes to zero, the
entry is dropped on the end of the list, and when you need a new
mapping, you pull from the front of the list, vm_deallocate the
specified region, and then vm_map anew (with anywhere==TRUE, btw).

> Sure.  But a reference is held for a relatively short time (most
> mappings will have a reference count of zero).  Hence, if a page has
> no references it does not mean it will not be used in the near future.
> Consider how a sequential read works: client asks for a page, meta
> data is consulted (hence a reference is gotten), block list is
> obtained (reference is dropped), data is gotten and returned to user
> which process the data; repeat.  For most of the time, the reference
> on the page is zero.

Right; my point is that you still need a reference count.  The actual
handling of the reference count is identical for both your and my
strategy. 

> Why is the kernel dropping the page if it is active in the first
> place?  This logic seems suspect.  

ANY page may be paged out at ANY time.  You MUST make that assumption
and be extremely careful that you are not assuming ANYTHING about when
the kernel will schedule a pageout.  You should assume, when writing
for correctness, that a complete pageout happens of anything you care
about between every two instructions.  You cannot assume "oh, I just
used this in the last instruction, so it will still be around."

You don't need to worry about such cases for performance: they can be
assumed to be rare.  But you must worry about them for corrected: they
cannot be assumed to be impossible.

> The interesting corner case is what happens if there are user
> references and the kernel drops the page.  Well, in this case, we
> just leave the mapping in place based on the assumption that the
> page will be faulted back in momentarily (otherwise, why would there
> by a reference?).

This is incorrect.  You might be just about to drop the reference,
having already completed all the uses of the page, when the page gets
paged out.

Thomas