Re: Review of Thomas's >2GB ext2fs proposal

[Neal H. Walfield]

Thomas,

At 16 Aug 2004 13:09:42 -0700,
Thomas Bushnell BSG wrote:

> What I really mean is that I can see no good reason to break the basic
> structure of the memory object model.  For example, there is no good
> reason to assume that the filesystem runs its own pager: in fact this
> is really a mistake.  
> 
> A much better design would have the disk driver pager be used, and not
> the filesystem's at all.  The only reason we can't do that is because
> we need to control to deal with the file data pagers, especially since
> the same disk block might be file data one day, and metadata (an indir
> block) the next.
> 
> So what a nice interface would be (starting over in the design of
> Mach) would allow multiple pagers to own a single memory page in some
> fancy ass way.  Failing that, we have to lump the two pagers into one
> program to get the necessary control.

Interesting.  But can't we do this already?  Instead of using
store_read and store_write in pager code, we need only use store_map
to get a memory object to the disk.  Then when we get a page in
request, we do a vm_map on the supplied memory object and return that
to Mach.

> But I think it's a mistake to take such heavy advantage of that
> control, and it seems somewhat brittle to me as well.  
 
> The only argument in favor of this proposal that I can see is that if
> the paging tables in the kernel are unpageable, and if they are really
> big, then my version might suck.  But I don't think that's a big
> factor, because they are pageable, and because I don't think they are
> so big anyway.
> 
> There is a disadvantage of my proposal which is the number of extra
> memory objects that have to be kept track of: because it's an array of
> memory objects, not just one, it might use more resources that way.

I will study Mach some more to make a better analysis.

> But on the other hand, your version suffers (in my opinion very
> severely) because you basically can't allow the kernel to cache data.
> You have taken over the caching policy for the kernel.  Since the
> assumption we operate under is that the kernel can do better caching
> than any user process (because it has a global view of the resource
> situation) we don't want user processes taking over caching of the
> actual disk data.  

There must be a misunderstanding here.  I have gone to great lengths
to make use that the mapping cache is perfectly harmonized with Mach's
paging strategy; in fact, I rely nearly completely on Mach's to make
any mapping eviction.

> But because you must purge the data from memory any
> time you remove the mapping, you now must think up a caching strategy,
> compare it with the real ones in the kernel, and make sure it
> interacts well.

Yes, we remove the data from memory any time we remove a mapping but
we only remove mappings when Mach evicts the data.  The only exception
is if the physical cache is able to grow larger than the mapping
cache.  Currently, the mapping cache is limited to 2GB and physical
memory is limited to 750MB.  Let's assume that we can get Mach to work
with 4GB of memory: still, it seems that this is a rare case.  But
let's be fanciful and try to get up to 64GB of ram.  In this case,
yes, we might have a problem.  However, I think we will be running on
64-bit architectures by then and all of this work will have been
useless anyway.

> By contrast, mine also requires a caching strategy, but it is only a
> caching strategy for memory mappings, which, unlike core, are not a
> limited globally shared resource.

But what is your caching strategy?  My caching strategy has only
previous pages as its overhead.  It is completely synchronized with
Mach's eviction scheme.  What is yours?

Thanks,
Neal