Allocating many workspaces.

[John Darrington]

On Sat, Mar 17, 2012 at 12:15:17PM -0700, Ben Pfaff wrote:
     John Darrington <address@hidden> writes:
     
     Ah, yes.  I've been aware of related problems for a long time,
     but I haven't come up with a good solution.  One must limit the
     total memory allocated, not the memory allocated per-instance, of
     course, but the proper way to distribute the available memory
     among the competing users is not obvious.  I guess that the
     easiest way is first-come-first-served.  That might be just fine
     in the common case, so perhaps we should implement it that way as
     a first cut.

Unless the number of cases per instances is known a priori (which in general it 
isn't) I don't see any better alternative to first-come-first-served. -- perhaps
decadically decreasing might be one way, in the assumption that if there are 
many
instances, then hopefully they are small ones.

Is it feasible to have workspaces which dynamically change their allocation or
is that not possible?
     
     For categoricals, though, what's the fallback if the memory usage
     becomes too high?  Can we fall back to some kind of on-disk
     storage, or do we just fail?  "Just fail" is probably not a good
     way to go, if first-come-first-served is the strategy we use,
     because it means that unrelated memory use (e.g. for cases) can
     cause even small number of categories to break.

Maybe we should do the "just fail" option in the first instance and see
if we can improve it later.
     
     Here's another idea that comes to mind: is there a maximum number
     of categories that makes sense?  Would a "max categories" setting
     defaulting to, say, 1000, still allow most users to get real work
     done in realistic cases?

1000 would be much too high.  How many machines can allocate 64GB of heap?
"Realistic cases" is somewhat subjective.  But I cannot envisage that in 
most instances more than 20 categories would be involved - but who knows?


-- 
PGP Public key ID: 1024D/2DE827B3 
fingerprint = 8797 A26D 0854 2EAB 0285  A290 8A67 719C 2DE8 27B3
See http://keys.gnupg.net or any PGP keyserver for public key.

signature.asc
Description: Digital signature