[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Hurd Projects

From: Lars Weber
Subject: Re: Hurd Projects
Date: 22 Dec 2001 22:37:59 +0100
User-agent: Gnus/5.0808 (Gnus v5.8.8) Emacs/21.1

Adam Olsen <> wrote:
> I agree that seperating the httpfs translator and the caching
> mechanism would be worthwhile.  But why not just use an existing
> caching proxy such as squid?  Is there some specific functionality
> you'd want it to have?  A caching proxy would have to speak http
> anyway, not fs, and translating between them is what the httpfs is
> for.

But it wouldn't than be possible (without further hacks) to show the
contents of the cache directly in the http directory.  And my idea
actually was that the cachefs-translator would *not* need to speak http,
as otherwise it would again only be useful for http-translators.

What I originally had in mind was this:

  a) one translator (cachefs, though expirefs might fit better) sits on
  some node and behaves like a traditional directory except that it has
  some mechanism for expiring files based on certain configurable factors
  (like age).

  b) another translator (ftpfs/httpfs/whateverfs) sits on top of the other
  translator and simply does what it does best (like fetching files from
  an ftp-server) without caring about anything else (like caching).

This, as I realise now, is in some ways broken; or needs at least some
further refinement.

One option to make this actually work is to simply swap the order of the
two translators :).  This way the first tranlator would serve all the
files that are in the cache directly and would pass on requests for other
files to the second translator.  (If the Hurd doesn't support two
translators on the same node then the cacheing translator could instead
provide an option to search for missing requested files in some other

There are two problems with this option: 

  1. In the case of stacked translators the caching translator would only
  be able to write cached files to the translated directory itself if the
  underlying translator does support it; else it would have to create the
  actual cache somewhere else.  (Something one could possibly live with.)

  2. More serious: The caching translator would have to decide all by
  itself whether to serve a cached file or to pass the request on.  This
  would make it difficult (or sometimes even impossible without evil
  hacks) to cache things that don't fall into a simple "cache-for-a-
  certain-time"-category, like files that should be cached only until a
  newer version is available.

The other (and IMHO better) option is to retain my original ordering (the
cache is queried after the other translator) and have the top-translator
provide something like --write-files and --read-files options (or
--write-files-to and --read-files-from options for translators on
different nodes).  Before downloading a file the translator would check if
the file already exists and decide whether this is to be served instead.
Downloaded files would be written to the underlying (or specified)
directory and completely forgotten about.  Directory listing would simply
be passed on to the cache translator.

The obvious disadvantage of this approach is that the only functionality
the caching translator removes from the other translator is that of
expiring files (hence expirefs) and providing access to the contents of
the cache.  But as other functionality is protocol-dependent and the
translator is meant to be generic this seems to be the only correct option

One remaining question now is whether all this is worth it, or whether a
traditional proxy wouldn't just do the job as well?  But especially when
considering things like httpfs as an extension to the development platform
itself (e.g. something to base a web-browser on), being able to list the
contents of the cache inside of the directory used for access can really
make a difference I think.  (Which would still leave us with the option of
using a http translator with an integrated cache (or maybe integrated
access to an external cache :))).

Final note: an expirefs-translator could even be useful by itself for
things like the /tmp directory.

Ok, that's it for now.  Sorry for the long post!


[ Lars Weber ]-------< >-----[ GPG-ID: 1383B42E ]
+++ fingerprint: 44B1 1D23 DD53 E6B2 4AAB 4C36 0323 9141 1383 B42E +++
[ Using GNU ]----< | >---[ Running Debian ]

reply via email to

[Prev in Thread] Current Thread [Next in Thread]