Re: mod_lisp for guile

[Neil Jerram]

Alan Grover <address@hidden> writes:

> Mod_lisp is a module for the Apache web-server. Via a simple
> configuration, you can have Apache "forward" a request to a separate
> process. The mod_lisp protocol is very simple.
>
> The mod_lisp home is:
> http://www.fractalconcept.com/asp/tC2/sdataQvrZSvEqz7EuDM==/uC2sdataQuvY9x3g$ecX
>
> So, instead of writing a HTTP server, you write a mod_lisp server which
> is simpler. You can take advantage of all the features of Apache (ssl,
> filtering, rewriting, access control, aliasing, etc.).

Yes, this makes a lot of sense.  Thanks for your patience in
describing it, and for the pointer.

It seems to me, though, that there is nothing especially lispy about
the module (mod_lisp) which runs inside Apache.  Presumably the Apache
side of the protocol could be implemented in any language, or could be
provided "native" by Apache itself?

Is the form of the protocol itself lispy?  (I.e. parenthesised and
readable using read.)

> Or, instead of writing your web-application as a CGI, your mod-lisp
> server is a separate process that is always running. Thus, you avoid the
> delay of starting a process for each CGI, instead starting it once as a
> daemon. You could use threads to handle the requests, possibly reducing
> resource usage, and allowing simple inter-thread communication, but
> requiring consideration of races/locks. Forking could also be used.

How would using threads reduce resource usage?  Otherwise this all
makes sense.

>>>* Should I provide an interface to integrate with TTN's www-guile?
>> 
>> 
>> How would that help?  (Genuine question.)
>
> Several people have produced modules/code to help handle the CGI
> interface. TTN's, for example, parses the environment variables to
> produce a scheme friendly interface, and parses the query-string. Such
> modules have done the work to be standards compliant/complete.
>
> So, I'd have to figure a clever way for TTN's www-guile to see the
> mod-lisp data, or propose a patch that integrates with mod-lisp.

So you mean that if someone has already written an HTTP request
handler procedure for use with www-guile, it ought to be possible to
reuse exactly that same procedure with your mod-lisp also?  Sounds
good to me.

>>>* What do you think of providing a lazy-list of requests, rather than
>>>the current technique of passing control to a
>>>blocking-looping-function?
>> 
>> 
>> Don't understand enough yet to know!  Please explain further.
>
> The interface in the first release is a function that loops, calling
> your function for each request. So, it kind of takes over. It occurred
> to me that there might be more polite ways of working. At the very
> least, there should be mechanisms that impose the least policy.

Yes, indeed.  This problem isn't specific to web requests, of course.
It applies to pretty much any socket-driven or GUI (event-driven)
library.  (For example, the (ossau gds-client) module in my
guile-debugging package.)

It would be nice to have a SRFI which standardised the way of
specifying handler procedures for such events, and of determining when
each such event source had work to process, along with standard
procedures for doing blocking and non-blocking selects on all the
events, and so on.

> So, I'm going to work on (some of) the following interfaces:
> * (f port) -> headers-as-alist, does nothing else
> * (f port header-constructor) -> calls (header-constructor k v) for each
> header, does nothing else (maybe like fold?)
> * (f port) -> lazy list of (k v), suitable for whatever map/fold you
> want to do.
> * (f port) -> (headers content-length), convenience for post-data
> * Same, plus the header-constructor
> * (f port ...) -> returns a lazy list of something like (headers port),
> (one element for each request, blocking).
> * Perhaps versions of the above that return something like
>     (request-header-port    ; returns pairs (k v) till '()/#f
>     request-content-port    ; returns content part till eof-object
>     response-header-port    ; takes only pairs (k v)
>     response-content-port)   ; typical HTML body
> Where request-content-port would only return up-to content-length bytes,
> and response-content-port constructs the content-length header if needed.

These are all nice, but I don't see how they help with what I thought
you were saying was your problem, namely the taking control.  All of
these still need to be scheduled in some way.

> Lazy-lists (see "delay" and "force") are things that act like a list,
> but where the next-element isn't calculated until you "look" at it (this
> is only one kind of lazy-list, but close enough).
>
> So, assuming I provided lazy-mod-lisp:
>       (define lazy-results (lazy-mod-lisp socket-port))
>
>       (pair? lazy-results) ; -> #t. maybe not, I'd have to check
>       
>        ; actually does the work to figure out first mod-lisp request. could
> block!

Yes, and blocking is the problem.  The application usually needs to
know when it can call this without blocking.

>       (define first (car lazy-results)) ; might need "force"
>
>       ; Handle the request (and to loop here)
>       (my-handle-request first)
>
>       (set! lazy-results (cdr lazy-results)) ; never blocks!
>       (set! first (car lazy-results)) ; might need "force"
>
>       ; Loop back to my-handle-request (till '() )
>
> Or,
>       ; Could/will block, waiting for each mod-lisp request
>       (for-each my-handle-request lazy-results)

That's a nice result, but I'd prefer a more general API able to
multiple different sources of input, as described above.

>>>* Do you want to see a SRFI that provides _just_ the mod_lisp
>>>protocol?
>> 
>> 
>> Presumably you mean a SRFI defining a common Scheme API that people
>> could use to write code that plugs into your mod_lisp engine?  (I
>> guess I'm missing the architecture again here.)
>
> Right, convenient API to the scheme side of the protocol.

I see.  So combining my thoughts and yours, I think this would have
three levels to it.

- Generic API for specifying sources of input and how to handle them.

- The particular case of this API for an HTTP request handler.

- Your ideas for presenting the HTTP content in various useful ways.

Sorry that this mail is a bit random - I'm just thinking out loud
really, which isn't that constructive.  (And I also need to read your
code again in more detail, now I understand the architecture.)  I hope
it's of some interest though.

      Neil