Re: Guile fibers return values

[Chris Vine]

On Sun, 5 Jan 2020 19:22:14 +0100
Zelphir Kaltstahl <address@hidden> wrote:
> I think the decision tree calculations, which I want to parallelize, are
> not I/O related. However, I am not quite sure I understand the whole
> suspendable port thing, but here is what I think it is:
> 
> When some I/O happens in a fiber, the fiber is still able to pause
> (suspend, yield, …) at that point, because the suspendable ports work in
> such a way, that no harm is done in doing so. Later the I/O work can be
> resumed (woken up from suspension). This quality had to be built into
> Guile first, before fibers were able to take advantage of it.
> 
> Is this correct?

Yes, suspendable ports were first implemented in guile-2.2.  They are
used by 8-sync (https://www.gnu.org/software/8sync/), guile-a-sync2
(https://github.com/ChrisVine/guile-a-sync2/wiki) and fibers, and
possibly by other libraries.

The basic arrangement is that if a port's file descriptor is not ready,
then its continuation is saved, and is resumed by the scheduler when it
becomes ready.  fibers use epoll rather than POSIX select or poll for
this.

> But I do not understand, why this is not the case with normal OS
> threads. Maybe it can be done but is not convenient or difficult to get
> right, to work with suspendable ports, when not using the fibers library?
>
> And why is simple-format not "suspendable-port-safe"? (What does a
> procedure need to do, in order to be "suspendable-port-safe"?)

simple-format does not suspend as it is implemented in C in libguile and
its continuation is not available to scheme code.  There is a
list of those of guile's i/o procedures which do (and which do not)
suspend here, in the second and third paragraphs, although it does not
mention format/simple-format:
https://github.com/ChrisVine/guile-a-sync2/wiki/await-ports
(That library has nothing to do with fibers, but as mentioned above it
happens to use suspendable ports for similar purposes): 

None of this is likely to be relevant to your use case.

[snip]
> With the parallel forms, isn't it the case, that at each call of such a
> form, new OS threads are created? In this case it might be a good idea
> to create a fibers scheduler and reuse it, if that is possible, so that
> I do not need to create my own process pool kind of thingy.

guile's parallel forms are implemented using futures, which use an
internal thread pool according to this:
https://www.gnu.org/software/guile/docs/master/guile.html/Futures.html#Futures

"Internally, a fixed-size pool of threads is used to evaluate futures,
such that offloading the evaluation of an expression to another thread
doesn’t incur thread creation costs. By default, the pool contains one
thread per available CPU core, minus one, to account for the main
thread."

Chris