Re: Guile fibers return values

[Zelphir Kaltstahl]

Hey Chris!

Thanks for your informative reply! I did not remember, that the parallel
forms are already that clever. It might be the case, that I only need to
use futures or parallel forms then. I have a question regarding futures
though.

In Racket the futures have some limitations, where one needs to use a
different number type to enable them in some cases to run in parallel –
wait, I am looking for the link … here:
https://docs.racket-lang.org/guide/parallelism.html – Is there any
similar restriction for futures in Guile?

Regards,

Zelphir

On 1/5/20 10:45 PM, Chris Vine wrote:
> On Sun, 5 Jan 2020 19:22:14 +0100
> Zelphir Kaltstahl <address@hidden> wrote:
>> I think the decision tree calculations, which I want to parallelize, are
>> not I/O related. However, I am not quite sure I understand the whole
>> suspendable port thing, but here is what I think it is:
>>
>> When some I/O happens in a fiber, the fiber is still able to pause
>> (suspend, yield, …) at that point, because the suspendable ports work in
>> such a way, that no harm is done in doing so. Later the I/O work can be
>> resumed (woken up from suspension). This quality had to be built into
>> Guile first, before fibers were able to take advantage of it.
>>
>> Is this correct?
> Yes, suspendable ports were first implemented in guile-2.2.  They are
> used by 8-sync (https://www.gnu.org/software/8sync/), guile-a-sync2
> (https://github.com/ChrisVine/guile-a-sync2/wiki) and fibers, and
> possibly by other libraries.
>
> The basic arrangement is that if a port's file descriptor is not ready,
> then its continuation is saved, and is resumed by the scheduler when it
> becomes ready.  fibers use epoll rather than POSIX select or poll for
> this.
>
>> But I do not understand, why this is not the case with normal OS
>> threads. Maybe it can be done but is not convenient or difficult to get
>> right, to work with suspendable ports, when not using the fibers library?
>>
>> And why is simple-format not "suspendable-port-safe"? (What does a
>> procedure need to do, in order to be "suspendable-port-safe"?)
> simple-format does not suspend as it is implemented in C in libguile and
> its continuation is not available to scheme code.  There is a
> list of those of guile's i/o procedures which do (and which do not)
> suspend here, in the second and third paragraphs, although it does not
> mention format/simple-format:
> https://github.com/ChrisVine/guile-a-sync2/wiki/await-ports
> (That library has nothing to do with fibers, but as mentioned above it
> happens to use suspendable ports for similar purposes): 
>
> None of this is likely to be relevant to your use case.
>
> [snip]
>> With the parallel forms, isn't it the case, that at each call of such a
>> form, new OS threads are created? In this case it might be a good idea
>> to create a fibers scheduler and reuse it, if that is possible, so that
>> I do not need to create my own process pool kind of thingy.
> guile's parallel forms are implemented using futures, which use an
> internal thread pool according to this:
> https://www.gnu.org/software/guile/docs/master/guile.html/Futures.html#Futures
>
> "Internally, a fixed-size pool of threads is used to evaluate futures,
> such that offloading the evaluation of an expression to another thread
> doesn’t incur thread creation costs. By default, the pool contains one
> thread per available CPU core, minus one, to account for the main
> thread."
>
> Chris