[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Guile fibers return values

From: Zelphir Kaltstahl
Subject: Re: Guile fibers return values
Date: Sun, 5 Jan 2020 19:22:14 +0100
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.2.2

Hi Chris!

On 1/5/20 3:23 PM, Chris Vine wrote:
> On Sun, 5 Jan 2020 13:58:24 +0100
> Zelphir Kaltstahl <address@hidden> wrote:
>> Thank you for the detailed explanation!
>> By "process" I meant only "sequence of steps performed", the main thunk
>> in run-fibers, separate from the steps that are run in the spawned
>> fiber, not really OS process or thread.
>> I will take a look again at the parallel forms and think about whether I
>> want to use them or fibers.
>> Originally I had my algorithm in Racket and could not get it to work in
>> parallel, unless I explore places and serializable lambdas more.
>> I think fibers are more flexible than the parallel forms though, as one
>> could also build a pipeline using fibers or any kind of network of
>> connected computation tasks, while the parallel forms split a task
>> immediately and then join again. Not sure any of the additional
>> flexibility of fibers helps me. Perhaps I can use both and abstract from
>> it with an additional abstraction layer. Then my code could also be used
>> more easily in other Schemes.
> Fibers are more flexible than guile's parallel forms as you say (of
> course, if you have your own thread pool available that can also be
> more flexible than the parallel forms), but fibers' main other feature
> is that they will multi-plex i/o operations on the same native thread
> using guile's suspendable ports and coroutines.  Any one native OS
> thread run by the fibers' scheduler may therefore have more than one
> coroutine running on it.  So, beware having blocking operations when
> using fibers, because between fibers running on the same native OS
> thread, concurrency is co-operative - coroutine task switching occurs on
> a port suspending or the fiber yielding, although there is some work
> stealing implemented between OS threads which will help you out.  (From
> the blocking point of view, do not use the simple-format procedure as in
> my toy example because it is not suspendable-port-safe.)
> If your work is i/o-bound rather than cpu-bound, fibers are likely to be
> especially useful.

I think the decision tree calculations, which I want to parallelize, are
not I/O related. However, I am not quite sure I understand the whole
suspendable port thing, but here is what I think it is:

When some I/O happens in a fiber, the fiber is still able to pause
(suspend, yield, …) at that point, because the suspendable ports work in
such a way, that no harm is done in doing so. Later the I/O work can be
resumed (woken up from suspension). This quality had to be built into
Guile first, before fibers were able to take advantage of it.

Is this correct?

But I do not understand, why this is not the case with normal OS
threads. Maybe it can be done but is not convenient or difficult to get
right, to work with suspendable ports, when not using the fibers library?

And why is simple-format not "suspendable-port-safe"? (What does a
procedure need to do, in order to be "suspendable-port-safe"?)

>> This is my project:
>> I still am not sure though, if I can simply use any lambda I want and
>> send that to a fiber, or I need to look out for things like "What is in
>> the environment of the lambda?". It would be good to know that. I guess
>> it depends on how data sent on channels is handled in the fibers library.
> The lambda closures passed to run-fibers and spawn-fiber are like any
> other lambda closure.  In particular, beware of closures with shared
> mutable state.  Apart from that you should be fine as regards closures
> but I am not convinced that I sufficiently understand your use case to
> comment further.  On data sent through channels, I haven't looked at it
> but I have always assumed that objects such as lists, vectors, records
> and so forth which are passed through channels are passed by reference
> in the ordinary way, so don't mutate them after putting them in a
> channel if more than one fiber may subsequently have them in use
> concurrently.  The purpose of channels is to provide safe
> synchronization.

Good advice, thanks. I was not planning to do mutation on the things
received from channels in a fiber, but it is good to remember, that it
might be problematic to do so, even with vectors or structs. So I better
create new structs or vectors, when creating a return message.

With the parallel forms, isn't it the case, that at each call of such a
form, new OS threads are created? In this case it might be a good idea
to create a fibers scheduler and reuse it, if that is possible, so that
I do not need to create my own process pool kind of thingy.



reply via email to

[Prev in Thread] Current Thread [Next in Thread]