Not understanding spawn-fiber's parallel? keyword argument

[Zelphir Kaltstahl]

Hi Guile Users!

I am experimenting with guile-fibers and hit some behavior that seems weird.

Have a spawn-fiber call in a run-fibers call inside a
call-with-new-thread, to spawn and run fibers without blocking the whole
execution of the program. The code is probably very similar to
amirouche's babelia thread pool (I get a 404 currently, when trying to
visit it on Github and could not find it on Gitlab either, could it be
you are moving it elsewhere @amirouche?), except, that I am trying to
use fibers instead of threads:

~~~~~~~~8<~~~~~~~~8<~~~~~~~~
(define pool-initializer
  (lambda* (#:key (parallelism (current-processor-count)))
    (let ([channel-receive (make-channel)]
          [scheduler (make-scheduler #:parallelism parallelism)])
      ;; Start as many workers as are desired.
      ;; TODO: PROBLEM: ~run-fibers~ blocks. So we need a new thread to run the
      ;; fibers in a non-blocking way. LOOKUP: How to start fibers without
      ;; waiting for them to finish?
      (call-with-new-thread
       (lambda ()
         (run-fibers
          (lambda ()
            (let loop ([index parallelism])
              (unless (zero? index)
                (display "[POOL INIT THREAD]: will spawn fiber ") (displayln 
index)
                (spawn-fiber (lambda () (worker index channel-receive))
                             )  ; add #:parallel? #t keyword argument here
                ;; We do not need to spawn new fibers in the same scheduler 
later. The
                ;; fibers should stay alive for the whole duration the program 
is
                ;; running.
                (displayln "[POOL INIT THREAD]: fiber spawned")
                (loop (- index 1)))))
          #:scheduler scheduler)
         (displayln "[POOL INIT]: pool init thread returning")
         ))
      (displayln "[POOL INIT]: will start work-distributor")
      (call-with-new-thread
       (lambda ()
         (work-distributor channel-receive)))
      ;; Return the channel for receiving work, so that the outside context can
      ;; make use of it when calling ~publish~ to publish work.
      channel-receive)))
~~~~~~~~>8~~~~~~~~>8~~~~~~~~

So there is that call to spawn-fiber. This call starts executing
whatever is defined in ~worker~. However, when I add the #:parallel? #t
keyword argument, the fiber never starts running. This is confusing. I
thought (after reading the manual of fibers regarding this keyword
argument multiple times) that it would only change what scheduler
(scheduler or peer scheduler) the fiber is run on, but not whether it is
run at all. It seems, that somehow when I add the keyword #:parallel?
#t, it never is run.

Can anyone explain, why this happens?

I will send the complete code as attachment.

There is also the question, why the fibers do not run in parallel, as
they should, because I created the scheduler with #:parallelism 2.
Instead they run sequentially. But maybe that makes for another e-mail
later.

Btw.: Thanks @amirouche! After a while of looking at your thread pool
code and drawing some diagrams to help me understanding it, it seems
quite clever and just in the direction I probably need for a fibers
"thread pool".

Regards,
Zelphir

pool.scm
Description: Text Data