bug#43565: cuirass: Fibers scheduling blocked.

[Ludovic Courtès]

Hi,

Mathieu Othacehe <othacehe@gnu.org> skribis:

>> In cuirass/utils.scm:
>>    320:22  1 (_)
>> In unknown file:
>>            0 (make-stack #t)
>> ERROR: In procedure make-stack:
>> In procedure struct-vtable: Wrong type argument in position 1 (expecting 
>> struct): #f
>
> I think this error is caused by setting:
>
>   ;; STORE's socket is O_NONBLOCK but since we're
>   ;; not in a fiber, disable Fiber's handlers.
>   (current-read-waiter #f)
>   (current-write-waiter #f)
>
>
> where it should be:
>
>   ;; STORE's socket is O_NONBLOCK but since we're
>   ;; not in a fiber, disable Fiber's handlers.
>   (current-read-waiter
>    (lambda (port)
>      (port-poll port "r")))
>   (current-write-waiter
>    (lambda (port)
>      (port-poll port "w")))

Ooh, good catch.

> then this should also be done in "fetch-inputs" that is using non
> blocking ports outside of Fibers.
>
> However, I still have the following error:
>
> In ice-9/boot-9.scm:
>   1731:15 17 (with-exception-handler #<procedure 7fac67194000 at ic…> …)
>   1736:10 16 (with-exception-handler _ _ #:unwind? _ # _)
> In ice-9/eval.scm:
>     619:8 15 (_ #(#(#(#(#<directory (cuirass base) 7fac6b51c…>)) …) …))
> In unknown file:
>           14 (_ #<procedure 7fac69b10b20 at ice-9/eval.scm:330:13 ()> …)
>           13 (partition #<procedure 7fac69b10880 at ice-9/eval.scm:…> …)
> In guix/store.scm:
>    1008:0 12 (valid-path? #<store-connection 256.99 7fac6b3fd6e0> "/…")
> 2020-11-19T11:47:23 Failed to compute metric average-eval-build-start-time 
> (1).
>    717:11 11 (process-stderr #<store-connection 256.99 7fac6b3fd6e0> _)
> In guix/serialization.scm:
>     76:12 10 (read-int #<input-output: socket 49>)
> In ice-9/suspendable-ports.scm:
>    307:17  9 (get-bytevector-n #<input-output: socket 49> 8)
> 2020-11-19T11:47:23 Failed to compute metric average-eval-build-complete-time 
> (1).
> 2020-11-19T11:47:23 Failed to compute metric evaluation-completion-speed (1).
>    284:18  8 (get-bytevector-n! #<input-output: socket 49> #vu8(0 …) …)
>     67:33  7 (read-bytes #<input-output: socket 49> #vu8(0 0 0 0 0 …) …)
> In fibers/internal.scm:
>     402:6  6 (suspend-current-fiber _)
> In ice-9/boot-9.scm:
>   1669:16  5 (raise-exception _ #:continuable? _)
>   1764:13  4 (_ #<&compound-exception components: (#<&error> #<&orig…>)
> In cuirass/utils.scm:
>     319:8  3 (_ _ . _)
> In ice-9/boot-9.scm:
>   1731:15  2 (with-exception-handler #<procedure 7fac683ea300 at ic…> …)
> In cuirass/utils.scm:
>    320:22  1 (_)
> In unknown file:
>            0 (make-stack #t)
> ERROR: In procedure make-stack:
> Attempt to suspend fiber within continuation barrier
>
> that originates from "valid-path?" in "restart-builds", not sure how to
> fix it yet.

I think that’s because of the ‘partition’ call: ‘partition’ is currently
implemented in C and the stack cannot be captured if it contains C calls
in the middle.

The simplest fix is probably to have a Scheme implementation:

diff --git a/src/cuirass/base.scm b/src/cuirass/base.scm
index 5a0c826..99a17fa 100644
--- a/src/cuirass/base.scm
+++ b/src/cuirass/base.scm
@@ -632,6 +632,21 @@ This procedure is meant to be called at startup."
      db "UPDATE Builds SET status = 4 WHERE status = -2 AND timestamp < "
      (- (time-second (current-time time-utc)) age) ";")))
 
+(define (partition pred lst)
+  ;; Scheme implementation of SRFI-1 'partition' so stack activations can be
+  ;; captured via 'abort-to-prompt'.
+  (let loop ((lst   lst)
+             (pass '())
+             (fail '()))
+    (match lst
+      (()
+       (values (reverse pass) (reverse fail)))
+      ((head . tail)
+       (let ((pass? (pred head)))
+         (loop tail
+               (if pass? (cons head pass) pass)
+               (if pass? fail (cons head fail))))))))
+
 (define (restart-builds)
   "Restart builds whose status in the database is \"pending\" (scheduled or
 started)."

It’s a bummer that one has to be aware of all these implementation
details when using Fibers.  The vision I think is that asymptotically
these issues would vanish as more things move from C to Scheme.

Thanks,
Ludo’.