guile-user
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

reading from input pipe is slow?


From: Thomas Danckaert
Subject: reading from input pipe is slow?
Date: Fri, 22 Jun 2018 18:40:30 +0200 (CEST)

Dear guile users,

I've notice that reading a child process' stdout from a port opened with (open-input-pipe "command") is very slow. In fact, it's faster to redirect the child process' output to a temporary file, and then read this temporary file from Guile, than to directly read the child process' output using a port (I've added some example code below where you can see this).

Is there an explanation for this?  Or even better: a solution?

Thank you for any information!

Thomas

You can execute the following at the REPL. Here, I "cat" a big file to simulate a process generating a largish amount of text.

(use-modules (ice-9 textual-ports) (ice-9 popen))

(define big-file "/tmp/big-file.txt")

;; write 5MB to big-file
(with-output-to-file big-file
  (lambda ()
    (let put-line ((n 0))
(simple-format (current-output-port) "~a~%" (make-string 1024 #\a))
      (unless (> n (* 1024 5))
        (put-line (1+ n))))
    (simple-format (current-output-port) "This is the end")))

(define my-command (format #f "cat ~a" big-file))

;; 15 seconds on my machine:
,time (format #t "end of file: '~a'~%"
              (string-take-right
               (get-string-all (open-input-pipe my-command))
               15))

;; 1 second on my machine:
,time (let ((tmpfile (tmpnam)))
;; redirect my-command into a temporary file and read the file:
        (system (format #f "~a > ~a" my-command tmpfile))
        (format #t "end of file: '~a'~%"
                (string-take-right
                 (call-with-input-file tmpfile get-string-all)
                 15))
        (delete-file tmpfile))



reply via email to

[Prev in Thread] Current Thread [Next in Thread]