guile-user
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

How to read integers from file faster?


From: Darren Hoo
Subject: How to read integers from file faster?
Date: Sat, 31 Aug 2013 11:55:03 +0800
User-agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3.50 (darwin)

It is way too slow to read numbers from a file simply by using `read' 

for example a txt file contains 10,000,000 line of numbers:

    (define (gen-sample max k file)
      (with-output-to-file 
          file
        (lambda ()
          (let lp ((k k))
        (when (> k 0)
          (display (random max)) (newline)
          (lp (- k 1) ))))))
         
    (gen-sample 999999999 10000000 "rnd.txt")
    
 
And read the numbers in 

    (define (read-test1)
      (with-input-from-file
          "rnd.txt"
        (lambda ()
          (let lp ((i (read)))
        (if (not (eof-object? i))
            (lp (read)))))))
    
scheme@(guile-user)> ,time (read-test1)
;; 37.348000s real time, 37.340000s run time.  0.450000s spent in GC.


with rdelim's read-line, it's better but still slow.

    (import (ice-9 rdelim))
    (define (read-test2)
      (with-input-from-file
          "rnd.txt"
        (lambda ()
          (let lp ((i (read-line)))
        (if (not (eof-object? i))
            (begin 
              (string->number i)
              (lp (read-line))))))))
    
scheme@(guile-user)> ,time (read-test2)
;; 11.943000s real time, 11.930000s run time.  0.890000s spent in GC.


it only takes 1.8 seconds by using fscanf

    FILE *f = fopen("rnd.txt", "r");
    if (f == NULL) {
      printf("open failed!");
      exit(1);
    }
    long long i;
    while (fscanf(f, "%lld", &i) != EOF) {
      
    }

$ time ./read 

real    0m1.844s
user    0m1.803s
sys     0m0.032s

Are there any primitives in Guile that is equivalent to C's scanf?




reply via email to

[Prev in Thread] Current Thread [Next in Thread]