[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: How to read integers from file faster?

From: Pascal J. Bourguignon
Subject: Re: How to read integers from file faster?
Date: Sun, 01 Sep 2013 22:55:56 +0200
User-agent: Gnus/5.13 (Gnus v5.13) Emacs/24.2 (gnu/linux)

Darren Hoo <address@hidden> writes:

> It is way too slow to read numbers from a file simply by using `read' 
> for example a txt file contains 10,000,000 line of numbers:

To get speed, I would:

1- open a binary file,

2- perform double-buffering I/O,  (ie. read a buffer while you process

3- assume the input format is as specified, ie. no check.
   (let ((byte (vector-ref buffer i)))
     (if (= byte 10)
        (begin (set! values (cons value values))
               (set! value 0))
        (set! value (+ (* value 10) (mod byte 16)))))

Actually, only using buffered I/O matters.  For big files, what slows
down is I/O, not the handling of characters or checking of the syntax or
anything that's done in memory.  If you want to read your data faster,
keep it in RAM, or else in SSD.

__Pascal Bourguignon__

reply via email to

[Prev in Thread] Current Thread [Next in Thread]