Re: copy-file: why this value for IO_SIZE

[Pádraig Brady]

On 04/05/2012 01:30 PM, Křištof Želechovski wrote:
> <URL: 
> http://git.savannah.gnu.org/gitweb/?p=gnulib.git;a=blob_plain;f=lib/copy-
> file.c >
> 
> has IO_SIZE = 32 * 1024
> 
> This number is used to specify the buffer length for copying data from one 
> file to another, something that neither ANSI C nor POSIX directly supports.  
> It is not configurable.  How was that number obtained?  Is it because it is 
> half of memory segment?  But then, doesn’t it fail to allocate in 16-bit 
> applications that use most of their memory?  And is it the best choice for a 
> 32-bit architecture too?  I ask because I am looking for a good buffer size 
> for an application that copies data between files.

The number was obtained empirically, to reduce system call overhead,
while introducing reasonable space requirements.

http://git.sv.gnu.org/gitweb/?p=gnulib.git;a=commitdiff;h=c910e6ce
http://git.sv.gnu.org/gitweb/?p=coreutils.git;a=commitdiff;h=55efc5f3

Note the best value is dependent on the latency between source and dest,
so if you were copying between two locations on the same disk
(mechanical head moving around), or across the network (ACKs taking
time to transfer), you would probably benefit from a larger buffer.
Also concurrent access to this buffer could reduce latency also,
where the contended resource is on separate machines.

cheers,
Pádraig.