Re: double _close()?

[Bruno Haible]

Hi Gisle,

> > Thus, skipping the fclose_nothrow call introduces a memory leak.
> 
> Right. But I'd rather have leaks than a lot of exceptions.
> 
> So a 'diff -r dir1 dir2' is using mostly read() and
> close(). Changing to:
>   MSVC_INVALID_PARAMETER_HANDLING == HAIRY_LIBRARY_HANDLING
> 
> and counting number of 'rpl_flose()' calls and 'fclose_nothrow()'
> catches, I find only 2 (no matter how many files I'm diffing).

I agree that 2 file stream buffers is not a noteworthy memory leak.
Therefore it's OK for you to use the modifications that you propose.

But it's not something we can do in Gnulib, since the fact that there
are only 2 (not N) such lost buffers is something that comes from the
GNU diff source code; other programs will behave differently.

> I'm just trying to speed-up GNU-diff; using 'Process Explorer',
> I find that the read-rate is 300 kByte/s on average (with some
> peaks at ~5 MByte/s) and CPU < 1%. IMHO this is patheticly slow.

Since GNU diff is a single-threaded program, the low CPU percentage
indicates that it is mostly stuck in the I/O calls. Why these produce
only 300 Kb/s, may have various causes:
  - slow hardware (e.g. if you read from an USB 2.0 stick or some
    old hard disk),
  - Windows process management,
  - the libc that you are using (mingw or MSVC DLLs).

> I've been trying several things to speed it up:
>   1) SetPriorityClass (GetCurrentProcess(), REALTIME_PRIORITY_CLASS);
>   2) Hinting the Windows-cache manager with a 'O_SEQUENTIAL'
>      in open (file, O_RDONLY | O_SEQUENTIAL, 0);

Can't comment, as I am not a Windows expert.

> Maybe a memory-mapped I/O in GNU-diff could improve the speed?

The problem here is:
  - On one hand, we try to use POSIX APIs in GNU programs, i.e. it would be
    an mmap() function,
  - But we don't have a native Windows emulation of mmap() in Gnulib so far,
    because of different semantics between mmap() and VirtualAlloc
    (regarding "reserved" address ranges).
    A "limited-use" mmap() is possible, but we could not call it 'mmap' since
    it would not be a 100% mmap() emulation.

Bruno