bug#22344: 25.0.50 (and 24.5.1, 24.4); Sending 4097 bytes to a subprocess hangs Emacs on Windows 8 and 10

[Eli Zaretskii]

> Cc: 22344@debbugs.gnu.org
> From: Clément Pit--Claudel <clement.pit@gmail.com>
> Date: Wed, 13 Jan 2016 19:48:38 -0500
> 
> Eli, there's something magical about your debugging abilities.
> I downloaded and compiled the latest master on Windows 8; I could reproduce 
> the bug there. Then I checked out emacs-25 and recompiled, and the bug seems 
> to be gone! Amazing.

I wish I had such magic, but I don't.  It's just a lot of hard work,
that's all.  But thank you for your kind words.

> This is only for my personal curiosity, but would you mind expanding a bit on 
> what the issue was (and how you found out)? I read the commit message and 
> looked at the diff, but I don't think I understood everything from there. Did 
> you have to look at node.js' source code? Would it be helpful for me to file 
> a bug with node.js?

I'm not entirely sure what caused the issue.  From the Emacs side, it
looked like the other end of the pipe wasn't read at all: each time
Emacs called _write, it got the return value of -1 with errno set to
ENOSPC, which means the pipe's buffer is full.  No other program with
which I tried the same test case did this.  With those other programs,
after a few first attempts which returned -1, _write started to return
positive values, meaning that it succeeded to write some part of the
stuff.

I tried to google for similar problems related to node.js, but didn't
find anything that looked useful.

So I just went with the only workaround you reported: never write to
the pipe more than its buffer size.  I.e., subdivide the write into
chinks the size of the pipe's buffer.  This required a system call, to
find out what that size was, and it also required a small change in
the logic of the loop that writes the stuff in chunks, because it
wasn't ready for the situation that a chunk succeeds, but the very
next one returns -1 and ENOSPC -- again something that only node.js
causes somehow.

No, I didn't look into node.js's sources.  It's a large program with
whose design I'm not familiar, so I expected it won't be easy to find
the code that is responsible for this.  I took a brief look now, and I
see that my expectations were correct: I see some very elaborate "pipe
server" there, which I'm not sure is relevant to the original problem.

If we want to try to understand this deeper, we should be talking to
some expert on how node.js works on Windows wrt I/O on its standard
input and standard output.  Filing a bug report with node.js might be
useful, but again, only if someone there will want to talk to us so
that we could understand the problem.

If you want to file a node.js bug report, you can start by saying that
it reads its stdin very slowly: once it reads a 4K chunk from the
pipe, it takes about 40 to 60 msec before it is ready to read the next
chunk.  (I can tell because when Emacs receives ENOSPC from the pipe,
it loops retrying, waiting 20 msec between retries, and I see 2 to 3
retries, sometimes even 4, before the next successful write.)  I don't
know why it takes so long -- this was on a Core i7 system which was
otherwise very quiet.  Maybe if we understand what takes so long, we
will also understand what did Emacs do wrongly, and how to avoid that.

For now, I guess we can be happy with the workaround I committed.

Thanks.