Re: read-process-output-max

[Stefan Monnier]

>> I understand there is a per-chunk overhead which pushes towards a higher
>> value, but I'm not sure what factors are pushing towards smaller values.
> Values larger that some threshold will cause us use malloc instead of
> alloca, that's one reason to use smaller buffers for performance
> reasons.

That's indeed a good reason not to push the limit to far up, indeed.

> Another potential issue, in programs that display the arriving stuff
> as it is received, is that showing the text in small chunks is
> generally better (UX-wise) than blasting a 1MB chunk of text to the
> screen in one go.

If 64kB already arrived, will part of the result really reach the eyes
of the user faster if we process them as 16 chunks of 4kB or do we skip
redisplay between the chunks (just like we skip redisplay when there
are pending input events)?

> The effect of the value was never studied in detail, and so the
> optimal value could be different from the default.  The doc string
> says what it says to prevent people from getting themselves into
> trouble when they don't really need to change the value for their
> application: after all, the default value was used for many years, and
> should be considered safe enough.

Makes sense, indeed.  It might be worth having people play with the
value for a while (and not just for bulk-bandwidth tests but also for
interactive use like `M-x shell`, `M-x compile`, `M-x grep`, including
with largish outputs) and report back.

I just put (setq read-process-output-max (max 65536 read-process-output-max))
into my init file, and I'll see if I notice a difference.
[ BTW, I think a value >=32kB would be desirable since it avoids
  breaking UDP datagrams.  ]


        Stefan