Re: Memory leaks in Make

[Jeffrey Walton]

On Tue, May 14, 2019 at 10:40 PM Martin Dorey
<address@hidden> wrote:
>
> > It would be nice if Make cleaned up its resources before exiting.
>
> Would it?  Linking all those blocks back on to free lists and coalescing 
> adjacent ones doesn't take a long time but it takes more time than just 
> tearing down the address space.  It can be a reasonable economy for 
> short-lived processes not to clean up before exiting.  Recursive make might 
> be considered harmful but it's still a common use-case with such short-lived 
> processes.

Interesting you bring up economic models for this. I've given that
quite a bit of thought, and even researched the numbers.

  * 25 million programmers in the world
  * 6 - 12 % are C and C++, depending on the source
  * 3 million QA folks in the world

That means there's a potential pool of (25M+3M) * 0.09 = 2.5 million
folks. Suppose 1/1000 perform acceptance testing, security testing and
evaluation, etc. That means about 3000 folks are experiencing this
problem.

I spent 5 days trying to test IDN and IDN2. It was not just the
LD_PRELOAD hail mary. It included a lot of other troubleshooting and
tinkering trying to get IDN and IDN2 to test.

Assume it takes the Make project 3 days to make the necessary changes
to avoid the leak. 3 days is 24 man-hours, and that beats the work I
put in by 16 hours. Assume the other 3000 folks try and give up after
a day. The world has spent 24000 man-hours on a problem that could
have been cleared in 3 days.

Which makes more sense to you? The Make project spending 3 days to fix
the problem, or the world servicing your technological debt at the
cost of 24,000 man hours or 11 man-years?

> > It looks like Make's memory leaks are killing the build process long before 
> > the test runners execute.
>
> I presume you have some compelling evidence for that and I appreciate not 
> being deluged with large attachments...

Yes, I have the build results that starts with 'make check', and end with:

    gmake: *** [Makefile:1495: check-recursive] Error 1
    Failed to test IDN

Not one test was run because the tools failed.

> but the trouble with what you have shared is that 89 bytes leaked in total 
> seems like it wouldn't be a problem for even the smallest system.

It taints results of other components. It does not matter if it is a 1
byte leak.

Jeff