[Bug binutils/23460] regression: ar can not create archive containing many (>1024) lto object files

[zenith432 at users dot sourceforge.net]

https://sourceware.org/bugzilla/show_bug.cgi?id=23460

--- Comment #4 from zenith432 at users dot sourceforge.net ---
FYI
I took a look at the code for GCC 8.2 plugin and LLVM 6.0.1 gold plugin.

As best as I can tell

- Neither plugins access the file through the descriptor passed to
claim_file_hook() after it returns.

  The llvm plugin uses the fd value as an index for determining which claimed
files come from the same archive, but does not later access the file through
the fd.  Moreover, if the llvm plugin is asked to register its 
all_symbols_all_read_hook(), it requires the linker to provide get_input_file()
and release_input_file() callbacks - none of which is done by bfd/plugin.c.

  The gcc plugin, for its all_symbols_all_read_hook() passes the input file
names and offsets to lto-wrapper and doesn't need any further descriptors.

- Calls to onload() only allocate memory if passing LDPT_OPTIONs to onload()
which bfd/plugin.c doesn't do.  Without options, onload() uses only static
memory.

- The gcc plugin never releases memory allocated for LDPT_OPTIONs passed to
onload().

- The gcc plugin retains all memory allocated in claim_file_hook() (including
the memory passed to add_symbols() callback - and frees it in the
cleanup_hook().  The cleanup_hook() is not called by bfd/plugin.c today.

- The llvm plugin is written in C++ so all memory it allocates is released by
the static destructors called when dlclose() is done on the plugin.  It appears
that its cleanup_hook() only deletes temporary files and releases some cached
memory - but does not fully release all memory previously allocated - including
memory allocated in claim_file_hook().

So to summarize:
- It's better to call dlopen() and onload() only once.  onload() doesn't leak
(due to the lack of LDPT_OPTION) - but it still violates the plugin API spec to
call onload() many times.

- The file descriptor passed to claim_file_hook() should be closed right after
the call or it's never closed.

- The memory allocated in claim_file_hook(), part of which is passed to the
all_symbols() callback - can be used by bfd/plugin.c.  However, this memory
accumulates more and more with each call to claim_file_hook().  For the gcc
plugin, it's enough to call its cleanup_hook() to release the memory and "start
over" to claim more input files.  For the llvm plugin this isn't enough - it
needs to be dlclose()-ed.

-- 
You are receiving this mail because:
You are on the CC list for the bug.