Re: [PATCH 1/2] Use POSIX nm to simplify AIX export_symbols_cmds.

[Michael Haubenwallner]

On 03/21/2016 03:47 PM, David Edelsohn wrote:
> On Mon, Mar 21, 2016 at 4:49 AM, Michael Haubenwallner
> <address@hidden> wrote:
>>
>> On 03/20/2016 01:04 AM, David Edelsohn wrote:
>>> I agree with this in principle, but I'm not convinced that the patch
>>> itself is correct.
>>
>> Thanks!
>>
>>> I also would have split the MS-compatible part of the patch as a
>>> separate step.  It is not listed in the ChangeLog and just confuses
>>> the patch.
>>
>> This is just a reordering of existing code - these lines are removed
>> below, and the ChangeLog part is "Reorder to allow for platform specific
>> hooks during transformation of global_symbol_pipe into C source code."
>>
>>> I don't see that this emits TLS symbols.  Global TLS symbols are not
>>> in the symcode list.
>>
>> Isn't the "L" for TLS symbols?
>>
>> +    symcode='[[BDLTVWZ]]'
> 
> Sorry, I was confused by the two cases for "aix" and the case for
> "GNU" within AIX.
> 
>>
>> But indeed I've failed to identify the symcode of "weak TLS" symbols,
>> except for "L*" with AIX nm in unspecified mode (nm -l, no -B or -P)
>> Neither AIX nm nor GNU nm docs explicitly tell about weak TLS symbols.
>> Is it possible for TLS symbols to be weak at all?

For GNU nm I've identified the "C" (common) symcode to be "weak" as well.

But still I'm unsure about TLS symbols: Always import them as "strong"?

>>> Global symbol pipe and exclude symbols somehow restricts the symbols
>>> to the non "dot" symbols?
>>
>> Yes - but this is what the existing code does as well:
>> '... && ([substr](\$ 1,1,1) != ".") ..'
> 
> Is that in your patch or existing code?  Your patch removes the
> current substr($3,1,1).

In current code there is substr($3,1,1) for GNU nm (BSD mode) and
substr($1,1,1) for AIX nm (POSIX mode), both identifying the "dot"
symbols as to be *removed* from the list of exported symbols.

This patch leaves removal of the "dot" symbols up to $global_symbol_pipe,
which does remove them due to not being a valid C symbol name.

>>> The export_symbols_cmds awk command does not have matching quotes, so
>>> I don't know what is going on.
>>>
>>> awk '\''{ kw = "" } /^[[VWZ]] / { kw = " weak" } { print $ 3 kw }'\''
>>>
>>> you're starting and ending with '\" which seems odd. why aren't these
>>> complementary?
>>
>> The awk argument is passed between ['], as it contains ["].
>> Escaping [\] the ['] is done outside any other string. Read as:
>> _LT_TAGVAR(...)=['$NM ... | awk '][\']['{ ... }'][\'][' | sort ...']
>> Indeed I can add [] here if you like.
> 
> Ah, well, this is turning into a very complex statement.  GCC requires
> GNU Awk and most developers use Bash, but does this work correctly
> with AIX awk and AIX Ksh?

This awk program is quite simple and works with AIX awk too. The AWK
variable is not promoted into the final libtool script, and I don't
want to have projects using libtool to require GNU Awk in general.

Most shells do not take [\] within two ['] as escapement,
so I do the escaping of ['] outside any string. Usually,
 $ echo '\''
will leave you with a string open to be continued, while
 $ echo \'
gives a single ['].

And this one doesn't feel more readable either:
_LT_TAGVAR(...)='$NM ... | awk '"'"'{ ... }'"'"' | sort ...'

What about this one:
_LT_TAGVAR(...)='$NM ... | awk '[\']'{ ... }'[\']' | sort ...'

Thanks!
/haubi/

>>
>>>
>>> Thanks, David
>>>
>>> On Wed, Mar 2, 2016 at 11:19 AM, Michael Haubenwallner
>>> <address@hidden> wrote:
>>>> * m4/libtool.m4 (LT_PATH_NM): Detect POSIX-compatible nm for AIX.  In
>>>> BSD mode, the AIX nm does not tell whether a symbol is weak, need to use
>>>> POSIX mode instead.
>>>> (_LT_CMD_GLOBAL_SYMBOLS): Support POSIX-compatible nm.  Reorder to allow
>>>> for platform specific hooks during transformation of global_symbol_pipe
>>>> into C source code.  For AIX, set hook to transform even weak text
>>>> symbols as text symbols.
>>>> (_LT_LINKER_SHLIBS): Use global_symbol_pipe to simplify forming the
>>>> export_symbols_cmds for AIX.
>>>> ---
>>>>  m4/libtool.m4 | 101 
>>>> ++++++++++++++++++++++++++++++++--------------------------
>>>>  1 file changed, 55 insertions(+), 46 deletions(-)
>>>>
>>>> diff --git a/m4/libtool.m4 b/m4/libtool.m4
>>>> index 2c0e657..6134522 100644
>>>> --- a/m4/libtool.m4
>>>> +++ b/m4/libtool.m4
>>>> @@ -3755,10 +3755,10 @@ _LT_DECL([], [want_nocaseglob], [1],
>>>>
>>>>  # LT_PATH_NM
>>>>  # ----------
>>>> -# find the pathname to a BSD- or MS-compatible name lister
>>>> +# find the pathname to a BSD-, POSIX- or MS-compatible name lister
>>>>  AC_DEFUN([LT_PATH_NM],
>>>>  [AC_REQUIRE([AC_PROG_CC])dnl
>>>> -AC_CACHE_CHECK([for BSD- or MS-compatible name lister (nm)], 
>>>> lt_cv_path_NM,
>>>> +AC_CACHE_CHECK([for BSD-, POSIX- or MS-compatible name lister (nm)], 
>>>> lt_cv_path_NM,
>>>>  [if test -n "$NM"; the
>>>>    # Let the user override the test.
>>>>    lt_cv_path_NM=$NM
>>>> @@ -3808,6 +3808,26 @@ else
>>>>    : ${lt_cv_path_NM=no}
>>>>  fi])
>>>>  if test no != "$lt_cv_path_NM"; then
>>>> +  case $host_os in
>>>> +  aix[[4-9]]*)
>>>> +    # With AIX nm we need the '-l' flag to get the "weak" information
>>>> +    # for the Import File, but '-l' is ignored with the '-B' flag.  So
>>>> +    # we use the '-P' (POSIX) flag instead.  As users often provide the
>>>> +    # '-B' flag, which conflicts with '-P', we drop any provided flag.
>>>> +    # AIX nm needs the '-C' flag to disable demangling.  For both GNU
>>>> +    # and AIX nm, the '-g' flag shows public (global) symbols only,
>>>> +    # and the '-p' flag disables sorting to improve performance.
>>>> +    set dummy $lt_cv_path_NM
>>>> +    case address@hidden|@2 -V 2>&1` in
>>>> +    *GNU* | *'with BFD'*)
>>>> +      lt_cv_path_NM="@S|@2 -Bgp"
>>>> +      ;;
>>>> +    *)
>>>> +      lt_cv_path_NM="@S|@2 -PlCgp"
>>>> +      ;;
>>>> +    esac
>>>> +    ;;
>>>> +  esac
>>>>    NM=$lt_cv_path_NM
>>>>  else
>>>>    # Didn't find any BSD compatible name lister, look for dumpbin.
>>>> @@ -3832,7 +3852,7 @@ fi
>>>>  test -z "$NM" && NM=nm
>>>>  _LT_SET_TOOL_ABI_FLAG([NM])
>>>>  AC_SUBST([NM])
>>>> -_LT_DECL([], [NM], [1], [A BSD- or MS-compatible name lister])dnl
>>>> +_LT_DECL([], [NM], [1], [A BSD-, POSIX- or MS-compatible name lister])dnl
>>>>
>>>>  AC_CACHE_CHECK([the name lister ($NM) interface], [lt_cv_nm_interface],
>>>>    [lt_cv_nm_interface="BSD nm"
>>>> @@ -3847,6 +3867,8 @@ AC_CACHE_CHECK([the name lister ($NM) interface], 
>>>> [lt_cv_nm_interface],
>>>>    cat conftest.out >&AS_MESSAGE_LOG_FD
>>>>    if $GREP 'External.*some_variable' conftest.out > /dev/null; then
>>>>      lt_cv_nm_interface="MS dumpbin"
>>>> +  elif $GREP '^[[       ]]*_*some_variable' conftest.out > /dev/null; then
>>>> +    lt_cv_nm_interface="POSIX nm"
>>>>    fi
>>>>    rm -f conftest*])
>>>>  ])# LT_PATH_NM
>>>> @@ -4012,8 +4034,33 @@ symcode='[[BCDEGRST]]'
>>>>  # Regexp to match symbols that can be accessed directly from C.
>>>>  sympat='\([[_A-Za-z]][[_A-Za-z0-9]]*\)'
>>>>
>>>> +if test "$lt_cv_nm_interface" = "MS dumpbin"; then
>>>> +  # Gets list of data symbols to import.
>>>> +  lt_cv_sys_global_symbol_to_import="sed -n -e 's/^I .* \(.*\)$/\1/p'"
>>>> +  # Adjust the below global symbol transforms to fixup imported variables.
>>>> +  lt_cdecl_hook=" -e 's/^I .* \(.*\)$/extern __declspec(dllimport) char 
>>>> \1;/p'"
>>>> +  lt_c_name_hook=" -e 's/^I .* \(.*\)$/  {\"\1\", (void *) 0},/p'"
>>>> +  lt_c_name_lib_hook="\
>>>> +  -e 's/^I .* \(lib.*\)$/  {\"\1\", (void *) 0},/p'\
>>>> +  -e 's/^I .* \(.*\)$/  {\"lib\1\", (void *) 0},/p'"
>>>> +else
>>>> +  # Disable hooks by default.
>>>> +  lt_cv_sys_global_symbol_to_import=
>>>> +  lt_cdecl_hook=
>>>> +  lt_c_name_hook=
>>>> +  lt_c_name_lib_hook=
>>>> +fi
>>>> +
>>>>  # Define system-specific variables.
>>>>  case $host_os in
>>>> +aix[[4-9]]*)
>>>> +  case `$NM -V 2>&1` in
>>>> +  *GNU* | *'with BFD'*) ;;
>>>> +  *)
>>>> +    symcode='[[BDLTVWZ]]'
>>>> +    lt_cdecl_hook=" -e 's/^W/T/p'" # weak text symbol
>>>> +  esac
>>>> +  ;;
>>>>  aix*)
>>>>    symcode='[[BCDT]]'
>>>>    ;;
>>>> @@ -4054,23 +4101,6 @@ case `$NM -V 2>&1` in
>>>>    symcode='[[ABCDGIRSTW]]' ;;
>>>>  esac
>>>>
>>>> -if test "$lt_cv_nm_interface" = "MS dumpbin"; then
>>>> -  # Gets list of data symbols to import.
>>>> -  lt_cv_sys_global_symbol_to_import="sed -n -e 's/^I .* \(.*\)$/\1/p'"
>>>> -  # Adjust the below global symbol transforms to fixup imported variables.
>>>> -  lt_cdecl_hook=" -e 's/^I .* \(.*\)$/extern __declspec(dllimport) char 
>>>> \1;/p'"
>>>> -  lt_c_name_hook=" -e 's/^I .* \(.*\)$/  {\"\1\", (void *) 0},/p'"
>>>> -  lt_c_name_lib_hook="\
>>>> -  -e 's/^I .* \(lib.*\)$/  {\"\1\", (void *) 0},/p'\
>>>> -  -e 's/^I .* \(.*\)$/  {\"lib\1\", (void *) 0},/p'"
>>>> -else
>>>> -  # Disable hooks by default.
>>>> -  lt_cv_sys_global_symbol_to_import=
>>>> -  lt_cdecl_hook=
>>>> -  lt_c_name_hook=
>>>> -  lt_c_name_lib_hook=
>>>> -fi
>>>> -
>>>>  # Transform an extracted symbol line into a proper C declaration.
>>>>  # Some systems (esp. on ia64) link data and code symbols differently,
>>>>  # so use this general approach.
>>>> @@ -4128,6 +4158,9 @@ for ac_symprfx in "" "_"; do
>>>>  "     s[1]~/address@hidden/{print f,s[1],s[1]; next};"\
>>>>  "     s[1]~prfx {split(s[1],t,\"@\"); print 
>>>> f,t[1],substr(t[1],length(prfx))}"\
>>>>  "     ' prfx=^$ac_symprfx]"
>>>> +  elif test "$lt_cv_nm_interface" = "POSIX nm"; then
>>>> +    symxfrm="\\2 $ac_symprfx\\1 \\1"
>>>> +    lt_cv_sys_global_symbol_pipe="sed -n -e 's/^[[      
>>>> ]]*$ac_symprfx$sympat[[         ]][[    ]]*\($symcode$symcode*\)[[      
>>>> ]][[    ]]*.*$opt_cr$/$symxfrm/p'"
>>>>    else
>>>>      lt_cv_sys_global_symbol_pipe="sed -n -e 's/^.*[[    
>>>> ]]\($symcode$symcode*\)[[       ]][[    
>>>> ]]*$ac_symprfx$sympat$opt_cr$/$symxfrm/p'"
>>>>    fi
>>>> @@ -5009,19 +5042,7 @@ m4_if([$1], [CXX], [
>>>>    _LT_TAGVAR(exclude_expsyms, 
>>>> $1)=['_GLOBAL_OFFSET_TABLE_|_GLOBAL__F[ID]_.*']
>>>>    case $host_os in
>>>>    aix[[4-9]]*)
>>>> -    # If we're using GNU nm, then we don't want the "-C" option.
>>>> -    # -C means demangle to GNU nm, but means don't demangle to AIX nm.
>>>> -    # Without the "-l" option, or with the "-B" option, AIX nm treats
>>>> -    # weak defined symbols like other global defined symbols, whereas
>>>> -    # GNU nm marks them as "W".
>>>> -    # While the 'weak' keyword is ignored in the Export File, we need
>>>> -    # it in the Import File for the 'aix-soname' feature, so we have
>>>> -    # to replace the "-B" option with "-P" for AIX nm.
>>>> -    if $NM -V 2>&1 | $GREP 'GNU' > /dev/null; then
>>>> -      _LT_TAGVAR(export_symbols_cmds, $1)='$NM -Bpg $libobjs $convenience 
>>>> | awk '\''{ if (((\$ 2 == "T") || (\$ 2 == "D") || (\$ 2 == "B") || (\$ 2 
>>>> == "W")) && ([substr](\$ 3,1,1) != ".")) { if (\$ 2 == "W") { print \$ 3 " 
>>>> weak" } else { print \$ 3 } } }'\'' | sort -u > $export_symbols'
>>>> -    else
>>>> -      _LT_TAGVAR(export_symbols_cmds, $1)='`func_echo_all $NM | $SED -e 
>>>> '\''s/B\([[^B]]*\)$/P\1/'\''` -PCpgl $libobjs $convenience | awk '\''{ if 
>>>> (((\$ 2 == "T") || (\$ 2 == "D") || (\$ 2 == "B") || (\$ 2 == "L") || (\$ 
>>>> 2 == "W") || (\$ 2 == "V") || (\$ 2 == "Z")) && ([substr](\$ 1,1,1) != 
>>>> ".")) { if ((\$ 2 == "W") || (\$ 2 == "V") || (\$ 2 == "Z")) { print \$ 1 
>>>> " weak" } else { print \$ 1 } } }'\'' | sort -u > $export_symbols'
>>>> -    fi
>>>> +    _LT_TAGVAR(export_symbols_cmds, $1)='$NM $libobjs $convenience | 
>>>> $global_symbol_pipe | $EGREP -v " ($exclude_expsyms)$" | awk '\''{ kw = "" 
>>>> } /^[[VWZ]] / { kw = " weak" } { print $ 3 kw }'\'' | sort -u > 
>>>> $export_symbols'
>>>>      ;;
>>>>    pw32*)
>>>>      _LT_TAGVAR(export_symbols_cmds, $1)=$ltdll_cmds
>>>> @@ -5464,19 +5485,7 @@ _LT_EOF
>>>>         exp_sym_flag='-Bexport'
>>>>         no_entry_flag=
>>>>        else
>>>> -       # If we're using GNU nm, then we don't want the "-C" option.
>>>> -       # -C means demangle to GNU nm, but means don't demangle to AIX nm.
>>>> -       # Without the "-l" option, or with the "-B" option, AIX nm treats
>>>> -       # weak defined symbols like other global defined symbols, whereas
>>>> -       # GNU nm marks them as "W".
>>>> -       # While the 'weak' keyword is ignored in the Export File, we need
>>>> -       # it in the Import File for the 'aix-soname' feature, so we have
>>>> -       # to replace the "-B" option with "-P" for AIX nm.
>>>> -       if $NM -V 2>&1 | $GREP 'GNU' > /dev/null; then
>>>> -         _LT_TAGVAR(export_symbols_cmds, $1)='$NM -Bpg $libobjs 
>>>> $convenience | awk '\''{ if (((\$ 2 == "T") || (\$ 2 == "D") || (\$ 2 == 
>>>> "B") || (\$ 2 == "W")) && ([substr](\$ 3,1,1) != ".")) { if (\$ 2 == "W") 
>>>> { print \$ 3 " weak" } else { print \$ 3 } } }'\'' | sort -u > 
>>>> $export_symbols'
>>>> -       else
>>>> -         _LT_TAGVAR(export_symbols_cmds, $1)='`func_echo_all $NM | $SED 
>>>> -e '\''s/B\([[^B]]*\)$/P\1/'\''` -PCpgl $libobjs $convenience | awk '\''{ 
>>>> if (((\$ 2 == "T") || (\$ 2 == "D") || (\$ 2 == "B") || (\$ 2 == "L") || 
>>>> (\$ 2 == "W") || (\$ 2 == "V") || (\$ 2 == "Z")) && ([substr](\$ 1,1,1) != 
>>>> ".")) { if ((\$ 2 == "W") || (\$ 2 == "V") || (\$ 2 == "Z")) { print \$ 1 
>>>> " weak" } else { print \$ 1 } } }'\'' | sort -u > $export_symbols'
>>>> -       fi
>>>> +       _LT_TAGVAR(export_symbols_cmds, $1)='$NM $libobjs $convenience | 
>>>> $global_symbol_pipe | $EGREP -v " ($exclude_expsyms)$" | awk '\''{ kw = "" 
>>>> } /^[[VWZ]] / { kw = " weak" } { print $ 3 kw }'\'' | sort -u > 
>>>> $export_symbols'
>>>>         aix_use_runtimelinking=no
>>>>
>>>>         # Test if we are trying to use run time linking or normal
>>>> --
>>>> 2.4.6
>>>>