bash cores if nscd disabled on Solaris LDAP sasl/gssapi client

[Serge Dussud - Sun Microsystems]

Hello bug-bash,

please find attached a bashbug report. I am not sure how to follow-up 
then, could you advise ?

Thanks,
Serge

From: root
To: bug-bash@gnu.org
Subject: bash cores if nscd disabled on Solaris LDAP sasl/gssapi client

Configuration Information [Automatically generated, do not change]:
Machine: sparc
OS: solaris2.11
Compiler: gcc
Compilation CFLAGS:  -DPROGRAM='bash' -DCONF_HOSTTYPE='sparc' 
-DCONF_OSTYPE='solaris2.11' -DCONF_MACHTYPE='sparc-sun-solaris2.11' 
-DCONF_VENDOR='sun' -DLOCALEDIR='/usr/local/share/locale' -DPACKAGE='bash' 
-DSHELL  -DHAVE_CONFIG_H   -I.  -I. -I./include -I./lib -I./lib/intl 
-I/share/bld/u/sdussud/workspaces/onnv/sfwNV-clone.1-23961034/usr/src/cmd/bash/bash-3.2/lib/intl
  -g -O2
uname output: SunOS vertebrae4 5.11 snv_98 sun4u sparc SUNW,UltraAX-i2
Machine Type: sparc-sun-solaris2.11

Bash Version: 3.2
Patch Level: 0
Release Status: release

Description:
        This issue is seen on Solaris 10 or Nevada systems when
        configrued as Native LDAP name service clients, using 
        sasl/gssapi authentication, also known as per-user credential
        level. See [1] for more details on the feature.

        When name service cache daemon, nscd(1M), is not running,
        all naming lookups are performed by the application itself,
        in this case bash. But bash core dumps -or loops receiving
        SEGV without core dumping-, having issues with its malloc/free
        internal routines as far as I can tell. 

        For the purpose of this bug report, I reproduced the issue on
        a recent Solaris nevada build (snv_98), with latest bash
        release:

#  bash --version
GNU bash, version 3.2.25(1)-release (sparc-sun-solaris2.11)
Copyright (C) 2005 Free Software Foundation, Inc.
# 
        I can provide a gcore of bash while the problem happens. Here
        are below outputs for pstack, pldd, pmap as well as an truss output
        extract when the problem occurs, here when doing a getpwuid(3C) call:

core 'core.17747' of 17747:     bash
 ff137068 sigacthandler (efe08, f7065555, 0, f8ca0, 0, 0)
 --- called from signal handler with signal 982536 (SIG Unknown) ---
 ff0e62a4 cleanfree (0, 8, ff1e64f8, ff1de63c, f83ec, 2) + 58
 ff0e53d8 _malloc_unlocked (20, f7065554, efe00, f7065555, ff1e0468, ffffffdf) 
+ 104
 ff0e52b8 malloc   (20, 0, 1, ff1de63c, f93c8, ff1e4864) + 48
 feb1d5f8 get_bigint_attr_from_template (f9d28, ffbfc400, f3848, 6c174, 
ffbfc400, ffbfc400) + 24
 feb1fe78 soft_build_secret_key_object (0, f3848, 0, 2d70, 0, 1f) + 3bc
 feb20e9c soft_build_object (ffbfc3c4, 6, f4e08, ffffffff, 0, f4f81) + 11c
 feb2546c soft_add_object (ffbfc3c4, 6, f36dc, f3108, f4e08, f3108) + 2c
 feb17c5c C_CreateObject (f3108, ffbfc3c4, 6, f36dc, 72414, 0) + 74
 fe9b5ddc init_key_uef (f3108, f36c8, 4553, fea3a798, fea36000, 4798) + 11c
 fe9b4d2c krb5_c_make_random_key (f1808, 12, f36c8, 0, f3808, fea390d8) + 14c
 fe9f19d0 krb5_generate_subkey (f1808, f9f94, f9f20, 0, ea338, b9dd8) + 34
 fe9f4724 krb5int_generate_and_save_subkey (f1808, f9f08, f9f94, ffbfc638, 
f368c, 13a) + 28
 fe9f4958 krb5_mk_req_extended (f1808, f3360, 20000001, f9f88, f9f94, ffbfc680) 
+ 1c8
 fe9a5090 make_ap_req_v1 (f1808, f3308, f9b88, f9f88, 11, fecf6450) + 130
 fe9a55c0 new_connection (ffbfc934, f9b88, ebf6c, 0, fecf6450, d0000) + 270
 fe9a5f84 krb5_gss_init_sec_context (ffbfc934, 0, ebf6c, f9948, fecf6450, 3a) + 
3a8
 fe9a2bb4 k5glue_init_sec_context (ffbfc940, 0, 0, 0, f9948, 0) + 50
 feca4a44 gss_init_sec_context (ebf68, 0, f840c, ebd28, d0000, 3a) + 1e8
 fece32ec gssapi_client_mech_step (f8408, f8008, 0, ffff0000, fecbeab4, 
ffbfcab0) + 520
 fed373dc sasl_client_step (f2008, 0, 0, ffbfcabc, ffbfcab0, ffbfcaa8) + 104
 fed372a8 sasl_client_start (f2008, 1, ffbfcabc, ffbfcab0, ffbfcab0, ffbfcab4) 
+ 4c8
 fef37374 nsldapi_sasl_do_bind (f1408, ffbfcab4, fefa2984, 1, fef998e8, 
ffbfcb94) + 14c
 fef37af4 ldap_sasl_interactive_bind_s (f1408, 0, fefa2984, 0, 0, fef998e8) + 
1f0
 fef8e470 doSASLBind (0, f1408, fef998e8, eed4c, ffbfcb94, 1) + 584
 fef8c878 openConnection (ffbfe468, 0, ec488, 32, eed4c, 1) + 1e8
 fef8bcd4 makeConnection (fefb9388, ec388, 1, eb3a8, fefb9384, eed4c) + 428
 fef8d22c getConnection (0, 0, eb2a8, ffbfede4, 0, eed4c) + 5a4
 fef8d384 __s_api_getConnection (0, 0, 0, ffbfede4, 1, eed4c) + 30
 fef80040 get_current_session (eed08, 0, c, ffffffff, 4, fef80c38) + 40
 fef80f1c search_state_machine (eed08, eed80, 0, ffffff3c, fefb6000, 1) + 288
 fef82150 ldap_list (0, fefe10ac, eed08, fefdc870, fefe0c2c, 0) + 2bc
 fef8225c __ns_ldap_list (fefe10ac, ffbff940, fefdc870, fefe0c2c, 0, 0) + 84
 fefdbe80 _nss_ldap_lookup (ec288, ffbffb18, fefe10ac, ffbff940, ffbff7dc, 
fefdc870) + 40
 fefda11c getbyuid (ec288, ffbffb18, ffbffb2c, 5f68, fefe0000, ffbff840) + 88
 ff0edb00 nss_search (1, ff1e0570, 5, ffbffb18, ff300240, ff1e76f0) + 1f4
 ff0da480 getpwuid_r (1869f, f0014, f0038, 400, ff0da8d4, 104208) + 50
 00030e80 get_current_user_info (eb068, d5fa8, d5c00, 653400, ff00, 0) + 18
 000310c0 shell_initialize (1, d5c00, ea400, ea400, 0, d5c00) + c0
 0002f170 main     (1, ffbffe0c, ffbffe14, d5c00, 10e250, ff300180) + 490
 0002e8c0 _start   (0, 0, 0, 0, 0, 0) + 108


core 'core.17747' of 17747:     bash
00010000     728K r-x--  /usr/bin/bash
000D4000      88K rwx--  /usr/bin/bash
000EA000     808K rwx--    [ heap ]
FE920000     264K r-x--  /lib/libresolv.so.2
FE972000      16K rwx--  /lib/libresolv.so.2
FE980000     664K r-x--  /usr/lib/gss/mech_krb5.so.1
FEA36000      24K rwx--  /usr/lib/gss/mech_krb5.so.1
FEAB0000      64K rwx-- 
FEAD0000      16K r-x--  /usr/lib/sasl/plain.so.1
FEAE4000       8K rwx--  /usr/lib/sasl/plain.so.1
FEB00000     488K r-x--  /usr/lib/security/pkcs11_softtoken.so.1
FEB8A000     160K rwx--  /usr/lib/security/pkcs11_softtoken.so.1
FEBB2000       8K rwx--  /usr/lib/security/pkcs11_softtoken.so.1
FEBD0000       8K rwx-- 
FEBE0000      40K r-x--  /usr/lib/libcryptoutil.so.1
FEBFA000       8K rwx--  /usr/lib/libcryptoutil.so.1
FEC00000     104K r-x--  /usr/lib/libpkcs11.so.1
FEC2A000      32K rwx--  /usr/lib/libpkcs11.so.1
FEC40000      48K r-x--  /usr/lib/sasl/digestmd5.so.1
FEC5C000       8K rwx--  /usr/lib/sasl/digestmd5.so.1
FEC70000      16K r-x--  /usr/lib/sasl/crammd5.so.1
FEC84000       8K rwx--  /usr/lib/sasl/crammd5.so.1
FECA0000      56K r-x--  /usr/lib/libgss.so.1
FECBE000       8K rwx--  /usr/lib/libgss.so.1
FECD0000       8K rwx-- 
FECE0000      24K r-x--  /usr/lib/sasl/gssapi.so.1
FECF6000       8K rwx--  /usr/lib/sasl/gssapi.so.1
FED10000      24K r-x--  /platform/sun4u/lib/libmd_psr.so.1
FED26000       8K rwx--  /platform/sun4u/lib/libmd_psr.so.1
FED30000      80K r-x--  /usr/lib/libsasl.so.1
FED54000       8K rwx--  /usr/lib/libsasl.so.1
FED70000      24K r-x--  /lib/libgen.so.1
FED86000       8K rwx--  /lib/libgen.so.1
FED90000      32K r-x--  /lib/libuutil.so.1
FEDA8000       8K rwx--  /lib/libuutil.so.1
FEDB0000     104K r-x--  /lib/libscf.so.1
FEDDA000       8K rwx--  /lib/libscf.so.1
FEDE0000       8K rwx-- 
FEDF0000      80K r-x--  /lib/libmd.so.1
FEE14000       8K rwx--  /lib/libmd.so.1
FEE20000      16K r-x--  /lib/libmp.so.2
FEE34000       8K rwx--  /lib/libmp.so.2
FEE50000       8K r-x--  /usr/lib/mps/cpu/sparcv8plus/libnspr_flt4.so
FEE60000       8K rwx--  /usr/lib/mps/cpu/sparcv8plus/libnspr_flt4.so
FEE80000       8K r-x--  /lib/librt.so.1
FEE90000      16K r-x--  /lib/libthread.so.1
FEEA0000       8K rwx-- 
FEEB0000     232K r-x--  /usr/lib/mps/libnspr4.so
FEEF8000      16K rwx--  /usr/lib/mps/libnspr4.so
FEEFC000       8K rwx--  /usr/lib/mps/libnspr4.so
FEF00000      16K r-x--  /lib/libpthread.so.1
FEF10000     256K r-x--  /usr/lib/libldap.so.5
FEF50000      16K rwx--  /usr/lib/libldap.so.5
FEF70000     216K r-x--  /usr/lib/libsldap.so.1
FEFB6000      16K rwx--  /usr/lib/libsldap.so.1
FEFD0000      64K r-x--  /usr/lib/nss_ldap.so.1
FEFE0000       8K rwx--  /usr/lib/nss_ldap.so.1
FF000000       8K rwx-- 
FF010000      32K r-x--  /lib/nss_files.so.1
FF028000       8K rwx--  /lib/nss_files.so.1
FF040000      64K rwx-- 
FF060000      24K r-x--  /usr/lib/locale/en_US.ISO8859-1/en_US.ISO8859-1.so.3
FF074000       8K rwx--  /usr/lib/locale/en_US.ISO8859-1/en_US.ISO8859-1.so.3
FF080000    1336K r-x--  /lib/libc.so.1
FF1DE000      32K rwx--  /lib/libc.so.1
FF1E6000       8K rwx--  /lib/libc.so.1
FF200000     616K r-x--  /lib/libnsl.so.1
FF2AA000      32K rwx--  /lib/libnsl.so.1
FF2B2000      24K rwx--  /lib/libnsl.so.1
FF2D0000      64K rwx-- 
FF2EA000       8K rwx-- 
FF2F0000      16K r-x--  /platform/sun4u/lib/libc_psr.so.1
FF300000      24K rwx-- 
FF320000       8K rwx-- 
FF330000      48K r-x--  /lib/libsocket.so.1
FF34C000       8K rwx--  /lib/libsocket.so.1
FF350000     184K r-x--  /lib/libcurses.so.1
FF38E000      32K rwx--  /lib/libcurses.so.1
FF396000       8K rwx--  /lib/libcurses.so.1
FF3A0000       8K r-x--  /lib/libdl.so.1
FF3B0000     216K r-x--  /lib/ld.so.1
FF3F6000       8K rwx--  /lib/ld.so.1
FF3F8000       8K rwx--  /lib/ld.so.1
FFBF8000      32K rw---    [ stack ]
 total      7896K


core 'core.17747' of 17747:     bash
/lib/libcurses.so.1
/lib/libsocket.so.1
/lib/libnsl.so.1
/lib/libdl.so.1
/lib/libc.so.1
/platform/sun4u/lib/libc_psr.so.1
/usr/lib/locale/en_US.ISO8859-1/en_US.ISO8859-1.so.3
/lib/nss_files.so.1
/usr/lib/nss_ldap.so.1
/usr/lib/libsldap.so.1
/usr/lib/libldap.so.5
/usr/lib/mps/libnspr4.so
/lib/libpthread.so.1
/lib/libthread.so.1
/lib/librt.so.1
/usr/lib/mps/cpu/sparcv8plus/libnspr_flt4.so
/lib/libmp.so.2
/lib/libmd.so.1
/lib/libscf.so.1
/lib/libuutil.so.1
/lib/libgen.so.1
/usr/lib/libsasl.so.1
/platform/sun4u/lib/libmd_psr.so.1
/usr/lib/sasl/gssapi.so.1
/usr/lib/libgss.so.1
/usr/lib/sasl/crammd5.so.1
/usr/lib/sasl/digestmd5.so.1
/usr/lib/libpkcs11.so.1
/usr/lib/libcryptoutil.so.1
/usr/lib/security/pkcs11_softtoken.so.1
/usr/lib/sasl/plain.so.1
/usr/lib/gss/mech_krb5.so.1
/lib/libresolv.so.2


# more truss.bash.17747
17747/1:        lwp_sigmask(SIG_SETMASK, 0x6801FEFB, 0x00000010) = 0xFFBFFEFF 
[0x0000FFFF]
17747/1:        setcontext(0xFFBFB9D0)
17747/1:            Incurred fault #6, FLTBOUNDS  %pc = 0xFF0E59C8
17747/1:              siginfo: SIGSEGV SEGV_MAPERR addr=0xF715535C
17747/1:            Received signal #11, SIGSEGV [caught]
17747/1:              siginfo: SIGSEGV SEGV_MAPERR addr=0xF715535C
17747/1:        lwp_sigmask(SIG_SETMASK, 0x6801FEFB, 0x00000010) = 0xFFBFFEFF 
[0x0000FFFF]
17747/1:        setcontext(0xFFBFB9D0)
17747/1:            Incurred fault #6, FLTBOUNDS  %pc = 0xFF0E59C8
17747/1:              siginfo: SIGSEGV SEGV_MAPERR addr=0xF715535C
17747/1:            Received signal #11, SIGSEGV [caught]
17747/1:              siginfo: SIGSEGV SEGV_MAPERR addr=0xF715535C
17747/1:        lwp_sigmask(SIG_SETMASK, 0x6801FEFB, 0x00000010) = 0xFFBFFEFF 
[0x0000FFFF]

        Compiling bash with option --without-bash-malloc, that is, telling
        bash to use system's malloc routines, fixes the issue. But this tells
        also that there is very likely a bug in bash's internal malloc
        routines.

[1] http://docs.sun.com/app/docs/doc/819-3194/ldapsecure-66?a=view


Repeat-By:
        - Configure a Solaris system to be a Native LDAP cllient using 
        sasl/gssapi credentails, as described in [2].

        - disable nscd(1M) by running:
        # svcadm disable name-service-cache

        - login or su to a regular LDAP user, and run bash.

[2] http://docs.sun.com/app/docs/doc/819-3194/gdzpf?a=view


Fix:
        Compiling bash with option --without-bash-malloc, that is, telling
        bash to use system's malloc routines, fixes the issue.

        However, I wonder if it's the appropriate way to fix and wonder
        if there could be side effects, possibly performance penalties,
        in doing so.