Re: remaking bash, trying static, glibc refuses static...?

[Linda Walsh]

Mike Frysinger wrote:
> On 18 Aug 2015 21:41, Linda Walsh wrote:
> > Mike Frysinger wrote:
> > > On 18 Aug 2015 13:34, Linda Walsh wrote:
> > > (2) it's using the nss system which lets people drop modules into the system
> > > at anytime and change the overall lookups to use that.  statically linking a
> > > specific subset would block that ability.
> > ---
> > The linux kernel is a perfect example of a statically linked program that
> > can dynamically load plugins to provide authorization data from external
> > sources.  Static doesn't mean you can't support 3rd party plugins/libs --
> > like LDAP.
>
> which is what the current glibc code already does and why you get a warning
> when you link things staticlly.  i'm not sure how you can argue both sides
> of the fence on this.
> -mike
---
        The current glibc code doesn't allow static linking (apart
from a special static glibc build that I'd have to build myself).  If the
kernel can be linked statically, and yet load 3rd party plugins dynamically,
why can't programs be linked statically with glibc (this usually pops up as
an Error -- though that could be because many or most build procedures treat
warnings as Errors).  Also, you don't need a dynamic linker at run time
to load plugins -- but glibc does because it patches addresses in the code
to make direct calls into a dynamically loaded lib transparent.  With the kernel
it performs callouts to 'helpers' -- usually giving 1 key to a helper with
the helper returning the associated value(s).  It doesn't require dynamic
linking -- but a helper script.

Examples
cifs.idmap (8)       - Userspace helper for mapping ids for Common Internet F...
cifs.upcall (8)      - Userspace upcall helper for Common Internet File Syste...
cifscreds (1)        - manage NTLM credentials in kernel keyring
getcifsacl (1)       - Userspace helper to display an ACL in a security descr...
idmapwb (8)          - winbind ID mapping plugin for cifs-utils
pam_cifscreds (8)    - PAM module to manage NTLM credentials in kernel keyring
setcifsacl (1)       - Userspace helper to alter an ACL in a security descrip...

None of those are required to be present when the kernel runs and
the kernel doesn't die if something that would use one of those helpers
is used but it isn't there. 

There are two types of dynamic linking... one where all libs must be
present when the program loads, the other -- linux makes far too little
use of.  I got spoiled at SGI, as they used runtime dynamic linking
to support different feature sets with different products.  

If you have an admin tool to add users, it might try to dlopen various
independently offered products -- if they are not there, the program
didn't show them on the menu -- it wasn't an error to have a dl-failure as
it usually is w/glibc.  Right now, if you try to load 'vim/gvim' on opensuse,
you also have to have perl "libified" and openable at program invocation.

You can't edit w/o the perl lib installed, but gvim doesn't require perl 
any more than it requires ruby, tcl or python that it can script with if
such libs are present.  If the libs are not present -- the core feature(s)
of vim are still there -- you can edit.  You may not be able to use alternate
scripting languages while you edit, but you can at least start the editor
and use it's standard features.

W/glibc, if you wanted to do the same, you'd have it link and programs 
runnable with whatever was statically built in -- just like gvim --
if I don't have ruby/tcl/whatever installed, it says the language isn't
available.  glibc could return a list of available modules for a specific
function and return soft-errors for missing components, with things
like falling back to /etc/passwd &/| shadow being hard coded alternatives.

But even if the dlopen call *can* be done right, I've yet to see a
linux prog do it right and dynamically configure itself -- try an ldap
auth w/no ldap plugin script? -- get a permission denied.  glibc could use
scriptable plugins like the kernel, but uses the more arcane dlopen for
runtime loading -- that most projects get wrong -- and fall back to 
load-time (non-lazy) loading of dynamic libs.

If you wanted to convince me glibc could be distributed statically, 
you could have it interface (as an example or proof of concept) with
the userspace helpers the kernel uses for alternate authentication and
permission mechanisms.  But with the current mechanisms, if I'm not using
NIS or LDAP, but "winbind" for login, I still need to have the NIS
and LDAP libs available to run login. 

Right now it's like most progs require static linking at *load time*,
which is the least flexible linking (programs don't auto-config "down"
services that aren't available -- but you have to have all the kerberos
and SASL, and every other possible security lib installed -- and that
also allows separation of binaries and libraries to happen as people
use symlinks in /bin/ls -> /usr/bin/ls when /usr isn't mounted and 
won't work without it's load-time dynamic libraries.

Right now most distros don't encourage users booting "from disk" -- but
from a ramdisk.  Vs. I statically link in all the things I need for my
HW, and leave optional modules as runtime loadable.  The big unix-iron
vendors supported people building static kernels for their specific HW
and setup -- but with no static drivers linked into the kernel, you'd
need to first boot from a ram disk, then boot again (or the rest of the
way) once disk is available. Because of the ram-disk solution, they
pile boot programs onto the ramdisk, but then are stuck if a needed
driver or program wasn't included.  

One program, I think dracut -- determines the minimum modules you need
for boot to go on a ramdisk -- but that 'list' could easily be used to
build in those drivers statically -- and a separate ramdisk boot 
wouldn't be needed.

Even bash can load many of it's builtin's dynamically -- but if they
aren't there, it could use the same named-programs -- and I don't
think it "falls-over" and dies if it can't dynamically load a builtin --
it just looks for the same name in the path.  So what part of that
can't be in a static lib that can dynamically configure itself based on
libs available?