Re: Calculating Perl module installation directory

[Adam Spiers]

On Wed, Dec 7, 2011 at 4:54 PM, Stefano Lattarini
<address@hidden> wrote:
> On Wednesday 07 December 2011, Adam Spiers wrote:
>> On Tue, Dec 6, 2011 at 7:16 PM, Stefano Lattarini
>>
>> [SNIP]
>>
>> > Warning: the ${var#pattern} substitution is unfortunately unportable to
>> > Bourne shells that lack full POSIX compliance, such as Solaris 10 /bin/sh
>> > (and I guess many other vendor /bin/sh).  While almost all systems have
>> > a (mostly-)POSIX-compliant shell installed somewhere (for example, on
>> > Solaris 10 there is /usr/xpg4/bin/sh), and while autoconf-generated
>> > configure scripts usually take care of re-executing themelves with a
>> > such a better shell when /bin/sh is not POSIX, you can't be sure that
>> > this will always happen.  If you still want to use POSIX shell constructs
>> > not portable to traditional Bourne shells in your configure script, you
>> > might want to ask on the autoconf list whether and how it is possible to
>> > force configure to re-execute itslef with a POSIX-complant shell, or give
>> > a clear error message when no one is found.
>>
>> Hrmph :-)  How about this as a workaround?
>>
>>   pmdir_relative_path=$( echo "${installsitelib}" | sed -e 
>> "s!^$siteprefix/!!" )
>>
> Alas, command substitutions with $(...) are as well unportable to
> old-style Bourne shells (like Solaris 10 /bin/sh).  You'll have to
> use backticks instead: `...`

OK.

> Apart from that, what you do should be mostly OK, assuming that:
>  1. "${installsitelib}" does not contain backslashes nor starts with a `-'
>    (and I assume these are fair assumptions), otherwise echo might get
>    confused; and
>  2. `$siteprefix' does not contain regular expression characters that
>    might mess up the sed invocation (this too could be safely assumed
>     I guess; even the case in which $siteprefix conains literal dots
>    `.' should be safe).

Yes, I think these are both safe assumptions.

> But then it occurs to me that, since your configure script assumes the
> presence of perl, you could simplify all of this by accessing and
> munging the values of $installsitelib and $siteprefix from perl itself,
> using the `Config' built-in module; these values are indeed accessible:
>
>  $ perl -e 'use Config; print $Config{siteprefix} . "\n"'
>  /usr/local
>  $ perl -e 'use Config; print $Config{installsitelib} . "\n"'
>  /usr/local/share/perl/5.12.4
>
> Details about how to do so are left as an excercise to the reader :-)

pmdir_relative_path=`\
  $PERL -MConfig \
        -wle '($_ = $Config{installsitelib})
              =~ s!^\Q$Config{siteprefix}/!!; \
              print'`

>> Awesome! ... but $pmdir_relative_path doesn't get expanded inside the
>> help string.
>>
> This sounds strange... can you copy and past the output from the help
> screen?

  --with-pmdir=DIR        Perl modules are in DIR
                          [PREFIX/${pmdir_relative_path}]

>>      second argument of `AS_HELP_STRING' is treated as a whitespace
>>      separated list of text to be reformatted, and is not subject to
>>      macro expansion.  Since it is not expanded, it should not be
>>      double quoted.
>>
> This means that you should drop the outmost pair of square brackets from
> `[[LIBDIR/$pmdir_relative_path]]' the, to avoid getting spurious `[' and
> `]' in the output.

They are not spurious - all options in the help use [] to denote the
default values.

> But this has nothing to do with the fact that
> $pmdir_relative_path is not expanded (it is not an m4 macro).

True, thanks for reminding me!  However ...

> There should be no problem with such an expansion ... can you give
> us more information about the details of the failure, or even post
> the entire generated configure script (compressed please)?  It might
> be a mistake of mine, but I can't tell without more details.

Part of the problem is the backslash before the _ACEOF here-doc
delimiter:

    if test -n "$ac_init_help"; then
      case $ac_init_help in
         short | recursive ) echo "Configuration of stow 2.1.2:";;
       esac
      cat <<\_ACEOF

    Optional Packages:
      --with-PACKAGE[=ARG]    use PACKAGE [ARG=yes]
      --without-PACKAGE       do not use PACKAGE (same as --with-PACKAGE=no)
      --with-pmdir=DIR        Perl modules are in DIR
                              [PREFIX/${pmdir_relative_path}]

    Report bugs to <address@hidden>.
    _ACEOF
    ac_status=$?
    fi

POSIX dictates that:

    If any character in word is quoted, the delimiter shall be formed
    by performing quote removal on word, and the here-document lines
    shall not be expanded.

    -- 
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_07_04

So I can't see how it is possibly to dynamically generate help text
:-(  The second part of the problem is that regardless where the
pmdir_relative_path assignment appears in configure.ac, it always
appears after the --help code, so even if the here doc *was* expanded,
it would expand to null.

This really looks like a dead-end, and the only solution is to not
show a default value in the help text.  But if you have another
solution I'd love to know ...

>> In this case the installation only works if the user set
>> PERL5LIB, or did
>>
>>   eval `perl -V:siteprefix`
>>   ./configure --prefix=$siteprefix
>>
>> My first inclination was to add:
>>
>>   AC_PREFIX_DEFAULT($siteprefix)
>
> Evil!  This would mean that, if I'd do a simple:
>
>  ./configure && make && sudo make install
>
> your package would install itself in /usr on my system.  I would
> definitely not like that, and probably many users agree with me.

Well, if your Perl has siteprefix set to /usr, then that's where it's
expecting site-wide installations to go.  So far I have only seen that
on openSUSE, so if you think that's an inappropriate location, maybe
you should submit a bug to openSUSE explaining why.

>> but it turns out that this gets expanded near the top of ./configure,
>> which is much earlier than where siteprefix gets set.  I found an evil
>> hack which works around this:
>>
>>   ac_default_prefix=$siteprefix
>>
>> but ac_default_prefix is not documented and presumably not part of the
>> public API.
>>
> Yes, this all seems rather brittle and a source of potential problems.
> I'd advise you not to follow this road.

OK.  FWIW I *think* I agree with you that /usr is a bad prefix
default, so I've scrapped this approach.

> Maybe you can recognize a special argument for `--with-pmdir' [1]
> that will cause configure to bypass the munging of $installsitelib,
> and just define PMDIR to this unmodified value.

That's exactly what --with-pmdir=/foo already does!  The munging of
$installsitelib is purely to calculate a sensible *default* for PMDIR.

> This is something that should be documented in the INSTALL and
> README file of course; and probably added to the configure help
> screen as well.

I've reworded the help string from "Perl modules are in DIR" to
"Install Perl modules in DIR" to make it clearer, and mentioned
--with-pmdir in INSTALL.  However I am deliberately avoiding
duplicating installation information between README and INSTALL.

> Thanks for the reference.  A couple of random remarks:
>
>> AC_PATH_PROGS([PERL], [perl] [perl5], [false])
>> if test "x$PERL" = xfalse
>> then
>>  AC_MSG_WARN([WARNING: Perl not found; you must edit line 1 of 'stow'])
>> fi
>>
> IMO you should just abort the configure process if perl cannot be found.
> This is especially true since later in the script you use a call to perl
> to determine the default directory where to install *.pm files -- so that
> you might end up installing to sem-random directories if a perl command
> isn't available!

Done.

>> # N.B. ${var#pattern} will not work with some shells, such as
>> # Solaris 10's /bin/sh :-(
>> #
>> # 
>> http://thread.gmane.org/gmane.comp.sysutils.automake.general/13203/focus=13204
>> #
> For issues like this, you might want to reference the autoconf
> manual as well, which describes in detail many quirks, limitations
> and portability problems of various UNIX tools (and especially of
> the shell).  For example, about this isue, it says:
>
>  ${#var}
>  ${var%word}
>  ${var%%word}
>  ${var#word}
>  ${var##word}
>
>  Posix requires support for these usages, but they do not work with many
>  traditional shells, e.g., Solaris 10 /bin/sh.  Also, pdksh 5.2.14
>  mishandles some word forms. For example if ‘$1’ is ‘a/b’ and ‘$2’ is
>  ‘a’, then ‘${1#$2}’ should yield ‘/b’, but with pdksh it yields the
>  empty string.
>
> It's a very recommended reading!
> <http://www.gnu.org/software/autoconf/manual/autoconf.html#Portable-Shell>

Thanks, I changed the link.

>> !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
>> !! WARNING!
>> !!
>> !! Perl modules will be installed to $PMDIR.
>> !! Unless you override the value of prefix at make-time,
>> !! this will expand to
>> !!
>> !!   $pmdir
>> !!
>> !! which is not in $PERL's built-in @INC.
>> !!
>> !! This means you will have to set PERL5LIB appropriately
>> !! before running Stow, e.g.
>> !!
>> !!   export PERL5LIB=$pmdir
>> !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
>>
> This is suboptimal, and I also find it very user unfriendly.  IMHO you
> should add to your scripts a BEGIN block where the value of @INC is
> extended with the value of $PMDIR determined at configure time.

Any reason not to

  use lib "...";

rather than

  BEGIN { unshift @INC, "..."; }

?

> This is what automake (itself a perl program) does to ensure it can
> find its private libraries after installation.

Hmm, well I originally wanted to avoid hard-coding paths, but I guess
it's OK for the front-end scripts (after all, we already hardcode the
#! line).  I'll do that.  Thanks!

Again here are the latest versions ...

  https://github.com/aspiers/stow/blob/master/configure.ac
  https://github.com/aspiers/stow/blob/master/Makefile.am

P.S. All this seems remarkably hard for such a conceptually simple
task.  I know most people are distributing Perl modules/scripts via
the native CPAN route, but it seems to me that Autotools could use a
little polish with regard to Perl support (assuming that Perl support
is considered relevant).  A dedicated section in the manual would
help, as would a few macros.  And sorry but no, I'm afraid this
observation can't count as volunteering ;-)  I'm already *way* over my
time budget, and even with all your valuable help there are still
unresolved issues:

  - how to generate dynamic help text

  - the .dirstamp mess I explained in another thread

Many thanks again!