[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
RE: MSYS+MSVC for libtool branch-2-0, take 2
From: |
Peter Ekberg |
Subject: |
RE: MSYS+MSVC for libtool branch-2-0, take 2 |
Date: |
Tue, 5 Jul 2005 23:01:37 +0200 |
Ralf Wildenhues wrote:
> Hi Peter,
>
> I'm very sorry it took me so long to write this email.
> Nice to see you are making progress. :)
Doing my best, it's not always so easy to decypher what's
going on in guts of libtool...
Besides, I'm taking revenge by sending a mail that I've been
writing on for about three weeks, so it's probably long and
winding and twisted in various ways :-)
> * Peter Ekberg wrote on Fri, Jun 17, 2005 at 03:15:33PM CEST:
> > Ralf Wildenhues wrote:
> > > * Peter Ekberg wrote on Fri, Jun 10, 2005 at 10:57:33PM CEST:
> > > > Ralf Wildenhues wrote:
> > > > >
> > > > > `link -LIB' can be a problem if the cygwin `link' (to create
> > > > > hard links)
> > > > > comes before the win32 paths. `LIB' works here.
> > > >
> > > > I have a fix for that in the attached patch, where you can
> > > > override by setting MSLINK to the appropriate executable.
> > >
> > > Hmm. Can't we just move all this stuff to the section in
> libtool.m4
> > > where $AR is set? It already allows user override (by setting AR
> > > appropriately during configure). We could test whether `ar'
> > > works, and
> > > if not, whether either one of `link -lib' or `lib' work.
> > > (This is where we then should also test whether `ar'
> > > understands the `S'
> > > option, and adjust archive_cmds/postinstall_cmds accordingly,
> > > but that's
> > > totally unrelated to this patch.)
> >
> > Of course, reuse the AR variable. *smacks forehead*
>
> :)
>
> > Looking closer reveals a couple of problems though...
> > $AR is used in three ways if I read it correctly:
> > 1. "$AR $AR_FLAGS archive files..." to create archives.
> > 2. "$AR x archive" to extract archives.
> > 3. "$AR t archive" to list archive content.
> >
> > I see no way to set AR and AR_FLAGS so that 1. is expanded
> > to "lib -OUT:archive files..." as is needed here (-OUT:
> > has to be prepended to the archive name without space).
>
> Hmm. See below.
>
> > "$AR x archive" has to be hidden in some construct, as there is
> > the problem that there is no way to extract all members from
> > a MSVC .lib in one go, you have to loop over the archived
> > files and extract them one by one.
>
> We have more than one variable which allows a set of commands.
> extract_old_archive_cmds comes to mind as a new name; and in fact, I
> have suggested this before in another mail in this thread.
> Seeded with '$AR x $oldlib'. Would you like me to show this
> in a patch?
Ah, I must have missed that. I'll look into changing the old archive
extraction part of the patch if you set up the framework for me.
So a patch will indeed be helpful, I will probably have a hard time
getting it right and will probably wait for (more) hours while
bootstrap is running in the background... :-/
> (Don't worry too much about global variable names in shell functions.
> This is orthogonal, and I have an idea to hide the resulting
> ugliness in ltmain.m4sh.)
This I don't understand, so I'm not worrying, ok? :-)
> Alternatively, if you don't like the extract_old_archive_cmds
> idea, I'd
> still prefer something like
> $AR $AR_FLAGS$AR_SEP$oldlib ..
> over your solution. Have AR_SEP=' ' by default.
> Rationale: Less variables to keep track of.
Hmmm, Automake (?) seems to set up a rule to use $AR, but
that rule uses "$AR $ARFLAGS foo..." w/o underscore in
AR_FLAGS (and of course no $AR_SEP).
I noticed this when I added AR_FLAGS and AR_SEP with AC_SUBST
in an attempt to pass one of the failed tests. (I added AR_SEP
to AC_SUBST thinking that Automake would catch up and make use
of it in the future...)
> > 3 is easy to solve, just introduce $AR_LFLAGS and set it to
> > "t" by default and to "-NOLOGO -LIST" for MSVC.
> >
> > I have made an attempt to solve these issues in
> > msys_msvc-4.patch.
>
> By looking at `lib -link' I believe issue 1. is actually the most
> problematic, since `lib' encodes the path into the archive by default.
> I have not found a switch to turn this off (which does not imply there
> is none). But that means we have to symlink/copy objects before
> archiving them. (Could be done in the same loop as the renaming, but
> will be expensive anyway.)
>
> Otherwise, all use of convenience archives breaks (and running the
> Libtool test suite really shows that).
If I do
$ lib -OUT:foo.lib a/b/c.obj
$ lib -EXTRACT:a/b/c.obj foo.lib
I get a fresh copy of c.obj in cwd
If I do
$ ar cru bar.lib a/b/d.obj
$ ar x bar.lib
I get a fresh copy of d.obj in cwd
Where's the difference?
> > (It turns out that both ar and lib works as the archiver
> > since MSVC does not actually see the generated archives.
> > ar is selected by default, so you have to specify AR=lib
> > or AR="link -lib" to get the Microsoft tool.)
>
> The "encodes paths" is one reason to prefer `ar' over `lib' any time.
See above, and some tests that pass with lib seem to fail when
ar is used. Haven't looked closely though.
> > > > > After working around these, the next failure is when
> > > linking libltdl:
> > > > >
> > > > > | libltdl_la-ltdl.obj : error LNK2019: unresolved external
> > > > > symbol _lt_libltdl_LTX_preloaded_symbols referenced in
> > > > > function _lt_dlinit.
> > > > > | .libs/ltdl-6.dll : fatal error LNK1120: 1
> unresolved externals
> > > >
> > > > Haven't tried libltdl, I do not depend on it for the project
> > > > I'm targeting and I have a hard time bootstrapping libtool
> > > > so I haven't tried the testsuite myself.
> > >
> > > Is that only because you'd need recent Autoconf/Automake
> (from CVS)?
> >
> > Yes. That, and I'm unsure of if there are any patches needed for
> > Cygwin (which I guess I would be using to bootstrap) and since
> > Cygwin has a strange side-by-side installation of autotools.
>
> I vaguely remember that you need to have only one of the packages
> installed (e.g., only Autoconf-2.59, but not 2.13).
I got autoconf-cvs running and can now bootstrap
> > No, I tried bootstrapping with Cygwin and ac 2.59 and it of
> > course didn't work...
>
> Copy and paste output?
Can't remember, no longer a problem for me so I'm not wasting time
on it... :-P
> > > > > The `mv' command in this line
> > > > > | + _LT_TAGVAR(archive_cmds, $1)='$CC -o $lib $libobjs
> > > > > $compiler_flags `$ECHO "X$deplibs" | $Xsed -e '\''s/
> > > > > -lc$//'\''` -link -dll~linknames=~mv .libs/${libname}`$ECHO
> > > > > "X${release}" | $Xsed -e +s/[.]/-/g`${versuffix}.lib
> > > > > .libs/${libname}.lib'
> > > > >
> > > > > is also broken w.r.t lib name.
> > > >
> > > > I don't see what you mean here. It does what I intend it to do,
> > > > it renames the import lib to not have a version number. Why
> > > > is that a problem? It matches the content of the .la file.
> > >
> > > I didn't keep a note on this due to time constraints,
> sorry. Was a
> > > problem with the paths again, I think (i.e., correct quoting).
> >
> > Well, I didn't even try to solve problems caused by spaces in paths,
> > I used my little -L/msvc/lib workaround instead.
>
> OK.
I no longer need that with this patch, it scans the LIB variable.
> > > > - Add paths given with -L to the LIB variable so that the linker
> > > > actually sees them. This should be done after converting them
> > > > to win32 paths, which is easy on Cygwin and harder on MSYS.
>
> > Maybe not so hard on MSYS after all, how about:
> > win32=$(cmd \\/C echo "$unix " | sed -e "s/\"\(.*\) \" /\1/")
> >
> > (cmd is not directly CMD.EXE, but a shell script provided by MSYS
> > that invokes $COMSPEC. One question is how portable /C is as an
> > option to $COMSPEC? I would guess that /C works for all command
> > interpreters from Microsoft, but I'm unsure about others...)
>
> This /might/ be a question for the msys/mingw mailing lists.
> What is COMSPEC, by the way?
COMSPEC is an env variable that is the full path of the windows
command line interpreter, e.g. C:\WINDOWS\command.com or
C:\WINXP\System32\cmd.exe.
> The `echo' might be problematic, if any of the shells' builtin and/or
> external echo commands interpret backslashes. The MinGW
> shell should be
> fine with `printf %s ".."', I believe.
That echo will be fed to the builtin echo in the windows command
line interpreter, so it should not be a problem. Or where you
referring to something else?
The unknown is what happens if there are some non-MS command
line interpreter installed. I remember using DRDOS or something
~10 years ago, and don't have a clue if those alternative command
line interpreters are still in use or if they even work in a
Windows environment. I'm willing to take a slight risk and
assume that they are compatible in this regard...
> Also I don't like the fact that we do not reuse
> fix_srcfile_path. This
> should set the right thing for cygwin/mingw and we should use
> it in the
> appropriate places. Anything more than a
> test -z "$fix_srcfile_path" \
> && $cmd
The problem is that fix_srcfile_path is not always the best thing
to do for MSYS. It must be used in sooo many more places on Cygwin
than on MSYS. MSYS != Cygwin, contrary to recent posts on this
list, this is one area where their design goals differ vastly.
So, if this functionality is moved to a func/var, there still has
to be some other logic involved than a simple flag that says yes/no
on whether to invoke the functionality.
> should IMHO be separated out into a function, in order to keep clutter
> out of the main (and IMVHO already too large) functions.
I don't see an obvious solution, so I'll leave it for now.
> > > > - libtool at some point thinks /mingw/bin/ld is the linker and
> > > > uses that to deduce stuff, perhaps other configure checks are
> > > > also a problematic, I haven't looked closely since it works
> > > > for me.
> > >
> > > Show `./libtool --config'. In order to use only the msvc
> tools, it
> > > might be necessary to configure somehow like
> > > CC=cl CXX=cl LD=link AR=link\ -lib
> > > iff the respective mingw/cygwin tools are available as well.
> > > I regard this as a feature, BTW, not a bug.
> > >
> > > It'll likely be some time before I can look at your patch again.
> > > It's good to see that it's getting smaller, though. :)
> > >
> > > Another couple of hints: if you intend to work on the
> path conversion
> > > thing, please separate that out into a shell function if at all
> > > possible. (And have a reasonably fast default path so that
> > > not everyone
> > > has to suffer from it.)
> >
> > One more thing that perhaps should be handled is that the content
> > of the LIB variable should be added to sys_lib_search_path (or
> > some other variable?) This would require converting paths from
> > win32 style to unix style, which again is easy on Cygwin, but
> > much harder on MinGW (the workaround above does not work).
> > But I would not consider this a showstopper. The user can always
> > add the needed paths with -L as I have done.
> >
> > PS. With msys_msvc-4.patch I configure with
> > ./configure CC="cl -MD" AR=lib LDFLAGS=-L/msvc/lib
> >
> > I also have this line in my msys /etc/fstab
> > C:/PROGRA~1/MICROS~4/VC98 /msvc
>
> Ahh. Guess that might be a good idea to put in corresponding
> documentation.
No longer needed with this patch. I'm feeding the paths in LIB
to sys_lib_search_path_spec. Those paths can't have spaces in
them, so that's what should go into the docs I suppose...
> > PPS. The patch does not solve as many issus on Cygwin as it
> > does on MSYS. Therefore, please don't judge it by its
> > performance on Cygwin. You have to walk before you can run...
>
> Sure.
>
> More comments below inline.
>
> Peter, I'm sorry to sound so picky at times. If you'd like
> me to, I can
> eventually do some work on this as well, but it will be a little more
> while at least. Thanks again for your work.
I have full understanding and realize that my patches are more
than likely to not conform with various criteria. If I fail to
change anything, I either did not understand what you meant or
did not know how to change it. Just keep flinging it my way, I
can take it. If there were no complaints I'd assume that I was
being ignored, it feels as if I'm just gussing most of the
time...
> | @@ -6008,11 +6094,15 @@
> | # we should really use a build-platform specific compiler
> | # here, but OTOH, the wrappers (shell script and this C one)
> | # are only useful if you want to execute the "real" binary.
> | # Since the "real" binary is built for $host, then this
> | # wrapper might as well be built for $host, too.
> | - $run $LTCC $LTCFLAGS -s -o $cwrapper $cwrappersource
> | + if test "$with_gcc" = "yes"; then
> | + $run $LTCC $LTCFLAGS -s -o $cwrapper $cwrappersource
> | + else
> | + $run $LTCC $LTCFLAGS -o $cwrapper $cwrappersource
>
> Why is this necessary?
Because MSVC does not have a -s option. It's not strictly
necessary since a warning is the only evidence, but I feel
uncomfortable with unrecognized options...
Besides, I think MSVC creates "release" binaries unless asked
to do differently. Not sure if they can be "stripped" further
though...
New issues:
In the project where I first developed the patch, functions
and variables are marked with the appropriate
__declspec(dllimport) and __declspec(dllexport), so exporting
and importing variables for shared libs failed miserably for
the general case for patches up to an including -4. In -5 this
is fixed. But I think I've hit a bit of a snag, importing a
variable from a dll requires that the "data item in the client
program must be declared using extern __declspec(dllimport)".
The snag is that it isn't known at compile time what the
compiled unit will be used for, so this has to be solved in
the project using libtool, if a clean solution is desired.
I found this out when fixing problems found by the tests. This
version of the patch exports functions and variables correctly.
It also imports function correctly, but importing variables are
cludgy. I always say extern __declspec(dllimport), even if
linking statically or exporting. This works, but results in
various linker warnings. At least it works if I use lib (or
"link -lib") as archiver. There is some failure when using ar
as archiver that I haven't investigated, so I'm avoiding that
ATM...
How do I get configure to not find g77 when it looks for a
Fortran compiler?
Version -5 of the patch has problems exporting symbols from C++,
I need to adjust to MSVC style symbol mangling, I think, but
I'm not sure what's really going on.
Also missing is support for NM=dumpbin, which needs something
along the lines of what is done for AR=lib, methinks.
The changes to libltdl/Makefile.am needs to be done properly.
My environment:
I have killed /etc/config.site from the MSYS/MinGW install.
I have made sure that the paths in the LIB variable for MSVC
does not contain any spaces (uses PROGRA~1 etc).
I configure with
./configure CC=cl CFLAGS=-MD CXX=cl CXXFLAGS=-MD LD=link AR=lib STRIP=:
Here is an annotated dump of make check results:
$ make check
Making check in .
make[1]: Entering directory `/home/peda/libtool/2-0/libtool/msvc'
make check-TESTS
make[2]: Entering directory `/home/peda/libtool/2-0/libtool/msvc'
PASS: tests/link.test
PASS: tests/link-2.test
PASS: tests/nomode.test
PASS: tests/objectlist.test
PASS: tests/quote.test
FAIL: tests/sh.test
This is due to the use of `...` instead of $(...)
in func_msvc_dashL_to_LINK when assigning to LINK.
I'm not fluent enough in shell programming to know
how to properly quote to get a working version
that does not fail this test...
PASS: tests/suffix.test
SKIP: tests/tagtrace.test
PASS: tests/cdemo-static.test
PASS: tests/cdemo-make.test
PASS: tests/cdemo-exec.test
PASS: tests/demo-static.test
PASS: tests/demo-make.test
PASS: tests/demo-exec.test
PASS: tests/demo-inst.test
PASS: tests/demo-unst.test
PASS: tests/depdemo-static.test
PASS: tests/depdemo-make.test
PASS: tests/depdemo-exec.test
PASS: tests/depdemo-inst.test
PASS: tests/depdemo-unst.test
PASS: tests/mdemo-static.test
PASS: tests/mdemo-make.test
PASS: tests/mdemo-exec.test
PASS: tests/mdemo-inst.test
PASS: tests/mdemo-unst.test
PASS: tests/cdemo-conf.test
PASS: tests/cdemo-make.test
PASS: tests/cdemo-exec.test
PASS: tests/demo-conf.test
PASS: tests/demo-make.test
PASS: tests/demo-exec.test
PASS: tests/demo-inst.test
PASS: tests/demo-unst.test
FAIL: tests/demo-deplibs.test
Rule for $AR by automake? See comments above...
PASS: tests/depdemo-conf.test
PASS: tests/depdemo-make.test
PASS: tests/depdemo-exec.test
PASS: tests/depdemo-inst.test
PASS: tests/depdemo-unst.test
PASS: tests/mdemo-conf.test
PASS: tests/mdemo-make.test
PASS: tests/mdemo-exec.test (runs .lib?)
PASS: tests/mdemo-inst.test (runs .lib?)
Even though the above two tests pass, a dialog pops up
with a failure, haven't looket further...
PASS: tests/mdemo-unst.test
PASS: tests/mdemo-dryrun.test
PASS: tests/mdemo2-conf.test
PASS: tests/mdemo2-make.test
PASS: tests/mdemo2-exec.test
PASS: tests/pdemo-conf.test
FAIL: tests/pdemo-make.test
link.exe does not behave as ld...
Probably need to teach libtool to use shorter command
lines, right?
SKIP: tests/pdemo-exec.test
SKIP: tests/pdemo-inst.test
PASS: tests/demo-nofast.test
PASS: tests/demo-make.test
PASS: tests/demo-exec.test
PASS: tests/demo-inst.test
SKIP: tests/demo-unst.test
PASS: tests/depdemo-nofast.test
PASS: tests/depdemo-make.test
PASS: tests/depdemo-exec.test
PASS: tests/depdemo-inst.test
SKIP: tests/depdemo-unst.test
PASS: tests/demo-pic.test
PASS: tests/demo-make.test
PASS: tests/demo-exec.test
PASS: tests/demo-nopic.test
PASS: tests/demo-make.test
PASS: tests/demo-exec.test
PASS: tests/cdemo-shared.test
PASS: tests/cdemo-make.test
PASS: tests/cdemo-exec.test
PASS: tests/demo-shared.test
PASS: tests/demo-make.test
PASS: tests/demo-exec.test
PASS: tests/demo-inst.test
FAIL: tests/demo-hardcode.test
Rule for $CC by automake?
FAIL: tests/demo-relink.test
foo not found in hello-2.dll.
PASS: tests/demo-noinst-link.test
PASS: tests/demo-unst.test
PASS: tests/depdemo-shared.test
PASS: tests/depdemo-make.test
PASS: tests/depdemo-exec.test
PASS: tests/depdemo-inst.test
FAIL: tests/depdemo-relink.test
func_l3 not found in l3-0.dll.
PASS: tests/depdemo-unst.test
PASS: tests/mdemo-shared.test
PASS: tests/mdemo-make.test
PASS: tests/mdemo-exec.test (runs .lib?)
PASS: tests/mdemo-inst.test (runs .lib?)
Even though the above two tests pass, a dialog pops up
with a failure, haven't looket further...
PASS: tests/mdemo-unst.test
PASS: tests/cdemo-undef.test
PASS: tests/cdemo-make.test
PASS: tests/cdemo-exec.test
PASS: tests/duplicate_members.test
PASS: tests/tagdemo-static.test
PASS: tests/tagdemo-make.test
PASS: tests/tagdemo-exec.test
PASS: tests/tagdemo-conf.test
FAIL: tests/tagdemo-make.test
Does not build shared libs?
Problem with MSVC style C++ symbol mangling?
SKIP: tests/tagdemo-exec.test
PASS: tests/tagdemo-shared.test
FAIL: tests/tagdemo-make.test
SKIP: tests/tagdemo-exec.test
PASS: tests/tagdemo-undef.test
PASS: tests/tagdemo-make.test
PASS: tests/tagdemo-exec.test
FAIL: tests/f77demo-static.test
g77 probably does not work with lib and other
Microsoft tools.
SKIP: tests/f77demo-make.test
SKIP: tests/f77demo-exec.test
FAIL: tests/f77demo-conf.test
SKIP: tests/f77demo-make.test
SKIP: tests/f77demo-exec.test
FAIL: tests/f77demo-shared.test
SKIP: tests/f77demo-make.test
SKIP: tests/f77demo-exec.test
====================================
11 of 103 tests failed
(13 tests were not run)
Please report to address@hidden
====================================
make[2]: *** [check-TESTS] Error 1
make[2]: Leaving directory `/home/peda/libtool/2-0/libtool/msvc'
make[1]: *** [check-am] Error 2
make[1]: Leaving directory `/home/peda/libtool/2-0/libtool/msvc'
make: *** [check-recursive] Error 1
Cheers,
Peter
msys_msvc-5.patch
Description: msys_msvc-5.patch
- RE: MSYS+MSVC for libtool branch-2-0, take 2,
Peter Ekberg <=