Absolute paths generated by libtool.

[Benoit Sigoure]

Hello folks.

I'm developing Qt-based applications. The build system is controlled by the
autotools rather than by Qmake.

I'm porting our projects on Windows. We're using an automated build system
(buildbot.sf.net) to build every commit on Linux/OSX/Windows. Automating the
process on Windows proved to be really annoying. I looked at how other
projects do it (Python, KDE, Subversion etc.) they all seem to use hand
written .bat files and/or Makefile.vc (for Visual C++) with nmake and co. I
didn't want to deal with special Makefiles or setting up a special build
system for each project, so I tried to do something more generic.

Things are rather complicated but I'll try to explain them here. Everything
might not be relevant to the problem described below but I suppose you people
have a good experience about porting applications so you might be able to
advise me.

First off, I chose to use a Cygwin environment for the buildfarm. This was
convenient since Cygwin comes with lots of packages (python, vim, zsh etc)
and make it easy to have a complete environment on the Windows machine. The
buildbot (which is a python app) runs in that environment.

Qt distributes Windows binaries for mingw only so I ended up installing
mingw. This entails that everything is built by the mingw-gcc toolchain rather
than by the cygwin-gcc. Anyway, I *guess* that building mingw apps in a
Cygwin environment isn't a problem since there is probably not much
differences between both.

However, doing this made me run into several problems. For instance, Qt
generates Makefiles that have rules such as:
C:\Foo\Bar: C:\Baz\Qux
Which is a problem for cygwin's GNU make (since it sees "C" as a target). For
some reason the mingw GNU make has no problem with this kind of rule, I guess
they patched it. Nevermind, let's use mingw GNU make. The problem is that
when make runs commands such as C:\Qt\4.2.1\bin\moc.exe, the backslashes are
interpreted by the shell (cygwin's shell) which leads to the execution of the
command C:Qt4.2.1binmoc.exe. I tried to use MSYS's shell (the shell provided
with mingw) or cmd.exe directly but this wouldn't work (for some reason it
always ended up running C:Qt4.2.1binmoc.exe).

My idea was to write a perl script and to invoke make SHELL=my_shell.pl. That
script would rewrite the command properly so that it works within the Cygwin
environment. I succeeded and I can now run configure scripts and compile my
projects using that shell-wrapper.
In order to make this automatic and transparent, I have ~/bin first in my
PATH and I wrote a shell script named make.exe there that forwards the
invocation of make to mingw's GNU make with the SHELL variable set.

However, on last problem remains, and that's where libtool comes in. MingW
applications don't know anything about the cygwin environment, they're not
aware of the pseudo filesystem emulated by cygwin. At the end of the
compilation, libtool is invoked to link the final executable but for some
reason, it transforms a relative path in an absolute one:

'C:/cygwin/bin/perl' -w C:/cygwin/home/build/bin/my_shell.pl ../libtool
--tag=CXX   --mode=link g++ -O2 -O2 -frtti -fexceptions -Wall -DUNICODE
-DQT_LARGEFILE_SUPPORT -DQT_DLL -DQT_NO_DEBUG -DQT_XML_LIB -DQT_GUI_LIB
-DQT_CORE_LIB -DQT_THREAD_SUPPORT -DQT_NEEDS_QMAIN -Wall -W -Wold-style-cast
-Woverloaded-virtual -pipe -g -O2 -lqscintilla2 -mthreads
-Wl,-enable-stdcall-fixup -Wl,-enable-auto-import
-Wl,-enable-runtime-pseudo-reloc -Wl,-s -Wl,-s -Wl,-subsystem,windows   -o
urbidev.exe [MANY.o] -L"c:\Qt\4.2.1\lib" -lmingw32 -lqtmain -lQtXml4 -lQtGui4
-lQtCore4 ../behavior-graphs/src/libfsm.la
../behavior-graphs/src/xml_parser/libfsm_xml_parser.la
C:/cygwin/home/build/bin/my_shell.pl: Rewrote the command to: sh ../libtool
--tag=CXX   --mode=link g++ -O2 -O2 -frtti -fexceptions -Wall -DUNICODE
-DQT_LARGEFILE_SUPPORT -DQT_DLL -DQT_NO_DEBUG -DQT_XML_LIB -DQT_GUI_LIB
-DQT_CORE_LIB -DQT_THREAD_SUPPORT -DQT_NEEDS_QMAIN -Wall -W -Wold-style-cast
-Woverloaded-virtual -pipe -g -O2 -lqscintilla2 -mthreads
-Wl,-enable-stdcall-fixup -Wl,-enable-auto-import
-Wl,-enable-runtime-pseudo-reloc -Wl,-s -Wl,-s -Wl,-subsystem,windows   -o
urbidev.exe [MANY.o] -L"c:/Qt/4.2.1/lib" -lmingw32 -lqtmain -lQtXml4 -lQtGui4
-lQtCore4 ../behavior-graphs/src/libfsm.la
../behavior-graphs/src/xml_parser/libfsm_xml_parser.la
mkdir .libs
g++ -O2 -O2 -frtti -fexceptions -Wall -DUNICODE -DQT_LARGEFILE_SUPPORT -DQT_DLL
-DQT_NO_DEBUG -DQT_XML_LIB -DQT_GUI_LIB -DQT_CORE_LIB -DQT_THREAD_SUPPORT
-DQT_NEEDS_QMAIN -Wall -W -Wold-style-cast -Woverloaded-virtual -pipe -g -O2
-mthreads -Wl,-enable-stdcall-fixup -Wl,-enable-auto-import
-Wl,-enable-runtime-pseudo-reloc -Wl,-s -Wl,-s -Wl,-subsystem -Wl,windows -o
urbidev.exe [MANY.o]  -lqscintilla2 -Lc:/Qt/4.2.1/lib -lmingw32
../behavior-graphs/src/.libs/libfsm.a
/home/build/build/urbidev_winxp/build/behavior-graphs/src/xml_parser/.libs/libfsm_xml_parser.a
../behavior-graphs/src/xml_parser/.libs/libfsm_xml_parser.a -lqtmain -lQtXml4
-lQtGui4 -lQtCore4
g++.exe:
/home/build/build/urbidev_winxp/build/behavior-graphs/src/xml_parser/.libs/libfsm_xml_parser.a:
No such file or directory

(Note that I stripped the numerous .o)
g++.exe (mingw's g++) does not know how to access things from /. Prefixing
/home by C:/cygwin works. However, since the original command line was using
a relative path (which works fine), I don't see why libtool rewrote it as an
absolute path.

Note that although I simply ran ./configure which leads configure to guess
that host = build = i686-pc-cygwin, running ./configure --with-gnu-ld
--host=mingw32 --build=mingw32 leads to the exact same problem.

My final questions are: is this necessary on Windows? Can't you keep the
relative path? What do you think about my weird setup mixing
Cygwin/mingw/perl hooks? Are you aware of The Right Way of doing it when it
comes to automated builds on Windows?

I saved various logs and useful files you might want to read at:
http://www.tsunanet.net/~tsuna/libtool_abs_path/
This includes the bootstrap and configure and make process, the debug output
of libtool and the sources of my configure.ac and some M4 macros I defined as
well as my Perl script (just in case). Some files are available in .html so
that you can read them with syntax highlighting in your navigator.

Here are the version of the various tools involved:
autoconf (GNU Autoconf) 2.60
automake (GNU automake) 1.10
ltmain.sh (GNU libtool) 1.5.23a (1.1220.2.412 2006/10/13 14:13:30)
GNU Make 3.80
g++.exe (GCC) 3.4.5 (mingw special)

Sorry for the long mail and thanks for reading it.

-- 
SIGOURE Benoit aka Tsuna
   _____
  /EPITA\ Promo 2008, LRDE