Attempt GUB build on 12.04

Started apt-get install for git and emacs.
Completed.

Started git clone

git clone git://github.com/colinghall/gub.git

Git clone completed

Attempted make help but make isn't installed.

$ make help
The program 'make' is currently not installed.  You can install it by typing:
sudo apt-get install make
address@hidden:~/gub$ sudo apt-get install make

Make installed. Try make help again

$ make bootstrap
python bin/gub  --platform=tools git
calculating dependencies
 *** Stage: download (make, tools)
downloading http://ftp.gnu.org/pub/gnu/make/make-3.81.tar.gz -> /home/colin/gub/downloads/make/
....................

I wonder if I can skip another monster download session by copying archives from the 32-bit VM?
Disconnected from screen session, closed the 64-bit VM.
Monday 4th June
Opened (resumed) the lilydev64 VM, ssh in, re-attached to screen session

Small shell script to do the whole build

#!/bin/sh
wget http://lilypond.org/download/binaries/test-output/lilypond-2.15.38-1.test-output.tar.bz2
mv lilypond-2.15.38-1.test-output.tar.bz2 gub/regtests/
touch gub/regtests/ignore
time ( cd gub && make bootstrap && make LILYPOND_BRANCH=release/unstable lilypond)

Started wget of regtest tarball
Done. Installing tarball
Issued command to make bootstrap and then make lilypond
Failed. Missing gcc. I'm not using LilyDev this time so I have to install all the dependencies.
List of build dependencies here: http://lilypond.org/doc/v2.15/Documentation/contributor/requirements-for-compiling-lilypond
Let's try build-dep lilypond
```
sudo apt-get build-dep lilypond
```
apt-get for 177 packages started.
Apt is running ok
Sent update to dev list
The apt-get install is complete

Re-issued timed build command

#!/bin/sh
time ( cd gub && make bootstrap && make LILYPOND_BRANCH=release/unstable lilypond)

Build chugging along happily through tools::make, tools::perl etc.

Still building

$
$ find gub/  -type f -printf '%TY-%Tm-%Td %TT %p\n' | sort | tail
2012-06-04 11:47:11.0877517240 gub/target/linux-x86/build/glibc-core-2.3/locale/broken_cur_max.os.dt
2012-06-04 11:47:11.0957517250 gub/target/linux-x86/build/glibc-core-2.3/locale/broken_cur_max.os
2012-06-04 11:47:11.0957517250 gub/target/linux-x86/build/glibc-core-2.3/locale/libBrokenLocale_pic.a
2012-06-04 11:47:11.1277517250 gub/target/linux-x86/build/glibc-core-2.3/locale/libBrokenLocale.so
2012-06-04 11:47:11.1317517250 gub/target/linux-x86/install/glibc-core-2.3-root/lib/libBrokenLocale-2.3.6.so
2012-06-04 11:47:11.1357517260 gub/target/linux-x86/build/glibc-core-2.3/elf/symlink.list
2012-06-04 11:47:11.1357517260 gub/target/linux-x86/log/glibc-core.log
2012-06-04 11:47:11.3717517330 gub/target/linux-x86/build/glibc-core-2.3/locale/broken_cur_max.o.dt
2012-06-04 11:47:11.3797517330 gub/target/linux-x86/build/glibc-core-2.3/locale/broken_cur_max.o
2012-06-04 11:47:11.3877517340 gub/target/linux-x86/build/glibc-core-2.3/locale/libBrokenLocale.a
$

Found the build terminated.

Looks like a configuration test for 32-bit-ness put the binary in the wrong place. Undefined environment variable? Console output was:

 building package: darwin-ppc::odcctools
 *** Stage: download (odcctools, darwin-ppc)
 *** Stage: untar (odcctools, darwin-ppc)
 *** Stage: patch (odcctools, darwin-ppc)
sh: 1: ./32bit: not found
error: cannot run 32 bit executable: 32bit


Tail of target/darwin-ppc/log/odcctools.log >>>>>>>>
    invoking cd /home/colin/gub/target/darwin-ppc/src/odcctools-278 && patch -p1 < /home/colin/gub/patches/odcctools-config-Wno-long-double.patch
    patching file configure
    patching file configure.ac
    invoking setup_linux_x86 ()
<<<<<<<< Tail of target/darwin-ppc/log/odcctools.log

Traceback (most recent call last):
  File "bin/gub", line 233, in exceptional_build
    build (settings, options, files)
  File "bin/gub", line 229, in build
    b.build_source_packages (names)
  File "bin/../gub/buildrunner.py", line 334, in build_source_packages
    self.spec_build (spec_name)
  File "bin/../gub/buildrunner.py", line 262, in spec_build
    deferred_runner.execute_deferred_commands ()
  File "bin/../gub/runner.py", line 167, in execute_deferred_commands
    cmd.execute (self.logger)
  File "bin/../gub/commands.py", line 172, in execute
    return self.func (logger, *self.args)
  File "bin/../gub/cross.py", line 230, in setup_linux_x86
    raise Exception ('Package %(name)s depends on 32 bit libraries''' % locals ())
Exception: Package Odcctools depends on 32 bit libraries
*** Failed target: darwin-ppc::odcctools
make: *** [cross-compilers] Error 1

real    42m32.265s
user    25m1.390s
sys     4m26.353s

Examining the last few modded files:

address@hidden:~$
address@hidden:~$ find gub/  -type f -printf '%TY-%Tm-%Td %TT %p\n' | sort | tail
2012-06-04 12:05:15.7957865480 gub/target/darwin-ppc/root/etc/gup/dicts.db
2012-06-04 12:05:15.7957865480 gub/target/darwin-ppc/root/etc/gup/files.db
2012-06-04 12:05:16.0077865540 gub/target/darwin-ppc/status/odcctools-278-odcctools-iphone-dev-278
2012-06-04 12:05:16.7637840890 gub/target/darwin-ppc/src/odcctools-278/as/read.c
2012-06-04 12:05:17.0317883250 gub/target/darwin-ppc/src/odcctools-278/configure
2012-06-04 12:05:17.0317883250 gub/target/darwin-ppc/src/odcctools-278/configure.ac
2012-06-04 12:05:17.0357881240 gub/32bit.c
2012-06-04 12:05:17.0597869150 gub/32bit
2012-06-04 12:05:17.0637868220 gub/target/darwin-ppc/log/odcctools.log
2012-06-04 12:05:17.0677866900 gub/log/gub.log
address@hidden:~$
address@hidden:~$

What are 32bit.c and 32bit doing in the root of gub directory?

Posted to lilypond-devel reporting build failure
Taking a break
Reading feedback on lilypond-dev.
Did some web research and found: http://devs.openttd.org/~truebrain/compile-farm/apple-darwin9.txt http://stackoverflow.com/questions/3024255/building-odcctools-in-a-64-bit-os
This needs some dedicated time to debug. Leaving it for now.
Tuesday 5th June. A little time to spare so resuming work.

Build failed on odcctools. Taking a look at the log:

 *** Stage: patch (odcctools, darwin-ppc)
invoking cd /home/colin/gub/target/darwin-ppc/src/odcctools-278 && patch -p1 < /home/colin/gub/patches/odcctools-r211-word.patch
patching file as/read.c
invoking cd /home/colin/gub/target/darwin-ppc/src/odcctools-278 && patch -p1 < /home/colin/gub/patches/odcctools-config-Wno-long-double.patch
patching file configure
patching file configure.ac
invoking setup_linux_x86 ()
Traceback (most recent call last):
  File "bin/gub", line 233, in exceptional_build
    build (settings, options, files)
  File "bin/gub", line 229, in build
    b.build_source_packages (names)
  File "bin/../gub/buildrunner.py", line 334, in build_source_packages
    self.spec_build (spec_name)
  File "bin/../gub/buildrunner.py", line 262, in spec_build
    deferred_runner.execute_deferred_commands ()
  File "bin/../gub/runner.py", line 167, in execute_deferred_commands
    cmd.execute (self.logger)
  File "bin/../gub/commands.py", line 172, in execute
    return self.func (logger, *self.args)
  File "bin/../gub/cross.py", line 230, in setup_linux_x86
    raise Exception ('Package %(name)s depends on 32 bit libraries''' % locals ())
Exception: Package Odcctools depends on 32 bit libraries

Patch went ok.

Finding the function setup_linux_x86()

Reading setup_linux_x86 I came across:

def setup_linux_x86 (logger, name, x86_cross):
    '''Hack for using 32 bit compiler on linux-64.

    Use linux-x86 cross compiler to compile non-64-bit-clean packages such
    as nsis and odcctools.  A plain 32 bit compiler could also be used,
    but we do not have such a beast.  Make sure to have 32-bit
    compatibility installed:
    apt-get install ia32-libs
    '''

Well, I don't have ia32-libs installed. I wonder if that was documented?
Break for lunch. After lunch, check doc link in recent email from Graham.

Found it, a recent post on lilypond-devel: http://lists.gnu.org/archive/html/lilypond-devel/2012-06/msg00088.html

On Mon, Jun 04, 2012 at 11:15:34AM +0100, Colin Hall wrote:
> 
> I'm attempting GUB on 64-bit VM runing Ubuntu Server 12.04 LTS
> 
> Progress so far attached. Advice welcome.

sorry, one-handed typing while eating.

see gub README for list of requirements.  much less than lilypond
build requirements.

- graham

From that README file:

INSTALLING

* You need
  - about 9 GB of free space (for all platforms)
  - standard unix shell utilities: cat, cp, install, mv, rm, sed, ...
  - a standard unix development environment with GCC and G++
  - Python 2.4 or newer (2.5, 2.6, 3.0 are known to work)
  - the libmpfr library (and development headers)

Python 2.7.3 and the build tools are all present. Need to add libmpfr and the ia32-libs. Issued the following command:
```
sudo apt-get install ia32-libs libmpfr-dev
```
Apt running, reporting over 300 packages to install. Breaking off for now as that will take a while.
Libs are all downloaded, unpacking.
apt-get completed successfully.

Probably a good idea to start afresh.

rm -rf uploads
rm -rf target  
time ( cd gub && make bootstrap && make LILYPOND_BRANCH=release/unstable lilypond)

Build running

Found the build had failed

building package: darwin-x86::cross/gcc
 *** Stage: download (cross/gcc, darwin-x86)
 *** Stage: untar (cross/gcc, darwin-x86)
 *** Stage: patch (cross/gcc, darwin-x86)
 *** Stage: autoupdate (cross/gcc, darwin-x86)
 *** Stage: configure (cross/gcc, darwin-x86)
 *** Stage: compile (cross/gcc, darwin-x86)
Command barfed: cd /home/colin/gub/target/darwin-x86/build/cross/gcc-4.3.2 && make -j2  tooldir='/home/colin/gub/target/darwin-x86/root/usr/cross/i686-apple-darwin8' gcc_tooldir='/usr/i686-apple-darwin8'

Tail of target/darwin-x86/log/cross/gcc.log >>>>>>>>
    make[1]: *** [configure-target-libgcc] Error 1
    make[1]: Leaving directory `/home/colin/gub/target/darwin-x86/build/cross/gcc-4.3.2'
    make: *** [all] Error 2
    Command barfed: cd /home/colin/gub/target/darwin-x86/build/cross/gcc-4.3.2 && make -j2  tooldir='/home/colin/gub/target/darwin-x86/root/usr/cross/i686-apple-darwin8' gcc_tooldir='/usr/i686-apple-darwin8'
<<<<<<<< Tail of target/darwin-x86/log/cross/gcc.log

*** Failed target: darwin-x86::cross/gcc
make: *** [cross-compilers] Error 1

real    59m39.139s
user    39m5.219s
sys     6m5.879s

Looking at recent files:

~$
~$ find gub/  -type f -printf '%TY-%Tm-%Td %TT %p\n' | sort | tail -20
2012-06-05 15:31:11.0753026810 gub/target/darwin-x86/build/cross/gcc-4.3.2/gcc/include/xmmintrin.h
2012-06-05 15:31:11.0793026840 gub/target/darwin-x86/build/cross/gcc-4.3.2/gcc/include/emmintrin.h
2012-06-05 15:31:11.0873026900 gub/target/darwin-x86/build/cross/gcc-4.3.2/gcc/include/pmmintrin.h
2012-06-05 15:31:11.0913026910 gub/target/darwin-x86/build/cross/gcc-4.3.2/gcc/include/tmmintrin.h
2012-06-05 15:31:11.0953026900 gub/target/darwin-x86/build/cross/gcc-4.3.2/gcc/include/ammintrin.h
2012-06-05 15:31:11.1033026870 gub/target/darwin-x86/build/cross/gcc-4.3.2/gcc/include/smmintrin.h
2012-06-05 15:31:11.1073026860 gub/target/darwin-x86/build/cross/gcc-4.3.2/gcc/include/nmmintrin.h
2012-06-05 15:31:11.1113026860 gub/target/darwin-x86/build/cross/gcc-4.3.2/gcc/include/bmmintrin.h
2012-06-05 15:31:11.1153026860 gub/target/darwin-x86/build/cross/gcc-4.3.2/gcc/include/mmintrin-common.h
2012-06-05 15:31:11.1233026860 gub/target/darwin-x86/build/cross/gcc-4.3.2/gcc/include/tgmath.h
2012-06-05 15:31:11.1473026820 gub/target/darwin-x86/build/cross/gcc-4.3.2/gcc/include/mm_malloc.h
2012-06-05 15:31:11.1673026870 gub/target/darwin-x86/build/cross/gcc-4.3.2/gcc/include/unwind.h
2012-06-05 15:31:11.2473026840 gub/target/darwin-x86/build/cross/gcc-4.3.2/gcc/include-fixed/limits.h
2012-06-05 15:31:11.2913026800 gub/target/darwin-x86/build/cross/gcc-4.3.2/gcc/include-fixed/README
2012-06-05 15:31:11.2953026790 gub/target/darwin-x86/build/cross/gcc-4.3.2/gcc/stmp-int-hdrs
2012-06-05 15:31:11.3073026800 gub/target/darwin-x86/build/cross/gcc-4.3.2/i686-apple-darwin8/libgcc/multilib.out
2012-06-05 15:31:11.5753026680 gub/target/darwin-x86/build/cross/gcc-4.3.2/i686-apple-darwin8/libgcc/config.cache
2012-06-05 15:31:11.7433026600 gub/target/darwin-x86/build/cross/gcc-4.3.2/i686-apple-darwin8/libgcc/config.log
2012-06-05 15:31:11.8153026550 gub/log/gub.log
2012-06-05 15:31:11.8153026550 gub/target/darwin-x86/log/cross/gcc.log
~$

Investigating

Looking at the last modified file, gcc.log, I found:

checking for i686-apple-darwin8-ranlib... /home/colin/gub/target/darwin-x86/root/usr/cross/i686-apple-darwin8/bin/ranlib -c
checking for i686-apple-darwin8-strip... /home/colin/gub/target/darwin-x86/root/usr/cross/i686-apple-darwin8/bin/strip
checking whether ln -s works... yes
checking for i686-apple-darwin8-gcc... /home/colin/gub/target/darwin-x86/build/cross/gcc-4.3.2/./gcc/xgcc -B/home/colin/gub/target/darwin-x86/build/cross/gcc-4.3.2/./gcc/ \
-B/home/colin/gub/target/darwin-x86/root/usr/cross/i686-apple-darwin8/bin/ -B/home/colin/gub/target/darwin-x86/root/usr/cross/i686-apple-darwin8/lib/ -isystem /home/colin/\
gub/target/darwin-x86/root/usr/cross/i686-apple-darwin8/include -isystem /home/colin/gub/target/darwin-x86/root/usr/cross/i686-apple-darwin8/sys-include
checking for suffix of object files... configure: error: cannot compute suffix of object files: cannot compile
See `config.log' for more details.
make[1]: *** [configure-target-libgcc] Error 1

So it's a failure in configuration. Searched back from the end of that config.log for the text reported in the gcc.log file, and I came across:

configure:2567: checking for suffix of object files
configure:2588: /home/colin/gub/target/darwin-x86/build/cross/gcc-4.3.2/./gcc/xgcc -B/home/colin/gub/target/darwin-x86/build/cross/gcc-4.3.2/./gcc/ -B/home/colin/gub/targe\
t/darwin-x86/root/usr/cross/i686-apple-darwin8/bin/ -B/home/colin/gub/target/darwin-x86/root/usr/cross/i686-apple-darwin8/lib/ -isystem /home/colin/gub/target/darwin-x86/r\
oot/usr/cross/i686-apple-darwin8/include -isystem /home/colin/gub/target/darwin-x86/root/usr/cross/i686-apple-darwin8/sys-include -c -O2 -g -g -O2    conftest.c >&5
/home/colin/gub/target/darwin-x86/build/cross/gcc-4.3.2/./gcc/cc1: error while loading shared libraries: libmpfr.so.1: cannot open shared object file: No such file or dire\
ctory
configure:2591: $? = 1
configure: failed program was:
| /* confdefs.h.  */
|
| #define PACKAGE_NAME "GNU C Runtime Library"
| #define PACKAGE_TARNAME "libgcc"
| #define PACKAGE_VERSION "1.0"
| #define PACKAGE_STRING "GNU C Runtime Library 1.0"
| #define PACKAGE_BUGREPORT ""
| /* end confdefs.h.  */
|
| int
| main ()
| {
|
|   ;
|   return 0;
| }
configure:2605: error: cannot compute suffix of object files: cannot compile
See `config.log' for more details.

Well, that's clear then. The linker can't find the libmpfr library. Odd, as I apt-get installed libmpfr-dev earlier today. Investigating.

Checked that the libmpfr runtime was a dependency of the dev package:

$
$ apt-cache depends libmpfr-dev
libmpfr-dev
  Depends: libgmp-dev
  Depends: libmpfr4
  Suggests: libmpfr-doc
  Conflicts: libgmp3-dev
  Conflicts: libgmp3-dev:i386
  Replaces: libgmp3-dev
  Replaces: libgmp3-dev:i386
  Replaces: libmpfr-dev:i386
  Breaks: libmpfr-dev:i386
$
$

Ah! Has the name of this library changed? Check.

$
$ find /usr/lib/ -iname \*mpfr\*
/usr/lib/x86_64-linux-gnu/libmpfr.a
/usr/lib/x86_64-linux-gnu/libmpfr.so.4
/usr/lib/x86_64-linux-gnu/libmpfr.so.4.1.0
/usr/lib/x86_64-linux-gnu/libmpfr.so
$
$

Perhaps we need the 32-bit version? Complete hunch.

$ apt-cache search mpfr
lib32gmp-dev - Multiprecision arithmetic library developers tools (32bit)
lib32mpfr-dev - multiple precision floating-point computation developers tools (32bit)
lib32mpfr4 - multiple precision floating-point computation (32bit)
libgmp-dev - Multiprecision arithmetic library developers tools
libmpc-dev - multiple precision complex floating-point library development package
libmpc2 - multiple precision complex floating-point library
libmpfr-dev - multiple precision floating-point computation developers tools
libmpfr-doc - multiple precision floating-point computation documentation
libmpfr4 - multiple precision floating-point computation
libmpfr4-dbg - multiple precision floating-point computation (debug symbols)
libgmpada-dbg - Ada binding to the GNU MultiPrecision library: debug symbols
libgmpada2 - Ada binding to the GNU MultiPrecision library: shared library
libgmpada3-dev - Ada binding to the GNU MultiPrecision library: development
libmpfi-dev - multiple precision floating-point interval computation library
libmpfi0 - multiple precision floating-point interval computation library
$

Could try apt-get on those 32bit versions and then try to build the cross compiler again.
```
sudo apt-get install lib32mpfr-dev
(cd gub && bin/gub --fresh darwin-x86::cross/gcc)
```
Apt-get completed ok. Issuing GUB invocation.
GUB running.
In the compilation phase. This is where it errored out before.

Opened annother ssh session on lilydev64:

$
$ cat gub/target/darwin-x86/build/cross/gcc-4.3.2/config.log | wc -l
786
$ tail gub/target/darwin-x86/build/cross/gcc-4.3.2/config.log
#define PACKAGE_BUGREPORT ""
#define PACKAGE_NAME ""
#define PACKAGE_STRING ""
#define PACKAGE_TARNAME ""
#define PACKAGE_VERSION ""
#endif
#ifdef __cplusplus
extern "C" void exit (int) throw ();

configure: exit 0
$

So it looks like the compilation of the cross compiler is continuing. Good.
No, hunch was wrong, it has failed in exactly the same way as before.
I have to find where the libmpfr dependency is coming from and track down the origin of it in the autoconf system. Then I can fix it. How about checking which of the (successfully linked and compiled) binaries in the 32-bit cross toolchain is linked with libmpfr?
Booted lilydev (the 32-bit VM) and started looking for the darwin-x86 cross-compiler.

Found it!

$
$ find gub/target/darwin-x86/root/usr/cross/libexec/gcc/i686-apple-darwin8/
gub/target/darwin-x86/root/usr/cross/libexec/gcc/i686-apple-darwin8/
gub/target/darwin-x86/root/usr/cross/libexec/gcc/i686-apple-darwin8/4.3.2
gub/target/darwin-x86/root/usr/cross/libexec/gcc/i686-apple-darwin8/4.3.2/cc1obj
gub/target/darwin-x86/root/usr/cross/libexec/gcc/i686-apple-darwin8/4.3.2/cc1plus
gub/target/darwin-x86/root/usr/cross/libexec/gcc/i686-apple-darwin8/4.3.2/collect2
gub/target/darwin-x86/root/usr/cross/libexec/gcc/i686-apple-darwin8/4.3.2/cc1
gub/target/darwin-x86/root/usr/cross/libexec/gcc/i686-apple-darwin8/4.3.2/cc1objplus
gub/target/darwin-x86/root/usr/cross/libexec/gcc/i686-apple-darwin8/4.3.2/install-tools
gub/target/darwin-x86/root/usr/cross/libexec/gcc/i686-apple-darwin8/4.3.2/install-tools/fixincl
gub/target/darwin-x86/root/usr/cross/libexec/gcc/i686-apple-darwin8/4.3.2/install-tools/mkheaders
gub/target/darwin-x86/root/usr/cross/libexec/gcc/i686-apple-darwin8/4.3.2/install-tools/fixinc.sh
gub/target/darwin-x86/root/usr/cross/libexec/gcc/i686-apple-darwin8/4.3.2/install-tools/mkinstalldirs
$

Alright, what does it link with?

$
$ ldd gub/target/darwin-x86/root/usr/cross/libexec/gcc/i686-apple-darwin8/4.3.2/cc1
        linux-gate.so.1 =>  (0x00d95000)
        libmpfr.so.1 => /usr/lib/libmpfr.so.1 (0x00c6f000)
        libgmp.so.3 => /usr/lib/libgmp.so.3 (0x00110000)
        libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0x00171000)
        /lib/ld-linux.so.2 (0x004f1000)
$

Right, so I need to find the autoconf source for the cross compiler.

Wait a minute. The mpfr library is one of the tools that GUB builds:

$ cat gub/gub/specs/mpfr.py
from gub import tools

class Mpfr__tools (tools.AutoBuild):
      source = 'http://www.mpfr.org/mpfr-2.3.2/mpfr-2.3.2.tar.gz'
      dependencies = ['libtool', 'gmp']
$

And the cross compilers targeting darwin have a dependency on it:

$
$ cat gub/gub/specs/darwin/cross/gcc.py | tail -15 | head -6

class Gcc__darwin__x86 (Gcc__darwin):
    source = 'ftp://ftp.fu-berlin.de/unix/languages/gcc/releases/gcc-4.3.2/gcc-4.3.2.tar.bz2'
    patches = ['gcc-4.3.2-no-fixed-includes.patch']
    dependencies = Gcc__darwin.dependencies + ['tools::mpfr']

$

So perhaps the problem is that the build of the cross compiler is failing to find the mpfr library built by GUB?
Taking a break.

From:	Colin Hall
Subject:	Re: GUB on recent Ubuntu release
Date:	Tue, 5 Jun 2012 18:47:28 +0100
User-agent:	Mutt/1.5.20 (2009-06-14)

Re: GUB on recent Ubuntu release

Attempt GUB build on 12.04