[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Fortran libraries on the Blue Gene with mpi

From: Ralf Wildenhues
Subject: Re: Fortran libraries on the Blue Gene with mpi
Date: Wed, 15 Apr 2009 23:35:23 +0200
User-agent: Mutt/1.5.18 (2008-05-17)

Hello John,

* John R. Cary wrote on Tue, Apr 14, 2009 at 07:23:41PM CEST:
> Ralf Wildenhues wrote:
>> * John R. Cary wrote on Mon, Apr 13, 2009 at 10:19:49PM CEST:
>>> I am trying to link some Fortran code on the Blue Gene (P) using
>>> the xl compilers.
>> Probably Libtool needs to be ported fully to this system first, or,
>> just the name matching be adjusted (libtool matches compilers by their
>> names).  Can you point us to documentation of the compiler and linker
>> suite of this system?  Would you be willing to go through the porting
>> effort with us (you'll have to answer a few questions and try out some
>> things on this system while we work on a patch).

> Sure, I would be happy to do this.


> First, the compilers are those at

Hehe.  Now the BG/L is a nice system: no virtual memory pages on the
compute nodes, so memory leaks won't be reaped.  Memory between the
application and the kernel is not separated.  A system for Real
Programmers[tm].  In comparison, BG/P seems harmless.  :-)

> But I think it is the linking that gets to be a pain.  This is a
> cross-compile situation, in which one compiles on 64-bit login nodes
> but runs on 32-bit compute nodes.  The former can have shared objects,
> but the latter cannot.

OK.  That's fairly easy to support in Libtool then: the only thing we
need to get right is to turn off shared library support completely for

> login3.intrepid$ \ls /opt/ibmcmp/xlf/bg/11.1/bin/bg*
> /opt/ibmcmp/xlf/bg/11.1/bin/bgf2003     
> /opt/ibmcmp/xlf/bg/11.1/bin/bgxlf2003_r
> login3.intrepid$ \ls /opt/ibmcmp/xlf/bg/11.1/bin/bg*
> /opt/ibmcmp/xlf/bg/11.1/bin/bgf2003     
> /opt/ibmcmp/xlf/bg/11.1/bin/bgxlf2003_r
> /opt/ibmcmp/xlf/bg/11.1/bin/bgf77      /opt/ibmcmp/xlf/bg/11.1/bin/bgxlf90
> /opt/ibmcmp/xlf/bg/11.1/bin/bgf90      /opt/ibmcmp/xlf/bg/11.1/bin/bgxlf90_r
> /opt/ibmcmp/xlf/bg/11.1/bin/bgf95      /opt/ibmcmp/xlf/bg/11.1/bin/bgxlf95
> /opt/ibmcmp/xlf/bg/11.1/bin/bgfort77   /opt/ibmcmp/xlf/bg/11.1/bin/bgxlf95_r
> /opt/ibmcmp/xlf/bg/11.1/bin/bgxlf      /opt/ibmcmp/xlf/bg/11.1/bin/bgxlf_r
> /opt/ibmcmp/xlf/bg/11.1/bin/bgxlf2003

Can we assume bg* produces code for the compute nodes only?  Let's turn
the question around: how can we reliably detect that code is being
compiled for the compute nodes?

What do

  CC=/opt/ibmcmp/xlc/bg/9.0/bin/bgcc path/to/config.guess

output for the system triplets?  Please download the latest config.guess
and config.sub files from <> for
this test, and adjust CC so that it points to a C compiler (or MPI
wrapper compiler like mpixlc) defaulting to code for the compute nodes.

Compiler names:
  BG/L: blrts_*
  BG/P: bg*
  bgc89 bgc99 bgcc bgxlc bgxlc++ bgxlC *_r 
  bgf200? bgf9? bgfort bgf77 bgxlf* *_r
  mpicc  (GCC)
  mpixl*(_r)? (XL)

Option -q64 is not supported.

Page 102 of <>
documents library creation.

It hints that one host_alias for the system is powerpc-bgp-linux.

For XL:

The flag to create a shared library is -qmkshrobj.
The flag to prefer (or force?) linking against shared libs is
-qnostaticlink.  If libgcc is to be linked shared, one needs
The flag to generate PIC is -qpic and seems to be necessary at compile
time only.
-Wl,-soname,SONAME works.

For GCC:
Flags are mostly the known ones: -shared, -fPIC
To prefer (or force?) linking against shared libs, pass -dynamic.

The soname naming scheme seems to be the plain Linux one, with two
symlinks set up.

In the appendix of the above-mentioned PDF, there is a link to

This document has more information on porting autotools-based packages
to BG/P.  This document has some smaller errors (namely it has no idea
of cross compiling support in configure scripts).  I will write to its
author about them.

It also lists some things differently:

-dy instead of -dynamic (gcc)
-G instead of -qmkshrobj (xlc)
-qnostaticlink is for xlc/xlC, -Wl,-dy is for xlf*

It documents LD_LIBRARY_PATH as suitable for shlibpath_var.
No mention of hardcoding facilities.

I am unsure whether -dy/-dynamic/-qnostaticlink is the default mode of
operation.  Can you find this out for me, for both compiler suites?
(Create a shared library without an accompanying static one, link a
program with it and find out whether you need to pass the flag).
  echo 'int foo () { return 0; }' > foo.c
  xlc -qpic -c foo.c
  xlc -qmkshrobj -o foo.o
  echo 'extern int foo (); int main () { return foo (); }' > main.c
  xlc -c main.c
  xlc -o main main.o -L. -lfoo  # without/with -qnostaticlink

Neither of these documents provide any indication as to hardcoding or
run paths into libraries or programs.  Can you find out whether the
runtime linker and the link editor have any kind of run path support?
You might need to look into the man pages for ld and (or whatever
its name is on this system) and search for "run path", "rpath",
"DT_RPATH", "-R", or similar names.

> which are probably also wrappers that ensure that one generates
> 32-bit code and does not link   So in the end, I think these are all just
> regular xl compilers.
> My solution, btw, is simply to pull any '-R' args out after libtool:

Yes, that's good as a workaround, but then you may need to remember to
set LD_LIBRARY_PATH appropriately.


reply via email to

[Prev in Thread] Current Thread [Next in Thread]