cons-discuss
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: Modularizing cons


From: Greg Spencer
Subject: RE: Modularizing cons
Date: Fri, 31 Aug 2001 10:55:16 -0600

Attached are a couple of messages I sent to Steven and Tony about eight
months ago (when I had more time to work on these sorts of things).  I
didn't really get a lot of feedback then, but I think maybe it was too
abstract at the time.

I pulled that stuff out of mothballs, and my first attempt is attached,
although you should treat it as a prototype, since I haven't tested it
much at all.  You can run the test from the top level directory with the
command "perl ./cons -f mytest/Construct suba/foo.exe".  The only
objects that are currently implemented are the win32 VC++ compiler and
linker.  Obviously, the other platforms and objects would have to be
fleshed out for this to be useful to anyone.  The code is based on Cons
2.1.2.

Also, any feedback on my methods and decisions would be really helpful.
I stopped working on this because I didn't have the time to really flesh
it out so that it could be tested, and because I needed to get feedback
from the community before putting that much effort into it.

                                        -Greg.
--- Begin Message --- Subject: Refactoring Cons Date: Thu, 16 Nov 2000 15:19:21 -0600
Hi Steven and Tony,

So, I've started implementing a "builder" based version of
cons.  It doesn't really work yet, but I think I've
successfully separated out the legacy stuff from the "new"
stuff, and I've decided that I can make it so that all
people have to do is require "cons-legacy.pm" in their top
level Construct to get the old behavior back (in addition to
the new bahavior, actually).

I've started implementation using Steven's idea that there
shouldn't be a distinction between "Converters" and
"Condensers", since I think he's probably right (and I
figured that if he wasn't, it would be impossible to
implement correctly, and be a good proof).

The main part of the interface so far is a cons package
method "Process" (I wanted to use Build, but that was too
much like main::Build).  It takes one or two arguments: if
it has an arrayref as the first argument, and no second
argument, then it assumes that it is a list of source files
that need to have corresponding files created for them (a
one-to-one mapping, i.e. a compiler).  If it has an arrayref
and a scalar as arguments, then is assumes that it is to
"condense" the array of files into the output scalar (a
many-to-one mapping, i.e. a linker).  It can also be just a
single scalar, which it treats as the "one-to-one" case, or
two scalars, in which case it treats it like the
"many-to-one" case, but where "many" is just one.  Hopefully
this won't be too confusing, and will just "do what you
want".

In order to configure this in a more individualized way,
I've removed all of the configuration in "@param::defaults",
and moved it into the legacy stuff (where it pushes stuff
onto @param::defaults), and replaced it with the following:

---------------------------
# Default environment.
# Includes maps for commonly built targets, using
# the builders defined in the "Builder" package.
# If you're looking for defaults for the individual builders,
# look in the builder class you're interested in.
{
    my $CPPScanner = new Scanner::CPP;
    my ($CXXCompiler, $CCompiler, $Linker, $Library, $SharedLib);

    if ($_WIN32) { # use different defaults if we're on Windows.
        $CXXCompiler = new Builder::Compiler::Win32CXXCompiler;
        $CCompiler = new Builder::Compiler::Win32CCompiler;
        $Linker = new Builder::Linker::Win32Linker;
        $Library = new Builder::Archiver::Win32Library;
        $SharedLib = new Builder::Linker::Win32SharedLibrary;
    }
    else {
        $CXXCompiler = new Builder::Compiler::UNIXCXXCompiler;
        $CCompiler = new Builder::Compiler::UNIXCCompiler;
        $Linker = new Builder::Linker::UNIXLinker;
        $Library = new Builder::Archiver::UNIXLibrary;
        $SharedLib = new Builder::Linker::UNIXSharedLibrary;
    }
    
    @param::defaults = (
       'BUILDERMAPS' => [ 'OUTPUTMAP', 'INPUTMAP', 'SCANNERMAP' ],
       'OUTPUTMAP'   => {
                         "$_o" => $CXXCompiler,
                         "$_exe" => $Linker,
                         "$_a" => $Library,
                         "$_so" => $SharedLibrary,
                        },
       'INPUTMAP'   => {
                        ".c" => $CCompiler,
                       },
       'SCANNERMAP'   => {
                          '.c'  => $CPPScanner,
                          '.s'  => $CPPScanner,
                          '.S'  => $CPPScanner,
                          '.C'  => $CPPScanner,
                          '.cc' => $CPPScanner,
                          '.cxx'=> $CPPScanner,
                          '.cpp'=> $CPPScanner,
                          '.c++'=> $CPPScanner,
                          '.C++'=> $CPPScanner,
                         },
      );
}
---------------------------

This allows the handler for each type of input file to be
specified as a result of the output that builder creates.
In the "Process" funtion, the builder that is used for a
particular file type is determined as follows:

    1) for each input file, if there is an entry in the
       "INPUTMAP" for that filetype, then that builder is
       used.
    2) if not, then each of the output handlers is asked if
       it can take that kind of input.  If there is more
       than one builder that can take that type, cons dies:
       it must be explicitly set out in the input map.
    3) for each input file, the set of output files is
       determined by asking the newly determined builder,
       and,
    4) the union of all the output files from all the input
       sources is returned as an array.

Scanners for each of the input file types are determined by
the scanner map.

Now, I've reached a crossroads, and I'm wondering which way
to go...  In order to configure each of the builders
separately (which is actually quite necessary, in order to
allow the full range of possibilities), I can either 1) go
the OOD way, and provide accessors for each parameter,
hiding the actual command-line flags from the user (and
instantly making it a lot easier to do multi-platform
builds), or 2) I can just provide a way to modify the
"environment" for each of the builders, and allow them to
interpret it any way they want, maybe making some standard
variable names along the way.

I'm thinking that if I follow the OOD way to modify a
builder, you do something like this in the Conscript files:

--------------
Import qw(INCLUDE LIB BIN CONS BUILD DEBUG);

@SRCS = qw(foo.cxx bar.cxx);

$cons = $CONSBASE->clone();

$cons->GetBuilder('.cxx')->PushInclude("#sdk/include");
$cons->GetBuilder('.cxx')->SetPrecompiledHeader("LibPCH.h",
                                                "LibPCH.cxx");
$cons->GetBuilder('.cxx')->SetOptimized($DEBUG);

@OBJS = $cons->Process(address@hidden);  # build the OBJs
$cons->Process(address@hidden,'foo.exe');  # build the executable
--------------

Now, I know it seems a lot more verbose than what we've had
before, but I'm pretty sure its a lot more powerful.  For
instance, there's no indication of what platform this is, or
what compiler -- that's all handled when the builders are
configured in the mappings.  Also, some elements could still
come out of the cons environment.  For instance, we could
have a standard cons environment variable for preprocessor
defines, and we would still be using the PATH and other
things out of the cons ENV (just as we do now).

Of course, we could also define some shortcut methods for
building programs, shared libs, etc, that would condense the
last two lines into one line.

On the other hand, here's how I think an environment-based
configuration would look:

--------------
Import qw(INCLUDE LIB BIN CONS BUILD DEBUG);

@SRCS = qw(foo.cxx bar.cxx);

$cons = $CONSBASE->clone(
        '.cxx' => [ PCHHEADER => 'StdAfx.h',
                    PCHSRC => 'StdAfx.cxx',
                    OPTIMIZE => $DEBUG,
                    CPPPATH => $cons->GetBuilder('.cxx')->
                               {CPPPATH}.";sdk/include" ]
        );

@OBJS = $cons->Process(address@hidden);  # build the OBJs
$cons->Process(address@hidden,'foo.exe');  # build the executable
--------------

I think that although I can see that we could do similar
things with this method as we could with the OOD method, it
seems easier to standardize an interface on an object
(i.e. PushInclude, SetOptimize) that it is to standardize
the environment variables (CPPPATH, OPTIMIZE).  In reality,
it's not really all that different, it just feels like an
interface is more "permanent" than a varaible name (Note
that in the OOD example, the methods there are probably
methods on the Builder::Compiler base class, and not on
Builder.)

Things I really like about the OOD approach:

1) it's more platform independent
2) I really like the idea of not having to worry
   about path separators in an include path.
3) It's easier to make changes to all builders in a class at
   once, (assuming the changes can be made in a base
   class).
4) there's a clear hierarchy and API for builders, so adding
   in new ones should be more straightforward.
5) Adding in things that aren't compilers/linkers/archivers
   should be easier, since they can each have their own API.
6) Adding new compilers/linkers/archivers should be easier,
   because a base class should be able to handle most of the
   basic API for you, and you just need to implement the
   differences.
7) it can make things more "automatic", so that when you, say,
   turn on debug mode, it can automatically change preprocessor
   defines.

Things I don't like:

1) it's more verbose.
2) it's different than methods we've been using (i.e. hash
   array assignment lists).
3) a plethora of API could make things harder to keep in
   your head (although a plethora of environment variables
   could be just as bad, and people are more likely to
   document API).

I'm still leaning towards to OOD method, because of its
power, but I'm not totally conviced that others will want
that -- they may be more comfortable with the environment
list.

Any opinions/comments?

                        -Greg.

Attachment: smime.p7s
Description: S/MIME cryptographic signature


--- End Message ---
--- Begin Message --- Subject: Modular Cons efforts Date: Fri, 1 Dec 2000 15:54:56 -0600
Hi Guys,
 
    I've gotten the newly factored cons to work now, and I think I'm ready to start populating the classes so that there are basic compiler configurations for Win32 and UNIX.
 
    It's going to need a lot of testing, of course, but a basic Conscript looks like this:
 

Import qw(CONS);
 
@SRCS = qw(foo.cxx);
@LIBS = qw(kernel32.lib);
 
$cons = $CONS->clone();
 
$cons->GetBuilder('.cxx','.obj')->AppendIncludeDirs("#include");
$cons->GetBuilder('.obj','.exe')->SetLibs(@LIBS);
 
@OBJS = $cons->Process(address@hidden);  # build the OBJs
@PRODUCTS = $cons->Process(address@hidden,'foo.exe');  # build the executable

 
I had to add  a second argument the the GetBuilder routine, since in order to specify it completely, you have to have a source and a destination type (since an .obj could be an input to either a linker or an archiver, for instance).
 
I've decided to go the OOD way that I described before, so I need to come up with what a useful set of methods would be on a compiler, linker and archiver.  So far, I'm thinking that the base Compiler methods would be:
For linkers, the base methods could be:
I've implemented most of these already, and it seems to work OK.  All of the methods that take directories as strings will take dir refs too, or a mixture of the two.  the "GetxxxDirs" methods return a list of dir refs, and dir refs are stored internally.
 
The class hierarchy I have so far is:
 
Builder
Builder::Compiler
Builder::Compiler::VCXX
Builder::Linker
Builder::Linker::VCXX
Builder::Archiver
Builder::Archiver::VCXX
 
Many more should be created: shared libraries, install builders, other platforms, etc.  I'd like to have an easy out-of-the-box solution for most folks on most platforms, but let it be infinitely customizable for anyone who needs it.
 
It's separated into four modules so far:
 
cons (of course)
cons-legacy.pm (for legacy code)
cons-builder.pm (for the builder base classes)
cons-win32.pm (for win32 related classes)
 
You include what you want.  I can even include the cons-legacy by default, and it doesn't change how things work, since all my code uses the same internal mechanisms that already exist (like build::command, the cpp scanner, the file class, etc.), it's just crufty to include the legacy stuff if we aren't using it anymore.  Also, since there's only code for the relevant platform being included, there's less perl parsing startup cost (not that it ever was an issue).
 
I haven't implemented any of the scanner stuff yet, but that should be straightforward.
 
How does that look so far?  I can still change things quite easily, so go ahead and criticize.  Later on, when I've fleshed out all the classes, it will be a lot harder to make fundamental changes (or maybe not -- depends on the change).
 
                            -Greg.
 

Attachment: smime.p7s
Description: S/MIME cryptographic signature


--- End Message ---

Attachment: ModularCons.tar.gz
Description: Macintosh archive


reply via email to

[Prev in Thread] Current Thread [Next in Thread]