RE: [Axiom-developer] Minor bugs

[Bill Page]

Tim,

On Sunday, December 19, 2004 3:17 PM you wrote:
> 
> The regression test process has not yet been recovered.
> I'll document it when I do recover it. Like everything
> else it "should" be working now but it all takes time.
> You've followed the pattern so your input file will be
> picked up in the regression process when it works.
> 
> The regression testing is done by hand for the moment.
>

But there is a lot of regression stuff in the

 src/input/Makefile.pamphlet

that is trying to run every time I do a build ...

> 
> ><soapbox>
> >BTW, all of this stuff seems so excruciatingly low-level to
> >me and SO EASY to break one way or another
> >
> > <b>like most of the Axiom build process --  !!</b>
> >
> ><sigh />
> 
> Yes, but trivial to fix because it's just a missing character or two.
> The reliability of a process is a ratio of the mean-time-to-failure
> and the mean-time-to-repair.

For me, mean-time-to-repair is VERY HIGH. It takes the system
on which I am running these builds about 1 hour to get around
to pointing out to me that I used a space instead of a comma.
I have repeated this process (as time permitted) about 10
times over the last two days, each time repairing just one
little thing that I had not idea I had broken.

> The build process is easily broken but trivial to fix,
> thus highly reliable.

After 2+ years of doing this stuff with Axiom I STRONGLY
disagree.

> Note that I've included explicit debugging information
> (those pesky unique echo line numbers) so the location of
> a mistake is painfully obvious and so is the fix.

Yes, useful if it even gets to that stage.

> 
> >I am becoming used to the idea of automated tools that make
> >this "messing around with picky details" less unnecessary.
> >Documentation is of course is still important but when it is
> >at such a low level and burdened with many details, just the
> >"on ramp" to the learning curve becomes very high even if the
> >process itself is quite simple.
> 
> If you have to work "thru a tool" and there is a bug in the
> tool you can spend hours (a) deciding it's a tool bug and
> (b) fixing the tool. Just look at the struggles of line-breaking
> and texmacs.

I don't understand this argument. We do not build houses any
more with sticks and rock hammers ...

> 
> Axiom is a finely tuned, high performance, very large machine.
> I'm working to include every bit of knowledge about the build
> process into the makefiles so (a) it works, (b) it's automated
> and (c) it's documented.
> 

Yes but it is so huge that it is usually like looking at an
elephant's tail and deciding that it is a rope ... etc. you
know that story.

> The makefiles are not hard as evidenced by the fact that
> i gave you NO instructions on how to make the changes and
> you did everything correctly.

I don't know anyone else on this project that is doing
this... :(

> The makefiles are tedious, I'll give you that.
> 
> >I find I spend an enormous amount of time just correctly
> >stupid simple silly mistakes like the use of a space some
> >place in a Makefile rather than a tab. Using "stupid tools"
> >always makes me feel stupid...
> >
> >Since a lot of the "pickiness" is just in the use of the
> >noweb/pamphlet low-level literature programming stuff, we
> >could for example (if we had infinite time and resources :),
> >invest in a more high-level literate programming tool such
> >as Leo.
> >
> 
> [.....SNIP.....]
> 
> I downloaded and used Leo when I started this process but
> Leo does not really understand the pamphlet structure and
> Leo isn't emacs.

Leo has a better approach than "pamphlets".

> Editors are like wives; you only get one,
> you learn its quirks, you learn to love its features and
> flaws, it is very costly to switch, your new editor 
> hates it when you use expressions from the previous editor,
> and you can get in deep trouble asking the new editor to
> do what the old editor used to do. I'm married to emacs.

That is a bad state to be in. As far as I am concerned
editors are "a dime a dozen" I probably use half a dozen
different editors all in the same day. I am more particular
about my relations with other people, however ... :)

> 
> Leo was not effective for me. If you can use it without
> breaking the pamphlet structure it should be transparent
> to everyone else.
> 

No, the pamphlet structure is too particular to Axiom.
But it is possible to do the same thing with Leo, only
better.

> In the fullness of time I'm hoping that your mathaction
> site (and the doyen local version of it) understands the
> pamphlet file structure and we can use it directly to 
> maintain the sources.
>

Perhaps, I like the idea of a collaboration project
environment mediated by something like MathAction, but
I think that is quite a long way off. In the mean time
Leo has a lot of useful features that one can get just
by downloading it to your desktop.
 
> >At the very least I think we need to spend more time
> >structuring the whole build process in a simpler way (i.e.
> >make more use of the advanced features of gnu make). I think
> >the whole thing from start to finish could be MUCH simpler
> >and the Makefiles much smaller. But then, what we have now
> >works (at least for those who designed it ... :) so of course
> >the incentive is not so great to change it. Still it seems
> >to me to be a significant barrier to greater participation
> >by other developers.
> ></soapbox>
> 
> Ummm, your definition of 'simpler' is different from mine.
> Using "advanced" features in gmake is NOT making it 'simpler'.

For the most part the "advanced" features of gmake are
quite trivial to learn - no more difficult than programming
in lisp. And if by learning one or two things it is possible
to reduce 3000 lines of makefile stanzas to just three lines
that it seems to me that the gain is enormous. Yes it takes
be a little longer to understand exactly what those 3 lines
are doing, but when I do it is equivalent to having read
and understood 3000 lines of other approach.

> Up until the introduction of $(shell..) the makefiles were
> "vanilla" and did not depend on any version of make. The
> size of the makefile is not a measure of complexity as you'll
> discover that there is a simple, repetitive pattern used in
> every makefile.

A simple repetitive pattern is a sure sign that you
are coding at too low a level.

> 
> My other open source project (magnus) used a makefile scheme
> that used every feature of gnu-make. In fact, it used the gcc
> compiler with the -MM switch to automatically generate the
> stanzas and dynamically load and evaluate the make dependencies.
> It was 'simple' in some sense but it took several days to decode
> how it worked because everything about the makefile was dynamic.
> unless you understood the gcc -MM switch, dynamic stanza loading,
> filename completion rules, and were a gnu-make wizard it was 
> unreadable and unmaintainable. 

I would VERY MUCH prefer that this approach be used with
AXIOM. Compared to the time it might take to become a "wizard"
working repeatedly with the low-level "keep it simple" approach
eats up far more of my time.

> 
> My makefiles are 'dumb' by design. Keep it dirt simple.  For
> every file there is a stanza.  The makefile structure follows
> the directory structure.  You don't even need to guess which
> makefile to modify as the makefile is always in the current
> directory.
> 
> Each directory has a makefile which has 2 responsibilities:
> (a) 'make' all subdirectories and (b) build its piece of
> the final build. it's a tree rooted at the top level.
>

That is a commonly used design but it has been strongly
criticized as fundamentally defeating the purpose for which
make was designed.
 
> And within each makefile there is a stanza for each final
> file in the build that gets made. No implicit rules are used.
> Every stanza is uniquely tagged by number and a comment of
> intention.  The makefiles follow the "pull" model. that is,
> 
>   final file in mnt   DEPENDS-ON file in obj    (the sys dep files)
> 
>   file in obj         DEPENDS-ON file in int    (the sys indep files)
> 
>   file in int         DEPENDS-ON pamphlet file in src
> 
>   doc file in mnt/doc DEPENDS-ON pamphlet file in src
> 
> I've previously explained the reasoning behind the src/int/obj/mnt
> structure. make (and gmake and configure) are based on the C 
> programmers world-view. They are not really capable of cleanly
> handling cross-directory dependencies although they can be
> coerced into it. 
>

Really I can only say that this just looks like a big mess
every time I leave it for a month and do something else and
later come back to it.
 
> In the fullness of time I'd like the whole system to be in
> common lisp. At that point i believe we should rewrite the
> build machinery so it uses 'asdf' or 'mk:defsystem' which
> follow the lisp worldview.
> 

Yes, we do certainly have different "world views"!!

--------------

Tim, please excuse the tone of this reply. I know that
it sounds rather adversarial. But keep in mind that I am
writing this while waiting for the 11th repeat of make
to come back and tell me I left out one more comma or
another. ...

Regards,
Bill Page.