Re: [Axiom-developer] RE: algebra Makefiles with explicit dependencies, bootstrap, fixed-points etc.

[Stephen Wilson]

Bill, Tim,

The current makefile system, in my opinion, deserves the title "dirt
simple". As a new developer, I have had no problems understanding or
modifying the current build environment to suit my own experiments and
tests. Granted, I work mostly in the src/interp directory, but I have
made notes regarding how one would build only a minimal subset of the
algebra. Studying the algebra makefile has not left me with the
impression that it is difficult to work with. Whenever I have broken a
makefile, I have found it easy to repair.

Adding bootstrap lisp code to every .spad file also strikes me as
being unnecessary. Generally, if domain A depends on domain B, domain
A will need to be recompiled only if the signature of B is
modified. This rule is not sacrosanct, but will hold `most' of the
time. This is applicable to the case of circular dependencies as
well. Code will converge to a fixed point rapidly.

If the signature of a domain A is modified, and it participates in a
circular chain of dependencies, then it is probably necessary to break
the cycle by building a twin domain A' and hand edit the generated
lisp to reconnect the cycle. No makefile will help here. 

I am of the opinion that the current bootstrapping process is a
hack. It "just works", and for now I am happy with it. The real
solution to the problem is to either rewrite the algebra into a true
lattice, or to teach the compiler how to handle mutually recursive
structures. Both are clearly heavy tasks, and as I continue to parse
the src/interp code, perhaps I can help in realizing the latter
alternative. 

Ultimately, the makefiles should go away. Everything should be done at
the lisp level, hopefully guided by the logic of some future
compiler. For now, I feel the current "dirt simple" makefiles will be
capable of supporting such a transition.


Sincerely,
Steve



On Mon, Jan 03, 2005 at 09:19:17PM -0500, Bill Page wrote:
> Tim,
> 
> On Monday, January 03, 2005 6:29 PM you wrote:
> > 
> > Ah, you've hit the nail in my memory. The cyclic dependencies
> > will cause the system to build in an arbitrary way. Adding the
> > dependencies causes the algebra to violate the build order
> > because make thinks it knows better.
> 
> If you give make the correct dependencies then it *does* know
> better. I do not agree that "cyclic dependencies will cause the
> system to build in an arbitrary way."
> 
> > 
> > Once the algebra exists you can rebuild in any order so the bug
> > only shows up in the first build.
> 
> What "bug"? I do not agree that "you can rebuild in any order".
> 
> > Thus the algebra does not "really" depend on the dependencies you
> > are trying to hard code.
> 
> Yes it does.
> 
> > You can remake any algebra file without making the files it
> > depends on if you have a running axiom with already built
> > algebra.
> 
> No, that does not solve the problem. It only hides it.
> 
> > Indeed, if you modify one of the .spad.pamphlet files then typing
> > "make" will do the minimum amount of work correctly.
> 
> I do not believe that.
> 
> > 
> > re: BOOTSTRAP everything
> > 
> > The problem with this idea is that it is not needed.
> 
> I think you are wrong. You do not seem to understand what the
> existence of a cyclic dependency implies.
> 
> > The bootstrap files are very fragile but only for the first build.
> 
> I am not sure what you mean by "fragile". They are lisp source code
> files that should always compile the way they were intended.
> 
> > Once the algebra files have been built it is possible to change the
> > .spad files any way you'd like.
> 
> It is possible to make such a change, but as a consequence Axiom
> will no longer be in a consistent state. It will depend on the
> order in which the changes have been made rather than just on
> the spad source code itself. As a minimum all of the spad files
> that depend on the file that you changed, and those that depend
> on these in turn, need to be recompiled. But life in not even
> this simple because there are cyclic dependencies and this
> could potentially lead to an infinite loop. Just breaking this
> loop in an arbitrary manner the way the Axiom build does now
> is not enough.
> 
> > The 'hole' that a user can fall thru happens when they change an
> > algebra file that has BOOTSTRAP code (which they did NOT update)
> > and then try to build the system from scratch. In this case
> > the first copy of the system will be built with no-longer-valid
> > code.
> > 
> > What the user needs to do to change algebra which has 
> > BOOTSTRAP code is:
> > 
> > build the system
> > change the .spad file
> > recompile the .spad file
> > replace the BOOTSTRAP section of code with the newly
> > generated .lsp
> > 
> > now the BOOTSTRAP code mirrors the algebra code and all
> > is well.
> >
> 
> All is not well even though the all of the code has
> compiled.
> 
> This is a necessary step but it does not guarantee that the
> Axiom system is fully consistent because this is a *cyclic*
> dependency. Once you replace the bootstrap code, then you
> must (once again) rebuild all of the code that depends on
> it... potentially ad in finitum, except that you (usually
> but I can not prove always) will reach a "fixed-point" after
> the changes have propagated throughout each of the modules
> in the cycle. The convergence to a fix-point might involve
> at least as many repetitions as the longest cyclic dependency
> in the code.
> 
> > I don't ever expect a user to reach that far down into the
> > algebra. Making BOOTSTRAP files for all of the algebra opens
> > this hole everywhere.
> 
> The hole already exists everywhere. The current make process
> just does not address it. I doubt that the current build of
> Axiom is self-consistent.
> 
> Whenever there is a "cyclic dependency" it is just like
> solving a set of interdependent equations
> 
>    ax+b=y
>    cy+d=x
> 
> i.e. y depends on x and x depends on y. We want to find values
> for x and y that simultaneously satisfy both equations. We know
> how to solve this for the general case when the relations are
> linear. But if we didn't, we could try a "bootstrap" approach.
> We could start with some (bootstrap) value for x, compute y and
> they use y and the old value of x to compute a new x, etc. In
> complex systems of equations, this is a fairly efficient
> algorithm with which to solve the system.
> 
> The same thing is happening in the Axiom algebra spad code, but
> solving this (i.e. producing the correct binary code for all
> of the spad files in the cycle) is not so easy because the
> relationships between the spad files can be quite "non-linear".
> changes could potentially propagate in rather complex ways.
> A single pass through the bootstrap loop such as you describe
> above is quite unlikely to produce a consistent solution for
> all of the modules in a cycle except where the changes actually
> fail to propagate (possible in some cases where the dependency
> is conditional).
> 
> Regards,
> Bill Page.
> 
> 
> 
> _______________________________________________
> Axiom-developer mailing list
> address@hidden
> http://lists.nongnu.org/mailman/listinfo/axiom-developer