RE: [Axiom-developer] database fixes

[Bill Page]

On October 19, 2006 10:38 PM Waldek Hebisch wrote:
> ...
> Bill Page wrote: 
> > When you say "as a reader", I wonder what you are reading:
> > the Lisp code or the dvi document that you generated from the
> > pamphlet file? In terms of creating a document that can be read
> > while sitting in a comfortable chair with your feet warmed by
> > the fireplace, I think your 2nd example is much better. :-)
> 
> Of course when reading paper copy first version is even more
> preferable -- searching electronic version is easier so one
> can somewhat tolerate jumping from labels to corresponding
> chunks. Reading paper copy one wants thing in linear order,
> to minimize searching of cross-references.
>

As I suspected, you are not thinking about reading a final
document, you are thinking only about reading the source.

Thus you would prefer that journals be printed in the original raw
LaTeX source format, omitting everything outside of \begin{equation}
... \end{equation}? Why bother to go to all that trouble to typeset
it, generate tables of contents, footnotes, equation references and
indexes when it is so much easier to read the equations in the
linear form? ;)

> > Of course to make good sense you must also consider the overall
> > structure of the document. The paragraph starting "To clean
> > old data" should be located in the section of the document where
> > you describe why it is desirable to clean up the database in the
> > first place, i.e. including at least some of the text in the
> > email that you sent introducing this patch.
> > 
> 
> Hmm, I do not think information from the e-mail belongs to
> the document -- for example writing a paper I discard many
> alternatives and failed attempts and publish only final version.

Thus you throw away the most important part! :-( One of the goals of
open source software development is collaboration - allowing many
people to contribute simultaneously to the same project. To do this
it is important to "release early and release often". In fact this
is a common motto of most open source projects. It means that we *do*
publish the many alternatives and failed attempts as well as the
"final version". Very few projects ever reach the stage of a final
version.

Axiom's pamphlet files should be living, changing things that reflect
our current state of knowledge about the system as well as containing
the actual code for the system.

The idea of publishing only final versions of things is obsolete.

> IMHO information about specific problem belongs to the ChangeLog
> (in form of a pointer to the message). Now if you think that what
> I wrote in the email needs extra record or additions I may register
> a bug.
> 

I have no objection to including such information in the ChangeLog
and capturing the detailed change history in a version control system.
But that is not an alternative to updating the pamphlet file itself.

We have a serious problem right now. Much of the most interesting
work on Axiom over the last 3 years is recording only in the email
archive of this email list. Many people have written many interesting
emails about Axiom. But it is embedded along with a lot of other
material that is not so interesting or of any lasting value. Yes
it is possible to search this linear chronology by keywords, author etc.
But in general it is hard to find the "good things" when you need to.

Tim Daly is the only person who has made a serious attempt to distill
what he has written into documentation that is available in pamphlet
files. He has even attempted to collect some of the useful material
from other people's emails but it is a difficult task to find and
extract and elaborate on just the right parts.

In some respects email is just "too easy"... on the other hand
people usually do find it much easier to write when their audience
is will defined - their email respondee and people presumed to be
reading the exchange from the public email list.

In my case I have frequently tried to extract such material and
format it for access on the Axiom wiki web site. In a way the wiki
web site represents an intermediate and alternate form of such a
"living" pamphlet document.

I am afraid that by neglecting to update the explanations in the
pamphlet files you are modifying you are condemning the interesting
arts of your emails to the same fate.

> > I probably would not have written:
> > 
> >   We also reset [boot::*allconstructors*]] to [[nil]].
> > 
> > Instead the text should say why we do that.
> 
> You, Gaby and Tim ask way. I am not sure if I can give good
> explanation.

So far you have given a better explanations in your email than
anything that I can find in the existing daase.lisp.pamphlet file.

> Basically, database is a single logical entity spread out into
> few variable and symbol properties (this is actually quite well
> explained in daase.lisp.pamplet).  All parts of database must
> be kept in sync.  Now, why my specific code?  The code follows
> database structure, the best explanation I can give just repeats
> database description.

No, not why your specific code. Instead, why you are writing such
code in the first place. This is better explained in your email
than in your final patch to daase.lisp.pamphlet. Even the above
paragraph of your email contains this useful information.

> 
> I can think of two changes to documentation: one is to say that
> all five openxxxx routines must be called together (because
> otherwise database gets out of sync). The second is that there
> is no need to have cleanup in browseOpen, because interpOpen
> should have already cleaned up old data.
> 

Yes, please include the comment also in the pamphlet file.

> ... 
> The point is that I did what I consider a self-contained change 
> (as I explained I do not think that _this_ change needs extra
> documentation).  While it is good to improve documentation
> part, that would be a separate change.  Keeping first change
> private and accumulating changes both slows progress and
> increases probability of conflicts during merge.
> 

I do not think you (or anyone else) should keep any of their work
private. I am very glad that you submitted a patch when you did.
I just think that you have much more that you could contribute
to improving the Axiom pamphlet files when it comes to the non-
code parts.

> Maybe another point: Axiom has parts that really need better
> documentation -- Bill Burge parser would be first example,
> Spad compiler the second. Compared to them daase.lisp.pamphlet
> contains straightforward and well documented code.  Also
> daase.lisp.pamphlet seem to work quite well compared to other
> parts of Axiom, so I see little reason to change it _now_ (except
> for this single patch which affects other things that I want
> to do) -- other parts need more care. 
>

The reason to change it now is that you understand it now. And
then other people will read it and begin to understand it. It
will save them time since they will not have to repeat your effort
of analysing the code and deducing it's purpose etc. And then they
will be able to direct their efforts to make contributions to those
"other parts that need more care".
 
> ...
> > 
> > Certainly it is possible to use diff on pamphlet files but in
> > some cases the result might include more than you are interested
> > in if someone has taken the time to add a lot of new (important!)
> > documentation to the file. Just like sometimes it is interesting
> > to run diff on a latex file, but most times not, because what we
> > are (usually) interested in when writing a latex document is the
> > final result not the details of source line indentation and coding.
> > 
> 
> I routinely run diff of latex files: when changing documents
> I want to avoid re-reading old parts and concentrate on changes.
>

You are still talking about reading the source of the document and
not about reading the presentation form of the document itself. It is
expected in a successful literate programming project however that
most people would begin by reading the actual literate documents in
their final format as dvi or pdf files or in printed form like in a
journal or a book.
 
> > > The patch I wrote changed the pamphlet file.  I admit that I did
> > > not use dvi viewer to check formatting, but since the parts I
> > > changed follow textual format using only text terminal should be
> > > OK.  
> > 
> > Actually I agree. Your original patch was not so bad - it is
> > just that it did not add any explanation to the document itself.
> > It improved the program but did not improve the document.
> >
> 
> I respectfully disagree: my change added extra explanations (and
> corrected a wrong explanation). Just is was a very small
> improvement (both to code and document aspect).
>

I am sorry. Perhaps I missed that part.
  
> > I would say that the main point of using a dvi viewer is not just
> > to check the formatting, but rather to focus on the document the
> > way new reader of the document would. You can ask yourself: Does
> > this document describe properly what this program is doing and why
> > I made the changes that I did?
> 
> Well my latex workflow is as follows: I compose text at a tty,
> I use xdvi to check formatting, for deeper revision I print
> the document.  When I have a lot of formulas I use xdvi much
> more, but for text I prefer tty.  Tty have two advantages:
> first is that I find tty fonts more pleasant to read, second
> is that knowing that text will be reformatted I can better
> concentrate on meaning.
>

Is this also how you would like to read other people's work in their
published articles?
 
> > 
> > Maybe the change in attitude from "documenting a program" to
> > "writing a literate program" is rather subtle. If you read the
> > literate programming news groups and web sites you will see that
> > many people have tried to explain this difference many times. So
> > I am not so confident that my attempt to do it again will be much
> > clearer. If not, I am sorry.
> > 
> 
> I have read Knuth literate programming text, later I read TeX
> (the program) and (at similar time) time Andrew Tanenbaum 
> "Operating Systems" which contained Minix sources.  Frankly, I
> find Tanenbaum's book better document then TeX.
> 

As I said, I did not really expect to be able to convince you about
the value of literate programming. :-(

And I agree that it's full actual value is yet to be proven in any
major computer programming project (except perhaps TeX). There are
other approaches to documenting programs such as javadoc, aldordoc
and related methods of trying to extract useful documentation from an
automated analysis of program structured and well-structured comments
embedded in the code. Or even just the discipline of attempting to
write the program code itself in a manner that is simple and easy for
a human being to read (within the constraints of the syntax of a
particular programming language). But Knuth's literate programming
methodology (as generalized by Norman Ramsey's "noweb" is what
Tim Daly chose as the preferred methodology for Axiom when he first
made Axiom available as an open source project. I think this bold
step deserves at least our "best effort" to see if such literate
programming can really make a long term contribution sometime within
the current "30 year horizon".

Regards,
Bill Page.