[Axiom-developer] Re: notangle

[Waldek Hebisch]

C Y wrote:
> --- Waldek Hebisch <address@hidden> wrote:
> 
> > Below is a newer version of CL notangle, this time handling recursive
> > references.  There are still significant differences compared to
> > real notangle (lack of @ escapes, different end of line handling,
> > ...).
> 
> Well, we probably don't actually need the full power of noweb notangle
> - I think it makes more sense to enforce sane pamphlet writing style
> instead (improves human readability, for that matter - stays
> consistent).
> 
> By lack of @ escapes, do you mean that any @ in the file is treated as
> the end of a code chunk, e.g. no awareness of position?  (e.g. it won't
> be able to process itself as a pamphlet?)  From what I saw earlier the
> only two things that are really needed are recognizing an illegal chunk
> name due to newline inclusion (you may not even need that one the way
> you've defined your start and end chars) and making sure @ is right
> after a newline (at least for parsing Axiom files, which is the main
> goal.)  I'll have a go at it later and see how it behaves.
> 

Look at the following C program:
<<*>>=
int
fun(int x)
{
   return (x@<<8) | (x>>8);
}
@
<<8) | (x>>=
+
@

Normal notangle gives:

int
fun(int x)
{
   return (x<<8) | (x>>8);
}

while CL version gives:

int
fun(int x)
{
   return (address@hidden);
}

Without the "8) | (x" chunk CL notangle would just halt on error.

> > I also fixed performace problem in scan-for-chunks (previous version
> > was using 'equal' and sbcl emited actual call to 'equal', current
> > version uses 'eql' which generates inline code).  Another performance
> > problem was using arrays of bytes as hash keys (completly killed
> > performance for pamphlets with many small chunks, or chunks
> > split into many small pieces).  I solved this problem converting
> > arrays to strings before hashing.
> 
> Wow.  VERY impressive work.  One question Waldek - I'm willing to try
> to convert this code into a literate document for purposes of Axiom
> inclusion, but I fear I'm not competent to describe these issues and
> reasons for specific choices.  I'll take a stab at it using your emails
> as a starting point (indeed, that might be a good way to ensure the
> documentation covers enough for other clueless beginners) but are you
> up for a few painfully ignorant questions? ;-)
>

Ask, I will try too answer.
 
> Also, are you OK with the modified BSD license for this?  It's
> essentially all your code at this point so I don't want to presume.
>

Sure, I prefer to omit license from files to avoid clutter, but use
the code any way you wish.  For Axiom distribution probably putting
a single line like "For legal info see Axiom.Licence" could be a good
compromise between beetween legal pedantry and practicality.
 
> > This version seem to be significantly faster than notangle: it is
> > able to process simple 8 Mb file (containing few big chunks) in 0.14s
> > while notangle needs 2s.  On 8Mb file containing 1000 chunks, each
> > consisting of 100 pieces 10 lines each, my version needs 0.24s, while
> > notangle uses 2.85s. 
> 
> You may have just become the poster child for those trying to debunk
> the "lisp is slow" myth ;-).
>
 
Do not be too eager with this.  It is clear that even for strings
hashtables take a lot of time.  We are forced to do an otherwise
useless copy to string to avoid huge slowdon.  And in the main
loop the code is clearly worse then the code from a C compiler.
So the program is probably faster than similar Perl or Python
programs, but slower than good C version.

> > BTW. Timing was done on 2GHz Athlon 64.  Notangle came from Axiom
> > build, Lisp is sbcl 1.0.  I tried also clisp, but it is quite slow
> > (need 8.73 senconds on my second test file) -- no wonder since I use
> > a low level code.  This version (like the previous one) does not
> > work with gcl (ANSI 2.6.5 interpreter claims that 'read-sequence'
> > is undefined, non-ANSI interpreter chokes on 'defclass').
> 
> Um.  Clisp doesn't surprise me, and normally if speed is a concern
> people don't run on Clisp anyway.  I also ran into the defclass thing
> on GCL - perhaps we could use defstruct instead?  I don't know that
> there's any particular reason to use CLOS other than convenience and
> trying to be "modern".
> 

Yes, defstruct should be enough.  'read-sequence' (and I guess also
'write-sequence') causes more problems, because performance depends on
it.  For gcl we probably must provide our own (calling C) versions.

-- 
                              Waldek Hebisch
address@hidden