Re: New read/eval Scheme syntax inconsistent in handling existing code

[David Kastrup]

Valentin Villenave <address@hidden> writes:

> On Fri, Dec 2, 2011 at 1:05 AM, David Kastrup <address@hidden> wrote:
>> The reason is that eval-string in Guile does not set
>> port-line/port-filename to sensible values and I use them for making
>> sensible error messages actually pointing to the source of the error.
>
> What would be "sensible values" here?

The port-line/port-filename values that it had when reading a literal
string.  Of course, this approach would not work when we are talking
about a constructed string.  In any way, at least the Guile 1.8
documentation does not mention port-filename returning #f:

 -- Scheme Procedure: port-filename port
 -- C Function: scm_port_filename (port)
     Return the filename associated with PORT.  This function returns
     the strings "standard input", "standard output" and "standard
     error" when called on the current input, output and error ports
     respectively.

     PORT must be open, `port-filename' cannot be used once the port is
     closed.

One can guess that on some ports, it might not be a string.  But there
is nothing in the docs that would particularly suggest that.

> Does it imply that the standard syntax documented in Guile's manual no
> longer apply here?

No.

> In other words: what concrete changes could make this work?

Upgrading to the current development version now in staging and master.

> I don't doubt it. What I'd need is a few pointers as to how.

Upgrade, rinse and repeat.  Up to now, your problems have exclusively
been unrelated to the nature of the patches, but rather because of bugs
slipping through the current regtest net that could be fixed with few
lines of code.

>> Anyway, we are talking about a
>> _development_ version.  The changes _all_ passed _all_ regression
>> tests.  If you want to avoid breakage in development versions, you have
>> two options:
>> a) contribute regression tests for your use cases
>
> We've talked about that, and I will indeed.
>
>> c) make a proper bug report once things break for you
>
> See above in the thread. How is my first post here not a valid bug
> report?

It it based on a real bug.  But that's about it.

    Subject: New read/eval Scheme syntax inconsistent in handling
             existing code

    Greetings everybody,

    David's new code with #/$(scheme) styles does offer a lot of exciting
    possibilities... but it is also hugely disruptive when it comes to
    existing code! Consider for instance the following:

[...]

    Thoughts?

"Consider for instance the following" tries to present the bug as
endemic and a systematic part of the changes.  You don't ask for a fix,
but ask for "thoughts".  Not me as the author of the changes, but
everybody else.

>> Personally, I consider option
>> d) Try to discredit work being done with ominous postings missing useful
>>   information
>> not helpful.  I understand that you are pissed off when your scores and
>> work patterns (and apparently nobody else's) stop working.  You know how
>> to keep this from happening in future.
>
> I wish I knew.

Contribute regtests, participate in testing.  It might also be a good
idea to ask people on this list instead of Lilypond whether a particular
approach or hack is a good idea.  Like Picasso once stated: "Computers
are useless.  They can only give you answers."  Lilypond's answers are
strictly yes/no and, while mostly reliable (but see our recent segfault
problems) valid for a single version.

> Not pissed off, though. I think I get what you've been trying to
> achieve wrt LilyPond syntax and I have no doubt it is an improvement
> compared to how we dealt with Scheme previously.

Fine.  Now get yourself out of the "lambs to the slaughter" mind frame
and report bugs as bugs.  I am not, I repeat _not_ trying to replace one
set of weird, illogical behavior with another.  I am doing large changes
in order to get sane, predictable behavior.

If there are unexpected side-effects, they are bugs.  Skip the drama and
report them.  It's not like I have a history of ignoring reports.

> If you think preserving location data is preferable, then by no means
> should you change it only for my hacks to keep working.

There is nothing to preserve here.  Guile has already thrown it away.  I
just need to deal with that.  This happens when evaluating material
coming from a source completely disconnected from Lilypond's input
channels (if Lilypond provides the data port, it now keeps track of
where it is from).

> As I said, all I'm asking for in that case would be a way to work
> around the new behavior. (If at all possible.)

Sigh.  There is no "new behavior".  There _was_ a bug.  There is nothing
to "work around" here.

>> So you are making an appeal to "everybody" to join into a campaign
>> against "inconsistent" handling of "existing code", due to the bad
>> disruptive things David does.  Why doesn't that existing code exist
>> in the regtests?
>
> That BS and I hope you know it. It is customary for me (and others) to
> introduce my postings with "hi everbody", that doesn't make it a
> campaign. And "disruptive" doesn't necessarily mean "bad". But it _is_
> disruptive from where I'm standing (where I'm standing is: on top of a
> pile of dirty, hackish code, and I haven't denied that once).

Once you get into the habit of treating bugs like bugs instead of a
personal attack on your code base, you might change your mind.  At least
in this case.

>> Valentin, bugs will happen.  Using eval-string is actually bypassing
>> the normal interplay of Scheme and Lilypond interpreters, and it is
>> quite likely that there are much better ways for doing what you are
>> doing.
>
> There certainly are. Since I haven't been able (yet) to learn how to
> do delayed evaluation in Guile (with hooks and everything), this is my
> way of working around my limited skills so far.

Why do you want delayed evaluation in Guile?  Can you put together an
example demonstrating the technique you employ here?  Not just something
demonstrating a bug (which should be gone by now) but rather showing
what you use this for?

I am not, I repeat _not_ suggesting that what you do right now should be
allowed to break, or that such breakage be taken lightly.  I just
consider it likely that it isn't a good idea in the first place and
might have been necessary for historic reasons.

> I (think I) get it. I used to assume lacking location data was just
> normal in this kind of cases.
> Do you think there could be proposals to be made upstream about that?
> (i.e. fixing Guile itself instead of compensating within Lily?)

Would not make much sense, I guess.  Keeping location data for literal
strings so that one can use them at evaluation time is rather pointless
since reading and evaluating literal strings instead of just compiling
them into a function and calling that is just asking for trouble,
anyway.  A compiled function has location data for every component, and
is verified syntactically at compile time.

The use of "eval-string" does not just stick out like a sore thumb in
Lilypond.  It would also be considered really bad style in most Scheme
programs.  So making a limited unnecessary subset of its use cases give
better error messages is not going to be worth anybody's trouble.
Blaming Guile for something that is not really all that surprising might
not have been the smartest move for me to make.

>> Those should certainly be reported and fixed.  But I am also
>> currently working on changes that will indeed be more disruptive to
>> existing code relying on less than pretty or comprehensible behavior
>> of Lilypond.
>
> Oh. Though times ahead for me, then :-)

Not all that tough, I suppose.  Music functions will have less need to
"unpeal" their arguments, but that also means that there is a larger
variety of input they might need to deal with.  "Everything is an
EventChord" is what I am trying to get rid of right now.

> Thanks for the heads-up. I guess there's be a lot of work involved for
> Scheme-based frameworks compatibility (such as Nicolas' or Reinhold's,
> even though their code certainly is cleaner than mine).

No.  Intricate Scheme code will require attention, but not "lot of
work".  Nicolas or Reinhold will not be fazed with the few changes (more
often than not, _removing_ code at the right place).  The people that
will be worst affected are those that _inherited_ code from somebody
else and use it as a black magic box.  For them, it implies their
documents getting bit rot.  Some will ask for advice, get it, and be
able to continue.  Some will just silently assume that the documents are
to be discarded.  That's actually worst for reputation and maintenance
of the existing _passive_ corpus of Lilypond documents.

> I don't mind changes, as long as they're documented and have as few
> unfortunate side-effects as possible. For example, replacing # with $
> whereever needed wasn't that big of a deal; what proved "disruptive"
> in this case was the change in ly:parser-include-string behavior
> (which you fixed), and in eval-string (which you seem to have
> addressed, and I'm on my way to check it now). It turns out the
> eval-string thing was done on purpose to preserve location data, but I
> couldn't find the relevant documentation for this change (if there's
> any).

There isn't since it was not supposed to be a change requiring user
attention.  Error messages from embedded Scheme #{ ... #} will pinpoint
file and line number properly where available.  I just did not fully
cater for the case where they are _not_ available.

> Now, using old versions is also an option -- especially for now, since
> we're talking 2-months old or so.

There is not much of a point thinking about 2-month periods.  Either the
changes are due to bugs, in which case the bugs will still be there
after 2 months if you did not bother reporting them.  Or they will be
permanent, in which case you can wait all you want.

So while you might want to keep a remotely old version for meeting
deadlines, not looking at newer versions is not a good option, either.

-- 
David Kastrup