Re: Identify included files

[Fr. Samuel Springuel]

> On 18 May, 2020, at 11:30 AM, David Wright <address@hidden> wrote:
> 
> If you were compiling a C program foo.c into an executable, what you
> would be trying to avoid is recompiling fnbar.c into fnbar.o over
> and over again (and all of its similar companions).
> 
> Can you explain which of your files are the equivalents of fnbar.o.
> (In python, for another example, they'd be your *.pyc files, and
> python looks after all this housekeeping itself.)

As I see it, the filetype equivalencies roughly map as this:

.ily -> .h
.ly -> .c
.pdf -> .o

To this I would also add one more file type for storing dependency information
.dly -> .d


> On 18 May, 2020, at 11:35 AM, David Wright <address@hidden> wrote:
> 
> If you follow some simple rules in your source layout, constructing
> such a list might be most straightforward to do in the shell, using
> grep. For example,
> 
> $ grep -e '^[[:space:]]*\\include ' foo-top-level.ly
> 
> will find all the includes in the source, restricting it to the start
> of lines, so that you can avoid false hits from, say, internal
> documentation or test includes.
> 
> You would insert the resulting filenames into two lists, files visited
> and all files. Iterate on the files list, skipping those visited,
> until the files to visit reaches zero (lists are of equal length).
> 
> One complication is applying the search of libraries in the same
> manner as LP, particularly if you rely on the order of searching.
> Another is if you're using external libraries where you have no
> control over their include format.
> 
> If you're not comfortable with shell programming, obviously it would
> be preferable to use a language like Python or Perl for coding.
> 
> The resulting list could be used for, say, making a tarball of source
> files for distribution, or for driving make to produce PDFs that are
> assembled by, say, TeX programs into final documents.


My original thoughts were to write something in Python or bash and it probably 
would have looked something like what you propose.  Being a firm believer in 
not reinventing the wheel whenever possible, I decided to ask the list to see 
if this had been done already first.


> 
> On 18 May, 2020, at 12:22 PM, antlists <address@hidden> wrote:
> 
> Okay, does that mean you want a list of all the lilypond include files, or 
> just your own?
> 
> If you're digging into the internals of lilypond, then it'll be a big job, 
> and makes it dangerous to upgrade lilypond. Because I assume that means 
> you're actually modifying lilypond's internal structure. What I do, anything 
> of lilypond's that I modify, I split it out into my directory structure, and 
> then over-ride lilypond's version from within my source. That way, what 
> happens to lilypond during an upgrade is no concern of mine (other than 
> changes to syntax which mess up my source :-)
> 
> If it's just your own files you're concerned about, then grep will probably 
> do the trick. I don't know how to recurse directories, but making it search 
> all the files in a directory for "include", and tell you which file it found 
> that line in, is pretty basic.
> 
> You can then read up on make for how to create those virtual objects, and use 
> an editor to put them in your makefile. Not totally automated, I admit, but 
> fairly simple grunt-work. The sort of thing I could have written a BASIC 
> program to do in 10 minutes, except I don't have access to a BASIC I 
> understand how to use any more - ironic - as computers have got easier to 
> use, so the programming languages have got harder...
> 

I’m not playing with the lilypond internals (and don’t need them in my list of 
prerequisites), but I do occasionally have files which make use of oll packages 
(I’ve been playing with the Edition Engraver recently).  Those packages have 
their own hierarchy outside my project hierarchy.  Further, the location of 
their hierarchy may be different on different machines (I always try to set 
things up with collaboration in mind, even when I’m not currently collaborating 
with anyone).  As a result, scripts which don’t understand how LilyPond finds 
files and simply look for “\include” statements are likely to come up short and 
not find all prerequisites.



> On 18 May, 2020, at 11:47 AM, Urs Liska <address@hidden> wrote:
> 
> I'm pretty sure that the only two ways to reasonably go forward is to
> either invoke LilyPond to produce this list or to use a library that
> already knows how LilyPond works. As I mentioned somewhere at the top
> of this thread, Frescobaldi knows pretty well how to recurse into the
> include files.
> 
> I would write a tool in Python that calls python-ly (well, probably
> quickly, it's designated successor (
> https://github.com/frescobaldi/quickly)) to produce the list of
> dependencies and feed that into Make.

I’ve reached a similar conclusion.  Right now I’m going forward with the 
lilypond route simply because David K.’s parse-only.ly means I’m closer to that 
solution than I am with python-ly or quickly (as I’ve never really worked with 
these packages before and would be starting from scratch).

In fact, I’ve attached my first go at it.  I’ve modified parse-only.ly to solve 
the context mod problem and to change the output format.  First off, 
parse-only.ly itself is removed from the list (as it’s not actually a 
prerequisite for building the pdf).  Second, the output is formatted in a 
make-friendly way with the files separated by a space (instead of a new line) 
and the name of the expected pdf prepended (and separated from the file list by 
a colon)

The makefile then contains a rule for creating .dly files from .ly files; an 
include statement for reading in all the dly files; and a rule for creating pdf 
files from .ly files.  My preliminary testing has everything working, but I’m 
fairly new to writing makefiles myself, so I’d appreciate anyone with more 
experience giving this a once over and critique.


✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝✝
Fr. Samuel, OSB
(R. Padraic Springuel)
St. Anselm’s Abbey
4501 South Dakota Ave, NE
Washington, DC, 20017
202-269-2300
(c) 202-853-7036

PAX ☧ ΧΡΙΣΤΟΣ

makefile
Description: Binary data

parse-only.ly
Description: Binary data