Re: Identify included files

[David Wright]

On Thu 21 May 2020 at 13:10:40 (-0400), Fr. Samuel Springuel wrote:
> > On 21 May, 2020, at 9:36 AM, David Kastrup <address@hidden> wrote:
> > antlists <address@hidden> writes:
> >> On 21/05/2020 01:49, David Wright wrote:
> >>> I don't understand your equivalence between .ily and .h files. The
> >>> .ily file(s) can contain just as much code as the .ly file(s), whereas
> >>> .h files don't contain any code at all (in the sense of producing
> >>> executable code for the next stage.
> >> 
> >> a) just like .h files, .ily files can't be passed directly to a
> >> compiler for compilation.
> >> 
> >> b) who says .h files don't produce executable code? They often contain
> >> code, sometimes entire functions.
> > 
> > The point of .h files is that they are intended to be included by more
> > than one compilation unit resulting in an executable.
> > 
> > As such, they are limited with regard to the code they can contain.
> > They can contain
> > 
> > a) inline function definitions: those can occur in multiple compilation
> > units as long as they are identical.
> > b) static inline: do not even need to be identical, are per-unit.
> > c) static data definitions, are per-unit.
> > d) template function definitions
> > 
> > I don't think that .ly/.ily is all that comparable here...
> 
> The analogy is not perfect, but then no analogy is (otherwise it wouldn’t be 
> an analogy).  Lilypond doesn’t enforce a distinction between .ly and .ily 
> files in the same way that C enforces the distinction between .h and .c 
> files.  Further, .pdf files are stand alone in a way that .o files are not.  
> The linker level which combines .o files into an executable would have as its 
> closest approximation in the LilyPond world assembling several pdfs into some 
> kind of portfolio.  And this is another point where the analogy breaks in 
> that an executable cannot contain incompatible .o files but a portfolio can 
> contain pdfs of scores with vastly different styles.  The result may not be 
> as professional looking, but it’s possible.
> 
> Here are the parts of the analogy that do work:
> 
> 1) .ily files cannot produce scores.  Sure, I can call LilyPond on them 
> directly, but regardless of the options I use, no score will ever result.  
> This may be purely by convention (as in, LilyPond itself doesn’t enforce 
> this, it really couldn’t care what the extension on its input files is), but 
> it’s a convention which I rigorously apply in the files I write.  In this 
> respect, .ily files are like .h files which also cannot, by themselves, ever 
> produce a .o file. For .h files, at least a dummy.c file is needed to include 
> the .h file.  My .ily files likewise need a .ly file to include them to 
> produce a score.

As you say, these file extensions are just conventions.

$ cat scale.ily 
\relative { c' d e f g \include "included.pdf" a b c }
$ cat included.pdf 
{
  f e f g
}
$ ~/lilypond-2.21.0-1.linux-64/bin/lilypond scale.ily
GNU LilyPond 2.21.0
Processing `scale.ily'
Parsing...
scale.ily:1: warning: no \version statement found, please add

\version "2.21.0"

for future compatibility
Interpreting music...
Preprocessing graphical objects...
Finding the ideal number of pages...
Fitting music on 1 page...
Drawing systems...
Layout output to `/tmp/lilypond-50Ds6o'...
Converting to `scale.pdf'...
Deleting `/tmp/lilypond-50Ds6o'...
Success: compilation successfully completed
$ 

Only the final .pdf (in scale.pdf) is unavoidable.

> 2) .ly files do produce scores.  If I call LilyPond on an .ly file, then I 
> expect a score to come out (in pdf format by default, but other formats are 
> possible too if I use the right options or flags).  Again, this is primarily 
> a convention, as LilyPond won’t throw an error on a .ly file which doesn’t 
> produce a score (but has no other problems).  However, it is a convention 
> that I strictly apply to my files.  In this respect .ly files are like .c 
> files, which can be compiled to create .o files.
> 
> 3) Further the relation between the .ly and .ily files is similar to the 
> relation between .c and .h files.  .ly (.c) files do not, themselves, depend 
> on the .ily (.h) files that they include.  There is no build process which 
> creates a .ly (.c) file from the included .ily (.h) files.  However, if I 
> make a change to a .ily (.h) file, then the .pdf (.o) file which is produced 
> when LilyPond (GCC) compiles a .ly (.c) file which includes that changed file 
> is out of date and needs to be recompiled. In this regard the directed 
> acyclic graph (DAG) which illustrates these relationships is the same, just 
> with one less level (unless I make use of a pdf merge tool to replace the 
> linker level).
> 
> 
> My goal is to inform make of these relationships so that when I tweak a 
> LilyPond source, only the scores affected by that source are recompiled.  
> This may be several pdfs, but it’s not all of them, even when I tweak an ily 
> file because not all ily files are included in all scores.  Furthermore, the 
> more I work on this project, the more I tend to split up the “formatting” 
> file so that I actually have several, each of which is responsible for a 
> single aspect of the formatting.  This makes it easier for me to find code 
> when I need to make adjustments to a particular aspect of the formatting.

If you follow some simple rules in your source layout, constructing
such a list might be most straightforward to do in the shell, using
grep. For example,

$ grep -r -l -e '^[[:space:]]*\\include[[:space:]]*"tweaked-file.¹ly"' 
top-of-tree/ … … ²

will find all the source files under all the tops-of-trees/ that pull
in the tweaked file. The files that are found will either be .ly files
that need recompiling, or .ily files whose names now need to be
searched for themselves:

$ grep -r -l -e '^[[:space:]]*\\include[[:space:]]*"found-file1.ily"' \
             -e '^[[:space:]]*\\include[[:space:]]*"found-file2.ily"' 
top-of-tree/ … …

(for all found-file*.ily files)

Repeat (while accumulating the .ly filenames) until no more .ily
files are being found. Now recompile all the .ly files that have
been accumulated.

If you've tweaked several files, the first grep will also have
multiple -e strings, one for each tweaked filename.

¹ insert "i" as appropriate.

² some people may need to add
  | grep -v '~$'
  to filter out their editor's backup files. I also added another [[:space:]]*
  to allow for variations in whitespace after the \include.

Cheers,
David.