Re: Testing a new compiler with Automake "simple tests"

[Stefano Lattarini]

On Wednesday 18 August 2010, Roberto Bagnara wrote:
> On 08/18/10 13:18, Stefano Lattarini wrote:
> > At Wednesday 18 August 2010, Roberto Bagnara wrote:
> >> # This does not work: when ../../bin/compiler changes, tests are
> >> # not recompiled.
> >> # TESTS_OBJS = $(TESTS:address@hidden@)
> > 
> > Typo here (and in Ralf example): should be $(TESTS:address@hidden@),
> > with a dot `.' before @OBJEXT@ (this is because @OBJEXT@ do not
> > contain a leading dot: it's either `o' or `obj').
> 
> Ah, right.
> 
>  >> # This does not work: Automake 1.11 rejects it with
>  >> #   Makefile.am:1148: bad characters in variable name `$(TESTS'
>  >> #$(TESTS:address@hidden@): ../../bin/compiler
> 
> The above is rejected even with the dot.
This seems like an automake limitation... no big deal though, as there 
is the simple workaround:

  TESTS_OBJS = $(TESTS:address@hidden@)
  $(TESTS_OBJS): ../../bin/compiler

which is also clearer to read IMO.

> > That said, you should consider Ralf's suggestion to use the
> > $(*_OBJECTS) variables instead, for more correctness.  In fact,
> > as Ralf pointed out in a previous message:
> >
> >   " ... this doesn't take into account that object file names are 
> >     an internal detail of Automake.  In practice, they might end
> >     in .obj, as Stefano already noted, which $(OBJEXT) or
> >     @OBJEXT@ can tell you, but also, object files may be renamed
> >     due to one of several reasons such as per-target flags,
> >     (obsolete) K&R support, and others. "
> > 
> > So, instead of doing simply e.g.:
> >    TESTS = t1 t2 t3 t4 t5 t6
> >    TESTS_OBJS = $(TESTS:address@hidden@)
> >    $(TESTS_OBJS): ../../bin/compiler
> > 
> > it's better to do:
> >    TESTS = t1 t2 t3 t4 t5 t6
> >    TESTS_OBJS = $(t1_OBJECTS) $(t2_OBJECTS) $(t3_OBJECTS) \
> >                 $(t4_OBJECTS) $(t5_OBJECTS) $(t6_OBJECTS)
> >    
> >    $(TESTS_OBJS): ../../bin/compiler
> > 
> > Unfortunately, this is more error-prone, since if you add, say,
> > `t7' to TESTS, but forget to add $(t7_OBJECTS) to TESTS_OBJS,
> > you'll have missing dependencies...  Well, you'll decide what's
> > better for you.
> 
> Hmm, it is error-prone indeed.  Moreover, we have more than one
> thousands tests and we could soon have two thousands of them.
> Said that, I will probably use this solution.
Well, you can always add a bit of indirection, and make the content of
$(TESTS_OBJS) auto-generated.  You can take a look at how Automake's
own `tests/Makefile.am' does something similar to generate the content
of the `$(parallel_tests)' variable.

> For the long term, given that the problem I am facing seems rather
> general (testing compilers or compiler-like tools), would you
> consider the possibility to extend Automate to simplify the job?
I think I definitely lack the insight and experience to get this 
right.  Let's wait what Ralf has to say about it... apart from 
"Patches are always welcome", obviously ;-)

Regards,
  Stefano