m4_map [was: Fix m4_join]

[Eric Blake]

On Oct 16 2007, Eric Blake <ebb9 <at> byu.net> writes:

> 
> Here's the patch for these macros, along with m4_apply, and semantics changes 
> to m4_map{,_sep} to make it reliably distinguish between an empty list of 
> arguments to apply, vs. a list containing a macro application with 0 
arguments, 
> vs. a list containing an argument list that contains an empty argument.
> 
> I'm not sure m4_map will be used much (partly because it requires so much 
> quoting!).  But I did find a use for it in my recent changes to 
> m4_version_unletter.
> 
> (m4_map): Change semantics to allow calling macro without
> arguments.
> (m4_map_sep): Likewise.  Also change semantics to quote separator,
> to match m4_join and m4_append.

I'm starting to have regrets about this semantic change to m4_map_sep.  At the 
time I made it, I did not know of any use of m4_map_sep in the wild (autoconf 
certainly did not use it, neither did libtool).  But I overlooked bison, which 
makes heavy use of m4_map_sep.

Here's my dilemma.  For m4_join, the separator is not expanded (and never was, 
even before I documented it for 2.62).  Let's see how this fares in the face of 
a macro that expands to an unboxed list of arguments (ie. the usage pattern 
common in bison):

m4_define([a], [[A]])m4_define([b], [B])dnl
m4_define([A], [C])m4_define([B], [D])dnl
m4_define([list], [[a], [b]])dnl

list
=> a, b
m4_count(list)
=> 2

By itself, list creates two arguments, which remain quoted (the first line 
expanded to "[a], [b]", before rescan stripped excess quotes).

m4_join([,], list)
=> a,b
m4_count(m4_join([,], list))
=> 1

Under m4_join, neither list elements nor the separator are expanded, so the 
effective literal string "[a,b]" results before the rescan.

m4_unquote(m4_join([,], list))
=> A,D
m4_count(m4_unquote(m4_join([,], list)))
=> 2

You've now generated a list of argument expansions (the first line resulted 
in "a,b" then "[A],B" before the final rescan).

m4_dquote(list)
=> [a],[b]
m4_count(m4_dquote(list))
=> 1

And finally, you've created a boxed list, ready to hand to m4_foreach or 
m4_map.  Four different uses of the list, all possible because we used the list 
elements as is.  And the lack of expansion in the separator is acceptable, 
since m4_join is intended to create a single string without altering its 
arguments.

On the other hand, m4_map_sep is designed to do some expansion on list 
elements.  Watch what happens in 2.62:

m4_map_sep([m4_echo], [,], m4_dquote(list))
=> C,D
m4_count(m4_map_sep([m4_echo], [,], m4_dquote(list)))
=> 1

Only one argument resulted, so this means that m4_echo(a) and m4_echo(b) were 
expanded, but the resulting string was "[A,D]".  But in bison (once m4_echo and 
m4_count are properly defined), this gives:

m4_map_sep([m4_echo], [,], m4_dquote(list))
=> A,D
m4_count(m4_map_sep([m4_echo], [,], m4_dquote(list)))
=> 2

That is, m4_map_sep is being used to generate a list of items (m4_echo(a) and 
m4_echo(b) resulting in "[A],B" before rescan).  The notion of converting one 
list into another is very powerful, but the 2.62 semantics of m4_map_sep 
blindly prevents this by refusing to expand the separator.

With the 2.62 implementation, the only way to get back to a list is to use 
m4_unquote on the result, but that over-expands the [A] unless every mapping is 
generated with excess quoting.  In this case, the way to generate the extra 
quoting is by using another macro, but that gets a bit heavy-handed (say I want 
to apply m4_eval in this manner - I would have to define a temporary macro 
m4_define([m4_qeval], [m4_dquote(m4_eval($@))]) and use [m4_qeval] as the 
argument to m4_map_sep rather than [m4_eval]):

m4_unquote(m4_map_sep([m4_dquote], [,], m4_dquote(list)))
=> A,D
m4_count(m4_unquote(m4_map_sep([m4_dquote], [,], m4_dquote(list))))
=> 2

I suppose there is another alternative, but it is also hairy to write (in other 
words, m4_map_sep with a sep of [,] is a nice shorthand for this):

m4_shift(m4_foreach([args], m4_dquote(list),
 [,m4_apply([m4_echo], m4_defn([args]))]))
=> A,D
m4_count(m4_shift(m4_foreach([args], m4_dquote(list),
 [,m4_apply([m4_echo], m4_defn([args]))])))
=> 2

At any rate, right now I'm debating about reverting the semantic change, and 
once again having m4_map_sep expand its argument between applications (you can 
always use m4_map_sep([m4_echo], [[,]], list) to result in the single string 
[A,D]).  I'm also thinking about a new macro (but what name to give it?) that 
takes an unboxed list, rather than a boxed one, for even less typing:

m4_...([m4_echo], [,], list))
=> A,D
m4_count(m4_...([m4_echo], [,], list)))
=> 2

Any thoughts on this?

-- 
Eric Blake