Fwd: multi-threaded compiling

[Mischa Baars]

Hi Martin,

On Fri, Mar 15, 2024 at 10:54 AM Martin D Kealey <martin@kurahaupo.gen.nz>
wrote:

> Hi Mischa
>
> It's already been mentioned several times, but I'll re-iterate now: I
> strongly suggest using the “-j” option for “make”.
>

That might be something for the future, but  for now my own approach
suffices. That's mainly because I've never seen a Makefile quite this
efficient. All other Makefiles I've seen over the years, were always more
complex.


> Writing one's own parallelization is error-prone, whereas “make” can
> parallelize operations while ensuring “before” and “after” constraints are
> met, provided that all dependencies are properly described in the Makefile.
>
> There is rarely any advantage to writing a sequential script for building
> a software module; “make” is almost always the better tool for the job.
>
> A partial exception is recording a “build ID”: in that case you need to
> write the new ID into a file, which then itself becomes input to the build
> process. Don't rush: making this work reliably requires clear-headed
> thinking and a stable design.
>
> I'm not really clear what your use-case is for having several sets of
> build flags; if you could explain that, then perhaps we could come up with
> a "better" option.
>

It's not a set of build flags. It's combinations, to get the result right.

Yes, we can try to make a start to using the "-j" option. Please show me
how to get #4 to work:

#1
#all: one two

#2
#BIN=one two
#all: $(BIN)

#3
#all:
# SRC=(*.c); BIN=$${SRC[*]%.c}; echo $${BIN};

#4
SRC=(*.c)
BIN=$${SRC[*]%.c}
all: $(BIN)



> -Martin
>
> On Fri, 15 Mar 2024 at 18:17, Mischa Baars <mjbaars1977.backup@gmail.com>
> wrote:
>
>> No? No further suggestions?
>>
>> Then I'd like to thank you all for your help and your input.
>>
>> I'll be off-list now.
>>
>> Kind regards,
>> Mischa Baars.
>>
>> On Thu, Mar 14, 2024 at 9:17 AM Mischa Baars <
>> mjbaars1977.backup@gmail.com> wrote:
>>
>>> Ok. Then this is what I was able to make of it. To be honest, I prefer
>>> the Makefile syntax over the script syntax, because of its direct
>>> substitutions.
>>>
>>> Other improvements that anyone likes to suggest? Is this the best bash
>>> can do?
>>>
>>>
>>> On Thu, Mar 14, 2024 at 3:47 AM Martin D Kealey <martin@kurahaupo.gen.nz>
>>> wrote:
>>>
>>>> On Wed, 13 Mar 2024, Mischa Baars wrote:
>>>>
>>>> > Date: Wed, 13 Mar 2024 22:52:16
>>>> > From: Mischa Baars <mjbaars1977.backup@gmail.com>
>>>> > To: alex xmb sw ratchev <fxmbsw7@gmail.com>
>>>> > Cc: psmith@gnu.org, bug-bash <bug-bash@gnu.org>, help-make@gnu.org
>>>> > Subject: Re: multi-threaded compiling
>>>> >
>>>> > I found another mistake of mine:
>>>> >
>>>> > #ifndef __STRINGIZED__
>>>> >
>>>> > should be
>>>> >
>>>> > #if __STRINGIZED__ == 0
>>>> >
>>>> > or it will always go for the second statement in the conditional.
>>>> >
>>>> > Can someone please correct my script??? Things are starting to itch.
>>>>
>>>> The fundamental limitation is that Bash only provides lists of scalars
>>>> masquerading as arrays, and lists of scalar-scalar pairs masquerading as
>>>> maps (aka hashes, dicts, or "associative arrays").
>>>>
>>>> It does not support any more complex data structures, and in particular,
>>>> does not have arrays-of-arrays or lists-of-lists.
>>>>
>>>> So I would change
>>>>
>>>>   CFLAGS[0]="-D__STRINGIZED__=0 -D__STRING__=\"${STR[0]}\""
>>>>   CFLAGS[1]="-D__STRINGIZED__=0 -D__STRING__=\"${STR[1]}\""
>>>>   CFLAGS[2]="-D__STRINGIZED__=1 -D__STRING__=\"${STR[0]}\""
>>>>   CFLAGS[3]="-D__STRINGIZED__=1 -D__STRING__=\"${STR[1]}\""
>>>>
>>>> to
>>>>
>>>>   CFLAGS0=( -D__STRINGIZED__=0 -D__STRING__="${STR[0]}" )
>>>>   CFLAGS1=( -D__STRINGIZED__=0 -D__STRING__="${STR[1]}" )
>>>>   CFLAGS2=( -D__STRINGIZED__=1 -D__STRING__="${STR[0]}" )
>>>>   CFLAGS3=( -D__STRINGIZED__=1 -D__STRING__="${STR[1]}" )
>>>>
>>>> and then change
>>>>
>>>>   gcc -o main main.c ${CFLAGS[0]} ; ./main
>>>>   gcc -o main main.c ${CFLAGS[1]} ; ./main
>>>>   gcc -o main main.c ${CFLAGS[2]} ; ./main
>>>>   gcc -o main main.c ${CFLAGS[3]} ; ./main
>>>>
>>>> to
>>>>
>>>>   gcc -o main main.c "${CFLAGS0[@]}" && ./main
>>>>   gcc -o main main.c "${CFLAGS1[@]}" && ./main
>>>>   gcc -o main main.c "${CFLAGS2[@]}" && ./main
>>>>   gcc -o main main.c "${CFLAGS3[@]}" && ./main
>>>>
>>>> If you absolutely must have some kind one indexed array for all the
>>>> permutations of cflags, then you could use a synthetic multidimensional
>>>> array, like this:
>>>>
>>>>   CFLAGS=( -D__STRINGIZED__=0 -D__STRING__="${STR[0]}"
>>>>            -D__STRINGIZED__=0 -D__STRING__="${STR[1]}"
>>>>            -D__STRINGIZED__=1 -D__STRING__="${STR[0]}"
>>>>            -D__STRINGIZED__=1 -D__STRING__="${STR[1]}" )
>>>>
>>>>   gcc -o main main.c "${CFLAGS[@]:0*2:2}" && ./main
>>>>   gcc -o main main.c "${CFLAGS[@]:1*2:2}" && ./main
>>>>   gcc -o main main.c "${CFLAGS[@]:2*2:2}" && ./main
>>>>   gcc -o main main.c "${CFLAGS[@]:3*2:2}" && ./main
>>>>
>>>> However, given the pattern, I'd have gone with:
>>>>
>>>>   for i in 0 1 2 3 ; do
>>>>     gcc -o main main.c -D__STRINGIZED__=$((i/2))
>>>> -D__STRING__="${STR[i%2]}" &&
>>>>     ./main
>>>>   done
>>>>
>>>> -Martin
>>>>
>>>

two.c
Description: Text Data

one.c
Description: Text Data

Makefile
Description: Binary data