Re: Parallel building of Linux Kernel is broken

[Masahiro Yamada]

Hi.


2018-07-12 14:13 GMT+09:00 Paul Smith <address@hidden>:
> On Thu, 2018-07-12 at 13:55 +0900, Masahiro Yamada wrote:
>> Please add V=1 to the build command, like this:
>>
>> $ make -j8 V=1 defconfig all
>>
>> It will emit _almost_ raw log.
>
> At this time I don't have the resources to do this.  It will take me
> some time if you're going to wait for me.

No problem.   No rush here.

The Linux build system is pretty complex.


I attached information that might be helpful.
Please take it FWIW.


>> >   That's what we need to see; the actual command that
>> > was used to recursively invoke make.
>>
>> Linux Makefile hides "Entering directory ..." messages.
>>
>> If you want to see them,
>> please comment out the following line of the top Makefile.
>>
>> MAKEFLAGS += --no-print-directory
>
> I'm not that interested in seeing these but it wouldn't hurt.  Mainly I
> need to see the manner in which the recursive make is invoked and the
> recipes used to do so.


OK, then V=1 is good enough
to show what you want.






How Linux Makefile works?
--------------------------


Linux needs the configuration before building any objects.


"make defconfig" configures the build setting
suitable for general usage.

"make all" actually builds the kernel.



"make defconfig all" is a useful shorthand
to do the two in a row.


[1] User run "make -j8 defconfig all"

[2] The configuration must finish before the Makefile
    starts building any objects.

    The following code divide "make defconfig all"
    into two sub-makes  "make defconfig" and "make all"
    These two are run one by one.

__build_one_by_one:
        $(Q)set -e; \
        for i in $(MAKECMDGOALS); do \
                $(MAKE) -f $(srctree)/Makefile $$i; \
        done



[3] Sub-make "make defconfig" will find a recipe here

%config: scripts_basic outputmakefile FORCE
        $(Q)$(MAKE) $(build)=scripts/kconfig $@


[4] Sub-make "make all" starts, but top Makefile
    needs to load include/config/auto.conf.

ifeq ($(dot-config),1)
-include include/config/auto.conf
endif


[5] A recipe to generate include/config/auto.conf
    is found here.

include/config/%.conf: $(KCONFIG_CONFIG) include/config/auto.conf.cmd
        $(Q)$(MAKE) -f $(srctree)/Makefile syncconfig


[6] include/config/auto.conf has been generated,
    so "make all" restarts.


I think the latest GNU Make emits
'jobserver unavailable: using -j1.  Add '+' to parent make rule'
just before "make all" restarts.



[7] '-j8 --jobserver-auth' disappears from MAKEFLAGS,
    then rest of the build process is serialized.





-- 
Best Regards
Masahiro Yamada