Re: Partial success building the trunk on MSYS2/Windows platform

[Gaius Mulley]

Benjamin Kowarsch <trijezdci@gmail.com> writes:

> On Fri, 28 Oct 2022 at 00:20, Gaius Mulley <gaiusmod2@gmail.com> wrote:
>
>  Runar Tenfjord <runar.tenfjord@gmail.com> writes:
>
>  > Some other issues:
>
>  > * BITSET64 seems to be missing. Is this temporary or some
>  >  fundamental reason for the omission?
>
>  BITSET was fixed to WORD/INTEGER/CARDINAL.  But you're right as gm2
>  supports CARDINAL64 (if the architecture supports it) and
>  INTEGER64/WORD64 - then it looks like an oversight that BITSET64 is
>  missing!  Thanks I will fix this and test.  I suspect it might need
>  some runtime support - (if I recall from past experimentation).
>
>  > * I find that the WORD type is 32bit on my 64bit installation.
>
>  Yes as SIZE(WORD) = SIZE(BITSET) = SIZE(INTEGER) and on a LP64 system
>  (or amd64) then sizeof(int) = 4.
>
> Keep in mind that only type BITSET is portable and then only for a
> maximum of 16 bits because Modula-2 implementations typically map type
> BITSET to the underlying machine architecture's standard register
> size. Any code written using BITSET with more than 16 bits will be
> non-portable. Any code written with extended BITSETnn types will be
> non-portable. They are best avoided.
>
>  >  Is this connected to the above issue? Or due to portability?
>
>  It is also connected to interfacing to C (SIZE(INTEGER) == SIZE(WORD) ==
>  sizeof(int)) necessary for the underlying libraries.  Hence we can use
>  CARDINAL64 or INTEGER64 without breaking anything (but we cannot
>  guarantee CARDINAL64 will exist).
>
> Here too, only CARDINAL, INTEGER and LONGINT are portable types. Any
> use of extended versions of these types will render the code
> non-portable. They, too are best avoided entirely.

Hi,

yes indeed any types (using the name FOOint) imported from SYSTEM are
potentially non-portable in gm2.  In gm2 the front end probes the back
end to see if these datatypes are supported and then creates them.

> In our M2BSK bootstrap kit, we address these issues as follows:
>
> (1) Avoid all use of BITSET and BITSETnn
>
> Instead, we developed a portable 96-bit bitset library that we use for sets 
> of tokens, thus called TokenSet.
>
> https://github.com/m2sf/m2bsk/blob/master/src/TokenSet.16bit.def
> https://github.com/m2sf/m2bsk/blob/master/src/TokenSet.32bit.def
> https://github.com/m2sf/m2bsk/blob/master/src/TokenSet.64bit.def

interesting!  In gm2 it chooses BITSET = unsigned int for smallsets <
sizeof (int) * 4 elements and otherwise contructs a large set which is
currently a record of multiple word sized sets.  I'll be changing this
implementation to use a pointer to an array of bits in the near future
(very similar to how ARRAYs OF type) are implemented.

The accessor functions (for large sets) are contained in SYSTEM.mod
and these are called from gcc/m2/gm2-gcc/m2expr.cc.

gm2 also needed a 96bit token set :-) and here it uses three 32bit word
sets to allow ease of bootstrapping with C/C++.  But in due course this
might also be changed (new longset implementation).

I'm hoping the new longset implementation will resolve a number of
longstanding portability issues on sparc and also enable minor/obvious
changes to gdb to support the data type.

> These work on any PIM3, PIM4 and ISO compiler. When the compilation
> target is 16-bit register based, the TokenSet type is implemented by
> six segments of type CARDINAL which is then 16-bit wide. When the
> target is 32-bit, the type is implemented by three segments of type
> CARDINAL, then 32-bit wide. And when the target is 64-bit, the type is
> implemented by two segments of type CARDINAL which is then 64-bit
> wide. The size of CARDINAL is determined by running a small program
> that counts the bits before building the code.
>
> (2) Avoid all use of CARDINALnn and INTEGERnn
>
> Instead, we developed our own portable LONGCARD library  
>
> https://github.com/m2sf/m2bsk/blob/master/src/lib/LONGCARD.def
>
> and a portable math and bit operations library that mirror operations in the 
> above library for type CARDINAL
>
> https://github.com/m2sf/m2bsk/blob/master/src/lib/CardBitOps.def
> https://github.com/m2sf/m2bsk/blob/master/src/lib/CardMath.def
>
> and likewise for type LONGINT
>
> https://github.com/m2sf/m2bsk/blob/master/src/lib/LongIntBitOps.def
> https://github.com/m2sf/m2bsk/blob/master/src/lib/LongIntMath.def
>
> The downside of this is that the operations are functions, not inline
> operators. But the benefit is that the types can be replaced with each
> other depending on the memory model of the build target and everything
> is portable across compilers and dialects PIM3, PIM4 and ISO.
>
> All the code is LGPL, so it can be used in both open source and closed
> source projects (as long as modifications to the incorporated
> libraries themselves are passed on in source form). So, if anybody
> wants to use them, you're welcome.
>
> The TokenSet library could easily be turned into a more generic
> 128-bit BitSet library by (1) renaming it and its type, (2) changing
> the value type to CARDINAL and (3) increasing the number of segments
> from 6 to 8 in the 16-bit version and respectively from 3 to 4 in the
> 32-bit version.
>
>  > * Can platform differences (Linux_x86_64, Windows64, etc) be switched
>
>  >  with the preprocessor or must this be handled by outer build system
>  >  like make?
>
>  The preprocessor can be used in user programs (some of the testsuite
>  does this for example).  The codebase of gm2 doesn't do this though as
>  the compiler is bootstrapped by converting it to C (or C++).  Is there a
>  particular problem you are trying to solve re: preprocessor?  I tend to
>  use the preprocessor in the underlying C code connecting to the system
>  calls though.  These choices were mainly down to keeping everything as
>  simple as possible (re: host/build/target differences).
>
> Using a C-like preprocessor has the disadvantage of cluttering the
> code and it diminishes the readability and clarity of the code.
>
> It is much better to treat pre-processing as a separate and
> independent editing session, just with the difference that it is an
> automated editing session rather than a manual editing session.
>
> This has the advantage that the output of such a preprocessing step
> will be a bespoke version of the code specific to the build target.
>
> For our M2BSK project we use this approach by furnishing build target
> specific versions for libraries that need to be build specific.
>
> <LibraryName>.<nn>bit.def
> <LibraryName>.<nn>bit.mod
>
> or platform specific
>
> <LibraryName>.posix.def
> <LibraryName>.posix.mod
>
> or dialect specific
>
> <LibraryName>.pim.def
> <LibraryName>.pim.mod
>
> or IO library specific
>
> <LibraryName>.posix.def
> <LibraryName>.posix.mod
>
> or even compiler specific (if it can't be helped)
>
> <LibraryName>.mocka.def
> <LibraryName>.mocka.mod
>
> I have written a templating pre-processor for this purpose:
>
> https://github.com/m2sf/m2pp
>
> CLI arguments are described here:
>
> https://github.com/m2sf/m2pp/wiki/CLI-Arguments
>
> The initial set of preprocessor directives is described here:
>
> https://github.com/m2sf/m2pp/wiki/Feature-Set-Of-Initial-Version
>
> As an added bonus, the placeholder expansion can also be used for generic 
> programming, even under PIM.
>
> Directives not yet implemented are described here:
>
> https://github.com/m2sf/m2pp/wiki/Preprocessor-Directives
>
> By generating target specific code using the preprocessor, the build system 
> can be kept as simple as a shell script to which the input parameters are the 
> desired memory model, the target compiler, the dialect and the
> choice of IO library
>
> https://github.com/m2sf/m2pp/blob/master/cfg/config.sh
> None of the holy mess of autosucks.

:-)

regards,
Gaius