Re: Target native layer

[Mark Wielaard]

Hi,

On Tue, 2004-08-10 at 13:30, Dr. Torsten Rupp wrote:
> I read many emails in the list about the TARGET_*-layer. I'm not very 
> happy about the discussion, because we discussed this already around 1 
> year ago and at that time it seemed everybody was happy to get the
> abstraction layer TARGET_*.

That is true, but since then we haven't (as GNU Classpath group) used
the abstraction layer at all to make our code more portable.

It was introduced by you in March 2003 and refined in July 2003. But
since then there hasn't been any real use/development of it. Michael
Koch made a couple of fixes late last year and during this year so that
it works on BSD systems and I made some some fixes this year to make
available() work on non-file streams (like system.in) and to remove
non-ansi C comments when the GNU Classpath code was converted to really
use ISO C 90 conventions only.

Both Michael and I take the libgcj code (that already works and is
ported to a few different systems) as inspiration for our changes. But
that code is written to use autoconf style checks. This is kind of hard
for us since the multi-line macro style used in the generic layer is
hard to read/write.

Since nobody ever commented on this and since except for all the empty
files in the target/Linux directory nobody ever made a real
implementation of the abstraction layer I always thought that we never
actually used it. That is why we have the discussion now. We have an
abstraction layer without a real implementation layer.

> 3. complexity of macros, debugging
> 
> It is true that #defines are difficult to debug. They are also difficult
> to write, but of course it depends always on the specific macro.

Yes, this is my main complaint personally. If we can get at least rid of
the multi-line defines then I think it would be much easier to debug and
write native code.

> #define 
> TARGET_NATIVE_FILE_OPEN(filename,filedescriptor,flags,permissions,result) \
>    #include <sys/types.h>
>    #include <sys/stat.h>
>    #include <fcntl.h>
>    do { \
>      filedescriptor=open(filename, \
>                          flags, \
>                          permissions \
>                          ); \
>      result=(filedescriptor>=0)?TARGET_NATIVE_OK:TARGET_NATIVE_ERROR; \
>    } while (0)

It is interesting that you quote this macro since this is actually one
which Michael Koch fixed so it now reads:

#ifndef TARGET_NATIVE_FILE_OPEN
  #include <sys/types.h>
  #include <sys/stat.h>
  #include <fcntl.h>
  #define 
TARGET_NATIVE_FILE_OPEN(filename,filedescriptor,flags,permissions,result) \
    do { \
      filedescriptor=open(filename, \
                          flags, \
                          permissions \
                          ); \
      if (filedescriptor >= 0) \
        fcntl (filedescriptor,F_SETFD,FD_CLOEXEC); \
      result=(filedescriptor>=0)?TARGET_NATIVE_OK:TARGET_NATIVE_ERROR; \
   } while (0)
#endif

Do you actually use the GNU Classpath version of the target/generic
implementation or do you use another implementation?

> Imho the complexity of the macros is usually not very high (if it 
> becomes complex also a function can be implemented; then a macro is only 
> an "alias"). They are multi-lined to make them more readable. The 
> "include"-statements are needed in the generic implementation and the
> "do...while" is a construct for safe usage of the macro.
>
> Debugging of macros is difficult - if they are complex. If a macro is 
> only a wrapper then only a OS-specific function is called with some 
> additional calculations, e. g. evaluation of the return value. It is a 
> good idea to keep the macros as simple as possible. And it is possible, 
> because the TARGET_*-layer does not add additional functionally, it only 
> "maps" some functionality.

One thing that worries me about a lot of these macro definitions is that
it seems they can trigger unexpected multiple evaluation of the
arguments. A couple of the macros their arguments multiple times this
might trip people who don't expect a function to be called multiple
times or who use such a macro with arguments such as (i++).
I guess you must be very careful when you use them.

> - for embeeded system it is not always feasible to check if a function
> exists by compiling and linking a small example-program, because 
> sometimes linkage is done only partially on the host. Final linkage is 
> done on the target when loading the program or when creating the system 
> image with the included application. Thus AC_CHECK_FUNC is not feasible. 
> The same problem occurs for other checks, e. g. constants or datatypes.

Kaffe seems to be able to handle that. You can give defaults for cases
where you need to cross-compile for example.

> - some systems have very strange and even wrong header-files. E. g. for
> Windows/MinGW the headers sys/stat.h, io.h, windows.h, winbase.h are
> needed for checking chsize() (truncate) or mkdir(). For embOS even some
> header-files can not be included, because they are wrong (but cannot be
> changed/fixed by aicas). These things make autoconf very complicate to
> use, because if there are a lot of possible functions which can be used 
> to implement some feature, it is not clear which function is detected by
> autoconf for some specific system.

That is handled in libgcj by having a hybrid system. autoconf for normal
Posix like systems and an os-layer for systems that are completely
different.

> There even could be very bad
> side-effects if more than one function is available (e. g. f1() and 
> f2()) and at some time f2() is used instead f1() (with different 
> behaviors or limitations), if there is some change for another target 
> system (e. g. you add some changes for RTEMS, but this will also have 
> effects on e. g. embOS. You will not detect this problem until testing 
> again all targets for any change in autoconf). It is a little bit 
> "undeterministic" which features are detected and if they are usable.

But on the other hand that is autoconf its strength. You put in tests
for features of different systems. If a platform has those features you
get your port for free.

> - some features are not detectable by autoconf, e. g. the ordering of
> parameters for functions like inb() and outb() (we had that problem) or
> additional parameters (which usually only produce a warning which is
> discarded), e. g. gethostbyname_r() under Solaris.

Have you looked at The GNU Autoconf Macro Archive?
http://www.gnu.org/software/ac-archive/
It has a macro specifically for this gethostbyname_r() parameter
detection for example. And there is also the ac-archive on sf.
http://ac-archive.sourceforge.net/

> There are much more difficult things which occur with autoconf. To 
> replace some target layer (e. g. TARGET_*) by autoconf only will imho 
> make implementations for non-Unix-like systems very difficult and will 
> only shift the so called "complex" C-macro-implementation into "complex"
> autoconf-macros-implementations (imho M4 is not much better then a
> C-preprocessor and difficult to debug).

But using autoconf has the advantage that "everybody" else is using it.
At least the two most important projects to GNU Classpath gcj and kaffe
are build around it. So solving something with autoconf helps everybody.
Having our own target porting layer will make it more difficult to work
together with others that do use autoconf like systems. And autoconf is
very well documented out own layer isn't really well documented and
actually has no examples how to use it properly.

> 4. Multiple code - some statistics:
> 
> In the current implementation we use at aicas we have the following
> systems. The numbers below count the number of macros at all (functions
> and constants) which are different from the standard (generic)
> implementation:
> 
> generic macros: 220
> 
> Linux: 0
> Solaris: 11
> RTEMS: 3
> MinGW: 46
> embOS: 10 (only partially implemented)

I think this sums up pretty well why most people don't really see the
point of the target layer. We only have the "empty implementation" for
GNU/Linux systems. 

It would be interesting to compare the MinGW implementation with the way
libgcj does it (mostly also through a os-layer).

Also you say "the current implementation we use at aicas". To really
consider the strength of the target-layer we must make it our
implementation (for GNU Classpath as group). Otherwise it is only this
abstraction that we don't really use.

> Efficient code: wrapper-functions are nice, but in some cases an 
> overkill, e. g. when calling simple function like sin(). In general C 
> compilers do not optimize this (imho that is some reason why "inline" 
> was introduced). If "inline" can be used, macros are almost not needed 
> anymore.

autoconf comes with AC_C_INLINE which handles this case.

> dead-code: the standard GNU linker does not remove functions which are 
> not used (dead-code). Thus if at least one function is needed from a 
> object file, all other function from that object file are linked to the
> application, too. There is only one automatic way to remove dead-code 
> (-ffunction-sections), but this have other disadvantages. Even the 
> man-page do not recommend it. Thus to remove dead-code of a function 
> some #ifdef-endif around a function is needed. On the other side: A 
> non-used macro will not produce any dead code.

If this matters wouldn't you just put these functions that might not be
used into their own source file so you have multiple object files?

> These are my thoughts to this topic. I hope all developers who are 
> interrested in some target native layer will reconsider the current 
> discussion. And I hope we will find a solution which can satisfy everybody.

I am very interested. And although I clearly favor some autoconf like
solution (since I know how to copycat that from libgcj, kaffe or other
GNU projects) I also don't provide any code at the moment.

I think that has been the real problem in this discussion. Neither the
supporters of the TARGET_NATIVE layer nor the supporters of a autoconf
approach have actually shown the code to really do the ports. Having
real code to argue about would make a lot of the arguments a lot easier
to settle.

Cheers,

Mark

signature.asc
Description: This is a digitally signed message part