[HOWTO] RE: dejagnu remote testing without rsh/rcp facilities on target.

[Dave Korn]

On 25 April 2006 07:54, J.J.Garcia wrote:

[ dejagnu list cc'd in for the sake of the archives, this is a useful 
technique.  note also that sourceware.org and sources.redhat.com are the same 
so there's no need to Cc both; please use the canonical 'sourceware.org' 
addresses. ]

> After googling a while and reading the dejagnu doc i found that it should be
> recommended to have the rcp/rsh/telnet facilities in target board, and i
> don't have these facilities running actually on this target.
> 
> What i have is a text/serial console/newlib with propietary application
> commands using jtag mode for that console. At the same time i have a telnet
> facility offered by target but not in the proper way, i mean it's the same
> input/output 
> as the console/newlib described b4 but replicated on telnet port. That
> means i (think i) have no way of compiling/uploading the objects to target,
> run them and check the results as supposed following the dejagnu procedures
> to do it using remoted targets.
> 
> My first approach is to embed the tests, i.e. memcheck.exp/*.c (gcc C only)
> in the src running app on target, compile the whole app with the toolchain,
> flash 
> it and finally using a custom command at target console to fire/start them
> remotelly to check after the results, dont know yet howto integrate with
> dejagnu all of this.

  Ok, so you have a command line listening over a serial port?  You could 
perhaps write a quick'n'dirty downloader using some ascii format (e.g. srec) 
couldn't you?  Anyway, even if you can't ...


  To integrate with dg, you need to add a baseboard file for your target, and 
set parameters in that that tell it how to interact with your remote.  I found 
it easiest to adapt the simulator support by inventing a fake sim protocol and 
using a little command line wrapper program to drive my target.  A rough 
outline of the steps I took: (replace $myboard with whatever name you prefer; 
this is the name you will pass to "make check" using 
RUNTESTFLAGS="--target_board=$myboard")

- in $prefix/share/dejagnu/baseboards, create a new file called $myboard.exp

------------------------------example------------------------------
set_board_info sim "$mywrapperprogram"
set_board_info isremote 0

# This is a list of toolchains that are supported on this board.
set_board_info target_install {$my-target-triplet}

# Load the generic configuration for this board, This will define a basic
# set of routines needed by the tool to communicate with the board.
load_generic_config "$myboardsim"

# basic-sim.exp is a basic description for the standard Cygnus simulator.
load_base_board_description "basic-sim"

# The name of the simulator is "$myboard_sim".
setup_sim $myboard_sim

# No multilib flags needed by default.
process_multilib_options ""

# The compiler used to build for this board. This has *nothing* to do
# with what compiler is tested if we're testing gcc.
set_board_info compiler  "[find_gcc]"

# Compiler and linker flags are set here.
set_board_info cflags "-I$whatever -Dmore_stuff --fbla-bla-bla"
# Our CRT/startup code enters at this label
set_board_info ldflags  "-e start"
# The default linker script is now correct for the simulator.
set_board_info ldscript  ""

#
# Here are some generic overall board characteristics
#

# We only have a small stack available to us
set_board_info gcc,stack_size 2048

# Nested functions and trampolines are known to not work
# but we don't use them at the moment anyway
set_board_info gcc,no_trampolines 1

# We don't have a math lib.
set_board_info mathlib ""

# The simulator now takes care of reporting testcase exit status for us.
##set_board_info needs_status_wrapper 1
# Can't pass argc/argv to a simulated program
set_board_info noargs 1
# The simulator isn't really remote.
set_board_info isremote 0

------------------------------example------------------------------

- then in $prefix/share/dejagnu/config, copy sim.exp to $myboardsim.exp (as 
seen without its .exp suffix in load_generic_config above)
- add a line reading 'set_board_info isremote 0' to the end of this new file
- replace the 'set_board_info protocol "sim"' line with a new function 
definition and set_board_info command, based on a template like this:

------------------------------example------------------------------
proc $myboardsimproto_load {dest prog args} {
    #puts ">>>>>>>YEEHAA! D/l $prog to $dest, run with $args"
    # what we actually do is run the wrapper executable which
    # we set in the baseboard file  using "set_board_info sim"
    if ![board_info $dest exists sim] {
        perror "no simulator defined for [board_info $dest name]"
        exit 1
    } else {
        set sim [board_info $dest sim]
    }
    # we ignore $dest, which is implictly known by the wrapper
    # exe, and $args can be ignored because the baseboard.exp
    # called "set_board_info noargs 1".  we could pass them to
    # the wrapper executable if we wanted to.
    verbose ">>>>>>>>exec $sim $prog" 2
    set result [catch "exec $sim $prog" output]
    verbose "Simulator: rc $result, '$output'"
    if { $result == 0 } {
    set result "pass"
    set output ""
    } else {
    set result "fail"
    }
        return [list $result $output]
    }
}

set_board_info protocol  "$myboardsimproto"
------------------------------example------------------------------

(note that IIRC it is important to have the function defined /before/ the 
"set_board_info protocol" command).

  At that point, all you need to do is throw together a quick C program or 
shell script for $mywrapperprogram.  It will be invoked by the dg framework and 
passed the filename of a test on the commandline, which will be the name of a 
fully-linked executable target binary in the current dir.  Your wrapper could 
download the executable to the target, or it could just pass the test name as a 
command to the target's command shell using the serial port or telnet if you've 
built them all in.  Your wrapper has to deduce whether the test completed 
successfully (by returning from main with a return value of zero, or calling 
exit(0)) or failed (by calling abort(), or returning from main or exit() with a 
non-zero status) and return that information to the dg framework by exiting 
with a zero status code for a PASS and a non-zero status code for a FAIL.  You 
could make this work by using the ld --wrap flag when you compile your 
testsuite files for the target to put wrappers around main, abort and exit, 
that would just print something back to the host down the serial/telnet 
connection and return to your commandline shell.  Your main wrapper function 
would setup a longjmp buffer and call on to the testcase's main, then your 
abort and exit wrappers would use setjmp to get back to the main wrapper which 
could indicate the final exit status and return to the command shell that 
invoked it.

  Finally you'd run the testsuite from the build directory where you compiled 
your cross-toolchain with command lines such as

make check RUNTESTFLAGS="--target_board=$myboard--tool gcc -v --all" \
 2>&1 | tee chk.log

make check RUNTESTFLAGS="--target_board=$myboard--tool gcc -v --all \
./gcc.c-torture/execute/execute.exp ./gcc.c-torture/execute/ieee/ieee.exp \
./gcc.c-torture/execute/memcheck/memcheck.exp" 2>&1 | tee chk.log


  Umm.  First draft, E&OE, I hope that should all work for you. :)

    cheers,
      DaveK
-- 
Can't think of a witty .sigline today....