Re: [lwip-users] inet_chksum.c misbehaving with compiler optimisation?

[Martin Velek]

Hi,

a simple code, tested on PC. The functions lwip_standard_chksumOpt and
lwip_standard_chksum are defined in separate files, a body is taken
from lwip . The  lwip_standard_chksumOpt has -0s flag. I have added
the gcc -v also.

It ends with OK everytime but it does not guarantee that everything is correct.

Regards
Martin
----------------------------------------------------------
#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
#include <time.h>

typedef uint8_t u8_t;
typedef int8_t s8_t;
typedef uint16_t u16_t;
typedef int16_t s16_t;
typedef uint32_t u32_t;
typedef int32_t s32_t;
typedef u32_t mem_ptr_t;

u16_t lwip_standard_chksumOpt(void *dataptr, int len);
u16_t lwip_standard_chksum(void *dataptr, int len);

#define ARRAY_SIZE 0x20000
int main(void)
{
        srand(time(NULL));
        uint8_t * array = (uint8_t *) malloc(ARRAY_SIZE * sizeof(uint8_t));
        for (int j = 0; j < 3; j++)
        {
                for (int i = 0; i < ARRAY_SIZE; i++)
                {
                        array[i] = (uint8_t)(rand() % 256);
                }

                for (int i = 0; i < ARRAY_SIZE - j; i++)
                {
                        if (lwip_standard_chksumOpt(array + j, i)
                                        != lwip_standard_chksum(array + j, i))
                        {
                                printf("ERROR\n");
                                exit(1);
                        }
                }
        }
        printf("OK\n");
        return 0;
}

OK

---------------------------------------------------------------------------------
**** Build of configuration Debug for project testChcksum ****

make all
Building file: ../src/chksum.c
Invoking: GCC C Compiler
gcc -O0 -g3 -pedantic -Wall -c -fmessage-length=0 -std=gnu99 -W -Wall
-Wextra -Wcast-qual -Wconversion -v -MMD -MP -MF"src/chksum.d"
-MT"src/chksum.d" -o "src/chksum.o" "../src/chksum.c"
Using built-in specs.
Finished building: ../src/chksum.c
Target: i686-linux-gnu

Configured with: ../src/configure -v --with-pkgversion='Ubuntu/Linaro
4.4.4-14ubuntu5'
--with-bugurl=file:///usr/share/doc/gcc-4.4/README.Bugs
--enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr
--program-suffix=-4.4 --enable-shared --enable-multiarch
--enable-linker-build-id --with-system-zlib --libexecdir=/usr/lib
--without-included-gettext --enable-threads=posix
--with-gxx-include-dir=/usr/include/c++/4.4 --libdir=/usr/lib
--enable-nls --with-sysroot=/ --enable-clocale=gnu
--enable-libstdcxx-debug --enable-objc-gc --enable-targets=all
--disable-werror --with-arch-32=i686 --with-tune=generic
--enable-checking=release --build=i686-linux-gnu --host=i686-linux-gnu
--target=i686-linux-gnu
Building file: ../src/chksumOpti.c
Invoking: GCC C Compiler
gcc -Os -pedantic -Wall -c -fmessage-length=0 -std=gnu99 -W -Wall
-Wextra -Wcast-qual -Wconversion -v -MMD -MP -MF"src/chksumOpti.d"
-MT"src/chksumOpti.d" -o "src/chksumOpti.o" "../src/chksumOpti.c"
Thread model: posix
gcc version 4.4.5 (Ubuntu/Linaro 4.4.4-14ubuntu5)
COLLECT_GCC_OPTIONS='-O0' '-g3' '-pedantic' '-c' '-fmessage-length=0'
'-std=gnu99' '-W' '-Wall' '-Wextra' '-Wcast-qual' '-Wconversion' '-v'
'-MMD' '-MP' '-MFsrc/chksum.d' '-MTsrc/chksum.d' '-o' 'src/chksum.o'
'-mtune=generic' '-march=i686'
Finished building: ../src/chksumOpti.c

Building file: ../src/main.c
Invoking: GCC C Compiler
gcc -Os -g3 -pedantic -Wall -c -fmessage-length=0 -std=gnu99 -W -Wall
-Wextra -Wcast-qual -Wconversion -v -MMD -MP -MF"src/main.d"
-MT"src/main.d" -o "src/main.o" "../src/main.c"
 /usr/lib/gcc/i686-linux-gnu/4.4.5/cc1 -quiet -v -MMD src/chksum.d
-MFsrc/chksum.d -MP -MTsrc/chksum.d -MQ src/chksum.o -dD
../src/chksum.c -D_FORTIFY_SOURCE=2 -quiet -dumpbase chksum.c
-mtune=generic -march=i686 -auxbase-strip src/chksum.o -g3 -O0
-pedantic -W -Wall -Wextra -Wcast-qual -Wconversion -std=gnu99
-version -fmessage-length=0 -fstack-protector -o /tmp/cc3PyXPs.s
ignoring nonexistent directory "/usr/local/include/i686-linux-gnu"
Finished building: ../src/main.c
ignoring nonexistent directory
"/usr/lib/gcc/i686-linux-gnu/4.4.5/../../../../i686-linux-gnu/include"

#include "..." search starts here:
Building target: testChcksum
Invoking: GCC C Linker
gcc  -o "testChcksum"  ./src/chksum.o ./src/chksumOpti.o ./src/main.o
Finished building target: testChcksum

#include <...> search starts here:
 /usr/local/include
 /usr/lib/gcc/i686-linux-gnu/4.4.5/include
 /usr/lib/gcc/i686-linux-gnu/4.4.5/include-fixed
 /usr/include/i686-linux-gnu
 /usr/include
End of search list.
GNU C (Ubuntu/Linaro 4.4.4-14ubuntu5) version 4.4.5 (i686-linux-gnu)
        compiled by GNU C version 4.4.5, GMP version 4.3.2, MPFR version 
3.0.0-p3.
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
Compiler executable checksum: 2c79c9815376ef8d35d3f163ae436fce
COLLECT_GCC_OPTIONS='-O0' '-g3' '-pedantic' '-c' '-fmessage-length=0'
'-std=gnu99' '-W' '-Wall' '-Wextra' '-Wcast-qual' '-Wconversion' '-v'
'-MMD' '-MP' '-MFsrc/chksum.d' '-MTsrc/chksum.d' '-o' 'src/chksum.o'
'-mtune=generic' '-march=i686'
 as -V -Qy -o src/chksum.o /tmp/cc3PyXPs.s
GNU assembler version 2.20.51 (i686-linux-gnu) using BFD version (GNU
Binutils for Ubuntu) 2.20.51-system.20100908
COMPILER_PATH=/usr/lib/gcc/i686-linux-gnu/4.4.5/:/usr/lib/gcc/i686-linux-gnu/4.4.5/:/usr/lib/gcc/i686-linux-gnu/:/usr/lib/gcc/i686-linux-gnu/4.4.5/:/usr/lib/gcc/i686-linux-gnu/:/usr/lib/gcc/i686-linux-gnu/4.4.5/:/usr/lib/gcc/i686-linux-gnu/
LIBRARY_PATH=/usr/lib/gcc/i686-linux-gnu/4.4.5/:/usr/lib/gcc/i686-linux-gnu/4.4.5/:/usr/lib/gcc/i686-linux-gnu/4.4.5/../../../../lib/:/lib/../lib/:/usr/lib/../lib/:/usr/lib/gcc/i686-linux-gnu/4.4.5/../../../:/lib/:/usr/lib/:/usr/lib/i686-linux-gnu/
COLLECT_GCC_OPTIONS='-O0' '-g3' '-pedantic' '-c' '-fmessage-length=0'
'-std=gnu99' '-W' '-Wall' '-Wextra' '-Wcast-qual' '-Wconversion' '-v'
'-MMD' '-MP' '-MFsrc/chksum.d' '-MTsrc/chksum.d' '-o' 'src/chksum.o'
'-mtune=generic' '-march=i686'
Using built-in specs.
Target: i686-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu/Linaro
4.4.4-14ubuntu5'
--with-bugurl=file:///usr/share/doc/gcc-4.4/README.Bugs
--enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr
--program-suffix=-4.4 --enable-shared --enable-multiarch
--enable-linker-build-id --with-system-zlib --libexecdir=/usr/lib
--without-included-gettext --enable-threads=posix
--with-gxx-include-dir=/usr/include/c++/4.4 --libdir=/usr/lib
--enable-nls --with-sysroot=/ --enable-clocale=gnu
--enable-libstdcxx-debug --enable-objc-gc --enable-targets=all
--disable-werror --with-arch-32=i686 --with-tune=generic
--enable-checking=release --build=i686-linux-gnu --host=i686-linux-gnu
--target=i686-linux-gnu
Thread model: posix
gcc version 4.4.5 (Ubuntu/Linaro 4.4.4-14ubuntu5)
COLLECT_GCC_OPTIONS='-Os' '-pedantic' '-c' '-fmessage-length=0'
'-std=gnu99' '-W' '-Wall' '-Wextra' '-Wcast-qual' '-Wconversion' '-v'
'-MMD' '-MP' '-MFsrc/chksumOpti.d' '-MTsrc/chksumOpti.d' '-o'
'src/chksumOpti.o' '-mtune=generic' '-march=i686'
 /usr/lib/gcc/i686-linux-gnu/4.4.5/cc1 -quiet -v -MMD src/chksumOpti.d
-MFsrc/chksumOpti.d -MP -MTsrc/chksumOpti.d -MQ src/chksumOpti.o
../src/chksumOpti.c -D_FORTIFY_SOURCE=2 -quiet -dumpbase chksumOpti.c
-mtune=generic -march=i686 -auxbase-strip src/chksumOpti.o -Os
-pedantic -W -Wall -Wextra -Wcast-qual -Wconversion -std=gnu99
-version -fmessage-length=0 -fstack-protector -o /tmp/cceG4T2w.s
ignoring nonexistent directory "/usr/local/include/i686-linux-gnu"
ignoring nonexistent directory
"/usr/lib/gcc/i686-linux-gnu/4.4.5/../../../../i686-linux-gnu/include"
#include "..." search starts here:
#include <...> search starts here:
 /usr/local/include
 /usr/lib/gcc/i686-linux-gnu/4.4.5/include
 /usr/lib/gcc/i686-linux-gnu/4.4.5/include-fixed
 /usr/include/i686-linux-gnu
 /usr/include
End of search list.
GNU C (Ubuntu/Linaro 4.4.4-14ubuntu5) version 4.4.5 (i686-linux-gnu)
        compiled by GNU C version 4.4.5, GMP version 4.3.2, MPFR version 
3.0.0-p3.
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
Compiler executable checksum: 2c79c9815376ef8d35d3f163ae436fce
COLLECT_GCC_OPTIONS='-Os' '-pedantic' '-c' '-fmessage-length=0'
'-std=gnu99' '-W' '-Wall' '-Wextra' '-Wcast-qual' '-Wconversion' '-v'
'-MMD' '-MP' '-MFsrc/chksumOpti.d' '-MTsrc/chksumOpti.d' '-o'
'src/chksumOpti.o' '-mtune=generic' '-march=i686'
 as -V -Qy -o src/chksumOpti.o /tmp/cceG4T2w.s
GNU assembler version 2.20.51 (i686-linux-gnu) using BFD version (GNU
Binutils for Ubuntu) 2.20.51-system.20100908
COMPILER_PATH=/usr/lib/gcc/i686-linux-gnu/4.4.5/:/usr/lib/gcc/i686-linux-gnu/4.4.5/:/usr/lib/gcc/i686-linux-gnu/:/usr/lib/gcc/i686-linux-gnu/4.4.5/:/usr/lib/gcc/i686-linux-gnu/:/usr/lib/gcc/i686-linux-gnu/4.4.5/:/usr/lib/gcc/i686-linux-gnu/
LIBRARY_PATH=/usr/lib/gcc/i686-linux-gnu/4.4.5/:/usr/lib/gcc/i686-linux-gnu/4.4.5/:/usr/lib/gcc/i686-linux-gnu/4.4.5/../../../../lib/:/lib/../lib/:/usr/lib/../lib/:/usr/lib/gcc/i686-linux-gnu/4.4.5/../../../:/lib/:/usr/lib/:/usr/lib/i686-linux-gnu/
COLLECT_GCC_OPTIONS='-Os' '-pedantic' '-c' '-fmessage-length=0'
'-std=gnu99' '-W' '-Wall' '-Wextra' '-Wcast-qual' '-Wconversion' '-v'
'-MMD' '-MP' '-MFsrc/chksumOpti.d' '-MTsrc/chksumOpti.d' '-o'
'src/chksumOpti.o' '-mtune=generic' '-march=i686'
Using built-in specs.
Target: i686-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu/Linaro
4.4.4-14ubuntu5'
--with-bugurl=file:///usr/share/doc/gcc-4.4/README.Bugs
--enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr
--program-suffix=-4.4 --enable-shared --enable-multiarch
--enable-linker-build-id --with-system-zlib --libexecdir=/usr/lib
--without-included-gettext --enable-threads=posix
--with-gxx-include-dir=/usr/include/c++/4.4 --libdir=/usr/lib
--enable-nls --with-sysroot=/ --enable-clocale=gnu
--enable-libstdcxx-debug --enable-objc-gc --enable-targets=all
--disable-werror --with-arch-32=i686 --with-tune=generic
--enable-checking=release --build=i686-linux-gnu --host=i686-linux-gnu
--target=i686-linux-gnu
Thread model: posix
gcc version 4.4.5 (Ubuntu/Linaro 4.4.4-14ubuntu5)
COLLECT_GCC_OPTIONS='-Os' '-g3' '-pedantic' '-c' '-fmessage-length=0'
'-std=gnu99' '-W' '-Wall' '-Wextra' '-Wcast-qual' '-Wconversion' '-v'
'-MMD' '-MP' '-MFsrc/main.d' '-MTsrc/main.d' '-o' 'src/main.o'
'-mtune=generic' '-march=i686'
 /usr/lib/gcc/i686-linux-gnu/4.4.5/cc1 -quiet -v -MMD src/main.d
-MFsrc/main.d -MP -MTsrc/main.d -MQ src/main.o -dD ../src/main.c
-D_FORTIFY_SOURCE=2 -quiet -dumpbase main.c -mtune=generic -march=i686
-auxbase-strip src/main.o -g3 -Os -pedantic -W -Wall -Wextra
-Wcast-qual -Wconversion -std=gnu99 -version -fmessage-length=0
-fstack-protector -o /tmp/ccx2IyBz.s
ignoring nonexistent directory "/usr/local/include/i686-linux-gnu"
ignoring nonexistent directory
"/usr/lib/gcc/i686-linux-gnu/4.4.5/../../../../i686-linux-gnu/include"
#include "..." search starts here:
#include <...> search starts here:
 /usr/local/include
 /usr/lib/gcc/i686-linux-gnu/4.4.5/include
 /usr/lib/gcc/i686-linux-gnu/4.4.5/include-fixed
 /usr/include/i686-linux-gnu
 /usr/include
End of search list.
GNU C (Ubuntu/Linaro 4.4.4-14ubuntu5) version 4.4.5 (i686-linux-gnu)
        compiled by GNU C version 4.4.5, GMP version 4.3.2, MPFR version 
3.0.0-p3.
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
Compiler executable checksum: 2c79c9815376ef8d35d3f163ae436fce
../src/main.c: In function ‘main’:
../src/main.c:20: warning: conversion to ‘unsigned int’ from ‘time_t’
may change the sign of the result
COLLECT_GCC_OPTIONS='-Os' '-g3' '-pedantic' '-c' '-fmessage-length=0'
'-std=gnu99' '-W' '-Wall' '-Wextra' '-Wcast-qual' '-Wconversion' '-v'
'-MMD' '-MP' '-MFsrc/main.d' '-MTsrc/main.d' '-o' 'src/main.o'
'-mtune=generic' '-march=i686'
 as -V -Qy -o src/main.o /tmp/ccx2IyBz.s
GNU assembler version 2.20.51 (i686-linux-gnu) using BFD version (GNU
Binutils for Ubuntu) 2.20.51-system.20100908
COMPILER_PATH=/usr/lib/gcc/i686-linux-gnu/4.4.5/:/usr/lib/gcc/i686-linux-gnu/4.4.5/:/usr/lib/gcc/i686-linux-gnu/:/usr/lib/gcc/i686-linux-gnu/4.4.5/:/usr/lib/gcc/i686-linux-gnu/:/usr/lib/gcc/i686-linux-gnu/4.4.5/:/usr/lib/gcc/i686-linux-gnu/
LIBRARY_PATH=/usr/lib/gcc/i686-linux-gnu/4.4.5/:/usr/lib/gcc/i686-linux-gnu/4.4.5/:/usr/lib/gcc/i686-linux-gnu/4.4.5/../../../../lib/:/lib/../lib/:/usr/lib/../lib/:/usr/lib/gcc/i686-linux-gnu/4.4.5/../../../:/lib/:/usr/lib/:/usr/lib/i686-linux-gnu/
COLLECT_GCC_OPTIONS='-Os' '-g3' '-pedantic' '-c' '-fmessage-length=0'
'-std=gnu99' '-W' '-Wall' '-Wextra' '-Wcast-qual' '-Wconversion' '-v'
'-MMD' '-MP' '-MFsrc/main.d' '-MTsrc/main.d' '-o' 'src/main.o'
'-mtune=generic' '-march=i686'

On 22 November 2011 21:17, FreeRTOS Info <address@hidden> wrote:
> On 22/11/2011 20:13, address@hidden wrote:
>> FreeRTOS Info wrote:
>>> Just in case anybody is watching this thread:
>>>
>>> As unlikely as this all seems, having previously isolated one file, I
>>> have now isolated it to a single function: lwip_standard_chksum().
>>>
>>> I have the entire applicatino running at maximum optimisation, except
>>> this function, and everything is happy.
>>>
>>> Put the lowest optimisation on just this single function (which is now
>>> in its own file), and the network works, but with a lot of corruption on
>>> the wire.
>> Like Kieran and Stephen, I wouldn't rule out the possibility of a timing
>> issue, too. Especially since checksumming was the most time consuming
>> task in lwIP when I last benchmarked (aside of memcpy, but I guess you
>> haven't changed the amount of optimization of memcpy in your tests as it
>> often comes precompiled with the C library).
>>
>> Nevertheless, to prove the function fails, why don't you run a test on
>> random data comparing the results of the optimized checksum function
>> with a non-optimized version? You can even let that test run for a while
>> if you put the two functions into the same binary. If it's a compiler
>> bug or anything in that area, this test should show it.
>
> Good thinking!  Will do and report back.
>
>
>
>>
>> BTW: what do mean with 'a lot of corruption on the wire', invalid
>> checksums or more errors?
>
> It was described in a previous email.  Sorry I have to run now but will
> be back online later this evening.  I will explain it again when I post
> the results of the above mentioned test.
>
> Regards,
> Richard.
>
>
> _______________________________________________
> lwip-users mailing list
> address@hidden
> https://lists.nongnu.org/mailman/listinfo/lwip-users
>