Re: macOS (Big Sur, Apple Silicon) 'make check' fails in test-crypto-tlscredsx509

[Eric Blake]

On 2/1/21 11:19 PM, Roman Bolshakov wrote:

> After a session of debugging I believe there's an issue with Clang 12.
> Here's a test program (it reproduces unexpected ASN1_VALUE_NOT_VALID
> from _asn1_time_der() in libtasn1):
> 
> #include <stdio.h>
> 
> static int func2(char *foo) {
>         fprintf(stderr, "%s:%d foo: %p\n", __func__, __LINE__, foo);
>         if (foo == NULL) {
>                 fprintf(stderr, "%s:%d foo: %p\n", __func__, __LINE__, foo);
>                 return 1;
>         }
>         return 0;
> }
> 
> int func1(char *foo) {
>         int counter = 0;
>         if (fprintf(stderr, "IO\n") > 0)
>                 counter += 10;
>         fprintf(stderr, "%s:%d foo: %p counter %d\n", __func__, __LINE__, 
> foo, counter);
>         if(!func2(foo + counter)) {

This line has unspecified behavior in the C standard.  Adding an integer
to a pointer is only well-specified if the pointer is to an array and
the integer is within the bounds or the slot just past the array.  But
since you called func1(NULL), foo is NOT pointing to an array, and
therefore foo+counter points to garbage, and the compiler is free to
optimize it at will.

>                 fprintf(stderr, "good\n");
>                 return 0;
>         } else {
>                 fprintf(stderr, "broken\n");
>                 return 1;
>         }
> }
> 
> int main() {
>         char *foo = NULL;
>         return func1(foo);
> }
> 
> 
> What return value would you expect from the program?

Because the code is not strictly compliant to the C standard, I'm not
sure what to expect.

> 
> If the program is compiled with -O0/O1 it returns zero exit code.
> Here's the output:
> IO
> func1:16 foo: 0x0 counter 10
> func2:4 foo: 0xa
> good
> 
> If it is compiled with -O2 it returns 1:
> IO
> func1:16 foo: 0x0 counter 10
> func2:4 foo: 0xa
> func2:6 foo: 0x0

And this proves the point that the compiler was able to exploit the
undefined behavior in your program.

> broken
> 
> That happens because clang uses register behind foo from func1 (it has zero
> pointer) inside inlined func2 (it should have non zero pointer).
> 
> So, immediate workaround would be to downgrade optimization level of libtasn1
> to -O1 in homebrew.
> 
> I've submitted the issue to Apple bugtracker:
> FB8986815

Yes, it's annoying that as compilers get smarter, it exposes the
presence of unspecified code in weird ways.  But I don't see this as a
bug in clang, but as a bug in libtasn1 for assuming undefined behavior
produces a sane result.

> 
> Best regards,
> Roman
> 

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org