Re: macOS (Big Sur, Apple Silicon) 'make check' fails in test-crypto-tlscredsx509

[Roman Bolshakov]

On Fri, Jan 29, 2021 at 09:53:27AM +0000, Daniel P. Berrangé wrote:
> On Fri, Jan 29, 2021 at 11:43:32AM +0300, Roman Bolshakov wrote:
> > On Wed, Jan 27, 2021 at 06:59:17PM +0000, Daniel P. Berrangé wrote:
> > > On Wed, Jan 27, 2021 at 07:56:16PM +0100, Stefan Weil wrote:
> > > > Am 27.01.21 um 19:17 schrieb Daniel P. Berrangé:
> > > > 
> > > > > On Wed, Jan 27, 2021 at 06:05:08PM +0100, Stefan Weil wrote:
> > > > > > Am 27.01.21 um 17:53 schrieb Daniel P. Berrangé:
> > > > > > 
> > > > > > > In $QEMU.git/crypto/init.c can you uncomment the "#define 
> > > > > > > DEBUG_GNUTLS"
> > > > > > > line and then re-build and re-run the test case.
> > > > > > > 
> > > > > > > There's a bunch of debug logs in code paths from 
> > > > > > > gnutls_x509_crt_privkey_sign
> > > > > > > that might give us useful info.
> > > > > > > 
> > > > > > > Regards,
> > > > > > > Daniel
> > > > > > 
> > > > > > % LANG=C.UTF-8 tests/test-crypto-tlscredsx509
> > > > > > # random seed: R02S9b95072a368ad370cdd4c780b8074596
> > > > > > 3: ASSERT: mpi.c[wrap_nettle_mpi_print]:60
> > > > > > 3: ASSERT: mpi.c[wrap_nettle_mpi_print]:60
> > > > > > 2: signing structure using RSA-SHA256
> > > > > > 3: ASSERT: common.c[_gnutls_x509_der_encode]:855
> > > > > > 3: ASSERT: sign.c[_gnutls_x509_pkix_sign]:174
> > > > > > 3: ASSERT: x509_write.c[gnutls_x509_crt_privkey_sign]:1834
> > > > > > 3: ASSERT: x509_write.c[gnutls_x509_crt_sign2]:1152
> > > > > > Bail out! FATAL-CRITICAL: Failed to sign certificate ASN1 parser: 
> > > > > > Value is
> > > > > > not valid.
> > > > > So it shows its failing inside a asn1_der_coding call, but I can't see
> > > > > why it would fail, especially if the same test suite passes fine on
> > > > > macOS x86_64 hosts.
> > > > 
> > > > 
> > > > It returns ASN1_MEM_ERROR, so the input vector is too small.
> > > 
> > > Hmm, that's odd - "Value is not valid" corresponds to
> > > ASN1_VALUE_NOT_VALID error code.
> > > 
> > 
> > Hi Daniel, Stefan,
> > 
> > It's interesting that "make check" of libtasn1 fails with three tests
> > and two of them produce VALUE_NOT_VALID error.
> > 
> > The failing tests are:
> >   FAIL: Test_parser
> >   FAIL: Test_tree
> >   FAIL: copynode
> 
> That's interesting. Assuming 'make check' for libtasn1 succeeeds on
> x86_64 macOS, then I'm inclined to blame this whole problem on
> libtasn1 not QEMU.
> 

'make check' of libtasn1 doesn't succeed on x86_64 either.

After a session of debugging I believe there's an issue with Clang 12.
Here's a test program (it reproduces unexpected ASN1_VALUE_NOT_VALID
from _asn1_time_der() in libtasn1):

#include <stdio.h>

static int func2(char *foo) {
        fprintf(stderr, "%s:%d foo: %p\n", __func__, __LINE__, foo);
        if (foo == NULL) {
                fprintf(stderr, "%s:%d foo: %p\n", __func__, __LINE__, foo);
                return 1;
        }
        return 0;
}

int func1(char *foo) {
        int counter = 0;
        if (fprintf(stderr, "IO\n") > 0)
                counter += 10;
        fprintf(stderr, "%s:%d foo: %p counter %d\n", __func__, __LINE__, foo, 
counter);
        if(!func2(foo + counter)) {
                fprintf(stderr, "good\n");
                return 0;
        } else {
                fprintf(stderr, "broken\n");
                return 1;
        }
}

int main() {
        char *foo = NULL;
        return func1(foo);
}


What return value would you expect from the program?

If the program is compiled with -O0/O1 it returns zero exit code.
Here's the output:
IO
func1:16 foo: 0x0 counter 10
func2:4 foo: 0xa
good

If it is compiled with -O2 it returns 1:
IO
func1:16 foo: 0x0 counter 10
func2:4 foo: 0xa
func2:6 foo: 0x0
broken

That happens because clang uses register behind foo from func1 (it has zero
pointer) inside inlined func2 (it should have non zero pointer).

So, immediate workaround would be to downgrade optimization level of libtasn1
to -O1 in homebrew.

I've submitted the issue to Apple bugtracker:
FB8986815

Best regards,
Roman