[Octave-bug-tracker] [bug #57591] Segmentation faults when running the test suite (mostly with clang)

[Markus Mützel]

Follow-up Comment #144, bug #57591 (project octave):

It might also be worth noting that initially the segfault seemed to be "all
over the place" and more frequent. But recently the number of segmentation
faults seems to have gone down. And the only(?) one that still comes up once
in a while is the one that seems to be related to the tests in "gmres.m". At
least, that functions shows predominantly as the last one in the the logs.

I built with ASan flags following the instructions on this page:
http://wiki.octave.org/Finding_Memory_Leaks

Compilation stopped when the images for the manual were built:

fatal: caught signal Aborted -- stopping myself...
/bin/bash: line 1: 501190 Aborted                 (core dumped) /bin/bash
run-octave --norc --silent --no-history --path
/home/osboxes/Documents/Repositories/Octave/octave-2/.build/../doc/interpreter/
--eval "geometryimages ('doc/interpreter/', 'voronoi', 'png');"
make[2]: *** [Makefile:31791: doc/interpreter/voronoi.png] Error 134
make[2]: *** Waiting for unfinished jobs....


Tail of asan log:

=================================================================
==507762==ERROR: AddressSanitizer: heap-buffer-overflow on address
0x6140000f7fe0 at pc 0x7fb9958216aa bp 0x7ffcc41c3cc0 sp 0x7ffcc41c3468
READ of size 68 at 0x6140000f7fe0 thread T0
    #0 0x7fb9958216a9 in __interceptor_memcpy
(/usr/lib/x86_64-linux-gnu/libasan.so.6+0x3a6a9)
    #1 0x7fb9798488a7  (/usr/lib/x86_64-linux-gnu/dri/swrast_dri.so+0x5cd8a7)
    #2 0x7fb979848bcb  (/usr/lib/x86_64-linux-gnu/dri/swrast_dri.so+0x5cdbcb)
    #3 0x7fb979847da4  (/usr/lib/x86_64-linux-gnu/dri/swrast_dri.so+0x5ccda4)
    #4 0x7fb97984560a  (/usr/lib/x86_64-linux-gnu/dri/swrast_dri.so+0x5ca60a)
    #5 0x7fb979845d9b  (/usr/lib/x86_64-linux-gnu/dri/swrast_dri.so+0x5cad9b)
    #6 0x7fb97989a297  (/usr/lib/x86_64-linux-gnu/dri/swrast_dri.so+0x61f297)
    #7 0x7fb97989a6b2  (/usr/lib/x86_64-linux-gnu/dri/swrast_dri.so+0x61f6b2)
    #8 0x7fb979852b79  (/usr/lib/x86_64-linux-gnu/dri/swrast_dri.so+0x5d7b79)
    #9 0x7fb97984c706  (/usr/lib/x86_64-linux-gnu/dri/swrast_dri.so+0x5d1706)
    #10 0x7fb979913322 
(/usr/lib/x86_64-linux-gnu/dri/swrast_dri.so+0x698322)
    #11 0x7fb9793bd647 
(/usr/lib/x86_64-linux-gnu/dri/swrast_dri.so+0x142647)
    #12 0x7fb979565b46 
(/usr/lib/x86_64-linux-gnu/dri/swrast_dri.so+0x2eab46)
    #13 0x7fb979564e57 
(/usr/lib/x86_64-linux-gnu/dri/swrast_dri.so+0x2e9e57)
    #14 0x7fb979608970 
(/usr/lib/x86_64-linux-gnu/dri/swrast_dri.so+0x38d970)
    #15 0x7fb994d1a230 in octave::opengl_functions::glDisable(unsigned int)
(/home/osboxes/Documents/Repositories/Octave/octave-2/.build/libgui/.libs/liboctgui.so.6+0x4c8230)
    #16 0x7fb993740d88 in
octave::opengl_renderer::set_linestyle(std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> > const&, bool, double)
../libinterp/corefcn/gl-render.cc:4325
    #17 0x7fb99370ce70 in
octave::opengl_renderer::draw_axes_boxes(axes::properties const&)
../libinterp/corefcn/gl-render.cc:1531
    #18 0x7fb99371bcb2 in octave::opengl_renderer::draw_axes(axes::properties
const&) ../libinterp/corefcn/gl-render.cc:2346
    #19 0x7fb9936ffba8 in octave::opengl_renderer::draw(graphics_object
const&, bool) ../libinterp/corefcn/gl-render.cc:724
    #20 0x7fb99373f6b2 in octave::opengl_renderer::draw(Matrix const&, bool)
../libinterp/corefcn/gl-render.cc:4146
    #21 0x7fb9937025af in
octave::opengl_renderer::draw_figure(figure::properties const&)
../libinterp/corefcn/gl-render.cc:791
    #22 0x7fb9936ffa41 in octave::opengl_renderer::draw(graphics_object
const&, bool) ../libinterp/corefcn/gl-render.cc:722
    #23 0x7fb994d16be3 in QtHandles::GLCanvas::do_getPixels(octave_handle
const&) ../libgui/graphics/GLCanvas.cc:125
    #24 0x7fb994d19b8c in QtHandles::Canvas::getPixels()
../libgui/graphics/Canvas.h:89
    #25 0x7fb994d06b42 in QtHandles::Figure::slotGetPixels()
../libgui/graphics/Figure.cc:342
    #26 0x7fb994dc0564 in QtHandles::Figure::qt_static_metacall(QObject*,
QMetaObject::Call, int, void**) libgui/graphics/moc-Figure.cc:105
    #27 0x7fb98e25b650 in QObject::event(QEvent*)
(/usr/lib/x86_64-linux-gnu/libQt5Core.so.5+0x2d7650)
    #28 0x7fb98ece5012 in QApplicationPrivate::notify_helper(QObject*,
QEvent*) (/usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5+0x16b012)
    #29 0x7fb994f59807 in octave::octave_qapplication::notify(QObject*,
QEvent*) ../libgui/src/octave-qobject.cc:133
    #30 0x7fb98e22f1c9 in QCoreApplication::notifyInternal2(QObject*, QEvent*)
(/usr/lib/x86_64-linux-gnu/libQt5Core.so.5+0x2ab1c9)
    #31 0x7fb98e231bc0 in QCoreApplicationPrivate::sendPostedEvents(QObject*,
int, QThreadData*) (/usr/lib/x86_64-linux-gnu/libQt5Core.so.5+0x2adbc0)
    #32 0x7fb98e2871c6  (/usr/lib/x86_64-linux-gnu/libQt5Core.so.5+0x3031c6)
    #33 0x7fb98ba3962a in g_main_context_dispatch
(/usr/lib/x86_64-linux-gnu/libglib-2.0.so.0+0x5362a)
    #34 0x7fb98ba398d7  (/usr/lib/x86_64-linux-gnu/libglib-2.0.so.0+0x538d7)
    #35 0x7fb98ba399a2 in g_main_context_iteration
(/usr/lib/x86_64-linux-gnu/libglib-2.0.so.0+0x539a2)
    #36 0x7fb98e286842 in
QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>)
(/usr/lib/x86_64-linux-gnu/libQt5Core.so.5+0x302842)
    #37 0x7fb98e22da4a in
QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>)
(/usr/lib/x86_64-linux-gnu/libQt5Core.so.5+0x2a9a4a)
    #38 0x7fb98e235fc5 in QCoreApplication::exec()
(/usr/lib/x86_64-linux-gnu/libQt5Core.so.5+0x2b1fc5)
    #39 0x7fb994f5ba12 in octave::base_qobject::exec()
../libgui/src/octave-qobject.cc:345
    #40 0x7fb994f7ebf3 in octave::qt_application::execute()
../libgui/src/qt-application.cc:73
    #41 0x556641034d9a in main ../src/main-gui.cc:106
    #42 0x7fb98fc50cb1 in __libc_start_main
(/lib/x86_64-linux-gnu/libc.so.6+0x28cb1)
    #43 0x55664103452d in _start
(/home/osboxes/Documents/Repositories/Octave/octave-2/.build/src/.libs/octave-gui+0x252d)

0x6140000f7fe0 is located 0 bytes to the right of 416-byte region
[0x6140000f7e40,0x6140000f7fe0)
allocated by thread T0 here:
    #0 0x7fb995897517 in malloc
(/usr/lib/x86_64-linux-gnu/libasan.so.6+0xb0517)
    #1 0x7fb979899ded  (/usr/lib/x86_64-linux-gnu/dri/swrast_dri.so+0x61eded)

SUMMARY: AddressSanitizer: heap-buffer-overflow
(/usr/lib/x86_64-linux-gnu/libasan.so.6+0x3a6a9) in __interceptor_memcpy
Shadow bytes around the buggy address:
  0x0c2880016fa0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c2880016fb0: fd fd fd fd fd fd fd fd fd fd fd fd fd fa fa fa
  0x0c2880016fc0: fa fa fa fa fa fa fa fa 00 00 00 00 00 00 00 00
  0x0c2880016fd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c2880016fe0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x0c2880016ff0: 00 00 00 00 00 00 00 00 00 00 00 00[fa]fa fa fa
  0x0c2880017000: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd
  0x0c2880017010: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c2880017020: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c2880017030: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fa
  0x0c2880017040: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
  Shadow gap:              cc
==507762==ABORTING


This looks like it is something different though. Maybe a mesa bug. (Or is
it?)

Anyway, I built again with the same flags on the stable branch.
That succeeded without issues.
Running `test gmres` works fine. Also `test eigs` (the test immediately before
gmres in the log) followed `test gmres` doesn't report any issues.
Running `make check` takes a lot of time (so I believe the flags were
correctly picked up). 
It finally failed with the following (after passing sparse/gmres.m):

  strings/strtrunc.m .............................................fatal:
caught signal Segmentation fault -- stopping myself...
/bin/bash: line 1: 664216 Segmentation fault      (core dumped) /bin/bash
../run-octave --no-init-file --silent --no-history -p
/home/osboxes/Documents/Repositories/Octave/octave-stable/.build/test/mex
/home/osboxes/Documents/Repositories/Octave/octave-stable/.build/../test/fntests.m
/home/osboxes/Documents/Repositories/Octave/octave-stable/.build/../test


The asan log was not very informative afaict. Maybe it capped out at approx.
620 MiB. The core dump from that crash is empty.

I still have both builds and the core dump of the crash on the default branch
lying around. If someone has hints what I should try with that, please let me
know.

    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?57591>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/