[Octave-bug-tracker] [bug #58641] Java crashes (OutOfMemoryError/StackOverflowError) when invoking a child process

[anonymous]

URL:
  <https://savannah.gnu.org/bugs/?58641>

                 Summary: Java crashes (OutOfMemoryError/StackOverflowError)
when invoking a child process
                 Project: GNU Octave
            Submitted by: None
            Submitted on: Mon 22 Jun 2020 03:43:01 PM UTC
                Category: Interpreter
                Severity: 3 - Normal
                Priority: 5 - Normal
              Item Group: Unexpected Error
                  Status: None
             Assigned to: None
         Originator Name: Benjamin Moody
        Originator Email: benjaminmoody@gmail.com
             Open/Closed: Open
                 Release: 5.2.0
         Discussion Lock: Any
        Operating System: GNU/Linux

    _______________________________________________________

Details:

When using a recent version of OpenJDK, if an Octave program invokes a Java
function that in turn creates a child process, it can cause a stack overflow
in an internal Java thread.  This can cause the program to crash or to freeze
when waiting for the child process to finish.

A simple example is to run the following:


b = javaObject('java.lang.ProcessBuilder', {'/bin/true'});
p = b.start();
p.waitFor()


On older versions of Octave and OpenJDK, this should return 0.  On newer
versions, it will either show an error message, or freeze the Octave process
completely.

The reason appears to be twofold:

* newer versions of Octave are using more thread-local storage than older
versions did;
* when creating a "process reaper" thread, newer versions of OpenJDK allocate
less stack space (which must include thread-local storage space) than older
versions of OpenJDK did.

More details in the Debian bug report:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=961681

Also see the Ubuntu bug report:
https://bugs.launchpad.net/ubuntu/+source/octave/+bug/1866453

It seems to me that the simplest solution would be for Octave to set the Java
system property "jdk.lang.processReaperUseDefaultStackSize".  If this property
is set to "true", Java will use the same stack size for "reaper" threads as it
uses for other threads.  Otherwise, "reaper" threads use a much smaller stack
size (currently 128k.)




    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?58641>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/