gforth
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Memory leak in unix/pthread.fs


From: nature
Subject: Memory leak in unix/pthread.fs
Date: Tue, 08 Nov 2022 15:38:54 +0100
User-agent: mu4e 1.8.11; emacs 28.2

Hi,

I am working on a web server written in pure Forth and I am using gforth
as the implementation I develop on.  Serving multiple requests without
blocking requires me to venture into the realm of multi-threading and
gforth's experimental support for pthread has been a blessing.  It may
be good for me to mention that I work on UNIX (OpenBSD).

Now instead of doing like in cilk.fs and spawning a few workers and
offloading requests to them, I prefer to create a separate thread each
time a request comes and this led me to notice that my server wasn't
able to serve more than ~3k requests before hanging and also the memory
usage of my process increased a lot with each new request comming in.  I
then proceeded to hunt down memory leaks due to the S" S\" and S+ words
with minimal impact.  That is what lead me to realize that creating new
threads with the PASS word was creating non-detached thread which never
freed the memory once the thread exited.

I am attaching a patch for unix/pthread.fs which fixed the big memory
leak for me, my server is no able to server well above 3k requests
without hanging/crashing (I tested 10 without any issues).

--- pthread.fs.old      Tue Nov  8 15:22:26 2022
+++ pthread.fs  Tue Nov  8 15:22:56 2022
@@ -247,7 +247,7 @@
     r> swap >r  save-task r@ 's !
     1+ dup cells negate  sp0 r@ 's @ -rot  sp0 r@ 's +!
     sp0 r@ 's @ swap 0 ?DO  tuck ! cell+  LOOP  drop
-    pthread-id r@ 's 0 thread_start r> pthread_create drop ; compile-only
+    pthread-id r@ 's pthread_detach_attr thread_start r> pthread_create drop ; 
compile-only
 
 : pass ( x1 .. xn n task -- ) \ gforth-experimental
     \G activates task, and passes n parameters from the data stack

Also optionally I am attaching another patch for cilk.fs just to
correctly get the number of cores on OpenBSD.

--- cilk.fs.old Tue Nov  8 15:31:33 2022
+++ cilk.fs     Tue Nov  8 15:32:43 2022
@@ -20,7 +20,7 @@
 
 require unix/pthread.fs
 
-e? os-type s" darwin" string-prefix? [IF]  s" sysctl -n hw.ncpu"
+e? os-type 2dup s" darwin" string-prefix? -rot s" openbsd" string-prefix? or 
[IF]  s" sysctl -n hw.ncpu"
 [ELSE]  s" nproc"  [THEN]
 r/o open-pipe throw slurp-fid s>number drop 1 max Value cores
 

Thank you for the great work on gforth, I can't wait for the official
1.0 to release so that OpenBSD ports won't be stuck with 0.7.3 forever! :D

Cheers,

-- 
nature

reply via email to

[Prev in Thread] Current Thread [Next in Thread]