guile-user
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: GC Warning: Repeated allocation of very large block


From: Damien Mattei
Subject: Re: GC Warning: Repeated allocation of very large block
Date: Mon, 19 Sep 2022 19:26:15 +0200

thank you, i think my algorithm is right but can be improved, i have to
work on it,activate tracing to see where is the big expression list and if
it is possible to correct that. There is a main loop in code and the
warning only appear when data is bigger so i do not think of any memory
leak (only have alittle doubt on an hastable which is global variable and
reused:
(when debug-mode (display-nl "Quine-Mc-Cluskey:"))
 {minterms-ht <- (make-hash-table)}  ;; need to be cleared at each run
(init-hash-table-with-set-and-value minterms-ht minterms #f)
this :
;; the hash table for minterms, better to be a top-level definition,it's
nightmare otherwise...
(declare minterms-ht)
{minterms-ht <- (make-hash-table)} ;; Scheme+  syntax
expression create a new hash table with pointer minterms-ht
this is done at each loop in the for each Cn
and i should clear it before but there is no instruction to do that in the
SRFI 69:
https://srfi.schemers.org/srfi-69/srfi-69.html
???
I will also check the still running memory used by program.
Damien

On Mon, Sep 19, 2022 at 1:44 PM Olivier Dion <olivier.dion@polymtl.ca>
wrote:

> On Mon, 19 Sep 2022, Damien Mattei <damien.mattei@gmail.com> wrote:
> > is this message appearing when a single scheme variable reach a given
> > size?
>
> This message is from the bdwgc and not from Guile itself.  From their
> documentation:
> --8<---------------cut here---------------start------------->8---
> The garbage collector generates warning messages of the form:
>
>
>     Repeated allocation of very large block ...
>     May lead to memory leak and poor performance
>
>
> when it needs to allocate a block at a location that it knows to be
> referenced
> by a false pointer. These false pointers can be either permanent (e.g.
> a static integer variable that never changes) or temporary. In the latter
> case, the warning is largely spurious, and the block will eventually
> be reclaimed normally. In the former case, the program will still run
> correctly, but the block will never be reclaimed. Unless the block is
> intended
> to be permanent, the warning indicates a memory leak.
>
>   1. Ignore these warnings while you are using GC_DEBUG. Some of the
> routines
>   mentioned below don't have debugging equivalents. (Alternatively, write
> the
>   missing routines and send them to me.)
>   2. Replace allocator calls that request large blocks with calls to
>   `GC_malloc_ignore_off_page` or `GC_malloc_atomic_ignore_off_page`. You
> may
>   want to set a breakpoint in `GC_default_warn_proc` to help you identify
> such
>   calls. Make sure that a pointer to somewhere near the beginning of the
>   resulting block is maintained in a (preferably volatile) variable as long
>   as the block is needed.
>   3. If the large blocks are allocated with realloc, we suggest instead
>   allocating them with something like the following. Note that the realloc
>   size increment should be fairly large (e.g. a factor of 3/2) for this to
>   exhibit reasonable performance. But we all know we should do that anyway.
>
>
>         void * big_realloc(void *p, size_t new_size) {
>             size_t old_size = GC_size(p);
>             void * result;
>             if (new_size <= 10000) return(GC_realloc(p, new_size));
>             if (new_size <= old_size) return(p);
>             result = GC_malloc_ignore_off_page(new_size);
>             if (result == 0) return(0);
>             memcpy(result,p,old_size);
>             GC_free(p);
>             return(result);
>         }
>
>
>   4. In the unlikely case that even relatively small object (<20 KB)
>   allocations are triggering these warnings, then your address space
> contains
>   lots of "bogus pointers", i.e. values that appear to be pointers but
> aren't.
>   Usually this can be solved by using `GC_malloc_atomic` or the routines
>   in `gc_typed.h` to allocate large pointer-free regions of bitmaps, etc.
>   Sometimes the problem can be solved with trivial changes of encoding
>   in certain values. It is possible, to identify the source of the bogus
>   pointers by building the collector with `-DPRINT_BLACK_LIST`, which will
>   cause it to print the "bogus pointers", along with their location.
>   5. If you get only a fixed number of these warnings, you are probably
> only
>   introducing a bounded leak by ignoring them. If the data structures being
>   allocated are intended to be permanent, then it is also safe to ignore
> them.
>   The warnings can be turned off by calling `GC_set_warn_proc` with
>   a procedure that ignores these warnings (e.g. by doing absolutely
>   nothing).
> --8<---------------cut here---------------end--------------->8---
>
> It's my understanding that either A) there's a leak or B) its a false
> positive.
>
> > i wanted to debug and trace because i know this algorithm expand a lot
> > expressions (perheaps too much) sometimes (but it is an NP-problem and
> > exponential, so perheaps stop the program is the only solution and run it
> > on more little data)
> > i do not think there could be a memory leak in a recursive scheme
> > program,
>
> Gotta be careful with a conservative GC I guess.  Your memory will get
> reclaim at some point, but it's not guaranteed when since any value on
> the C stack -- and global variables -- could reference your allocation
> by accident.  So if by accident there's a static constant value in C
> that has the value of an allocation by the GC, it will never reclaim it.
> Since it's the Scheme stack and I suppose it's tail call optimized, that
> should not impact the GC.
>
> The most important thing would be to check the memory usage of the
> program with a tool like `htop'.  If there's leak, you will see that the
> memory usage percentage keep increasing.
>
> --
> Olivier Dion
> oldiob.dev
>


reply via email to

[Prev in Thread] Current Thread [Next in Thread]