[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: GC Warning: Repeated allocation of very large block
From: |
Damien Mattei |
Subject: |
Re: GC Warning: Repeated allocation of very large block |
Date: |
Mon, 19 Sep 2022 19:26:15 +0200 |
thank you, i think my algorithm is right but can be improved, i have to
work on it,activate tracing to see where is the big expression list and if
it is possible to correct that. There is a main loop in code and the
warning only appear when data is bigger so i do not think of any memory
leak (only have alittle doubt on an hastable which is global variable and
reused:
(when debug-mode (display-nl "Quine-Mc-Cluskey:"))
{minterms-ht <- (make-hash-table)} ;; need to be cleared at each run
(init-hash-table-with-set-and-value minterms-ht minterms #f)
this :
;; the hash table for minterms, better to be a top-level definition,it's
nightmare otherwise...
(declare minterms-ht)
{minterms-ht <- (make-hash-table)} ;; Scheme+ syntax
expression create a new hash table with pointer minterms-ht
this is done at each loop in the for each Cn
and i should clear it before but there is no instruction to do that in the
SRFI 69:
https://srfi.schemers.org/srfi-69/srfi-69.html
???
I will also check the still running memory used by program.
Damien
On Mon, Sep 19, 2022 at 1:44 PM Olivier Dion <olivier.dion@polymtl.ca>
wrote:
> On Mon, 19 Sep 2022, Damien Mattei <damien.mattei@gmail.com> wrote:
> > is this message appearing when a single scheme variable reach a given
> > size?
>
> This message is from the bdwgc and not from Guile itself. From their
> documentation:
> --8<---------------cut here---------------start------------->8---
> The garbage collector generates warning messages of the form:
>
>
> Repeated allocation of very large block ...
> May lead to memory leak and poor performance
>
>
> when it needs to allocate a block at a location that it knows to be
> referenced
> by a false pointer. These false pointers can be either permanent (e.g.
> a static integer variable that never changes) or temporary. In the latter
> case, the warning is largely spurious, and the block will eventually
> be reclaimed normally. In the former case, the program will still run
> correctly, but the block will never be reclaimed. Unless the block is
> intended
> to be permanent, the warning indicates a memory leak.
>
> 1. Ignore these warnings while you are using GC_DEBUG. Some of the
> routines
> mentioned below don't have debugging equivalents. (Alternatively, write
> the
> missing routines and send them to me.)
> 2. Replace allocator calls that request large blocks with calls to
> `GC_malloc_ignore_off_page` or `GC_malloc_atomic_ignore_off_page`. You
> may
> want to set a breakpoint in `GC_default_warn_proc` to help you identify
> such
> calls. Make sure that a pointer to somewhere near the beginning of the
> resulting block is maintained in a (preferably volatile) variable as long
> as the block is needed.
> 3. If the large blocks are allocated with realloc, we suggest instead
> allocating them with something like the following. Note that the realloc
> size increment should be fairly large (e.g. a factor of 3/2) for this to
> exhibit reasonable performance. But we all know we should do that anyway.
>
>
> void * big_realloc(void *p, size_t new_size) {
> size_t old_size = GC_size(p);
> void * result;
> if (new_size <= 10000) return(GC_realloc(p, new_size));
> if (new_size <= old_size) return(p);
> result = GC_malloc_ignore_off_page(new_size);
> if (result == 0) return(0);
> memcpy(result,p,old_size);
> GC_free(p);
> return(result);
> }
>
>
> 4. In the unlikely case that even relatively small object (<20 KB)
> allocations are triggering these warnings, then your address space
> contains
> lots of "bogus pointers", i.e. values that appear to be pointers but
> aren't.
> Usually this can be solved by using `GC_malloc_atomic` or the routines
> in `gc_typed.h` to allocate large pointer-free regions of bitmaps, etc.
> Sometimes the problem can be solved with trivial changes of encoding
> in certain values. It is possible, to identify the source of the bogus
> pointers by building the collector with `-DPRINT_BLACK_LIST`, which will
> cause it to print the "bogus pointers", along with their location.
> 5. If you get only a fixed number of these warnings, you are probably
> only
> introducing a bounded leak by ignoring them. If the data structures being
> allocated are intended to be permanent, then it is also safe to ignore
> them.
> The warnings can be turned off by calling `GC_set_warn_proc` with
> a procedure that ignores these warnings (e.g. by doing absolutely
> nothing).
> --8<---------------cut here---------------end--------------->8---
>
> It's my understanding that either A) there's a leak or B) its a false
> positive.
>
> > i wanted to debug and trace because i know this algorithm expand a lot
> > expressions (perheaps too much) sometimes (but it is an NP-problem and
> > exponential, so perheaps stop the program is the only solution and run it
> > on more little data)
> > i do not think there could be a memory leak in a recursive scheme
> > program,
>
> Gotta be careful with a conservative GC I guess. Your memory will get
> reclaim at some point, but it's not guaranteed when since any value on
> the C stack -- and global variables -- could reference your allocation
> by accident. So if by accident there's a static constant value in C
> that has the value of an allocation by the GC, it will never reclaim it.
> Since it's the Scheme stack and I suppose it's tail call optimized, that
> should not impact the GC.
>
> The most important thing would be to check the memory usage of the
> program with a tool like `htop'. If there's leak, you will see that the
> memory usage percentage keep increasing.
>
> --
> Olivier Dion
> oldiob.dev
>