Re: [lwip-users] How to release all memory to track leaks

[Erik Ekman]

The lwIP unit tests use this code to verify that all memory has been returned:
https://git.savannah.nongnu.org/cgit/lwip.git/tree/test/unit/lwip_unittests.c#n48

On Fri, 4 Jun 2021 at 14:30, Dave Nadler <drn@nadler.com> wrote:
>
> R. Diez - While you are doing bare metal, you can find more info about newlib 
> memory use here:
> https://nadler.com/embedded/NXP_newlibAndFreeRTOS.html
> Hope that helps,
> Best Regards, Dave
>
> On 6/4/2021 7:43 AM, Trampas Stern wrote:
>
> In C there is a _sbrk(int incr) function.  This function is called when 
> memory allocation or deallocation happens.
> The newlib has their own version of this function, however via linker with 
> the --specs=nosys.spec you can disable the default version and add your own.
>
> You can google sbrk for more information.....
>
> What I do is implement a break point in the function to see who is allocating 
> memory. I also keep track of the memory allocation as the program runs.
>
> What I have found is that newlib memory allocation does not always free 
> memory and it also allocates memory if you are using floating point printf(). 
>  What happens is newlib has it's own internal memory management system.  For 
> example if you malloc(4) newlib might actually request a block of more than 4 
> bytes then when you free those 4 bytes newlib might not actually release 
> those bytes and instead mark the block as free internally and reuse it for 
> another malloc().
>
> What I do in my programs is implement a serial port command line interface 
> using a debug uart. Then I have a memory command where I print out the stack 
> and heap usage. I can also monitor the stack and heap usage as my program 
> runs to help handle stack and heap overflow more gracefully if needed. I 
> track maximum heap used as well as current heap usage.
>
> I have found that libraries like newlib standard C library will often malloc 
> memory.  This is highly dependent on the functions you use. For example if I 
> recall correctly when you enabled floating point on newlib it will allocate 
> memory when you do a floating point printf().  To avoid this and issues with 
> reentrant on printf()/sprintf() I just implemented my own version.  Every 
> line of code in the project becomes your problem, even standard C library 
> calls.
>
> As far as releasing all memory for lwip, I find this to be not needed. The 
> most important thing is to know what memory is being used and that it is not 
> growing. For example in C you could do this:
>
> uint8_t data[512];
> void main(){}
>
> This creates room in RAM for 512 bytes of data that is never released.  
> However this does the same thing:
>
> uint8_t *data;
> void main(){
> data=malloc(512);
> }
>
> That is both programs now need 512 bytes of RAM which is never released, the 
> only difference is one is in heap while the other is in the static allocation 
> (bss) section of RAM.   Hence malloc is not evil as many people have said, 
> rather it is a tool you have to know how to use correctly.
> I will say that 99% of the time the code I see that uses malloc() is evil, so 
> there exists a strong correlation to bad code and malloc, but it is not 
> memory allocation's (malloc) fault.  It is always the programmer not 
> understanding what they are doing in the code.
>
> Note I monitor stack and heap usage, not just for my code, but also for the 
> next guy.  For example projects live long after I am gone, so if the next 
> programmer inserts code that overflows stack or heap it is good that the 
> monitoring catches the error and lets them know before they release code to 
> production, i.e. defensive coding techniques.
>
> My sbrk function is below, but depends on the linker variables __HeapBase, 
> and __HeapLimit to work.  Note I often add a macro,  HALT_IF_DEBUGGING(), 
> which will insert a breakpoint into the code when a debugger is attached to 
> help track down which functions are doing memory allocations.
>
> Thanks
> Trampas
>
>
> extern uint32_t __HeapBase;
> extern uint32_t __HeapLimit;
> uint32_t getHeapUsed(void);
> uint32_t getHeapSize(void);
> int32_t getLastMalloc(void);
>
> uint32_t heapUsed=0;
> int32_t lastMalloc=0;
> uint32_t getHeapUsed(void)
> {
> return heapUsed;
> }
>
> uint32_t getHeapSize(void)
> {
> return (uint32_t)&__HeapLimit-(uint32_t)&__HeapBase;
> }
>
> int32_t getLastMalloc(void)
> {
> return lastMalloc;
>
> }
>
>
> static unsigned char *heap = (unsigned char *)&__HeapBase;
> static unsigned char *max_heap = (unsigned char *)&__HeapBase;
>
> uint32_t getMaxHeapUsed(void)
> {
> if (max_heap == NULL)
> {
> return 0;
> }
> return (uint32_t)max_heap - (uint32_t)&__HeapBase;
> }
>
> uint8_t *getHeapPointer(void)
> {
>
> return heap;
> }
> extern caddr_t _sbrk(int incr)
> {
>
> unsigned char *prev_heap;
>
> prev_heap = heap;
>
> heap += incr;
>
> if (heap > max_heap)
> {
> max_heap=heap;
> }
> lastMalloc = incr;
> heapUsed = (uint32_t) heap - (uint32_t) ((unsigned char *) &__HeapBase);
>
> //HALT_IF_DEBUGGING();//if you hit this something allocated memory, which 
> maybe you do not want...
>
> //for (;;) { }
>
> return (caddr_t) prev_heap;
> }
>
> On Fri, Jun 4, 2021 at 6:14 AM R. Diez via lwip-users <lwip-users@nongnu.org> 
> wrote:
>>
>> Hi all:
>>
>> I have a bare-metal (Newlib, no threads) firmware that uses lwIP and its 
>> httpd server to provide a simple web interface.
>>
>> The firmware is actually quite complex in other areas. I am trying to 
>> identify and track down memory leaks, so I implemented a "shutdown" command 
>> in the debug console that releases all resources and checks the remaining 
>> allocated bytes with mallinfo().
>>
>> I suspect some of the remaining memory "leaks" belong to lwIP and/or its 
>> httpd server, because they only come up after having downloaded the web page 
>> once. They are probably not real leaks, because memory usage does not grow 
>> over time.
>>
>> It is hard to say, because I have not figured out yet how to track memory 
>> allocations in an embedded Newlib firmware like this.
>>
>> In any case, I would like lwIP to release all memory on shutdown, in order 
>> to locate any memory leaks in other parts. The trouble is, there are 
>> routines like lwip_init() and httpd_init(), but no xxx_terminate() 
>> counterparts.
>>
>> I tried removing all interfaces with netif_remove(), but that is probably 
>> not enough.
>>
>> Is there some trick I could use? For example, something like ticking all 
>> state machines again after all interfaces have been removed could possibly 
>> trigger a complete memory release all over the place.
>>
>> Thanks in advance,
>>   rdiez
>>
>> _______________________________________________
>> lwip-users mailing list
>> lwip-users@nongnu.org
>> https://lists.nongnu.org/mailman/listinfo/lwip-users
>
>
> _______________________________________________
> lwip-users mailing list
> lwip-users@nongnu.org
> https://lists.nongnu.org/mailman/listinfo/lwip-users
>
>
> --
> Dave Nadler, USA East Coast voice (978) 263-0097, drn@nadler.com, Skype
>  Dave.Nadler1
>
> _______________________________________________
> lwip-users mailing list
> lwip-users@nongnu.org
> https://lists.nongnu.org/mailman/listinfo/lwip-users