Re: [lwip-users] ERR_MEM when sending large data, LWIP stats show no error

[Adrian Figueroa]

Hi Noam,

 

Thanks for your comments. I am using the RAW API.

 

The complete data array I want to send is 32k. I use individual packets of 1460 bytes each.

 

> I think that you have set a too large window size for the mem size you have.

 

You mean TCP_WND? It looks like only a fraction of my MEM_SIZE is actually used, judging from the lwIP stats output.

 

> If you work with RAW API you can and probably need to use the sent call-back to synchronize

when to send more data.

 

In that case I need to wait for the ACK of the server. I need to send data as fast as possible, so I want to send quickly without ACKs. As it is now, my Windows TCP server sends an ACK after two received packets only.

 

> you probably are sending data from outside the LwIP context

 

that should not be the case.

 

Let me briefly summarize how I send data.

 

1.)    Initiate connection with TCP server

2.)    Start transfer of data

3.)    If data amount > MSS remaining, transmit with tcp_write() and go to 3.) again, else go to 4.)

4.)    Use one last tcp_write() and tcp_output() to force out any last data

5.)    Wait for new data, if available go to 2.)

 

Find the full source code here: https://snip.coda.pw/?b582a6a527039785#71em5UiMnc13djv7QNH6H+OJ+t6ibUd0svsMWkiq+Z0=

 

Basically, the included “start_transfer()” is called in my main loop along with “sys_check_timeouts()” and “ethernetif_input(&gnetif)”. At some other place in my application, “csp->state” is set to “CS_DATA” for signaling the availability of new data. Then, transfer starts with “conn_send_next()”. This is the main transmit function.

 

Can you recommend some memory settings for my usecase? In the connection between client and server, only the client will ever transmit data. Large data arrays of 32k or more will be transmitted in blocks of 1460 bytes.

 

Thank you again,

Adrian

 

Von: lwip-users [mailto:lwip-users-bounces+address@hidden Im Auftrag von Noam Weissman
Gesendet: Friday, September 1, 2017 2:09 PM
An: address@hidden
Betreff: Re: [lwip-users] ERR_MEM when sending large data, LWIP stats show no error

 

Hi Adrian,

 

Are you using RAW, netcon or Socket API ?

 

I think that you have set a too large window size for the mem size you have.

You should not send 32K blocks in one go. You need to send smaller block, say 1K or 1.5K,

check available memory of send buffer and if available send another chunk of data.

 

If you work with RAW API you can and probably need to use the sent call-back to synchronize

when to send more data.

 

I suggest checking some examples and incorporate ideas from them.

 

Memory errors are normally happening when one does not follow the LwIP threading

requirements. If you are using RAW API and your module is the client (initiating the connection)

you probably are sending data from outside the LwIP context, you are not using it as it should be.

 

Hope that helps,

Noam.

 

---

From: lwip-users <lwip-users-bounces+address@hidden> on behalf of Adrian Figueroa <address@hidden>
Sent: Friday, September 1, 2017 11:15 AM
To: address@hidden
Subject: [lwip-users] ERR_MEM when sending large data, LWIP stats show no error

 

Hello all,

 

I am trying to send large amounts of data continuously. I use lwIP as a TCP client on an STM32F7 and Matlab on a PC as a TCP server.

 

I send blocks of 32768 bytes. Here are some settings for the memory sizes in lwIP:

 

#define MEM_ALIGNMENT 4

#define MEM_SIZE 17520

#define MEMP_NUM_TCP_SEG 65

#define PBUF_POOL_BUFSIZE 1516

#define TCP_WND 8760

#define TCP_MSS 1460

#define TCP_SND_BUF 8760

 

When sending with tcp_write(), I get ERR_MEM errors. So, I switched on LWIP_STATS for seeing memory statistics. When should I be using tcp_output()? Should I check for the remaining tcp_sndbuf and use tcp_output() if it gets low?

 

In Wireshark, I can see TCP Dup ACKs and “TCP Previous segment not captured” errors. Find the dump here:

 

https://sync.coda.pw/r/Z_uuIXRtl2#NT5OwZOSW4Nd8erfCRvLvSdAmMS1YODnWL3tP+dGRMk=

 

I send data every 2 seconds and it takes around 1.5 seconds to send it to the PC, which is too long.

 

Here are some of the results after ERR_MEM happened:

https://snip.coda.pw/?69c4627d150293fa#3cUX2gAqvAKaflXk8NTs7k+TxBAjQwmNT+IYu+GfiJE=

 

There is no Mem/Memp section that contains even one error. So, I am not sure where to look next.

 

Thank you in advance!

Adrian