[lwip-users] RFC: TCP issue with PBUF_REF [forwarded]

[Leon Woestenberg]

Hello all,

I have forwarded a mail from Anders Carlman describing
an issue where you might think PBUF_ROM should be replaced
by PBUF_REF, but where PBUF_ROM actually is more
efficient (and still safe). See bottom. Here is my interpretation
and further discussion:

When data is submitted to the TCP module for transmission,
using the non-copy method, it is placed in pbufs of type
PBUF_REF:

tcp_out.c/tcp_enqueue()/line 188

    else {
      /* Do not copy the data. */

      /* First, allocate a pbuf for holding the data. */
      if((p = pbuf_alloc(PBUF_TRANSPORT, seglen, PBUF_REF)) == NULL) {
        DEBUGF(TCP_OUTPUT_DEBUG, ("tcp_enqueue: could not allocate memory
for pbuf non-copy\n"));

The data referenced by the pbufs is static and constant during the complete
lifetime
that they live in lwIP. That is because the pbufs are freed once the remote
side has
ACKed them and the ACK is received on the local side.

Even if pbuf end up on the etharp.c queue for transmission, lwIP can safely
assume
that the data is available, constant and static. Having etharp.c call
pbuf_unref() will
therefore make unnecessary copies of PBUF_REF and type could as well be
PBUF_ROM saving some precious RAM.

Just for clarity, I think this must be in the documentation:

PBUF_ROM means:
- references constant data that has an infinite, or lwIP controlled,
life-time.

PBUF_REF means:
- references data that is available, and constant, only during the
function call that submitted the data to lwIP, but no longer.


OK, now for something I haven't fully thought out yet: what if a TCP
connection is
aborted by the application?

The associated pbufs are freed (their reference count is lowered by one, and
only
freed when reaching zero), but any pbuf on the ARP queue may still be sent
out.
Any pbuf of type PBUF_ROM may still refer to the application-provided
non-copy
data, which might by now be non-available now the application has aborted.

So, my thought is that pbuf_free() must actually traverse the ARP queue and
remove
the pbuf from the queue if it referenced there.

More simply said: the ARP queue does not know when its queued pbufs are
freed, so
the free'ing party must remove the pbufs from the ARP queue.

As usual, I welcome any feedback on this!

Regards,

Leon Woestenberg.

----- Original Message -----
From: "Anders Carlman"
Sent: Thursday, March 27, 2003


...

I've been looking at the changes in the PBUF-code and in the code that uses
pbuf. The changes in the UDP-code that uses PBUF:s makes sence, but the use
of the PBUF_REF in TCP-communications confuses me... As I understand it
TCP-pbufs aren't relased until they have been ACKed anyway, so copying the
contents for the pbuf (which is the consequence of using the PBUF_REF type)
seems to be quite unnecessary...
When the data is ACKed, the arp-queue is definitely done with the it, so to
me the change from PBUF_ROM to PBUF_REF in line 192 in tcp_out.c, seems to
cause nothing but unnessecary copying and memory usage...
If there's a good reason for this, that I have missed, I would be glad if
you could explain it to me...

Regards
Anders Carlman