RE: [lwip-devel] Packet queues (was Re: [bug #11400]ARP multi-packet-queue modifies TCP ...)

[Goldschmidt Simon]

Hi,

sorry I was posting to the bug yesterday evening, I just did not see these mails...

>The issue being that most sensible drivers will not poll and wait for the
>ethernet driver transmit to complete, but instead will hand off the packet
>data to the driver. Some of my hardware drivers copy the data first due to
>hardware restrictions, but that's time-consuming and more importantly takes
>more memory - one MTU size buffer for each tx buffer I allow outstanding.
>For the others I just point the driver at the pbuf chain, after having
>given it a pbuf_ref() first; but now I realise that is insufficient since
>control may be returned to the application before the packet has truly been sent.

I solved this in 2 ways:
1. I call pbuf_ref() on each pbuf I'm sending, calling pbuf_free() when sending is finished. For that to work, a TX-interrupt is needed and I need to work out the pbuf pointer from the data pointer (can be tricky).
That's why I was pushing on not to change a pbuf's contents while ref > 1.
2. With a PBUF_REF, I simply 'hope' the ethernet hardware is fast enough (I have only 20 packets on queue max, that's about 2.5 ms to send all packets)
This is true for _my_ system (which is a slow Altera NIOS-II @ only 60 MHz), also because the queue is empty most of the time which means the packet is sent with no delay. But it can (and will) be different on faster systems or 10-mbit ethernet.

I agree that implementing a new pbuf type is simpler (and also less intrusive to the rest of the stack) than changing the function parameters.


Generally, does it mean we have to copy each PBUF_REF (and PBUF_RAM coming from the application?) using memcpy() in the ethernet driver?
If so, it may be faster to wait in the network driver until the packet is finished sending... Not that I favour it, but copying using the processor is a lot slower in my system...

>>True, but there are very few standard APIs that are capable of it, which
>>is probably why there hasn't been much concern about it.

>But I had thought it was meant to be a feature of the lwip API :).

And if others can do it, why can't we? :)

>>Zero-copy
>>through the sockets API is nigh on impossible for example.

On RX side, it's impossible, but on TX side, why not? As long as the data is not being queued... (that takes out TCP, of course!)

>That's why the sockets API isn't very good for low memory use. But even
>there it is valuable - otherwise you risk copying not just once but twice:

It's not the memory I fear but the time the CPU spends copying!

Again, I think the different types of pbufs stated by Jonathan are a good idea. That way, the network driver can figure out if copying is necessary or not.
And provided that we supply a good documentation and spread it with the next releas and on lwip-users, the user is responsible for choosing the right pbuf type.
I think that's the best solution.