[lwip-users] Re: [lwip] pbuf alloc/free locks

[Adam Dunkels]

Hi Kieran!

On Friday 22 February 2002 18.44, you wrote:
> I'm having problems with the pbuf_pool_free_lock and pbuf_pool_alloc_lock.
> They're doing what they're supposed to, and stopping access to the
> pbuf_pool from two places at the same time (pbuf_refresh and
> pbuf_pool_alloc), but when pbuf_pool_alloc is prevented from getting
> access to the pool it pretends it is out of buffers and returns NULL.
>
> This is where thing start to go wrong as the code that has called
> pbuf_alloc doesn't then know if it has failed due to a real lack of
> memory, or if it's just not been able to get a lock, and it wants to
> behave differently in these two situations.
>
> If it were a rare occurence I'd be happy to just try again, but it seems
> to happen quite frequently (approx 1% of the time), and trying again won't
> help if the thread that has the lock hasn't been run in the meantime.
>
> (As an aside, it seems rather unlucky that pbuf_refresh is getting
> interrupted so frequently (which it must be if pbuf_pool_alloc can't
> obtain the lock) - perhaps there is a bug elsewhere, or a simple way to
> restructure the code so this doesn't happen).
>
> Ideally, in the case where it has buffers but can't get the lock, I'd
> like it to block until the lock is available.  However, there might be
> reasons why it's not done like this, so I thought I'd ask first.  (It
> would require some reworking to do this at it would be at risk of deadlock
> as it currently stands)

It is interesting to hear that as much as 1% of the allocations gets locked 
out. When I run it in Unix simluation under FreeBSD, I never get any lockouts 
because of the pthreads implementation for FreeBSD, which appears to be 
non-preemptive. Perhaps this depends on the application as well? You 
mentioned that you were using lwIP for high-speed networks, so your 
applications perhaps allocate and free alot of pbufs? I'll go over the code 
again when I get the time, and see if there is anything that can be done 
about it.

One way to distinguish between an out-of-memory situation and a lockout would 
be to let pbuf_alloc() return a special PBUF_ALLOC_LOCKED value when 
allocation fails because of a lockout. The reason for not blocking in 
pbuf_alloc() is that it should be a very quick function suitable for use in a 
high-priority thread and that the caller could implement a retry mechanism 
instead of doing this in pbuf_alloc(). A retrying mechanism could look like 
this, for instance:

p = pbuf_alloc();
while(p == PBUF_ALLOC_LOCKED) {
  yield();
  p = pbuf_alloc();
}

/adam
-- 
Adam Dunkels <address@hidden>
http://www.sics.se/~adam
[This message was sent through the lwip discussion list.]