bug#32234: Cuirass: The SQLite built in busy handler might block the Fibers scheduler

[Ludovic Courtès]

Clément Lassieur <address@hidden> skribis:

> Ludovic Courtès <address@hidden> writes:
>
>> Clément Lassieur <address@hidden> skribis:
>>
>>> Ludovic Courtès <address@hidden> writes:
>>
>> [...]
>>
>>>> Perhaps instead we need to set the timeout to a small value and handle
>>>> SQLITE_BUSY at the call site in our code.  We could define a macro that
>>>> automatically retries upon SQLITE_BUSY.
>>>
>>> That would limit the issue to the first timeout span: for that short
>>> time the scheduler would be blocked.  I think a timeout of 0 would be
>>> better.
>>
>> Yes, 0 is an acceptable “small value.”  ;-)  Perhaps 100ms would be
>> acceptable if the situation is rare enough, dunno.
>>
>>> Another solution would be to serialize all the database accesses as we
>>> do already with the url handler, and stop using the SQLITE
>>> multithreading features.  It would probably make the code simpler
>>> because we would use the same paradigm everywhere, and we would avoid
>>> looping until SQLITE isn't busy at each request.
>>
>> In essence we’d introduce a “database server” running as a fiber, and
>> everyone would talk to that server.
>
> Why not as a thread?  There would be only one thread dedicated to SQLITE
> (thus adding a constant overhead), and that would prevent the Scheduler
> from being bloqued by long SQLite queries.

Right, if we want to have a big timeout, it should be a thread, not a
fiber.  Sorry for the confusion.

>> I considered doing that before but then though sqlite would probably be
>> able to do better than this, but I don’t know.
>
> And it's hard to test...
>
>> What’s a bit annoying with switching to a database server model is that
>> we’d need to adapt every call site.
>>
>> Thoughts?
>
> I'm a tiny bit in favor of switching to a database server model because
> it's more consistent overall (we already use it for the url handler) and
> the scheduler wouldn't be bloqued by long SQLite queries.
>
> It's annoying to adapt every call site, but it would be done once and
> for all. :-)

Indeed.  If you want to take that route, that’s fine with me!

Thank you,
Ludo’.