bug#32234: Cuirass: The SQLite built in busy handler might block the Fibers scheduler

[Clément Lassieur]

Ludovic Courtès <address@hidden> writes:

> Clément Lassieur <address@hidden> skribis:
>
>> Ludovic Courtès <address@hidden> writes:
>
> [...]
>
>>> Perhaps instead we need to set the timeout to a small value and handle
>>> SQLITE_BUSY at the call site in our code.  We could define a macro that
>>> automatically retries upon SQLITE_BUSY.
>>
>> That would limit the issue to the first timeout span: for that short
>> time the scheduler would be blocked.  I think a timeout of 0 would be
>> better.
>
> Yes, 0 is an acceptable “small value.”  ;-)  Perhaps 100ms would be
> acceptable if the situation is rare enough, dunno.
>
>> Another solution would be to serialize all the database accesses as we
>> do already with the url handler, and stop using the SQLITE
>> multithreading features.  It would probably make the code simpler
>> because we would use the same paradigm everywhere, and we would avoid
>> looping until SQLITE isn't busy at each request.
>
> In essence we’d introduce a “database server” running as a fiber, and
> everyone would talk to that server.

Why not as a thread?  There would be only one thread dedicated to SQLITE
(thus adding a constant overhead), and that would prevent the Scheduler
from being bloqued by long SQLite queries.

> I considered doing that before but then though sqlite would probably be
> able to do better than this, but I don’t know.

And it's hard to test...

> What’s a bit annoying with switching to a database server model is that
> we’d need to adapt every call site.
>
> Thoughts?

I'm a tiny bit in favor of switching to a database server model because
it's more consistent overall (we already use it for the url handler) and
the scheduler wouldn't be bloqued by long SQLite queries.

It's annoying to adapt every call site, but it would be done once and
for all. :-)

So... as you wish!

Clément