Re: Against sqlite3!!!

[Tassilo Horn]

Qiantan Hong <qhong@mit.edu> writes:

>> I'm not very knowledgable with cl-* stuff but doesn't your
>> implementation load all key-value pairs at once?  That would be quite
>> a disadvantage compared to a DB approach where I'd naturally expect
>> that only the value I'm asking for is loaded.
> It does, but I’ve done some benchmark and it loads 10k entries in
> 0.02~0.03 seconds. 100k entries takes <0.5s.

But your values are very only small lists.

> I’d say it should be suffice for most Emacs application I know of.
> Nobody is using Emacs for trillions of business records.

I'd imagine that if such a feature became available, packages would
start using it and store more data that they do now for whatever
reasons.  Like eww/elpher could want to store the list of the last 100
visited URLs.

>> Also, how would it ensure consistency when I have 2 parallel emacs
>> sessions (like one for mail/irc and one for programming/editing) where
>> session 1 modifies the value of key A and the other of key B?  It looks
>> like the values of the kv-store that gets saved later will win.  In case
>> that's the emacs session which has modified B, it'll revert A to the
>> state before the other session modified it, no?
>
> Since it records a log of deltas instead of printing the whole data
> structure, different key won’t interfere.

Ah, allright.  By skimming the code I've first thought the log would
only be for analysis/debugging purposes.

> Being said that, currently it probably won’t work because UNIX append
> is not atomic and will probably be interleaved into nonsense.
> There’re various workarounds, lock file being one, but I like the idea
> of keeping only one “controller” instance with exclusive access to the
> file more.

Interesting, but how would emacs instances interact with that controller
instance?  And would that controller instance simply be the first emacs
instance of a user?  What if the controller blocks because Gnus is
running and currently downloading mails with huge attachments?

>> If that were true, I'd say your resist!.el is a non-starter in the
>> current form.  It should at least load only values explicitly asked for
>> and only persist/override actually changed values.  A trivial solution
>> could use one file per key.  Not sure how sensible that is.
> It does the latter.

No, it uses one file per kv-store but you can have as many kv-stores as
you like, e.g., one per package.  Or do you mean "it only overrides
changed values" with "the latter"?  Indeed, that's true.  I've played
with the code and now understand it.  Nice! :-) (kv-rem is missing a
kv-store arg.)

> As I’ve mentioned, from the benchmark results, the former doesn’t
> seem to be a big problem. You’ll do it at most once for every Emacs
> instance anyway.

Yeah, and since it's no global store in the sense of "every package
feeds its data into the same store", my argument is void.  If my emacs
session doesn't start Gnus, Gnus won't load its own store.

Bye,
Tassilo