bug#33275: 27.0.50; Image cache pruning

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> I think it's reasonable to expect 2GB worth of image files to take
> several times that in memory.  (Btw, visiting a 16GB file usually
> takes twice that much during insert-file-contents.)

It would be reasonable if the code had asked for those images to be kept
in memory, but it hasn't.  Emacs decides that on its own without saying
that it's doing that.

It'd be similarly surprising if

(dolist (file (directory-files "/directory/with/many/images" t "png$"))
  (with-temp-buffer
    (insert-file-contents-literally file)))

were to lead to Emacs growing uncontrollably.

At present, the image cache is purely based on timestamps, if I recall
correctly?  If we wish to fix this unexpected behaviour, we could also
look at the size of the image cache and perhaps try to limit it, or if
we wanted to be really fancy, we could somehow check whether there's any
references to the images anywhere (beyond the image cache itself).

I think the latter would mean adding some... fun... to the gc system?

Or does Emacs have weak hashes?  In which case the image cache could be
a weak hash and the problem would resolve itself?

> I think we should try to add some code that would call
> display_malloc_warning and/or memory_full, before the system runs out
> of memory.  That should be enough to prevent OOM in such cases.  Can
> you spot why we never called memory_full in this case?

Sorry, I'm not familiar with the memory_full stuff...

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no