You can try using the s-maxage setting for Cache Control, ala:
s-maxage=<seconds>
Overrides max-age or the Expires header, but only for shared caches (e.g., proxies). Ignored by private caches.
I would definitely test it before investing too much effort; in a past life, I spent waaay too much time dealing with the general internet's understanding/respect for the refinements of communication between clients, servers, proxies, and caches. (tl;dr: The internet is more broken than you expect. Yes, even you, for nearly all values of "you".)
If s-maxage doesn't work, you might have more luck with max-age (which I guess is consistent in the number of hyphens?) instead.
You should be able to use the Apache "Header" directive, something like:
Header set Cache-control "s-maxage=86400, public"
put inside the relevant <Directory "/mumble/badges"> or <Filesmatch "\.svg$"> or whatnot wrapper.
I haven't actually done this in a very long time, but a quick net-search suggests that it's still basically as functional. Hope this helps,
~Chad