At least for now I'm giving up on this and using png images, if it isn't displaying dependably for me it probably won't for some percentage of my site users. Probably fixable with CSS but I can't be bothered to mess with it, this has already taken way too long.
--
Seems to be related to firefox font scaling. I have my browser font
set larger as many pages are unreadable on this monitor without doing
that. It appears that crome does not apply browser font size to SVG
while firefox does.
I have also tested this using an SVG
from abcm2ps and it also displays fonts larger than intended, but not to
the extent that lilypond files are. abcm2ps is showing about 2x size,
lilypond is more like 10x for some reason.