Lilypond ships with a text font as well as a music font. I agree that I suspect that currently Lilypond's text font does not actually define these Unicode music characters, so it falls back on the OS to find them. Why not just add copies of Emmentaler glyphs to the Lilypond text fonts for characters within the Unicode spec? That seems like a pretty reasonable suggestion to me...
I also don't think it's unreasonable to suggest that the default typography should look good. I hope we can agree that the output on staves 1 & 2 is significantly worse than the output on staff 3 (attached picture if it doesn't display inline for you).
Staves 1&2 are Unicode and default markup glyphs as in my original post (compiled on Windows). To get staff 3 the following is required:
\new Staff {
c'1^\markup\concat\vcenter { B \hspace #0.2 \fontsize #-1.5 \flat }
c'1^\markup\concat\vcenter { C \hspace #0.1 \fontsize #-2 \sharp }
c'1^\markup\concat\vcenter { D \hspace #0.2 \fontsize #-1.5 \natural }
}
IMO Staff 3 should be the default output, not something that requires so much tweaking.