Hmm...
Does evaluating the following (with M-:) crash on your system?
(let ((str "The quick brown fox jumps over the lazy dog
´`''\"\"1lI|¦!Ø0Oo{[()]}.,:; "))
(insert (propertize str
'font-lock-face '(:family "charter")) "\n")
(insert (propertize str
'font-lock-face '(:family "charter" :slant italic))))
It doesn't crash here, and so I wonder what happens on your system
that causes the face's fontset ID to become -1. If you can figure
that out, maybe it will help; or maybe someone else will be able to
reproduce and investigate.
(I tried to do this with Caladings CLM font, but all the fonts I find
of that family are AFM or PFA/PFB fonts, and I cannot use them.)