If you're running emacs27 and later, and are still running comparative timings, try putting something like this into early-inti.el:
(push '(font . "Anonymous Pro-16") default-frame-alist)
This seemed to do the job of avoiding redraw/flicker when I set it up, but I'm using emacs almost entirely under unusual circumstances (involving VM's, multiple window systems, git head vs. pgtk branch, etc.) these days, and I don't have a good way to test a simple unix+X11 setup right now.
Hope that helps,
~Chad