Would allowing a regexp solve the expression \alpha-\beta,
\alpha+\beta, \alpha/\beta, so that \alpha and \beta are prettified
seperately?
That's not necessary. Just follow Stefan's advice:
(defun texinfo-prettify-symbols-compose-p (start end _match)
;; We know the matches all start with a backslash and end with
;; a word-element.
(not (or (memq (char-before start) '(?\\))
(memq (char-syntax (or (char-after end) ?\s)) '(?w))
(nth 8 (syntax-ppss)))))
(add-hook 'texinfo-mode-hook
(lambda ()
(push '("\\alpha" . ?α) prettify-symbols-alist)
(push '("\\beta" . ?β) prettify-symbols-alist)
(setq prettify-symbols-compose-predicate
'texinfo-prettify-symbols-compose-p)
(prettify-symbols-mode)))
This does what you want.