Function of nexttoken()

[Abdillah Fa'iz Hernawan]

I was exploring gnu global source code and find that it is
`nexttoken()` function which mostly essential.
I wonder a few question on it,
1. What the definition of token is?
2. And surely, why we need a list of `const char`, "interested" chars?
What's the relation between interested chars and a token?

Well, I have observing the source, but not yet understand of it.

Thanks.

Note : Sorry, as I don't find any mailing list suitable for a question
like this, I use a bug-mail to sent this to.