[PATCH 0/5] refactor command name parsing

[Ben Pfaff]

This is a respin of the patches that I sent out in September under
the title of "pre-lexer-2".  It adds UTF-8 support, at John's
suggestion.

Ben Pfaff (5):
  str: Inline some trivial functions.
  str: Change "char" to "byte" in function names.
  str: Add some functions for handling UTF-8.
  identifier: Add some functions for Unicode syntax.
  command: Factor command name matching out of command.c.

 Smake                                    |    4 +
 src/data/data-in.c                       |   76 +++---
 src/data/dictionary.c                    |    2 +-
 src/data/format-guesser.c                |   18 +-
 src/data/identifier.c                    |   82 ++++++-
 src/data/identifier.h                    |    9 +-
 src/data/settings.c                      |   10 +-
 src/data/sys-file-writer.c               |   16 +-
 src/language/command.c                   |  385 +++++-------------------------
 src/language/control/repeat.c            |    8 +-
 src/language/data-io/data-list.c         |    4 +-
 src/language/data-io/data-parser.c       |   14 +-
 src/language/data-io/data-reader.c       |    6 +-
 src/language/data-io/get-data.c          |    4 +-
 src/language/data-io/print.c             |    6 +-
 src/language/expressions/operations.def  |    2 +-
 src/language/expressions/parse.c         |    4 +-
 src/language/lexer/automake.mk           |    5 +-
 src/language/lexer/command-name.c        |  235 ++++++++++++++++++
 src/language/lexer/command-name.h        |   46 ++++
 src/language/lexer/format-parser.c       |   10 +-
 src/language/lexer/lexer.c               |   18 +-
 src/language/syntax-string-source.c      |    2 +-
 src/language/utilities/host.c            |    2 +-
 src/language/utilities/set.q             |   14 +-
 src/language/utilities/title.c           |    4 +-
 src/libpspp/argv-parser.c                |    8 +-
 src/libpspp/message.c                    |    2 +-
 src/libpspp/model-checker.c              |    4 +-
 src/libpspp/str.c                        |  306 ++++++++++++++----------
 src/libpspp/str.h                        |   59 ++++-
 src/output/ascii.c                       |    4 +-
 src/ui/gui/text-data-import-dialog.c     |   36 ++--
 src/ui/syntax-gen.c                      |   20 +-
 tests/automake.mk                        |   17 ++-
 tests/language/command.at                |    2 +-
 tests/language/data-io/inpt-pgm.at       |    4 +-
 tests/language/lexer/command-name-test.c |  149 ++++++++++++
 tests/language/lexer/command-name.at     |  234 ++++++++++++++++++
 39 files changed, 1200 insertions(+), 631 deletions(-)
 create mode 100644 src/language/lexer/command-name.c
 create mode 100644 src/language/lexer/command-name.h
 create mode 100644 tests/language/lexer/command-name-test.c
 create mode 100644 tests/language/lexer/command-name.at