[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
branch master updated: * tp/Texinfo/ParserNonXS.pm (_parse_texi_regex) (
|
From: |
Patrice Dumas |
|
Subject: |
branch master updated: * tp/Texinfo/ParserNonXS.pm (_parse_texi_regex) (_process_remaining_on_line), tp/Texinfo/XS/MiscXS.xs (xs_parse_texi_regex), tp/Texinfo/XS/parsetexi/menus.c (handle_menu), tp/Texinfo/XS/parsetexi/parser.c (process_remaining_on_line), tp/Texinfo/XS/parsetexi/separator.c (handle_separator): handle menu separator separatly from other separators (with a possible overlap for comma, considered both as a menu and other separator in the perl parser), also moving the code starting the menu_entry_se [...] |
|
Date: |
Thu, 02 Mar 2023 17:25:49 -0500 |
This is an automated email from the git hooks/post-receive script.
pertusus pushed a commit to branch master
in repository texinfo.
The following commit(s) were added to refs/heads/master by this push:
new ff02e6d456 * tp/Texinfo/ParserNonXS.pm (_parse_texi_regex)
(_process_remaining_on_line), tp/Texinfo/XS/MiscXS.xs (xs_parse_texi_regex),
tp/Texinfo/XS/parsetexi/menus.c (handle_menu), tp/Texinfo/XS/parsetexi/parser.c
(process_remaining_on_line), tp/Texinfo/XS/parsetexi/separator.c
(handle_separator): handle menu separator separatly from other separators (with
a possible overlap for comma, considered both as a menu and other separator in
the perl parser), also moving the code start [...]
ff02e6d456 is described below
commit ff02e6d456c825be70dff5f184f7a69014f0f01a
Author: Patrice Dumas <pertusus@free.fr>
AuthorDate: Thu Mar 2 23:25:39 2023 +0100
* tp/Texinfo/ParserNonXS.pm (_parse_texi_regex)
(_process_remaining_on_line), tp/Texinfo/XS/MiscXS.xs
(xs_parse_texi_regex), tp/Texinfo/XS/parsetexi/menus.c (handle_menu),
tp/Texinfo/XS/parsetexi/parser.c (process_remaining_on_line),
tp/Texinfo/XS/parsetexi/separator.c (handle_separator): handle menu
separator separatly from other separators (with a possible overlap for
comma, considered both as a menu and other separator in the perl
parser), also moving the code starting the menu_entry_separator
element together with the code handling menu entries.
* tp/Texinfo/ParserNonXS.pm (_process_remaining_on_line): code more
similar to XS parser code for accent argument without brace and for
menu_entry.
---
ChangeLog | 16 +++++++
tp/Texinfo/ParserNonXS.pm | 93 +++++++++++++++++++++++--------------
tp/Texinfo/XS/MiscXS.xs | 9 +++-
tp/Texinfo/XS/misc.c | 15 +++++-
tp/Texinfo/XS/miscxs.h | 1 +
tp/Texinfo/XS/parsetexi/menus.c | 23 ++++++++-
tp/Texinfo/XS/parsetexi/parser.c | 11 ++++-
tp/Texinfo/XS/parsetexi/separator.c | 18 -------
8 files changed, 125 insertions(+), 61 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index c13d6464f0..8d71ff3e8c 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,19 @@
+2023-03-02 Patrice Dumas <pertusus@free.fr>
+
+ * tp/Texinfo/ParserNonXS.pm (_parse_texi_regex)
+ (_process_remaining_on_line), tp/Texinfo/XS/MiscXS.xs
+ (xs_parse_texi_regex), tp/Texinfo/XS/parsetexi/menus.c (handle_menu),
+ tp/Texinfo/XS/parsetexi/parser.c (process_remaining_on_line),
+ tp/Texinfo/XS/parsetexi/separator.c (handle_separator): handle menu
+ separator separatly from other separators (with a possible overlap for
+ comma, considered both as a menu and other separator in the perl
+ parser), also moving the code starting the menu_entry_separator
+ element together with the code handling menu entries.
+
+ * tp/Texinfo/ParserNonXS.pm (_process_remaining_on_line): code more
+ similar to XS parser code for accent argument without brace and for
+ menu_entry.
+
2023-03-02 Patrice Dumas <pertusus@free.fr>
* tp/Texinfo/XSLoader.pm (init): call Texinfo::Parser->import() after
diff --git a/tp/Texinfo/ParserNonXS.pm b/tp/Texinfo/ParserNonXS.pm
index 81819e1ec1..1eb1d82daf 100644
--- a/tp/Texinfo/ParserNonXS.pm
+++ b/tp/Texinfo/ParserNonXS.pm
@@ -4490,12 +4490,13 @@ sub _parse_texi_regex {
# REMACRO
my ($at_command, $open_brace, $asterisk, $single_letter_command,
- $separator_match, $misc_text)
+ $separator_match, $menu_separator, $misc_text)
= ($line =~ /^\@([[:alnum:]][[:alnum:]-]*)
|^(\{)
|^(\*)
|^\@(["'~\@&\}\{,\.!\? \t\n\*\-\^`=:\|\/\\])
- |^([{}@,:\t.\f])
+ |^([{}@,\f])
+ |^([:\t.])
|^([^{}@,:\t.\n\f]+)
/x);
@@ -4503,10 +4504,12 @@ sub _parse_texi_regex {
$separator_match = $open_brace;
} elsif ($asterisk) {
($misc_text) = ($line =~ /^([^{}@,:\t.\n\f]+)/);
+ } elsif ($separator_match and $separator_match eq ',') {
+ $menu_separator = $separator_match;
}
return ($at_command, $open_brace, $asterisk, $single_letter_command,
- $separator_match, $misc_text);
+ $separator_match, $menu_separator, $misc_text);
}
sub _check_line_directive {
@@ -5029,7 +5032,7 @@ sub _process_remaining_on_line($$$$)
my $at_command_length;
my @line_parsing = _parse_texi_regex($line);
my ($at_command, $open_brace, $asterisk, $single_letter_command,
- $separator_match, $misc_text) = @line_parsing;
+ $separator_match, $menu_separator, $misc_text) = @line_parsing;
print STDERR "PARSED: "
.join(', ',map {!defined($_) ? 'UNDEF' : "'$_'"} @line_parsing)."\n"
if ($self->{'DEBUG'} and $self->{'DEBUG'} > 3);
@@ -5128,9 +5131,13 @@ sub _process_remaining_on_line($$$$)
# argument (an opening brace, or a character after spaces for
# accent commands) was not found and there is already a new command.
#
- # It would have been nice to allow for comments, but there is no
- # container in the tree to put them when after command and before brace
- # or argument for accent commands.
+ # NOTE the last element in the current command contents is an element that
+ # is transiently in the tree, and is put in the info hash by
+ # _gather_spaces_after_cmd_before_arg. It could therefore be possible
+ # to accept an @comment here and put it in this element. It would not
+ # necessarily be a good idea, as it would mean having an element
+ # in the info hash that holds something more complex than text and source
+ # marks.
if ($command
and $current->{'cmdname'}
and defined($self->{'brace_commands'}->{$current->{'cmdname'}})) {
@@ -5244,20 +5251,23 @@ sub _process_remaining_on_line($$$$)
# may possibly be empty in some specific case, without end of line.
} elsif ($accent_commands{$current->{'cmdname'}}
and $line =~ s/^([^@])//) {
- print STDERR "ACCENT following_arg \@$current->{'cmdname'}\n"
+ my $arg_char = $1;
+ print STDERR "ACCENT following_arg $arg_char \@$current->{'cmdname'}\n"
if ($self->{'DEBUG'});
if ($current->{'contents'}) {
_gather_spaces_after_cmd_before_arg($self, $current);
}
my $following_arg = {'type' => 'following_arg',
'parent' => $current};
- $following_arg->{'contents'} = [{ 'text' => $1,
- 'parent' => $following_arg } ];
$current->{'args'} = [ $following_arg ];
- if ($current->{'cmdname'} eq 'dotless' and $1 ne 'i' and $1 ne 'j') {
+ my $accent_arg = { 'text' => $arg_char, 'parent' => $following_arg };
+ $following_arg->{'contents'} = [ $accent_arg ];
+
+ if ($current->{'cmdname'} eq 'dotless'
+ and $arg_char ne 'i' and $arg_char ne 'j') {
$self->_line_error(sprintf(
- __("%c%s expects `i' or `j' as argument, not `%s'"),
- ord('@'), $current->{'cmdname'}, $1),
+ __("\@dotless expects `i' or `j' as argument, not `%s'"),
+ $arg_char),
$source_info);
}
$current = $current->{'parent'};
@@ -5276,7 +5286,7 @@ sub _process_remaining_on_line($$$$)
and ($current->{'parent'}->{'type'} eq 'menu_comment'
or $current->{'parent'}->{'type'} eq 'menu_entry_description')
and $asterisk
- and @{$current->{'contents'}}
+ and $current->{'contents'}
and $current->{'contents'}->[-1]->{'type'}
and $current->{'contents'}->[-1]->{'type'} eq 'empty_line'
and $current->{'contents'}->[-1]->{'text'} eq '') {
@@ -5298,7 +5308,8 @@ sub _process_remaining_on_line($$$$)
# this is the menu star collected previously
my $menu_star_element = _pop_element_from_contents($self, $current);
$line =~ s/^(\s+)//;
- my $leading_text = '*' . $1;
+ my $star_leading_spaces = '*' . $1;
+
if ($current->{'type'} eq 'preformatted'
and $current->{'parent'}->{'type'}
and $current->{'parent'}->{'type'} eq 'menu_comment') {
@@ -5324,18 +5335,21 @@ sub _process_remaining_on_line($$$$)
# close menu_entry (which cannot actually be empty).
$current = _close_container($self, $current);
}
- push @{$current->{'contents'}}, { 'type' => 'menu_entry',
- 'parent' => $current,
- };
- $current = $current->{'contents'}->[-1];
- $current->{'contents'} = [ { 'type' => 'menu_entry_leading_text',
- 'text' => $leading_text,
- 'parent' => $current },
- { 'type' => 'menu_entry_name',
- 'parent' => $current } ];
+
+ my $menu_entry = { 'type' => 'menu_entry',
+ 'parent' => $current,
+ };
+ my $leading_text = { 'type' => 'menu_entry_leading_text',
+ 'text' => $star_leading_spaces,
+ 'parent' => $menu_entry };
# transfer source marks from removed menu star to leading text
- _transfer_source_marks($menu_star_element, $current->{'contents'}->[0]);
- $current = $current->{'contents'}->[-1];
+ _transfer_source_marks($menu_star_element, $leading_text);
+ my $entry_name = { 'type' => 'menu_entry_name',
+ 'parent' => $menu_entry };
+ push @{$current->{'contents'}}, $menu_entry;
+ push @{$menu_entry->{'contents'}}, $leading_text;
+ push @{$menu_entry->{'contents'}}, $entry_name;
+ $current = $entry_name;
}
# after a separator in menu
} elsif ($current->{'contents'} and @{$current->{'contents'}}
@@ -5379,6 +5393,19 @@ sub _process_remaining_on_line($$$$)
print STDERR "MENU NODE done $separator\n" if ($self->{'DEBUG'});
$current = _enter_menu_entry_node($self, $current, $source_info);
}
+ # After a separator in a menu, end of menu entry node or menu
+ # entry name (. must be followed by a space to stop the node).
+ } elsif ($menu_separator
+ # if menu separator is not ':', it is [,\t.]
+ and (($menu_separator ne ':' and $current->{'type'}
+ and $current->{'type'} eq 'menu_entry_node')
+ or ($menu_separator eq ':' and $current->{'type'}
+ and $current->{'type'} eq 'menu_entry_name'))) {
+ substr ($line, 0, 1) = '';
+ $current = $current->{'parent'};
+ push @{$current->{'contents'}}, { 'type' => 'menu_entry_separator',
+ 'text' => $menu_separator,
+ 'parent' => $current };
# Any other @-command.
} elsif ($command) {
if (!$at_command) {
@@ -6638,16 +6665,6 @@ sub _process_remaining_on_line($$$$)
and $current->{'parent'}->{'cmdname'}
and $current->{'parent'}->{'cmdname'} eq 'node') {
$self->_line_warn(__("superfluous arguments for node"), $source_info);
- # After a separator in a menu, end of menu node
- # (. must be followed by a space to stop the node).
- } elsif (($separator =~ /[,\t.]/ and $current->{'type'}
- and $current->{'type'} eq 'menu_entry_node')
- or ($separator eq ':' and $current->{'type'}
- and $current->{'type'} eq 'menu_entry_name')) {
- $current = $current->{'parent'};
- push @{$current->{'contents'}}, { 'type' => 'menu_entry_separator',
- 'text' => $separator,
- 'parent' => $current };
} elsif ($separator eq "\f" and $current->{'type'}
and $current->{'type'} eq 'paragraph') {
# A form feed stops and restart a paragraph.
@@ -6661,6 +6678,10 @@ sub _process_remaining_on_line($$$$)
} else {
$current = _merge_text($self, $current, $separator);
}
+ # need to be after as , is in common with separators
+ } elsif ($menu_separator) {
+ substr ($line, 0, 1) = '';
+ $current = _merge_text($self, $current, $menu_separator);
# Misc text except end of line
} elsif (defined $misc_text) {
print STDERR "MISC TEXT: $misc_text\n" if ($self->{'DEBUG'});
diff --git a/tp/Texinfo/XS/MiscXS.xs b/tp/Texinfo/XS/MiscXS.xs
index e4849b51f1..4223eec88b 100644
--- a/tp/Texinfo/XS/MiscXS.xs
+++ b/tp/Texinfo/XS/MiscXS.xs
@@ -108,10 +108,12 @@ xs_parse_texi_regex (text)
char *asterisk;
char *single_letter_command;
char *separator_match;
+ char *menu_separator;
char *new_text;
PPCODE:
xs_parse_texi_regex(text, &at_command, &open_brace, &asterisk,
- &single_letter_command, &separator_match, &new_text);
+ &single_letter_command, &separator_match,
+ &menu_separator, &new_text);
EXTEND(SP,6);
PUSHs(sv_newmortal());
sv_setpv((SV*)ST(0), at_command);
@@ -129,8 +131,11 @@ xs_parse_texi_regex (text)
sv_setpv((SV*)ST(4), separator_match);
SvUTF8_on(ST(4));
PUSHs(sv_newmortal());
- sv_setpv((SV*)ST(5), new_text);
+ sv_setpv((SV*)ST(5), menu_separator);
SvUTF8_on(ST(5));
+ PUSHs(sv_newmortal());
+ sv_setpv((SV*)ST(6), new_text);
+ SvUTF8_on(ST(6));
SV *
xs_default_format_protect_text (self, text_in)
diff --git a/tp/Texinfo/XS/misc.c b/tp/Texinfo/XS/misc.c
index d81725ded1..de7fc5cb30 100644
--- a/tp/Texinfo/XS/misc.c
+++ b/tp/Texinfo/XS/misc.c
@@ -289,6 +289,7 @@ void xs_parse_texi_regex (SV *text_in,
char **asterisk,
char **single_letter_command,
char **separator_match,
+ char **menu_separator,
char **new_text)
{
char *text;
@@ -301,7 +302,7 @@ void xs_parse_texi_regex (SV *text_in,
text = SvPV_nolen (text_in);
*at_command = *open_brace = *asterisk = *single_letter_command
- = *separator_match = *new_text = 0;
+ = *separator_match = *menu_separator = *new_text = 0;
if (*text == '@' && isalnum(text[1]))
{
@@ -338,10 +339,20 @@ void xs_parse_texi_regex (SV *text_in,
a[1] = '\0';
}
- else if (strchr ("{}@,:\t.\f", *text))
+ else if (strchr ("{}@,\f", *text))
{
static char a[2];
*separator_match = a;
+ if (*text == ',')
+ *menu_separator = a;
+ a[0] = *text;
+ a[1] = '\0';
+ }
+
+ else if (strchr (":\t.", *text))
+ {
+ static char a[2];
+ *menu_separator = a;
a[0] = *text;
a[1] = '\0';
}
diff --git a/tp/Texinfo/XS/miscxs.h b/tp/Texinfo/XS/miscxs.h
index 9928595d8c..8dea52bffa 100644
--- a/tp/Texinfo/XS/miscxs.h
+++ b/tp/Texinfo/XS/miscxs.h
@@ -2,6 +2,7 @@ char *xs_unicode_text (char *, int);
char *xs_entity_text (char *);
char *xs_process_text (char *text);
void xs_parse_texi_regex (SV *text,
+ char **,
char **,
char **,
char **,
diff --git a/tp/Texinfo/XS/parsetexi/menus.c b/tp/Texinfo/XS/parsetexi/menus.c
index 600af4efd0..4bc04885b6 100644
--- a/tp/Texinfo/XS/parsetexi/menus.c
+++ b/tp/Texinfo/XS/parsetexi/menus.c
@@ -187,8 +187,7 @@ handle_menu (ELEMENT **current_inout, char **line_inout)
debug ("ABORT MENU STAR");
last_contents_child(current)->type = ET_NONE;
}
- /* After a separator in a menu, which would have been added in
- handle_separator in separator.c:
+ /* After a separator in a menu, which would have been added in code below
, tab or . after ET_menu_entry_node
: after ET_menu_entry_name */
else if (current->contents.number > 0
@@ -256,6 +255,26 @@ handle_menu (ELEMENT **current_inout, char **line_inout)
current = enter_menu_entry_node (current);
}
}
+ /* After a separator in a menu, end of menu node
+ (. must be followed by a space to stop the node). */
+ else if (*line != '\0'
+ && ((*line == ':' && current->type == ET_menu_entry_name)
+ || (strchr (",\t.", *line)
+ && current->type == ET_menu_entry_node)))
+ {
+ ELEMENT *e;
+ char menu_separator = *line;
+ line++;
+
+ debug ("MENU SEPARATOR %c\n", menu_separator);
+ current = current->parent;
+ e = new_element (ET_menu_entry_separator);
+ text_append_n (&e->text, &menu_separator, 1);
+ add_to_element_contents (current, e);
+
+ /* Note, if a '.' is not followed by whitespace, we revert was was done
here
+ in code above. */
+ }
else
retval = 0;
diff --git a/tp/Texinfo/XS/parsetexi/parser.c b/tp/Texinfo/XS/parsetexi/parser.c
index cca8aeaa91..35a9789ff2 100644
--- a/tp/Texinfo/XS/parsetexi/parser.c
+++ b/tp/Texinfo/XS/parsetexi/parser.c
@@ -2207,7 +2207,7 @@ process_remaining_on_line (ELEMENT **current_inout, char
**line_inout)
}
}
/* "Separator" character */
- else if (*line != '\0' && strchr ("{}@,:\t.\f", *line))
+ else if (*line != '\0' && strchr ("{}@,\f", *line))
{
char separator = *line++;
debug ("SEPARATOR: %c", separator);
@@ -2216,6 +2216,15 @@ process_remaining_on_line (ELEMENT **current_inout, char
**line_inout)
else
current = handle_separator (current, separator, &line);
}
+ else if (*line != '\0' && strchr (":\t.", *line))
+ {
+ /* merge menu separator (other than comma, done with other separators) */
+ char separator = *line++;
+ char t[2];
+ t[0] = separator;
+ t[1] = '\0';
+ current = merge_text (current, t, 0);
+ }
/* "Misc text except end of line." */
else if (*line && *line != '\n')
{
diff --git a/tp/Texinfo/XS/parsetexi/separator.c
b/tp/Texinfo/XS/parsetexi/separator.c
index 3f639b7748..dd646710a3 100644
--- a/tp/Texinfo/XS/parsetexi/separator.c
+++ b/tp/Texinfo/XS/parsetexi/separator.c
@@ -766,24 +766,6 @@ handle_separator (ELEMENT *current, char separator, char
**line_inout)
{
line_warn ("superfluous arguments for node");
}
- /* After a separator in a menu, end of menu node
- (. must be followed by a space to stop the node). */
- else if ((separator == ','
- || separator == '\t'
- || separator == '.')
- && current->type == ET_menu_entry_node
- || separator == ':' && current->type == ET_menu_entry_name)
- {
- ELEMENT *e;
-
- current = current->parent;
- e = new_element (ET_menu_entry_separator);
- text_append_n (&e->text, &separator, 1);
- add_to_element_contents (current, e);
-
- /* Note in 'handle_menu' in menus.c, if a '.' is not followed by
- whitespace, we revert was was done here. */
- }
else if (separator == '\f' && current->type == ET_paragraph)
{
ELEMENT *e;
| [Prev in Thread] |
Current Thread |
[Next in Thread] |
- branch master updated: * tp/Texinfo/ParserNonXS.pm (_parse_texi_regex) (_process_remaining_on_line), tp/Texinfo/XS/MiscXS.xs (xs_parse_texi_regex), tp/Texinfo/XS/parsetexi/menus.c (handle_menu), tp/Texinfo/XS/parsetexi/parser.c (process_remaining_on_line), tp/Texinfo/XS/parsetexi/separator.c (handle_separator): handle menu separator separatly from other separators (with a possible overlap for comma, considered both as a menu and other separator in the perl parser), also moving the code starting the menu_entry_se [...],
Patrice Dumas <=