Re: dired-do-find-regexp failure with latin-1 encoding

[Eli Zaretskii]

> Cc: emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sat, 28 Nov 2020 22:16:21 +0200
> 
> >>>    C-x RET c latin-1 RET A ä RET
> >>>
> >>> ?
> >>
> >> Yes (with -a added to the grep invocation, but not without it).  And
> >> then with either 'a' or 'ä' as the search term, *xref* displays 'aä'.
> >> So this seems to be the best workaround, though inconvenient for
> >> frequent uses
> > 
> > I really don't see any other way, especially if different files in the
> > directory have different encodings.  Grep looks for bytes, not
> > characters, and is agnostic to encoding.  And even if we'd do this in
> > Emacs Lisp, we'd still need to trust Emacs to guess/detect the correct
> > encoding of each file.
> 
> Ah, so this way the user explicitly searches for a regexp encoded as 
> latin-1?

More accurately, this is how to search in files encoded in Latin-1.
(The regexp also gets encoded in latin-1, but the important part is
the files' encoding.)

> > Adding -a probably cannot do any harm, but its support should be
> > detected, since I don't think it's portable enough (it isn't in the
> > latest Posix spec, at least).
> 
> Are you sure about that? Are we sure it won't make searching binary 
> files slower, for example?

It will be slower, but more useful: by default Grep just says "Binary
file foo matches".

> Also, the manual has this warning:
> 
> Warning: The -a option might output  binary  garbage,  which  can  have 
> nasty  side effects if the output is a terminal and if the terminal 
> driver interprets some of it as commands.
> 
> ...which might conceivably mess up our parsing of Grep output sometimes?

This is not relevant, since we read that output, there's no terminal
device driver to interpret it and get messed up.

I actually don't think I understand why we need -a in this case, since
Grep looks for null bytes to decide this is a binary file, and encoded
non-ASCII characters don't have null bytes 9except if they are in
UTF-16).

> P.S. Or we can forgo all that and ask the users who want to search for 
> non-ASCII strings to install ripgrep.

We should support Grep regardless, since not everyone will have
ripgrep.  And in any case, "C-x RET c" will be needed with it as well,
no?