Re: TRAMP problem with large repositories

[Michael Albinus]

Philippe Vaucher <address@hidden> writes:

> Hello,

Hi Philippe,

> While helping someone for a projectile issue
> (https://github.com/bbatsov/projectile/issues/1480), it seems that
> when `shell-command-to-string` tries to execute `git ls-files -zco -
> -exclude-standard` over TRAMP on a repository that has 85K files it
> takes forever to complete.
>
> We see that `tramp-wait-for-output` calls `tramp-wait-for-regexp`
> which calls `tramp-check-for-regexp`, and when looking at the source:
>
> My understanding is that it does a loop that reads a bit of what the
> commands outputs then tries to parse end of lines (or '\0') and
> repeats until the process died or that it found one. Because the
> command returns a huge string (85K files), this process of
> read-regexp-repeat takes all the CPU (compared to reading the whole
> chunk in one go and then trying to check for the regexp).
>
> My questions are the following:
>
> 1 Did I understand the problem right? Is this something known?

Yes, your analysis is right. And no, I haven't seen related reports yet.

> 2 Is there something to be done about this? Or maybe it would it
>   require too much refactoring / faster implementation?

I have appended a patch which should fix the problem. Could you, please,
(let) test?

Btw, the latest Tramp release is always available via GNU ELPA.

> Kind regards,
> Philippe

Best regards, Michael.

txtrcGfaTvbOT.txt
Description: Text Data