[Fab-user] contrib.files.append() and escaping madness

[Neilen Marais]

Hi,

I ran into some limitations with contrib.files.append(), and have now
managed to fix them, at least for me:

https://github.com/fabric/fabric/pull/487

For discussion's sake I repeat the diff at the end of the message.

The fix for single quote escaping seems to make sense, since the
original method resulted in invalid bash syntax:

-        line = line.replace("'", r'\'') if escape else line
+        line = line.replace("'", r"'\\''") if escape else line

The rest deal with grepping to check if the line is already present.
To be honest, I don't _quite_ know what I did to fix it, but it seems
to be a combination of shell escaping issues (dealt with by making
contains() bypass the shell), and differences in how the egrep command
and python's re module treat special character escaping.

The contains() function could perhaps use a bit of clarification. Is
it meant to be used with regexes (the docstring seems to indicate no)
or is it supposed to match text exactly? Perhaps we could add a kwarg
regexp, which defaults to False. If it is false, it tries to match the
exact string, if true it searches for a regex. There would of course
be escaping differences between these two cases.

Cheers
Neilen


diff --git a/fabric/contrib/files.py b/fabric/contrib/files.py
index d251b45..ea1d19b 100644
--- a/fabric/contrib/files.py
+++ b/fabric/contrib/files.py
@@ -281,10 +281,11 @@ def contains(filename, text, exact=False, use_sudo=False):
     if exact:
         text = "^%s$" % text
     with settings(hide('everything'), warn_only=True):
-        return func('egrep "%s" "%s"' % (
+        egrep_cmd = 'egrep "%s" "%s"' % (
             text.replace('"', r'\"'),
             filename.replace('"', r'\"')
-        )).succeeded
+        )
+        return func(egrep_cmd, shell=False).succeeded


 def append(filename, text, use_sudo=False, partial=False, escape=True):
@@ -323,8 +324,13 @@ def append(filename, text, use_sudo=False,
partial=False, escape=True):
         text = [text]
     for line in text:
         regex = '^' + re.escape(line) + ('' if partial else '$')
+        # Tripple-escaping seems to be required for $ signs
+        regex = regex.replace(r'\$', r'\\\$')
+        # Whereas single quotes should not be escaped
+        regex = regex.replace(r"\'", "'")
+
         if (exists(filename, use_sudo=use_sudo) and line
             and contains(filename, regex, use_sudo=use_sudo)):
             continue
-        line = line.replace("'", r'\'') if escape else line
+        line = line.replace("'", r"'\\''") if escape else line
         func("echo '%s' >> %s" % (line, filename))