Re: python-mode: make sure output is not eaten

[Slawomir Nowaczyk]

On Sat, 26 Aug 2006 10:48:25 -0400
Stefan Monnier <address@hidden> wrote:

#> > Have you installed the complete patch? Because current version of the
#> > CVS code doesn't quite work for me... it hangs emacs whenever I try
#> > python-send-buffer (I need C-g to get it make it responsive again),
#> > which seems to be related to the "while" loop in python-send-receive
#> > never finishing due to the (local-variable-p 'python-preoutput-result)
#> > in python-preoutput-filter never being true...
#> 
#> Hmm... looks like the two pieces of code don't run in the same buffer maybe?
#> I've just installed a patch which may fix it.

I works better -- no longer hangs my Emacs, but after python-send-buffer
the final prompt still doesn't appear.

Commenting out (set (make-local-variable 'python-preoutput-skip-next-prompt) t))
in python-preoutput-filter makes it work for me.

#> > I still think 
#> >   (python-send-string "import emacs")
#> >   (python-send-receive "print '_emacs_out ()'")
#> > is cleaner than
#> >   (python-send-receive "import emacs; print '_emacs_out ()'")
#> 
#> Again, here's the issue: the process's output can be absolutely
#> arbitrary. If we see "\n>>> " somewhere in the process's output we
#> have in general no guarantee that it's actually a prompt. Similarly
#> if we see "_emacs_out " we have in general no guarantee that it's
#> coming from python.el or from emacs.py. So we should be careful in
#> the process filter to be as selective as possible in what we remove.

Agreed. But why is removing "_emacs_out ()" any more selective than
removing ">>> _emacs_out ()"?

#> In interactive use, it's perfectly OK to do:
#> 
#>    >>> import emacs
#>    >>> print '_emacs_out ()'
#>    _emacs_out ()
#>    >>> 
#> 
#> but from python.el this basically turns into:
#> 
#>    >>> import emacs
#>    print '_emacs_out ()'
#>    >>> _emacs_out ()
#>    >>> 
#> 
#> because we don't wait for the prompt in between the two.  That's not
#> something you'd normally do interactively.  Instead if you didn't want to
#> wait for the prompt between the two, you'd do something like:
#> 
#>    >>> import emacs; print '_emacs_out ()'
#>    _emacs_out ()
#>    >>> 
#> 
#> which has the advantage that the _emacs_out () thingy appears at its normal
#> place so you can be more selective and risk misidentifying it in
#> fewer cases.

I think I understand your reasoning, but I do not think I agree with it.
If we *want* to wait for prompt after both "import ..." and "print ...",
we can always use python-send-receive twice, instead of
python-send-string followed by python-send-receive. But I still do not
see why one is supposed to be more robust than the other.

Also, I have finally found a case in which adding "; print ..." breaks
things: assume the string is "if False: print 1". Observe that "if
False: print 1; print '_emacs_out ()'" does *not* print anything.

It may not be a *real* problem, but it does -- IMHO -- show that sending
two separate commands *is* more robust.

#> > I do not understand why "command" python-send-command shouldn't
#> > contain newline...
#> 
#> It's not a command, so it can only called from elisp code. Check the
#> callers and you'll see it is never called with a newline. So it's OK
#> to make such an assumption.

Well, maybe. But still, either python-send-string works for multiline
strings (in which case python-send-command should work as well) or
python-send-string doesn't work for multiline strings, in which case the
assert in python-send-command doesn't help at all (as you wrote, nothing
can call it with multiline command anyway).

#> > It should work just fine with multiple lines, the only thing which
#> > might cause problems is indentation, and even that only if there is
#> > an indented block at the very end.
#> 
#> Sending a multiline thingy via python-send-string will look like
#> absolute crap in the resulting output,

I disagree...

#> because all the intermediate prompts will get concatenated rather
#> than interleaved with the input.

Right, but what's wrong with that? If I send a string like "a = 1\n
print 'Test'\n b = 2", what's wrong with getting output like
">>> Test\n>>> \n >>>"?

Sure, doing that may not be the smartest thing to do, comint buffers
work much better for this kind of stuff, but still... the output
actually makes sense.

#> So I see no point in trying to support this in a case such as
#> `python-send-command' where we don't use this flexibility anyway (and
#> where we need to analyze the output, so the more control over it the
#> better).

You may be right... but if you think it makes sense, the assert should
be in python-send-string, not in python-send-command.

#> > If you are thinking about conditionally adding one or two newlines after
#> > python-send-string, than the correct condition should be "add double
#> > newline if and only if the last line is indented", IMHO.
#> 
#> Sure, go ahead.  I just tried a simple heuristic to preserve the previous
#> "\n\n" thingy without imposing it in the case where I know it's unneeded and
#> even annoying.
#> 
#> > I do not understand the purpose of python-preoutput-result variable. It
#> > is used in python-send-receive and in python-preoutput-filter, but those
#> > uses seem unrelated...
#> 
#> They're very much related: the accept-process-output call in
#> python-send-receive will let process filters run, so it's basically an
#> indirect call to python-preoutput-filter.

OK, I see... I forgot the filter function can be called from within
accept-process-output.

#> > Also, python-mode-running doesn't seem to be used anywhere.
#> 
#> It's used right where it's used ;-)
#> 
#>   (unless (boundp 'python-mode-running)      ; kill the recursion from 
jython-mode
#>     (let ((python-mode-running t))
#>       (python-maybe-jython))))
#> 
#> i.e. it's used to prevent python-maybe-jython -> jython-mode ->
#> python-mode -> python-maybe-jython -> jython-mode -> python-mode ->
#> python-maybe-jython -> ...

OK, I was mislead by the name... I thought it was supposed to be an
implementation of a feature you mentioned earlier, a check whether
Python interpreter is busy so that things like eldoc won't be called
when they shouldn't.

#> > I do not think this is the right way to solve the problem. IMHO, the
#> > variable should be called python-inhibit-output and should prevent
#> > *anything* from being printed in Inferior Python buffer -- thus,
#> > functions like eldoc could bind it to 't and we would not need to worry
#> > about them at all.
#> 
#> > After all, it doesn't matter how the output of python-eldoc-function
#> > looks like, it should never appear in the comint buffer. And it would
#> > let us simplify the common case of *user* wanting to run Python code.
#> 
#> Sounds good. As long as you can be sure that there can be no other
#> output at the same time (i.e. the python process is idle, waiting at
#> the prompt when you send the eargs command). Currently, we do not
#> check that python is idle, so we have no guarantee that we can just
#> discard all the output between after we send the `eargs' (ou modpath,
#> or complete, ...) until after we see the _emacs_out.

I have hard time imagining how the current code could work at all if it
send anything to Python interpreter while it was busy, but I will take a
closer look at this.

#> > I do not quite understand... You never *know* that it is the internal
#> > thingy anyway (try typing "print '_emacs_out'" in the Inferior Python
#> > buffer)...
#> 
#> Thanks to the local-variable-p check, the filter should now correctly let it
#> go through, except for unusual cases.

Yes, I see, it's working now.

-- 
 Best wishes,
   Slawomir Nowaczyk
     ( address@hidden )

Hard Disk:  A device that allows users to delete vast quantities of
data with simple mnemonic commands.