Re: plz help with tcpserver

[Alex fxmbsw7 Ratchev]

i use gawk cause it has RT
to preserve RT \n or \n\n
but yours says \r ? lemme test mine no mine still bugs

i did t.. strace gawk 1
it reads newlines, but prints at end
gawk bug ?

read(0, "foo\n", 4096)                  = 4
fstat(1, {st_mode=S_IFSOCK|0777, st_size=0, ...}) = 0
read(0, "\n", 4096)                     = 1
read(0, "\n", 4096)                     = 1
read(0, "\n", 4096)                     = 1
read(0, "bar\n", 4096)                  = 4
read(0, "\n", 4096)                     = 1
read(0, "\n", 4096)                     = 1
read(0, "\n", 4096)                     = 1
read(0, "", 4096)                       = 0
openat(AT_FDCWD, "/dev/null", O_RDWR)   = 3
dup2(3, 0)                              = 0
close(3)                                = 0
write(1, "foo\n\n\n\nbar\n\n\n\n", 14)  = 14
exit_group(0)                           = ?
+++ exited with 0 +++
^C

On Mon, Jul 26, 2021 at 10:33 PM Alex fxmbsw7 Ratchev <fxmbsw7@gmail.com> wrote:
>
> and if the reader has no idea what a gawk -v RS=\\n\\n 1 is there wont
> be useful input for this thread by this reader, ..
>
> On Mon, Jul 26, 2021 at 10:32 PM Alex fxmbsw7 Ratchev <fxmbsw7@gmail.com> 
> wrote:
> >
> > sorry, run my gawk, not awk
> > i didnt write sh, did i
> >
> > or at least as i did with -v RS=\\n\\n
> >
> > in the end my gawk, to explain, merges two following newlines to one,
> > in a print process, this is short to test, and it failed
> > so its a gawk bug ?
> >
> > On Mon, Jul 26, 2021 at 10:27 PM Greg Wooledge <greg@wooledge.org> wrote:
> > >
> > > On Mon, Jul 26, 2021 at 09:46:38PM +0200, Alex fxmbsw7 Ratchev wrote:
> > > > > term 1
> > > > > tcpserver 0 2000 strace -f gawk -v RS=\\n\\n 1 &
> > > > >
> > > > > strace for seeing weirdness
> > > > > then
> > > > > term 2, no printf or pipe, non eof input
> > > > > cause, http clients dont end conn after printing req headers
> > > > > ( nc 0 2000 to connect via netcat .org or so telnet client to 
> > > > > localhost port 2000 )
> > > > >
> > > > > nc 0 2000
> > > > >
> > > > > then see strace began to show, read(
> > > > >
> > > > > then enter newlines some, it never prints, till control-c of nc
> > > > > and result, the printed input, wont ever be recieved
> > >
> > > Pretend for a moment that the reader has no idea what "gawk -v RS=\\n\\n 
> > > 1"
> > > is supposed to do.
> > >
> > > Let's first verify how our tools actually *work*, shall we?  I don't
> > > think you've programmed correctly for the line ending issue.
> > >
> > > ===== BEGIN EXAMPLE ONE =====
> > > term1> tcpserver 0 2000 bash -c 'while read -r; do printf "Got: <%s>\n" 
> > > "$REPLY"; done'
> > >
> > > term 2> telnet localhost 2000
> > > ... Escape character is '^]'.
> > > hello
> > > >ot: <hello
> > > goodbye
> > > >ot: <goodbye
> > > ===== END EXAMPLE ONE =====
> > >
> > > What happened?  The lines that I typed interactively in telnet were
> > > received and processed in real time by the program that tcpserver
> > > launched, which is good.  But there's an obvious carriage return issue
> > > going on there.  You can see it by the greater-than signs overwriting
> > > the first character of the line, instead of being at the end of the line.
> > >
> > > Remember, when you receive lines of text from a network socket, they may
> > > be encoded with CR-LF line ending characters.  They're "supposed" to be
> > > that way, because that's the "standard".  And it appears that telnet on
> > > my system is enforcing that standard.
> > >
> > > We can verify that it's telnet doing so, and not tcpserver, by running
> > > another experiment.
> > >
> > > ===== BEGIN EXAMPLE TWO =====
> > > term1> same as before
> > >
> > > term2> { echo hello; sleep 1; echo goodbye; } | nc -N localhost 2000
> > > Got: <hello>
> > > Got: <goodbye>
> > > ===== END EXAMPLE TWO =====
> > >
> > > That looks like what I was expecting to see!  So, we can conclude that
> > > telnet was adding carriage returns to what I was typing, but nc -N
> > > (using the implementation of nc on my system -- remember, nc is NOT
> > > standard, and there are numerous variants of it out there) passed the
> > > input through without modification.
> > >
> > > So, let's rewrite our server side, to handle the possible presence of
> > > carriage returns in the input.  This is a pain in the ass to write as
> > > a shell one-liner with bash -c, so I'll use a script.
> > >
> > > ===== BEGIN EXAMPLE THREE =====
> > > term1> cat foo
> > > #!/bin/bash
> > > while read -r; do
> > >   REPLY=${REPLY%$'\r'}
> > >   printf 'Got: <%s>\r\n' "$REPLY"
> > > done
> > > term1> tcpserver 0 2000 ./foo
> > >
> > > term2> telnet localhost 2000
> > > ... Escape character is '^]'.
> > > hello
> > > Got: <hello>
> > > goodbye
> > > Got: <goodbye>
> > > ===== END EXAMPLE THREE =====
> > >
> > > All is well.  Now we know that two combinations of tools
> > > (tcpserver, bash, telnet) and (tcpserver, bash, nc -N) both work fine.
> > >
> > > Now let's introduce awk.  Since I don't know what your thing does or is
> > > trying to do, I'll try to duplicate the bash script that I just wrote,
> > > using awk.  Let's start out simple:
> > >
> > > ===== BEGIN EXAMPLE FOUR =====
> > > term1> tcpserver 0 2000 awk '{print "Got <" $0 ">"}'
> > >
> > > term2> telnet localhost 2000
> > > ... Escape character is '^]'.
> > > hello
> > > goodbye
> > > ^]
> > > telnet> q
> > > ===== END EXAMPLE FOUR =====
> > >
> > > Well, that was a complete failure, wasn't it?  Maybe that just
> > > demonstrates that I don't know awk very well.  Maybe it's a buffering
> > > thing (BashFAQ 009).  The FAQ page says that for gawk, we have to use
> > > the fflush() function.  So let's try again:
> > >
> > > ===== BEGIN EXAMPLE FIVE =====
> > > term1> tcpserver 0 2000 awk '{print "Got <" $0 ">"; fflush()}'
> > >
> > > term2> telnet localhost 2000
> > > ... Escape character is '^]'.
> > > hello
> > > >ot <hello
> > > goodbye
> > > >ot <goodbye
> > > ===== END EXAMPLE FIVE =====
> > >
> > > Looks familiar!  My awk program isn't handling the carriage returns from
> > > telnet correctly.  That's a bug -- not in awk, but in my program, because
> > > a network service has to be able to handle them.  They may be there, or
> > > they may not.  But the good news is that, apart from the CR issue, it
> > > worked as expected.
> > >
> > > Now let's move on to nc -N.
> > >
> > > ===== BEGIN EXAMPLE SIX =====
> > > term1> same as above
> > >
> > > term2> { echo hello; sleep 1; echo goodbye; } | nc -N localhost 2000
> > > Got <hello>
> > > Got <goodbye>
> > > ===== END EXAMPLE SIX =====
> > >
> > > Looks reasonable to me.
> > >
> > > Well, you're the awk guy, not me, so you can take it from here.  Figure
> > > out how to write your program so that it correctly handles CR-LF or LF
> > > line endings, use the fflush() command, and use nc -N on the client side 
> > > if
> > > you're on the same version of netcat that I am (netcat-openbsd 1.217-3).
> > > Or if you're on a different version of netcat, figure out how to make 
> > > yours
> > > close cleanly when stdin hits EOF.
> > >
> > > Good luck.
> > >