Re: [lmi] Headless GUI tests

[Greg Chicares]

On 11/14/21 8:11 PM, Vadim Zeitlin wrote:
> On Sat, 13 Nov 2021 18:24:25 +0000 Greg Chicares <gchicares@sbcglobal.net> 
> wrote:
> 
> GC> On 11/13/21 5:37 PM, Greg Chicares wrote:
> GC> 
> GC> [...need to ^C because GUI test doesn't terminate normally...]
> [...]
> GC> and now, after all the output above:
> GC>   xwud -in /var/tmp/Xvfb_screen0
> GC> shows something that may actually be useful. What I see is:
> GC>  - a small background window with a "File Census Test Window" menubar.
> GC>      and "Added cell number 7." on its statusbar; and, on top,
> GC>  - a foreground window saying:
> GC>      "[blue '?' icon] Do you want to save changes to unnamed15?"
> GC>      "Yes No Cancel"
> GC> Everything else is just a black background, except that an "x" graphic
> GC> (black with white border, looking much like a close-button graphic),
> GC> several lines below the word "number".
> GC> 
> GC> If I run that 'xwud' command repeatedly while the GUI test is
> GC> running, I see numerous screenshots that look like what I'd normally
> GC> expect when running non-headlessly.
> GC> 
> GC> If I wait a couple minutes after "NOTE: 4 tests were skipped", then
> GC> I still see the "Yes No Cancel" stuff described above.
> GC> 
> GC> If I run 'xwud' after ^C-ing the GUI test, I see a black screen.
> GC> 
> GC> This may be enough information for me to find and fix whatever is
> GC> going wrong.

With your patch
  https://github.com/let-me-illustrate/lmi/pull/197
(cherry-picked verbatim and pushed as lmi commit d7c83565ef513),
now I observe something a bit different:

/opt/lmi/bin[0]$xvfb-run --server-args="-fbdir /var/tmp" wine 
/opt/lmi/bin/wx_test.exe --ash_nazg  --data_path=/opt/lmi/data --test 
paste_census
NOTE: starting the test suite
paste_census: started
paste_census: ERROR (Assertion '(lmi::ssize(grid_window->GetSelectedRows())) == 
(1)' failed (expected 1 vs observed 0). [wx_test_paste_census.cpp : 270] )
time=195ms (for all tests)
FAILURE: 1 out of 1 test failed.
Assertion '"Assert failure"' failed
(A message box "Do you want to save changes to unnamed1?" was shown 
unexpectedly.).
[file /opt/lmi/local/include/wx-3.1/wx/testing.h, line 426]

Here's what 'xwud' shows:
 - a small lmi MDI main window, with menubar
     "File Census Test Window [wrap to new line] Help"
   and one MDI child; largely covered by...
 - a messagebox containing the last three lines quote
   from the console above, with a red X-in-circle icon
   and an "OK" button; along with
 - a normal arrow cursor, pointing north-northwest,
   near the red X-in-circle icon.

I still need to use ^C because it doesn't terminate by itself.

>  Unfortunately I still can't reproduce the problem,

Perhaps it requires this exact version of 'wine':
  $wine --version
  wine-4.0.3 (Debian 4.0.3-1)
I can't think of any other dependency that seems likely to account for
the difference in what we're seeing.

> and I couldn't manage
> to understand how could it happen from reading the code neither. The dialog
> you see is shown by wxDocument::OnSaveModified(), and is expected, because
> the document is indeed modified. In fact, it's so expected, that there is
> special code for dealing with it in the dtor of wx_test_document_base which
> ensures that the document is closed after selecting "No" in the dialog if
> it appears.

If that is happening here, then it's obscured completely by the
assertion-failure messagebox.

I haven't found a way to prevent it, but I can get a different
outcome with this patch (which should at least do no harm: if
the mouse click selects row zero, then SelectRow(0) should have
no observable effect):

--8<---8<---8<---8<---8<---8<---8<---8<---8<---8<---8<---
diff --git a/wx_test_paste_census.cpp b/wx_test_paste_census.cpp
index 27b73863e..a09afc3c1 100644
--- a/wx_test_paste_census.cpp
+++ b/wx_test_paste_census.cpp
@@ -264,6 +264,7 @@ LMI_WX_TEST_CASE(paste_census)
             )
         );
     ui.MouseClick();
+    grid_window->SelectRow(0);
     wxYield();
 
     LMI_ASSERT_EQUAL(lmi::ssize(grid_window->GetSelectedRows()), 1);
--8<---8<---8<---8<---8<---8<---8<---8<---8<---8<---8<---

With that (experimental and non-committed) patch, in addition
to your (pushed) patch, I get:

$xvfb-run --server-args="-fbdir /var/tmp" wine /opt/lmi/bin/wx_test.exe 
--ash_nazg  --data_path=/opt/lmi/data --test paste_census
NOTE: starting the test suite
paste_census: started
paste_census: ERROR (Assertion failure: Expected class defaults dialog was not 
shown. [file /opt/lmi/src/lmi/wx_test_paste_census.cpp, line 317, in run()].)
time=207ms (for all tests)
FAILURE: 1 out of 1 test failed.
Assertion '"Assert failure"' failed
(A message box "Do you want to save changes to unnamed1?" was shown 
unexpectedly.).
[file /opt/lmi/local/include/wx-3.1/wx/testing.h, line 426]

^CX connection to :99 broken (explicit kill or server shutdown).

...so the
  (Assertion '(lmi::ssize(grid_window->GetSelectedRows())) == (1)' failed 
(expected 1 vs observed 0).
failure is avoided, but instead there's a different assertion failure.
I still need to ^C manually.