[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [PATCH] make test: Make failure results more verbose
From: |
Max Nikulin |
Subject: |
Re: [PATCH] make test: Make failure results more verbose |
Date: |
Sat, 15 Jan 2022 22:58:41 +0700 |
User-agent: |
Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.14.0 |
On 02/01/2022 20:12, Ihor Radchenko wrote:
In newer Emacs, ERT is capable of providing more info about FAILED
tests. Maybe we can enable this option by default in the Org test suite?
Thinking more, I have realized that something is wrong.
Behavior of tests in Org should be controlled by EMACS_TEST_VERBOSE
*environment* variable. Org makefiles may have it on by default, but it
should not override explicitly chosen value. Fortunately Emacs
repository contains no makefiles from Org, so it will not cause conflict
with Emacs builds.
The problem is that EMACS_TEST_VERBOSE interface is broken. In the
following case I expect that test summary should not be verbose:
ert-sample.el
---- >8 ----
(require 'ert)
(ert-deftest ert-sample ()
(should (equal 5 (* 2 2))))
(ert-run-tests-batch-and-exit)
---- 8< ----
(I am tried it with Emacs-27, so I put a copy of ert.el from git HEAD to
~/ert)
EMACS_TEST_VERBOSE= emacs --batch -Q -L ~/ert -l ert-sample.el
1 unexpected results:
FAILED ert-sample ((should (equal 5 (* 2 2))) :form (equal 5 4)
:value nil :explanation (different-atoms (5 "#x5" "?") (4 "#x4" "?")))
That is why I am going to file a bug against ert.
In a minimal variant empty string (`getenv' return value) should not be
considered as t.
For better user experience, I expect that the following case-insensitive
strings should be considered as false: "0", "false", "no", "n", "off",
"f" (borrowed from .ini), "none", "nil". Other values should be
considered as true or anything besides "1", "true", "yes", "y", "on",
"t" should be considered as invalid value.
I am attaching a tentative patch for Org that should make
EMACS_TEST_VERBOSE setting more transparent.
0001-make-test-Make-failure-summary-more-verbose.patch
Description: Text Data
- [PATCH] make test: Make failure results more verbose, Ihor Radchenko, 2022/01/02
- Re: [PATCH] make test: Make failure results more verbose, Max Nikulin, 2022/01/03
- Re: [PATCH] make test: Make failure results more verbose, Ihor Radchenko, 2022/01/07
- Re: [PATCH] make test: Make failure results more verbose, Max Nikulin, 2022/01/11
- Re: [PATCH] make test: Make failure results more verbose, Max Nikulin, 2022/01/15
- Re: [PATCH] make test: Make failure results more verbose, Max Nikulin, 2022/01/15
- Re: [PATCH] make test: Make failure results more verbose,
Max Nikulin <=
- Re: [PATCH] make test: Make failure results more verbose, Ihor Radchenko, 2022/01/21
- Re: [PATCH] make test: Make failure results more verbose, Max Nikulin, 2022/01/21
- Re: [PATCH] make test: Make failure results more verbose, Ihor Radchenko, 2022/01/23
- Re: [PATCH] make test: Make failure results more verbose, Ihor Radchenko, 2022/01/21