[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[O] [test] Mark tests with missing dependencies as "expected to fail"
From: |
David Maus |
Subject: |
[O] [test] Mark tests with missing dependencies as "expected to fail" |
Date: |
Tue, 18 Oct 2011 06:32:45 +0200 |
User-agent: |
Wanderlust/2.15.9 (Almost Unreal) SEMI/1.14.6 (Maruoka) FLIM/1.14.9 (Gojō) APEL/10.8 Emacs/23.2 (i486-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) |
Hi all,
Currently tests with missing dependency are silently skipped -- it
might be worth changing this behavior to not skip them, but mark them
as expected to fail. You can do this in ERT by placing the
keyword :expected-result followed by either :passed or :failed before
the test's body.
Benefit of this is that the tests w/ missing dependencies will show up
in the ERT result page (with a small letter f) but (obviously) don't
count as failures.
The following macro is a first shot at a convenient way to define
tests with dependencies:
#+begin_src emacs-lisp
(defmacro org-test-deftest-with-dependency (dependency &rest body)
(let* ((docstring (if (stringp (third body)) (list (third body))))
(deftest (nthcdr (if docstring 3 2) body))
(dependency-p (eval dependency)))
`(ert-deftest ,@(append (list (first body) (second body)) docstring)
,@(if dependency-p
'(:expected-result :passed)
'(:expected-result :failed (error "Missing dependency")))
,@deftest)))
#+end_src
Here DEPENDENCY is a simple form that evaluates to non-nil if the
dependency is met. If marking the tests this way is the agreed way to
go we can extend the syntax of a dependency to an alist with a
human-readable description of the dependency as key and a list of
forms that all must eval to non-nil as value. E.g.
#+begin_src emacs-lisp
(defvar test-ob-R-dependencies
'(("R executable" (org-test-for-executable "R"))
("Package ESS" (featurep 'ess))))
#+end_src
And change the expander code to map through this alist and as soon one
dependency evals to nil sets a variable `dependency-missing' to the
respective dependency.
Any comments on this?
Best,
-- David
--
OpenPGP... 0x99ADB83B5A4478E6
Jabber.... address@hidden
Email..... address@hidden
pgp5v2hDpInro.pgp
Description: PGP signature
- [O] [test] Mark tests with missing dependencies as "expected to fail",
David Maus <=