emacs-orgmode
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Orgmode] Re: Testing --- again...


From: Eric Schulte
Subject: Re: [Orgmode] Re: Testing --- again...
Date: Sat, 02 Oct 2010 12:12:37 -0600
User-agent: Gnus/5.13 (Gnus v5.13) Emacs/23.1.90 (gnu/linux)

Hi,

This is exciting.

Rather than impose a complete directory/layout schema before-hand I'd
lean towards starting with a little more chaos and then letting the
structure of the test directory develop /naturally/.  From the
discussion below it sounds like an initial structure of

testing/lisp/
testing/contrib/lisp/

may make sense, reserving the top level for "meta" testing stuff, like
functions for running tests, common fixtures, example files, etc...

I have two questions.

1) while waiting for ert to be included into Emacs, should we include an
   ert distribution as part of the Org-mode repository (maybe using git
   sub-modules) or should we just agree that users should have a certain
   version of ert installed locally?  I'm honestly not sure which of
   these options sounds preferable.

2) should the initial population of the testing/ directory take place in
   a separate branch of the repository or in the master branch?  Again I
   don't know which I would prefer, branches add complication but could
   result in cleaner commit histories.

Best -- Eric

Carsten Dominik <address@hidden> writes:

> Hi Sebastian,
>
> the lack of a testing suite for Org-mode is really frustrating,
> and even more frustrating is that we have had like seven attempts
> to start one, and each of these lead to nothing.  So I would
> be perfectly happy to give a free hand, write access to the repo
> and a full directory in the distribution to implement one.
> Once there is a framework, I am sure many people would be
> willing to contribute tests.
>
> More comments below.
>
> On Oct 2, 2010, at 5:51 AM, Sebastian Rose wrote:
>
>> Hi,
>>
>>
>> I thought about testing again recently.  This is something, that never
>> really got started.  For a reason:  there's no framework for testing.
>>
>> I therefore wrote a very rough proposal,  found  on
>> http://github.com/SebastianRose/org-test
>>
>> The idea is, to provide two simple commands:
>>
>>
>>  *  org-test-test-current-defun
>>     will search for tests for the defun point is in or behind
>>     (`beginning-of-defun') and execute them surrounded by
>>
>>  (let ((select (or selector "^org"))
>>      (deactivate-mark nil))
>>    (save-excursion
>>      (save-match-data
>>
>>
>>  *  org-test-test-buffer-file
>>     will search for tests for the entire file and execute them the
>> same
>>     way.
>
> FIrst:  I have *no* clue about testing.
>
> Second, I am surprised that you want to structure it by function.  I
> would have
> thought that it could be structure by file at the most.  And then
> there will
> be tests that involve code from many files.
>
> But I guess
>
>>
>> If you use one of these commands, all currently registered ERT tests
>> are
>> deleted, and files are reloaded (since you're likely to work on the
>> tests, too).  To repeat the tests without reloading, you will use the
>> ERT commands like `ert-results-rerun-all-tests', bound to `r' in the
>> ERT
>> results buffer.
>>
>>
>>
>> I choose ERT (git clone http://github.com/ohler/ert.git) because
>> that's
>> likely to go into Emacs core (or elpa.gnu.org).
>>
>>
>>
>>
>> The idea is to search the directory structure from the current source
>> file upwards for a directory named "tests/" if it exists.  Else ask
>> the
>> user.  Similar to what `add-change-log-entry' does.
>>
>> Below that directory, a tree like the source tree exists:
>>
>> project
>>   +-- lisp/
>>   |     +-- a.el
>>   |     `-- b/
>>   |         +-- b.el
>>   |
>>   `-- tests/
>>         +-- a.el/
>>         |     +-- tests.el
>>         |     `-- a-defun.el
>>         `-- b/
>>             +-- b.el/
>>                   +-- tests.el
>>                   `-- b-defun.el
>>
>> If this setup exists, when editing defun-x in lisp/a.el,
>> `M-x org-test-test-current-defun' will load tests/a.el/defun-x.el
>> (fallback: tests.el there) and execute all tests with selector
>> "^a-defun".
>
> Well, OK, this is fine.  But under a.el and b.el there should also be
> general tests that are not function dependent, and there should be a
> place
> to put tests that you do not want to assign to a specific file.
>
> We do have a "testing" directory already, you can use that.
> I would prefer the tests to be in testing, not in lisp/testing
> if possible. I would like to have the lisp directory contain
> only code.  If possible.
>
> It would be OK to have a lisp subdirectory in testing,
> just as it would be OK to have contrib/lisp in testing
> for the contributed packages.
>
> PLEASE, go ahead.  I do not think you have write access
> yet on repo - give me your user name and I'll activate you.
>
> - Carsten
>
>> `M-x org-test-test-buffer-file' in that same source file will load all
>> *.el files in tests/a.el/ and execute all ERT tests for selector "^a".
>>
>>
>> Thus tests for
>>    org-mode/lisp/org-protocol.el
>> will be searched in the directory
>>    org-mode/tests/lisp/org-protocol.el/*.el
>>
>>
>> Once the basic route of testing is clear, I'd like to "translate" the
>> existing tests for org-html.el to work with ERT, which will involve
>> writing more tools (create output buffers, compare output with control
>> files using ediff etc.).  I know Lennart Borgman has wrote that stuff
>> for nXhtml already.  I hope we can use his stuff and help here.
>>
>> The directory org-mode/lisp/tests/ would not need to be part of the
>> "official" Org mode package.  It could as well be checked out
>> separately, if "tests" is part of org-mode/lisp/.gitignore (e.g.).
>>
>>
>> Any thoughts?
>>
>>
>>
>>
>>  Sebastian
>
>
> _______________________________________________
> Emacs-orgmode mailing list
> Please use `Reply All' to send replies to the list.
> address@hidden
> http://lists.gnu.org/mailman/listinfo/emacs-orgmode



reply via email to

[Prev in Thread] Current Thread [Next in Thread]