[Orgmode] RFC: Improvements to org-remember

[James TD Smith]

I have a number of improvements to org-remember I am planning to implement. I
have briefly discussed some of them with Carsten, and I thought I should post a
detailed description here for discussion as I'm sure you will have ideas,
suggestions and criticisms of the proposed changes.

* Preliminary work
** Refactoring

  Carsten suggested that the remember handler could do with some refactoring
  before adding any new features, as it has gotten quite complicated in places.

  The longest functions are `org-remember-apply-template' and
  `org-remember-handler', so these are probably the best place to start.

  I think it would make sense to move the code to get values for remember
  expansions out of `org-remember-apply-template' into separate functions. These
  could be added to an association list indexed by the expansion character. This
  would also make it easier to add new expansions.

** Unit tests

   If we can decide on a unit testing framework, this would be an ideal
   opportunity to add some test coverage, as I'll need to do extensive testing
   to ensure I don't break anything with the refactoring or the new features. I
   have some experience with unit testing using JUnit.
   
** Plists for remember templates

   I want to change the format of remember templates to use plists. This is to
   allow introducing a number of optional parameters to control the new features
   I want to add. Org uses plists elsewhere, for example in the #+OPTIONS:
   configuration header, and #+PLOT: lines, so the syntax should already be
   familiar to org users.

   I also think it would make sense to to move some options which are currently
   set using escapes outside of the template, specifically "%!" (store
   template immediately) and "%&" (jump to entry after storing).

   For backwards compatibility, the current template format would still be
   supported, but the additional options would not be available. Defaults for 
the
   extra parameters would be set so if they are not present the templates would
   work as they do currently.

   The current options would be represented as below:
   - :template :: the template itself
   - :name :: the name of the template
   - :target :: The :target parameter takes the place of the current file and
                headline target specification. The parameter specifies only the
                default target; all the other options will remain available via
                numeric prefixes to C-c C-c. The available options are:
     - "<filename>:<heading>" :: a file target. If the heading is omitted,
                                 a top-level heading will be created.
     - clock :: currently clocked task
     - current :: add under the the org headline where the point is.
     - interactive :: select a location interactively using the appropriate
                      interface
     - <function> :: call function to get the target. The function can return
                     either a marker, or a file/headline string.
   - :test :: a function, or list of major modes. The template is only
              available if the function returns true, or the current
              buffer is in one of the appropriate modes
   - :immediate :: replaces the %! escape; if t, the template is stored as soon
                   as all escapes are replaced.
   - :jumpto :: replaces the %& escape; if t, org jumps to the target location
                when the note is stored.

* New features
** Adding non-headline items

   For some time I have wanted to be able to use remember to add checklist
   entries and table rows as well as org headlines. To configure this, a :type
   parameter will be added to the template, which can be either headline (the
   default), list, checklist or table.

   - Table rows.

     Currently if you want to use org to record periodic measurements (for
     example see the thread about using org to manage fitness training), you
     have to use properties and column view, which has a number of limitations
     (speed, calculations). Being able to add table rows via remember would make
     it much easier to do this.

     The simplest implementation would use a template containing the appropriate
     number of table columns, for example something like

     "| %U | %^{Value 1} | %^{Value 2} |"

     This would be added to the table at the appropriate position (depending on
     the :prepend value for the template, and then formatted properly using
     `org-table-align'.

     The handler would also need to ensure that table formulae get updated
     (increment row ranges etc) and that values are recalculated (if automatic
     recalculation is enabled) after the line is added.

     An extension to this would be to include a truncated copy of the table in
     the remember buffer, with just the headers (and possibly formulae) from the
     target table, so the user could add multiple lines in the remember buffer
     and then add them to the table.

   - Checklist entries

     I use checklists a lot for breaking down tasks. Being able to add checklist
     entries using remember, with the clocked task as a target, would make it
     possible to do this without having to switch away from the buffer I was
     working in.

   - Plain list entries.

     It also be useful to be able to add plain list items. This would be like a
     more featureful version of `org-add-note', with the full range of remember
     expansions available.

** Updating completion statistics

   If the target for a remember template contains either of the completion
   status tracking cookies ([/}, [%]), these should be updated after the entry
   has been added. This should apply to both entries containing checklists, and
   entries containing other headlines.

   Also, target matching for templates should ignore the completion status
   cookies.

** Per-template insertion order

   It is currently possible to configure a remember template to add entries at
   the start or end of the target only if the target is a file, or set globally
   with `org-reverse-note-order'. It would be useful to be able to set this for
   individual templates.

   An optional :prepend parameter would control where the entry would be
   inserted. If set to t the new entry would be the first under the heading, and
   the last the if nil. The default would be whatever `org-reverse-note-order'
   is set to.

** Automatic sorting

   It would be useful to be able to automatically sort the target's entries
   after adding one using remember, for example if you want to keep entries
   sorted by TODO keyword or in alphabetical order.

   This would require two parameters:
   - :sort :: if non-nil, sort the contents of the headline the item was
               added under with the specified function. valid options are
     - todoorder :: sort by TODO state
     - alpha :: sort alphabetically
     - numeric :: sort numerically
     - priority :: sort by priority
     - time :: sort by time
     - property :: sort by property
     - <function> :: use function to determine sort order
     - Additionally, any of the previous items postfixed with '-rev' will sort
       in reverse
   - :sortkey :: for tables, and property searches, this specifies either the
                 column or property to sort by.

* Development plan

  It will take me quite a while to implement all this so I plan on doing it in
  stages. After the preliminary work is done, each new feature will be
  implemented in a separate git branch which can be merged into the master
  branch when it's finished and properly tested. I'll post an update to the list
  when each feature is ready for testing.
  
James

--
|-<James TD Smith>-<email/address@hidden>-|