[Orgmode] Re: [BABEL] "unset" :var definitions for subtree

[Dan Davison]

Rainer M Krug <address@hidden> writes:

> On 02/11/2011 10:32 AM, Dan Davison wrote:
>> Rainer M Krug <address@hidden> writes:
>>
>>> On 02/10/2011 05:48 PM, Eric Schulte wrote:
>>>> Rainer M Krug <address@hidden> writes:
>>>>
>>>>> On 02/10/2011 02:27 AM, Eric Schulte wrote:
>>>>>> Rainer M Krug <address@hidden> writes:
>>>>>>
>>>>>>> Hi
>>>>>>>
>>>>>>> For one project, I am usinr org to write submit scripte to a cluster
>>>>>>> runing torqu. The important bit in this is, that between the shebang and
>>>>>>> the code, no other executable line must occur. As I am using variables
>>>>>>> in org (:var) they will occur just after the shebang, which causes a
>>>>>>> problem for torque. So, my question is, is there a way to "unset"
>>>>>>> variables defined by using :var for a subtree?
>>>>>>>
>>>>>>
>>>>>> Hi Rainer,
>>>>>>
>>>>>> Interesting question... unfortunately I don't think that removing
>>>>>> variables from header arguments is possible under the current setup.
>>>>>>
>>>>>> Perhaps in your case you could add a function to the post-tangle hook,
>>>>>> which recognizes when it is being called in a just-tangled torqu script
>>>>>> (maybe by searching for a series of #PBS lines), and then removes any
>>>>>> lines between the shebang and the first #PBS line?
>>>>>
>>>>> That is also an option - what I am using at the moment is to use
>>>>> :no-expand as a code block specific header argument. But this raises the
>>>>> other question:
>>>>>
>>>>> Can I set the :no-expand in a properties block? As far as I understand,
>>>>> in the properties block I have the argument and the value - but what do
>>>>> I do with :noexpand?
>>>>>
>>>>> :PROPERTIES:
>>>>> :var: A=13
>>>>> :no-expand
>>>>> :END:
>>>>>
>>>>
>>>> You can just set it to "yes" or really any value you like (the value
>>>> will be ignored).  I did however have to add "no-expand" to the list of
>>>> header argument names searched for in property blocks -- I just pushed
>>>> up a patch to this effect.
>>>
>>> Thanks - I'll try it today and come back if it does not work.
>>>
>>>>
>>>>>
>>>>>>
>>>>>> More generally, I wonder what a natural method would be to allow
>>>>>> unsetting of pre-set header arguments for local blocks or subtrees?
>>>>>> This may only apply to the :var header argument, as most others have a
>>>>>> default setting which can be actively set.  If you have any ideas for a
>>>>>> natural syntax for such an operation I'd be happy to hear it.
>>>>>
>>>>> First solution (probably the easiest to implement) would be along the
>>>>> lines of the :no-expand header argument -
>>>>>
>>>>> :expand-var yes
>>>>> and
>>>>> :expand-var no
>>>>>
>>>>> This could possibly be expanded to
>>>>> :expand-var A B C
>>>>>
>>>>> which would expand only the variables A B and C
>>>>>
>>>>> One step further: one could define groups of variables, like
>>>>> :var-group X=A,B,C
>>>>> or a similar syntax
>>>>>
>>>>> and then
>>>>> :expand-var X
>>>>> would expand A B and C
>>>>>
>>>>> This all would not be real unset - but a possibility for unsetting would 
>>>>> be
>>>>>
>>>>> :var B=
>>>>>
>>>>> or
>>>>>
>>>>> :var-unset B
>>>>>
>>>>> i.e. if no value is specified in :var, the variable will be removed
>>>>> (i.e. unset) - one could also use a null value (if it exists in elisp):
>>>>>
>>>>> :var B=(null)
>>>>>
>>>>
>>>> Thanks for the ideas,
>>>>
>>>> I think you're right that something along the lines of the above should
>>>> be the easiest to implement, however after reading these suggestions,
>>>> I'm thinking that more generally there are a couple of other header
>>>> arguments which could need to be unset, namely
>>>> - file
>>>> - dir
>>>> - session
>>>> - shebang
>>>> some of these (like session) accept a "none" value which has the effect
>>>> of un-setting the header argument.
>>>
>>> True - haven't thought about those (and did not know about :dir useful
>>> one!). And the :session might definitely come in handy - I have cases,
>>> where I reset it manually before evaluating certain sections of the block.
>>>
>>>>
>>>> It would be nice to generalize whatever solution we apply across all
>>>> types of header argument (both for implementation and for user
>>>> simplicity).
>>>
>>> Absolutely - coherent solutions are definitely the best.
>>>
>>>> The simplest option would probably be to ensure that
>>>> setting any header argument to :none would remove all instances of that
>>>> header argument.
>>>
>>> Agreed - makes perfect sense. But probably for readibility use something
>>> like:
>>>
>>> : header :remove()
>>>
>>> or
>>>
>>> :header :remove
>>>
>>>> The only problem there is cases like var, where you
>>>> might not want to remove all :var's.  Maybe this could be expanded
>>>> s.t. :none could take arguments, e.g.
>>>>
>>>> :header :none(A, B)
>>>>
>>>> which would remove all instances of the "header" header argument whose
>>>> value is or is named "A" or "B"?
>>>
>>> I would stick to the name of the variable - that is more consistent.
>>>
>>> But instead of :none() I would suggest :remove :
>>>
>>> :header :remove(A, B)
>>>
>>> and if one wants to remove all variables with *value "A"*, one could use
>>>
>>> :header :remove("A")
>>>
>>> Or does that look too funky?
>>>
>>> No - I like it.
>>
>> I'm concerned that all this is looking rather complex. And I'm a bit
>> dubious about the :xxx syntax -- those should correspond to keys in an
>> association list. Could we step back a moment -- would someone mind
>> giving me a concrete example of a problem whose solution requires these
>> new features?
>

Hi Rainer,

> Here is my situation:
>
> I am using org-babel to write analysis for a cluster environment, which
> uses torque. To make my scripts flexible, I am using variables defined
> in the beginning of the org document.
> Everything works fine for after tangling in most of the scripts, but in
> the submission script for the cluster (a nash scripd with definitions
> for processing of the cluster job, following *directly/* after the
> shebang, this causes problems, as the options are expected directly
> after the shebang, and when variables are defined in the org file, these
> will be included directly after the shebang. This results in torque not
> finding the parameter it needs to successfully process the job.
>
> So at the moment, I am using :no-expand to avoid this,

But don't you need the variables?

OK, so let's have a concrete example to play with. Here are two ideas --
are either of these any good for you?

1. Use noweb to include the variables

-------------------------------------------
#+srcname: set-variables
#+begin_src sh
A=13
B=14
#+end_src

#+begin_src sh :tangle script.sh :noweb yes :shebang #!/bin/bash
#$ -cwd
<<set-variables>>
#+end_src
-------------------------------------------

2. It should be possible somehow to include the options together with
   the shebang, or to replace :shebang with a more general "preamble"
   concept. Or to make :shebang accept a code block as its value.

--------------------------------------------
#+BABEL: :var A=13
#+BABEL: :var B=14

#+header: :shebang #!/bin/bash #$ -cwd
#+begin_src sh :tangle script.sh
# empty
#+end_src
--------------------------------------------

[...]

> I must admit, I am not clear what you men with "those should correspond
> to keys in an association list" - I am definitely not a lisp / elisp
> person and only a regular user of emacs / org mode / ESS.

In the header lines, words that start with a colon are the *names* of
options, and the bits inbetween are the *values* of those options. So
I'm just whinging about using a word starting with a colon for something
which is essentially a value.

I'm going to put some separate suggestions that involve code changes in
a separate reply.

Dan



>
> Hope this clarifies my suggestion a bit,
>
> Cheers,
>
> Rainer
>
>
>>
>>>
>>> For consistency, one could also have a function :set() which could be
>>> used as follow:
>>>
>>> :header :set(A=12, B=13)
>>>
>>> to set the "header" header argument A to 12 and B to 13.
>>>
>>> And then probably use :unset instead of :remove?
>>>
>>> Just thinking along while I am typing...
>>>
>>>
>>>>
>>>>>
>>>>> But this raises another question of mine:
>>>>>
>>>>> I tried to use two :var headers in a properties block, but it only used
>>>>> the first - did I miss something here?
>>>>>
>>>>
>>>> Nope, it appears that property blocks (like a hash) assume that there is
>>>> only one instance of each key->value pair, so once it is satisfied it
>>>> will quit looking for more instances.
>>>
>>> But not always - there are exceptions, e.g. #+LATEX_HEADER: of which I
>>> can specify more then one.
>>> I am asking, because I have a horribly long line of #+BABEL: :var ....
>>> and would really like to split it in two.
>>
>> I believe that I fixed that at 05ef2ae7c (2011/01/20) -- i.e., you can
>> split it in two.
>
> Org-mode version 7.4 (release_7.4.338.gda8dc) (updated this morning)
>
> Thanks - that makes life easier. When I use
>
> #+BABEL: :var A=13
> #+BABEL: :var B=14
>
> Both are defined - but it still does not work in a properties block (as
> in the Eris's example).
>
>
>
>>
>> Dan
>>
>>>
>>>> Maybe the following alternative
>>>> will suffice
>>>>
>>>> Best -- Eric
>>>>
>>>> ** two vars in a properties block -- not possible
>>>>    :PROPERTIES:
>>>>    :var:      test1=7
>>>>    :var:      test2=8
>>>>    :END:
>>>>
>>>> #+begin_src emacs-lisp
>>>>   (message "test1=%S test2=%S" test1 test2)
>>>> #+end_src
>>>>
>>>> results in Error
>>>> : let: Symbol's value as variable is void: test2
>>>>
>>>> *** an alternative
>>>>     :PROPERTIES:
>>>>     :var:      tests=all-tests
>>>>     :END:
>>>>
>>>> #+tblname: all-tests
>>>> - 7
>>>> - 8
>>>>
>>>> #+begin_src emacs-lisp :var eric=89
>>>>   (message "test1=%S test2=%S" (first tests) (second tests))
>>>> #+end_src
>>>>
>>>> #+results:
>>>> : test1=7 test2=8
>>>
>>> That's sneaky - I like that solution. But I would have to rename my
>>> variables in my code blocks - but I must say I like it. I'll probably
>>> use this - it keeps the variables nicely together.
>>>
>>> Just an idea:
>>>
>>> :var: :in_table(all-variables)
>>>
>>> #+tblname: all-tests
>>> - 7
>>> - 8
>>>
>>> #+tblname: all-variables
>>> | A    | 20           |
>>> | B    | "next value" |
>>> | C    | 13           |
>>> | test | all-tests    |
>>>
>>> which would create the variables A with value 20, B with value "next
>>> value" and C with value 13, and the variable tests would be based on the
>>> table all-tests (i.e. test1 = 7, test2 = 8).
>>>
>>> That would be *really* neat and useful to use. as one would have a
>>> nicely formatted table of all variables and their values.
>>>
>>> One could even go one step further and say that when no value is
>>> specified in the second column, the variable is removed and if the
>>> variable already exists it is overwritten.
>>>
>>> In this case, the limitation of one line :var only would not be a
>>> problem any more.
>>>
>>> While we are at variables, it could be useful to rename or copy
>>> variables as well:
>>>
>>> rename A to B:
>>>
>>> :var :rename(A, B)
>>>
>>> or copy A to B
>>>
>>> :var :copy(A, B)
>>>
>>> But probably not that useful.
>>>
>>> Cheers and thanks,
>>>
>>> Rainer