Re: [Orgmode] [PATCH] sha1 hash of latex fragments to avoid regeneration

[Eric Schulte]

Hi Carsten,

I just pulled, reloaded, and re-ran my simple tests and the patch
appears to have been applied successfully.

Thanks -- Eric

Carsten Dominik <address@hidden> writes:

> Hi Eric,
>
> I had a problem while pushing, please verify that the patch got in
> correctly.  Thanks!
>
> - Carsten
>
> On Nov 17, 2009, at 4:24 PM, Eric Schulte wrote:
>
>> Carsten Dominik <address@hidden> writes:
>>
>>> Wow, this is fantastic!
>>>
>>> Do you think it is ready to be included (because you say first
>>> pass...)
>>>
>>
>> Yes,
>>
>> I said first pass because I had only done minimal testing.  However
>> all
>> indications are that it works, and there are no further changes I
>> would
>> like to make, so if it looks good to you I would say "yes", please
>> apply
>> it.
>>
>> Thanks -- Eric
>>
>>>
>>> - Carsten
>>>
>>> On Nov 17, 2009, at 3:42 AM, Eric Schulte wrote:
>>>
>>>> "Eric Schulte" <address@hidden> writes:
>>>>
>>>>> Hi Carsten,
>>>>>
>>>>> Thanks for the feedback, I have comments inline below
>>>>>
>>>>> Carsten Dominik <address@hidden> writes:
>>>>
>>>> [...]
>>>>
>>>>>> Now, I am sure that you are already planning to do the same
>>>>>> for ditaa images etc?
>>>>>
>>>>> of course :)
>>>>
>>>> A first pass at a patch implementing caching of ditaa and dot images
>>>> generated by org-exp-blocks is attached.  It seems to work in all
>>>> initial tests, and it will only remove files which match the
>>>> following
>>>> pattern
>>>>
>>>> "beginning-of-file-name_\\([[:alnum:]]+\\)\\.extension"
>>>>
>>>> such that the length of the part matched by \\([[:alnum:]]+\\) is 40
>>>> characters.
>>>>
>>>> Best -- Eric
>>>>
>>>> From bd90d519a7d3e16732f4f377ccce51bdb97cef90 Mon Sep 17 00:00:00
>>>> 2001
>>>> From: Eric Schulte <address@hidden>
>>>> Date: Mon, 16 Nov 2009 19:33:11 -0700
>>>> Subject: [PATCH] use sha1 hash keys to cache ditaa and dot images
>>>> when exporting through org-exp-blocks
>>>>
>>>> ---
>>>> lisp/ChangeLog         |    6 ++++
>>>> lisp/org-exp-blocks.el |   60 +++++++++++++++++++++++++++++++++++++
>>>> +---------
>>>> 2 files changed, 54 insertions(+), 12 deletions(-)
>>>>
>>>> diff --git a/lisp/ChangeLog b/lisp/ChangeLog
>>>> index 5f83aaa..c2d44fa 100755
>>>> --- a/lisp/ChangeLog
>>>> +++ b/lisp/ChangeLog
>>>> @@ -1,3 +1,9 @@
>>>> +2009-11-17  Eric Schulte  <address@hidden>
>>>> +
>>>> +  * org-exp-blocks.el (org-export-blocks-format-ditaa): Use sha1
>>>> +  hash keys to cache and re-use images generated by the
>>>> +  org-exp-blocks interface to ditaa and dot.
>>>> +
>>>> 2009-11-16  Carsten Dominik  <address@hidden>
>>>>
>>>>    * org-html.el (org-export-html-home/up-format): Add an ID to the
>>>> diff --git a/lisp/org-exp-blocks.el b/lisp/org-exp-blocks.el
>>>> index 71e1608..72fe6c4 100644
>>>> --- a/lisp/org-exp-blocks.el
>>>> +++ b/lisp/org-exp-blocks.el
>>>> @@ -217,9 +217,15 @@ Specify the path at which the image should be
>>>> saved as the first
>>>> element of headers, any additional elements of headers will be
>>>> passed to the ditaa utility as command line arguments."
>>>>  (message "ditaa-formatting...")
>>>> -  (let ((out-file (if headers (car headers)))
>>>> -  (args (if (cdr headers) (mapconcat 'identity (cdr headers) " ")))
>>>> -  (data-file (make-temp-file "org-ditaa")))
>>>> +  (let* ((args (if (cdr headers) (mapconcat 'identity (cdr headers)
>>>> " ")))
>>>> +         (data-file (make-temp-file "org-ditaa"))
>>>> +         (hash (sha1 (prin1-to-string (list body args))))
>>>> +         (raw-out-file (if headers (car headers)))
>>>> +         (out-file-parts (if (string-match
>>>> "\\(.+\\)\\.\\([^\\.]+\\)
>>>> $" raw-out-file)
>>>> +                             (cons (match-string 1 raw-out-file)
>>>> +                                   (match-string 2 raw-out-file))
>>>> +                           (cons raw-out-file "png")))
>>>> +         (out-file (concat (car out-file-parts) "_" hash "." (cdr
>>>> out-file-parts))))
>>>>    (unless (file-exists-p org-ditaa-jar-path)
>>>>      (error (format "Could not find ditaa.jar at %s" org-ditaa-jar-
>>>> path)))
>>>>    (setq body (if (string-match "^\\([^:\\|:[^ ]\\)" body)
>>>> @@ -229,9 +235,21 @@ passed to the ditaa utility as command line
>>>> arguments."
>>>>                        "\n")))
>>>>    (cond
>>>>     ((or htmlp latexp docbookp)
>>>> -      (with-temp-file data-file (insert body))
>>>> -      (message (concat "java -jar " org-ditaa-jar-path " " args " "
>>>> data-file " " out-file))
>>>> -      (shell-command (concat "java -jar " org-ditaa-jar-path " "
>>>> args " " data-file " " out-file))
>>>> +      (unless (file-exists-p out-file)
>>>> +        (mapc ;; remove old hashed versions of this file
>>>> +         (lambda (file)
>>>> +           (when (and (string-match (concat (regexp-quote (car out-
>>>> file-parts))
>>>> +                                            "_\\([[:alnum:]]+\\)\
>>>> \."
>>>> +                                            (regexp-quote (cdr out-
>>>> file-parts)))
>>>> +                                    file)
>>>> +                      (= (length (match-string 1 out-file)) 40))
>>>> +             (delete-file (expand-file-name file
>>>> +                                            (file-name-directory
>>>> out-file)))))
>>>> +         (directory-files (or (file-name-directory out-file)
>>>> +                              default-directory)))
>>>> +        (with-temp-file data-file (insert body))
>>>> +        (message (concat "java -jar " org-ditaa-jar-path " " args "
>>>> " data-file " " out-file))
>>>> +        (shell-command (concat "java -jar " org-ditaa-jar-path " "
>>>> args " " data-file " " out-file)))
>>>>      (format "\n[[file:%s]]\n" out-file))
>>>>     (t (concat
>>>>     "\n#+BEGIN_EXAMPLE\n"
>>>> @@ -259,14 +277,32 @@ digraph data_relationships {
>>>> }
>>>> #+end_dot"
>>>>  (message "dot-formatting...")
>>>> -  (let ((out-file (if headers (car headers)))
>>>> -  (args (if (cdr headers) (mapconcat 'identity (cdr headers) " ")))
>>>> -  (data-file (make-temp-file "org-ditaa")))
>>>> +  (let* ((args (if (cdr headers) (mapconcat 'identity (cdr headers)
>>>> " ")))
>>>> +         (data-file (make-temp-file "org-ditaa"))
>>>> +         (hash (sha1 (prin1-to-string (list body args))))
>>>> +         (raw-out-file (if headers (car headers)))
>>>> +         (out-file-parts (if (string-match
>>>> "\\(.+\\)\\.\\([^\\.]+\\)
>>>> $" raw-out-file)
>>>> +                             (cons (match-string 1 raw-out-file)
>>>> +                                   (match-string 2 raw-out-file))
>>>> +                           (cons raw-out-file "png")))
>>>> +         (out-file (concat (car out-file-parts) "_" hash "." (cdr
>>>> out-file-parts))))
>>>>    (cond
>>>>     ((or htmlp latexp docbookp)
>>>> -      (with-temp-file data-file (insert body))
>>>> -      (message (concat "dot " data-file " " args " -o " out-file))
>>>> -      (shell-command (concat "dot " data-file " " args " -o " out-
>>>> file))
>>>> +      (unless (file-exists-p out-file)
>>>> +        (mapc ;; remove old hashed versions of this file
>>>> +         (lambda (file)
>>>> +           (when (and (string-match (concat (regexp-quote (car out-
>>>> file-parts))
>>>> +                                            "_\\([[:alnum:]]+\\)\
>>>> \."
>>>> +                                            (regexp-quote (cdr out-
>>>> file-parts)))
>>>> +                                    file)
>>>> +                      (= (length (match-string 1 out-file)) 40))
>>>> +             (delete-file (expand-file-name file
>>>> +                                            (file-name-directory
>>>> out-file)))))
>>>> +         (directory-files (or (file-name-directory out-file)
>>>> +                              default-directory)))
>>>> +        (with-temp-file data-file (insert body))
>>>> +        (message (concat "dot " data-file " " args " -o " out-
>>>> file))
>>>> +        (shell-command (concat "dot " data-file " " args " -o "
>>>> out-
>>>> file)))
>>>>      (format "\n[[file:%s]]\n" out-file))
>>>>     (t (concat
>>>>     "\n#+BEGIN_EXAMPLE\n"
>>>> --
>>>> 1.6.4.73.gc144
>>>>
>>>
>>> - Carsten
>
> - Carsten