Re: [O] Allow #+SETUPFILE to point to an URL for the org file

[Kaushal Modi]

On Thu, May 25, 2017, 6:15 AM Nicolas Goaziou <address@hidden> wrote:

Interactive functions do not have double-dashes in their names. However,
I have concerns about this interactive status. Given than the function
is not properly documented in the manual, there is little chance it will
be actually used. And if it isn't, it could return surprising results.

Another idea would be to replace NOCACHE with CLEAR-CACHE. When this is
non-nil, the cache is reset at the beginning of the function. The point
is to reset the cache the first time the function is called, but not on
recursive calls, which ensures any file is retrieved only once.

Here is my use case for org-file-clear-cache:

Let's say I have a file where I have a SETUPFILE retrieved from a URL. Now the upstream version changes but my cache is still on the older version. So I need to clear the hash. The org-file-clear-cache simply does that. 

With the function being interactive, I just do 

- M-x org-file-clear-cache

- C-c C-e h h (or whatever I am exporting to)

If you suggest a node where I should put that in the manual, I can add that to my updated patch. I'll all add more explanation to the doc-string of that function.  

Now, if the CLEAR-CACHE argument is added to org-file-clear-cache, how do we control the cache clearing interactively from outside? 

Also, how do we implement the resetting of the cache only the first time the function is called? Wouldn't that need an extra alist defvar to record the state of whether the function is already called specifically for that file? I think that would unnecessary complicate the logic. 

Another idea is that we have a defcustom like org-file-never-cache. When non-nil, that will always do a fresh URL download. This will be or'ed with the NOCACHE inside org-file-contents. This, though, makes it a bit inconvenient for the user to use the latest upstream version when they need... They might need to set org-file-never-cache to t momentarily, probably via Local Variables, before an export. 

 Of

course the cache doesn't survive to multiple exports, but at least it is
transparent to the user.

Sorry, I didn't follow that. Did you mean that the cache doesn't survive between emacs sessions? Because the cache will actually survive between multiple exports. 

Yet another idea is to add a time-to-live to cached values and remove
them from cache past it. However, I prefer the idea above.

The specific reason is, as you noticed, %S wraps file name within double
quotes. IMO, `...' would be for symbols or key bindings.

I'll replace `%s' with "%S".. I just liked the curly quotes created by ` '. But if that breaks a convention, I rather not do that. 

`prog1' is difficult to read when the first SEXP is large. Also, we
should avoid splitting error messages and constructing them back, for
hypothetical i18n considerations.

I suggest the following re-factoring, where I limited the number of
bindings and remove some trivial comments.

  (defun org-file-contents (file &optional noerror nocache)
    "Return the contents of FILE, as a string.

  FILE can be a file name or URL.

  If FILE is a URL, download the contents.  If the URL contents are
  already cached in the `org--file-cache' hash table, the download step
  is skipped.

  If NOERROR is non-nil, ignore the error when unable to read the FILE
  from file or URL.

  If NOCACHE is non-nil, do a fresh fetch of FILE even if cached version
  is available.  This option applies only if FILE is a URL."
    (let* ((is-url (org-file-url-p file))
           (cache (and is-url
                       (not nocache)
                       (gethash file org--file-cache))))
      (cond
       (cache)
       (is-url
        (with-current-buffer (url-retrieve-synchronously file)
          (goto-char (point-min))
          (if (re-search-forward "HTTP.*\\s-+200\\s-OK" nil t)
              ;; URL retrieved correctly.  Move point to after the
              ;; url-retrieve header, update the cache `org--file-cache'
              ;; and return contents.
              (progn
                (search-forward "\n\n" nil 'move)
                (puthash file
                         (buffer-substring-no-properties (point) (point-max))
                         org--file-cache))
            (funcall (if noerror #'message #'user-error)
                     (format "Unable to fetch file from %S" file)))))
       (t
        (with-temp-buffer
          (condition-case err
              (progn (insert-file-contents file) (buffer-string))
            (file-error
             (funcall (if noerror #'message #'user-error)
                      (error-message-string err)))))))))

I'm not sure about the return value when NOERROR is non-nil, but it may
not matter, per the suggestion above.

I'll integrate your suggestion.

> Do we need to update the code using org-file-contents in these places:

What do you meant by "update"?

For the listed locations of org-file-contents instances in my earlier email, I made no change as I think that those would work the same as before after this commit is applied. I just wanted you to verify if that's the case. The only "update" required around org-file-contents was where expand-file-name was used or default-directory was set (as my patch shows). 

Thanks!

--

Kaushal Modi