package.el 0.3

[Tom Tromey]

This is version 0.3 of package.el, a simple package manager for Emacs.

package.el is a simple way to download, install, and manage Emacs
packages.  It is connected to ELPA, the Emacs Lisp Package Archive:

    http://tromey.com/elpa/

The web pages have an auto-installer, which is by far the simplest way
to get up and running with package.el.  It literally takes less than a
minute, and you don't have to edit anything.

This is a relatively minor update.  New since the last release:

* On Emacs 22, no longer require an external 'tar' program
* Added bug-reporting function
* Show package description in package menu
* Added help key bindings to package menu mode

For future plans, see the to-do list in the code.

Please send comments, suggestions, complaints, etc, to me.
I will try to address them in future releases.  Advice on how to make
package.el and ELPA more successful is appreciated :-)

Package maintainers, please send me your packages.  There is some info
on the web page about formatting requirements (basically the same as
what is in the Elisp reference manual).

ELPA has seen a few updates and has some nice code in it now... a few
games, a few major modes, some minor modes.  I hope to upload a few
larger packages soon.

Tom

;;; package.el --- Simple package system for Emacs

;; Copyright (C) 2007 Tom Tromey <address@hidden>

;; Author: Tom Tromey <address@hidden>
;; Created: 10 Mar 2007
;; Version: 0.3
;; Keywords: tools

;; This file is not (yet) part of GNU Emacs.
;; However, it is distributed under the same license.

;; GNU Emacs is free software; you can redistribute it and/or modify
;; it under the terms of the GNU General Public License as published by
;; the Free Software Foundation; either version 2, or (at your option)
;; any later version.

;; GNU Emacs is distributed in the hope that it will be useful,
;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;; GNU General Public License for more details.

;; You should have received a copy of the GNU General Public License
;; along with GNU Emacs; see the file COPYING.  If not, write to the
;; Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
;; Boston, MA 02110-1301, USA.

;;; Change Log:

;;  2 Apr 2007 - now using ChangeLog file
;; 15 Mar 2007 - updated documentation
;; 14 Mar 2007 - Changed how obsolete packages are handled
;; 13 Mar 2007 - Wrote package-install-from-buffer
;; 12 Mar 2007 - Wrote package-menu mode

;;; Commentary:

;; To use this, put package.el somewhere on your load-path.  Then add
;; this to your .emacs:
;;
;;    (load "package")
;;    (package-initialize)
;;
;; This will automatically make available the packages you have
;; installed using package.el.  If your .emacs will refer to these
;; packages, you may want to initialize the package manager near the
;; top.
;;
;; Note that if you want to be able to automatically download and
;; install packages from ELPA (the Emacs Lisp Package Archive), then
;; you will need the 'url' package.  This comes with Emacs 22; Emacs
;; 21 users will have to find it elsewhere.
;;
;; If you installed package.el via the auto-installer:
;;
;;    http://tromey.com/elpa/
;;
;; then you do not need to edit your .emacs, as the installer will
;; have done this for you.  The installer will also install the url
;; package if you need it.

;; Other external functions you may want to use:
;;
;; M-x package-list-packages
;;    Enters a mode similar to buffer-menu which lets you manage
;;    packages.  You can choose packages for install (mark with "i",
;;    then "x" to execute) or deletion (not implemented yet), and you
;;    can see what packages are available.  You can update the list of
;;    packages from ELPA using "r".
;;
;; M-x package-install-from-buffer
;;    Install a package consisting of a single .el file that appears
;;    in the current buffer.  This only works for packages which
;;    define a Version header properly; package.el also supports the
;;    extension headers Package-Version (in case Version is an RCS id
;;    or similar), and Package-Requires (if the package requires other
;;    packages).

;; The idea behind package.el is to be able to download packages and
;; install them.  Packages are versioned and have versioned
;; dependencies.  Furthermore, this supports built-in packages which
;; may or may not be newer than user-specified packages.  This makes
;; it possible to upgrade Emacs and automatically disable packages
;; which have moved from external to core.  (Note though that we don't
;; currently register any of these, so this feature does not actually
;; work.)

;; This code supports a single package repository, ELPA.  All packages
;; must be registered there.

;; A package is described by its name and version.  The distribution
;; format is either  a tar file or a single .el file.

;; A tar file should be named "NAME-VERSION.tar".  The tar file must
;; unpack into a directory named after the package and version:
;; "NAME-VERSION".  It must contain a file named "PACKAGE-pkg.el"
;; which consists of a call to define-package.  It may also contain a
;; "dir" file and the info files it references.

;; A .el file will be named "NAME-VERSION.el" in ELPA, but will be
;; installed as simply "NAME.el" in a directory named "NAME-VERSION".

;; The downloader will download all dependent packages.  It will also
;; byte-compile the package's lisp at install time.

;; At activation time we will set up the load-path and the info path,
;; and we will load the package's autoloads.  If a package's
;; dependencies are not available, we will not activate that package.

;; Conceptually a package has multiple state transitions:
;;
;; * Download.  Fetching the package from ELPA.
;; * Install.  Untar the package, or write the .el file, into
;;   ~/.emacs.d/elpa/ directory.
;; * Byte compile.  Currently this phase is done during install,
;;   but we may change this.
;; * Activate.  Evaluate the autoloads for the package to make it
;;   available to the user.
;; * Load.  Actually load the package and run some code from it.

;;; Thanks (sorted by sort-lines):

;; Jim Blandy <address@hidden>
;; Karl Fogel <address@hidden>
;; Michael Olson <address@hidden>
;; Stefan Monnier <address@hidden>
;; Vinicius Jose Latorre <address@hidden>

;;; ToDo:

;; - tramp needs its files to be compiled in a certain order.
;;   how to handle this?  fix tramp?
;; - on emacs 21 we don't kill the -autoloads.el buffer.  what about 22?
;; - maybe we need separate .elc directories for various emacs versions
;;   and also emacs-vs-xemacs.  That way conditional compilation can
;;   work.  But would this break anything?
;; - should store the package's keywords in archive-contents, then
;;   let the users filter the package-menu by keyword.  See
;;   finder-by-keyword.  (We could also let people view the
;;   Commentary, but it isn't clear how useful this is.)
;; - Interface with desktop.el so that restarting after an install
;;   works properly
;; - Implement M-x package-upgrade, to upgrade any/all existing packages
;; - Use hierarchical layout.  PKG/etc PKG/lisp PKG/info
;;   ... except maybe lisp?
;; - It may be nice to have a macro that expands to the package's
;;   private data dir, aka ".../etc".  Or, maybe data-directory
;;   needs to be a list (though this would be less nice)
;; - need to implement package deletion
;; - package menu needs:
;;     font locking
;;     ability to know which packages are built-in & thus not deletable
;;     it can sometimes print odd results, like 0.3 available but 0.4 active
;;        why is that?
;; - Allow multiple versions on the server...?
;;   [ why bother? ]
;; - Don't install a package which will invalidate dependencies overall
;; - Allow something like (or (>= emacs 21.0) (>= xemacs 21.5))
;;   [ currently thinking, why bother.. KISS ]
;; - Allow optional package dependencies
;;   then if we require 'bbdb', bbdb-specific lisp in lisp/bbdb
;;   and just don't compile to add to load path ...?
;; - Have a list of archive URLs?  [ maybe there's no point ]
;; - David Kastrup pointed out on the xemacs list that for GPL it
;;   is friendlier to ship the source tree.  We could "support" that
;;   by just having a "src" subdir in the package.  This isn't ideal
;;   but it probably is not worth trying to support random source
;;   tree layouts, build schemes, etc.
;; - Our treatment of the info path is somewhat bogus


;;; Code:

(defconst package-archive-base "http://tromey.com/elpa/";
  "Base URL for the package archive.
Ordinarily you should not need to edit this.
The default points to ELPA, the Emacs Lisp Package Archive.
Note that some code in package.el assumes that this is an http: URL.")

(defconst package-archive-version 1
  "Version number of the package archive understood by this file.
Lower version numbers than this will probably be understood as well.")

;; Note that this only works if you have the password, which you
;; probably don't :-).  Also if you are using Emacs 21 then you will
;; need to hack ange-ftp-name-format to make this work.
(defconst package-archive-upload-base "/address@hidden@tromey.com:/"
  "Base location for uploading to package archive.")

(defconst package-el-maintainer "Tom Tromey <address@hidden>"
  "The package.el maintainer.")

(defconst package-el-version "0.3"
  "Version of package.el.")

;; Prime the cache.
(defvar package-archive-contents
  '((bubbles . [(0 2) nil "Bubbles (same game) puzzle" tar])
    (newsticker . [(1 10) nil "Headline news ticker" tar])
    )
  "A representation of the contents of the ELPA archive.
This is an alist mapping package names (symbols) to package
descriptor vectors.  These are like the vectors for `package-alist'
but have an extra entry which is 'tar for tar packages and
'single for single-file packages.")

(defvar package-user-dir (expand-file-name "~/.emacs.d/elpa")
  "Name of the directory where the user's packages are stored.")

(defvar package-directory-list
  (list (file-name-as-directory package-user-dir)
        "/usr/share/emacs/site-lisp/elpa/")
  "List of directories to search for packages.")

(defvar package-alist
  (if (>= emacs-major-version 22)
      ;; FIXME: emacs 22 includes tramp, ERC, rcirc, maybe other
      ;; things...
      `((emacs . [(,emacs-major-version ,emacs-minor-version) nil
                  "GNU Emacs"])
        ;; The external URL is version 1.15, so make sure the built-in
        ;; one looks newer.
        (url . [(1 16) nil "URL handling libary"]))
    `((emacs . [(,emacs-major-version ,emacs-minor-version) nil
                "GNU Emacs"])))
  "Alist of all packages available for activation.
Maps the package name to a vector [VERSION REQS DOCSTRING].")

(defvar package-activated-list
  (if (>= emacs-major-version 22)
      '(emacs url)
    '(emacs))
  "List of the names of all activated packages.")

(defvar package-obsolete-alist nil
  "Representation of obsolete packages.
Like package-alist, but maps package name to a second alist.
The inner alist is keyed by version.")

(defun package-version-split (string)
  "Split a package string into a version list."
  (mapcar 'string-to-int (split-string string "[.]")))

(defun package-version-join (l)
  "Turn a list of version numbers into a version string."
  (mapconcat 'int-to-string l "."))

(defun package-version-compare (v1 v2 fun)
  "Compare two version lists according to FUN.
FUN can be <, <=, =, >, >=, or /=."
  (while (and v1 v2 (= (car v1) (car v2)))
    (setq v1 (cdr v1))
    (setq v2 (cdr v2)))
  (setq v1 (if v1 (car v1) 0))
  (setq v2 (if v2 (car v2) 0))
  (funcall fun v1 v2))

(defun package-strip-version (dirname)
  "Strip the version from a combined package name and version.
E.g., if given \"quux-23.0\", will return \"quux\""
  (if (string-match "^\\(.*\\)-[0-9]+\\([.][0-9]+\\)*$" dirname)
      (match-string 1 dirname)))

(defun package-load-descriptor (dir package)
  "Load the description file for a package.
Return nil if the package could not be found."
  (let ((pkg-dir (concat (file-name-as-directory dir) package "/")))
    (if (file-directory-p pkg-dir)
        (load (concat pkg-dir (package-strip-version package) "-pkg") nil t))))

(defun package-load-all-descriptors ()
  "Load descriptors of all packages.
Uses `package-directory-list' to find packages."
  (mapc (lambda (dir)
          (if (file-directory-p dir)
              (mapc (lambda (name)
                      (package-load-descriptor dir name))
                    (directory-files dir nil "^[^.]"))))
        package-directory-list))

(defsubst package-desc-vers (desc)
  "Extract version from a package description vector."
  (aref desc 0))

(defsubst package-desc-reqs (desc)
  "Extract requirements from a package description vector."
  (aref desc 1))

(defsubst package-desc-doc (desc)
  "Extract doc string from a package description vector."
  (aref desc 2))

(defsubst package-desc-kind (desc)
  "Extract the kind of download from an archive package description vector."
  (aref desc 3))

(defun package-do-activate (package pkg-vec)
  (let* ((pkg-name (symbol-name package))
         (pkg-ver-str (package-version-join (package-desc-vers pkg-vec)))
         (dir-list package-directory-list)
         (pkg-dir))
    (while dir-list
      (let ((subdir (concat (car dir-list) pkg-name "-" pkg-ver-str "/")))
        (if (file-directory-p subdir)
            (progn
              (setq pkg-dir subdir)
              (setq dir-list nil))
          (setq dir-list (cdr dir-list)))))
    (unless pkg-dir
      (error "Internal error: could not find directory for %s-%s"
             pkg-name pkg-ver-str))
    (if (file-exists-p (concat pkg-dir "dir"))
        (progn
          ;; FIXME: not the friendliest, but simple.
          (require 'info)
          (info-initialize)
          (setq Info-directory-list (cons pkg-dir Info-directory-list))))
    (setq load-path (cons pkg-dir load-path))
    ;; Load the autoloads and activate the package.
    (load (concat pkg-dir (symbol-name package) "-autoloads")
          nil t)
    (setq package-activated-list (cons package package-activated-list))
    ;; Don't return nil.
    t))

;; FIXME: return a reason instead?
(defun package-activate (package version)
  "Try to activate a package.
Return nil if the package could not be activated.
Recursively activates all dependencies of the named package."
  (or (memq package package-activated-list)
      (let* ((pkg-desc (assq package package-alist))
             (this-version (package-desc-vers (cdr pkg-desc)))
             (req-list (package-desc-reqs (cdr pkg-desc)))
             (keep-going (package-version-compare this-version version '>=)))
        (while (and req-list keep-going)
          (or (package-activate (car (car req-list))
                                (car (cdr (car req-list))))
              (setq keep-going nil))
          (setq req-list (cdr req-list)))
        (if keep-going
            (package-do-activate package (cdr pkg-desc))))))

(defun package-mark-obsolete (package pkg-vec)
  "Put package on the obsolete list, if not already there."
  (let ((elt (assq package package-obsolete-alist)))
    (if elt
        ;; If this obsolete version does not exist in the list, update
        ;; it the list.
        (unless (assoc (package-desc-vers pkg-vec) (cdr elt))
          (setcdr elt (cons (cons (package-desc-vers pkg-vec) pkg-vec)
                            (cdr elt))))
      ;; Make a new association.
      (setq package-obsolete-alist
            (cons (cons package (list (cons (package-desc-vers pkg-vec)
                                            pkg-vec)))
                  package-obsolete-alist)))))

;; (define-package "emacs" "21.4.1" "GNU Emacs core package.")
;; (define-package "erc" "5.1" "ERC - irc client" '((emacs "21.0")))
(defun define-package (name-str version-string
                            &optional docstring
                            &optional requirements)
  "Define a new package.
NAME is the name of the package, a string.
VERSION-STRING is the version of the package, a dotted sequence
of integers.
DOCSTRING is the optional description.
REQUIREMENTS is a list of requirements on other packages.
Each requirement is of the form (OTHER-PACKAGE \"VERSION\")."
  (let* ((name (intern name-str))
         (pkg-desc (assq name package-alist))
         (new-version (package-version-split version-string))
         (new-pkg-desc
          (cons name
                (vector new-version
                        (if requirements
                            (mapcar
                             (lambda (elt)
                               (list (car elt)
                                     (package-version-split (car (cdr elt)))))
                             requirements))
                        docstring))))
    ;; Only redefine a package if the redefinition is newer.
    (if (or (not pkg-desc)
            (package-version-compare new-version
                                     (package-desc-vers (cdr pkg-desc))
                                     '>))
        (progn
          (when pkg-desc
            ;; Remove old package and declare it obsolete.
            (setq package-alist (delq pkg-desc package-alist))
            (package-mark-obsolete (car pkg-desc) (cdr pkg-desc)))
          ;; Add package to the alist.
          (setq package-alist (cons new-pkg-desc package-alist)))
      ;; You can have two packages with the same version, for instance
      ;; one in the system package directory and one in your private
      ;; directory.  We just let the first one win.
      (unless (package-version-compare new-version
                                       (package-desc-vers (cdr pkg-desc))
                                       '=)
        ;; The package is born obsolete.
        (package-mark-obsolete (car new-pkg-desc) (cdr new-pkg-desc))))))

;; From Emacs 22.
(defun package-autoload-ensure-default-file (file)
  "Make sure that the autoload file FILE exists and if not create it."
  (unless (file-exists-p file)
    (write-region
     (concat ";;; " (file-name-nondirectory file)
             " --- automatically extracted autoloads\n"
             ";;\n"
             ";;; Code:\n\n"
             "\n;; Local Variables:\n"
             ";; version-control: never\n"
             ";; no-byte-compile: t\n"
             ";; no-update-autoloads: t\n"
             ";; End:\n"
             ";;; " (file-name-nondirectory file)
             " ends here\n")
     nil file))
  file)

(defun package-generate-autoloads (name pkg-dir)
  (let* ((auto-name (concat name "-autoloads.el"))
         (ignore-name (concat name "-pkg.el"))
         (generated-autoload-file (concat pkg-dir auto-name))
         (version-control 'never))
    ;; In Emacs 22 'update-autoloads-from-directories' does not seem
    ;; to be autoloaded...
    (require 'autoload)
    (unless (fboundp 'autoload-ensure-default-file)
      (package-autoload-ensure-default-file generated-autoload-file))
    (update-autoloads-from-directories pkg-dir)))

(defun package-untar-buffer ()
  "Untar the current buffer.
This uses `tar-untar-buffer' if it is available.
Otherwise it uses an external `tar' program.
`default-directory' should be set by the caller."
  (require 'tar-mode)
  (if (fboundp 'tar-untar-buffer)
      (progn
        ;; tar-mode messes with narrowing, so we just let it have the
        ;; whole buffer to play with.
        (delete-region (point-min) (point))
        (tar-mode)
        (tar-untar-buffer))
    ;; FIXME: check the result.
    (call-process-region (point) (point-max) "tar" nil '(nil nil) nil
                         "xf" "-")))

(defun package-unpack (name version)
  (let ((pkg-dir (concat (file-name-as-directory package-user-dir)
                         (symbol-name name) "-" version "/")))
    ;; Be careful!!
    (make-directory package-user-dir t)
    (if (file-directory-p pkg-dir)
        (mapc (lambda (file) nil) ; 'delete-file -- FIXME: when we're
                                  ; more confident
              (directory-files pkg-dir t "^[^.]")))
    (let* ((default-directory (file-name-as-directory package-user-dir)))
      (package-untar-buffer)
      (package-generate-autoloads (symbol-name name) pkg-dir)
      (let ((load-path (cons pkg-dir load-path)))
        (byte-recompile-directory pkg-dir 0 t)))))

(defun package-unpack-single (file-name version desc requires)
  "Install the contents of the current buffer as a package."
  (let* ((dir (file-name-as-directory package-user-dir))
         (pkg-dir (file-name-as-directory
                   (concat dir file-name "-" version))))
          (make-directory pkg-dir t)
          (write-region (point-min) (point-max)
                        (concat pkg-dir file-name ".el")
                        nil nil nil 'excl)
          (write-region (concat
                         (prin1-to-string (list 'define-package
                                                file-name
                                                version
                                                desc
                                                requires))
                         "\n")
                        nil
                        (concat pkg-dir file-name "-pkg.el")
                        nil nil nil 'excl)
          (package-generate-autoloads file-name pkg-dir)
          (let ((load-path (cons pkg-dir load-path)))
            (byte-recompile-directory pkg-dir 0 t))))

(defun package-handle-response ()
  "Handle the response from the server.
This parse the HTTP response and throws if an error occurred.
The url package seems to require extra processing for this.
This should be called in a save-excursion, in the download buffer.
It will move point to somewhere in the headers."
  ;; We assume HTTP here.
  (let ((response (url-http-parse-response)))
    (when (or (< response 200) (>= response 300))
      (display-buffer (current-buffer))
      (error "Error during download request:%s"
             (buffer-substring-no-properties (point) (progn
                                                       (end-of-line)
                                                       (point)))))))

(defun package-download-single (name version desc requires)
  "Download and install a single-file package."
  (let ((buffer (url-retrieve-synchronously
                 (concat package-archive-base
                         (symbol-name name) "-" version ".el"))))
    (save-excursion
      (set-buffer buffer)
      (package-handle-response)
      (re-search-forward "^$" nil 'move)
      (forward-char)
      (delete-region (point-min) (point))
      (package-unpack-single (symbol-name name) version desc requires)
      (kill-buffer buffer))))

(defun package-download-tar (name version)
  "Download and install a tar package."
  (let ((tar-buffer (url-retrieve-synchronously
                     (concat package-archive-base
                             (symbol-name name) "-" version ".tar"))))
    (save-excursion
      (set-buffer tar-buffer)
      (package-handle-response)
      (re-search-forward "^$" nil 'move)
      (forward-char)
      (package-unpack name version)
      (kill-buffer tar-buffer))))

(defun package-installed-p (package version)
  (let ((pkg-desc (assq package package-alist)))
    (and pkg-desc
         (package-version-compare version
                                  (package-desc-vers (cdr pkg-desc))
                                  '>=))))

(defun package-compute-transaction (result requirements)
  (while requirements
    (let* ((elt (car requirements))
           (next-pkg (car elt))
           (next-version (car (cdr elt))))
      (unless (package-installed-p next-pkg next-version)
        (let ((pkg-desc (assq next-pkg package-archive-contents)))
          (unless pkg-desc
            (error "Package '%s' not available for installation."
                   (symbol-name next-pkg)))
          (unless (package-version-compare (package-desc-vers (cdr pkg-desc))
                                           next-version
                                           '>=)
            (error
             "Need package '%s' with version %s, but only %s is available."
             (symbol-name next-pkg) (package-version-join next-version)
             (package-version-join (package-desc-vers (cdr pkg-desc)))))
          ;; Only add to the transaction if we don't already have it.
          (unless (memq next-pkg result)
            (setq result (cons next-pkg result)))
          (setq result
                (package-compute-transaction result
                                             (package-desc-reqs
                                              (cdr pkg-desc)))))))
    (setq requirements (cdr requirements)))
  result)

(defun package-read-from-string (str)
  "Read a Lisp expression from STR.
Signal an error if the entire string was not used."
  (let* ((read-data (read-from-string str))
          (more-left
             (condition-case nil
                       ;; The call to `ignore' suppresses a compiler warning.
                       (progn (ignore (read-from-string
                                       (substring str (cdr read-data))))
                                   t)
                   (end-of-file nil))))
    (if more-left
        (error "Can't read whole string")
      (car read-data))))

(defun package-read-archive-contents ()
  "Re-read `archive-contents', if it exists.
Will set `package-archive-contents' if successful.
Will throw an error if the archive version is too new."
  (let ((filename (concat (file-name-as-directory package-user-dir)
                          "archive-contents")))
    (if (file-exists-p filename)
        (with-temp-buffer
          (insert-file-contents-literally filename)
          (let ((contents (package-read-from-string
                           (buffer-substring-no-properties (point-min)
                                                           (point-max)))))
            (if (> (car contents) package-archive-version)
                (error "Package archive version %d is greater than %d"
                       (car contents) package-archive-version))
            (setq package-archive-contents (cdr contents)))))))

(defun package-download-transaction (transaction)
  "Download and install all the packages in the given transaction."
  (mapc (lambda (elt)
          (let* ((desc (cdr (assq elt package-archive-contents)))
                 (v-string (package-version-join (package-desc-vers desc)))
                 (kind (package-desc-kind desc)))
            (cond
             ((eq kind 'tar)
              (package-download-tar elt v-string))
             ((eq kind 'single)
              (package-download-single elt v-string
                                       (package-desc-doc desc)
                                       (package-desc-reqs desc)))
             (t
              (error "Unknown package kind: " (symbol-name kind))))))
        transaction))

(defun package-install (name)
  "Install the package named NAME."
  (interactive
   (list (progn
           ;; Make sure we're using the most recent download of the
           ;; archive.  Maybe we should be updating the archive first?
           (package-read-archive-contents)
           (intern (completing-read "Install package: "
                                    (mapcar (lambda (elt)
                                              (cons (symbol-name (car elt))
                                                    nil))
                                            package-archive-contents)
                                    nil t)))))
  (let ((pkg-desc (assq name package-archive-contents)))
    (unless pkg-desc
      (error "Package '%s' not available for installation."
             (symbol-name name)))
    (let ((transaction
           (package-compute-transaction (list name)
                                        (package-desc-reqs (cdr pkg-desc)))))
      (package-download-transaction transaction)))
  ;; Try to activate it.
  (package-initialize))

(defun package-buffer-info ()
  "Return a vector of information about the package inn the current buffer.
The vector looks like [FILENAME REQUIRES DESCRIPTION VERSION]
FILENAME is the file name, a string.  It does not have the \".el\" extension.
REQUIRES is a requires list, or nil.
DESCRIPTION is the package description (a string).
VERSION is the version, a string.
Throws an exception if the buffer does not contain a conforming package.
If there is a package, narrows the buffer to the file's boundaries.
May narrow buffer or move point even on failure."
  (goto-char (point-min))
  (if (re-search-forward "^;;; \\([^ ]*\\)\\.el --- \\(.*\\)$" nil t)
      (let ((file-name (match-string 1))
            (desc (match-string 2))
            (start (progn (beginning-of-line) (point))))
        (if (search-forward (concat ";;; " file-name ".el ends here"))
            (progn
              ;; Try to include a trailing newline.
              (forward-line)
              (narrow-to-region start (point))
              (require 'lisp-mnt)
              ;; Use some headers we've invented to drive the process.
              (let* ((requires-str (lm-header "package-requires"))
                     (requires (if requires-str
                                   (package-read-from-string requires-str)))
                     ;; Can't just always use "version" because that might
                     ;; return an SCCS id or something equally un-useful
                     ;; for our purposes.
                     (version-str (lm-header "version"))
                     (pkg-version
                      (if (and version-str
                               (string-match "^[0-9.]*$" version-str))
                          version-str
                        (lm-header "package-version"))))
                (unless pkg-version
                  (error
                   "Package does not define a usable \"Version\" or 
\"Package-Version\" header."))
                (set-text-properties 0 (length file-name) nil file-name)
                (set-text-properties 0 (length pkg-version) nil pkg-version)
                (set-text-properties 0 (length desc) nil desc)
                (vector file-name requires desc pkg-version)))
          (error "Package missing a terminating comment.")))
    (error "No starting comment for package.")))

(defun package-install-from-buffer ()
  "Install a package from the current buffer.
The package is assumed to be a single .el file which
follows the elisp comment guidelines; see
info node `(elisp)Library Headers'."
  (interactive)
  (save-excursion
    (save-restriction
      (let* ((pkg-info (package-buffer-info))
             (file-name (aref pkg-info 0))
             (requires (aref pkg-info 1))
             (desc (if (string= (aref pkg-info 2) "")
                       "No description available."
                     (aref pkg-info 2)))
             (pkg-version (aref pkg-info 3)))
        ;; Download and install the dependencies.
        (let ((transaction (package-compute-transaction nil requires)))
          (package-download-transaction transaction))
        ;; Install the package itself.
        (package-unpack-single file-name pkg-version desc requires)
        ;; Try to activate it.
        (package-initialize)))))

(defun package-upload-from-buffer ()
  "Upload a single .el file to ELPA from the current buffer."
  (interactive)
  (save-excursion
    (save-restriction
      ;; Find the package in this buffer.
      (let* ((pkg-info (package-buffer-info))
             (file-name (aref pkg-info 0))
             (pkg-name (intern file-name))
             (requires (aref pkg-info 1))
             (desc (if (string= (aref pkg-info 2) "")
                       (read-string "Description of package: ")
                     (aref pkg-info 2)))
             (pkg-version (aref pkg-info 3))
             (split-version (package-version-split pkg-version))
             (pkg-buffer (current-buffer))

             ;; Download latest archive-contents.
             (buffer (url-retrieve-synchronously
                      (concat package-archive-base "archive-contents"))))

        ;; Parse archive-contents.
        (set-buffer buffer)
        (package-handle-response)
        (re-search-forward "^$" nil 'move)
        (forward-char)
        (delete-region (point-min) (point))
        (let ((contents (package-read-from-string
                         (buffer-substring-no-properties (point-min)
                                                         (point-max))))
              (new-desc (vector split-version requires desc 'single)))
          (if (> (car contents) package-archive-version)
              (error "Unrecognized archive version %d" (car contents)))
          (let ((elt (assq pkg-name (cdr contents))))
            (if elt
                (if (package-version-compare split-version
                                             (package-desc-vers (cdr elt))
                                             '<=)
                    (error "New package has smaller version: %s" pkg-version)
                  (setcdr elt new-desc))
              (setq contents (cons (car contents)
                                   (cons (cons pkg-name new-desc)
                                         (cdr contents))))))

          ;; Now CONTENTS is the updated archive contents.  Upload
          ;; this and the package itself.  For now we assume ELPA is
          ;; writable via file primitives.
          (write-region (concat (pp-to-string contents) "\n")
                        nil
                        (concat package-archive-upload-base
                                "archive-contents"))

          (set-buffer pkg-buffer)
          (kill-buffer buffer)
          (write-region (point-min) (point-max)
                        (concat package-archive-upload-base
                                file-name "-" pkg-version ".el")
                        nil nil nil 'excl))))))

(defun package-gnus-summary-upload ()
  "Upload a package contained in the current *Article* buffer.
This should be invoked from the gnus *Summary* buffer."
  (interactive)
  (save-excursion
    (set-buffer gnus-article-buffer)
    (package-upload-from-buffer)))

(defun package-refresh-contents ()
  "Download the ELPA archive description if needed.
Invoking this will ensure that Emacs knows about the latest versions
of all packages.  This will let Emacs make them available for
download."
  (interactive)
  (let ((buffer (url-retrieve-synchronously
                 (concat package-archive-base "archive-contents"))))
    (save-excursion
      (set-buffer buffer)
      (package-handle-response)
      (re-search-forward "^$" nil 'move)
      (forward-char)
      (delete-region (point-min) (point))
      (setq buffer-file-name (concat (file-name-as-directory package-user-dir)
                                     "archive-contents"))
      (let ((version-control 'never))
        (save-buffer))
      (kill-buffer buffer)))
  (package-read-archive-contents))

(defun package-initialize ()
  "Load all packages and activate as many as possible."
  (package-load-all-descriptors)
  (package-read-archive-contents)
  ;; Try to activate all our packages.
  (mapc (lambda (elt)
          (package-activate (car elt) (package-desc-vers (cdr elt))))
        package-alist))



;;;; Package menu mode.

(defvar package-menu-mode-map nil
  "Local keymap for `package-menu-mode' buffers.")

(unless package-menu-mode-map
  (setq package-menu-mode-map (make-keymap))
  (suppress-keymap package-menu-mode-map)
  (define-key package-menu-mode-map "q" 'quit-window)
  (define-key package-menu-mode-map "n" 'next-line)
  (define-key package-menu-mode-map "p" 'previous-line)
  (define-key package-menu-mode-map "u" 'package-menu-mark-unmark)
  (define-key package-menu-mode-map "\177" 'package-menu-backup-unmark)
  (define-key package-menu-mode-map "d" 'package-menu-mark-delete)
  (define-key package-menu-mode-map "i" 'package-menu-mark-install)
  (define-key package-menu-mode-map "g" 'package-menu-revert)
  (define-key package-menu-mode-map "r" 'package-menu-refresh)
  (define-key package-menu-mode-map "~"
    'package-menu-mark-obsolete-for-deletion)
  (define-key package-menu-mode-map "x" 'package-menu-execute)
  (define-key package-menu-mode-map "?" 'package-menu-quick-help)
  (define-key package-menu-mode-map "h" 'describe-mode)
  )

(put 'package-menu-mode 'mode-class 'special)

(defun package-menu-mode ()
  "Major mode for browsing a list of packages.
Letters do not insert themselves; instead, they are commands.
\\<package-menu-mode-map>
\\{package-menu-mode-map}"
  (kill-all-local-variables)
  (use-local-map package-menu-mode-map)
  (setq major-mode 'package-menu-mode)
  (setq mode-name "Package Menu")
  (setq truncate-lines t)
  (setq buffer-read-only t)
  ;; Support Emacs 21.
  (if (fboundp 'run-mode-hooks)
      (run-mode-hooks 'package-menu-mode-hook)
    (run-hooks 'package-menu-mode-hook)))

(defun package-menu-refresh ()
  "Download the ELPA archive.
This fetches the file describing the current contents of
the Emacs Lisp Package Archive, and then refreshes the
package menu.  This lets you see what new packages are
available for download."
  (interactive)
  (package-refresh-contents)
  (package-list-packages-internal))

(defun package-menu-revert ()
  "Update the list of packages."
  (interactive)
  (package-list-packages-internal))

(defun package-menu-mark-internal (what)
  (unless (eobp)
    (let ((buffer-read-only nil))
      (beginning-of-line)
      (delete-char 1)
      (insert what)
      (forward-line))))

;; fixme numeric argument
(defun package-menu-mark-delete (num)
  "Mark a package for deletion and move to the next line."
  (interactive "p")
  (package-menu-mark-internal "D"))

(defun package-menu-mark-install (num)
  "Mark a package for installation and move to the next line."
  (interactive "p")
  (package-menu-mark-internal "I"))

(defun package-menu-mark-unmark (num)
  "Clear any marks on a package and move to the next line."
  (interactive "p")
  (package-menu-mark-internal " "))

(defun package-menu-backup-unmark ()
  "Back up one line and clear any marks on that package."
  (interactive)
  (forward-line -1)
  (package-menu-mark-internal " ")
  (forward-line -1))

(defun package-menu-mark-obsolete-for-deletion ()
  "Mark all obsolete packages for deletion."
  (interactive)
  (save-excursion
    (goto-char (point-min))
    (forward-line 2)
    (while (not (eobp))
      (if (looking-at ".* obsolete$")
          (package-menu-mark-internal "D")
        (forward-line 1)))))

(defun package-menu-quick-help ()
  "Show short key binding help for package-menu-mode."
  (interactive)
  (message "n-ext, i-nstall, d-elete, u-nmark, x-ecute, r-efresh, h-elp"))

(defun package-menu-get-package ()
  (save-excursion
    (beginning-of-line)
    (if (looking-at ". \\([^ \t]*\\)")
        (match-string 1))))

(defun package-menu-get-version ()
  (save-excursion
    (beginning-of-line)
    (if (looking-at ". [^ \t]*[ \t]*\\([0-9.]*\\)")
        (match-string 1))))

(defun package-menu-get-status ()
  (save-excursion
    (if (looking-at ".*[ \t]\\([^ \t]*\\)$")
        (match-string 1)
      "")))

(defun package-menu-execute ()
  "Perform all the marked actions.
Packages marked for installation will be downloaded and
installed.  Packages marked for deletion will be removed.
Note that after installing packages you will want to restart
Emacs."
  (interactive)
  (goto-char (point-min))
  (forward-line 2)
  (while (not (eobp))
    (let ((cmd (char-after))
          (pkg-name (package-menu-get-package))
          (pkg-vers (package-menu-get-version))
          (pkg-status (package-menu-get-status)))
      (cond
       ((eq cmd ?D)
        (if (string= pkg-status "")
            ;; FIXME: ask for confirmation.
            nil))
       ((eq cmd ?I)
        (package-install (intern pkg-name)))))
    (forward-line))
  (package-menu-revert))

(defun package-print-package (package version key desc)
  (insert "  ")
  (insert (symbol-name package))
  (indent-to 20 1)
  (insert (package-version-join version))
  (indent-to 30 1)
  (insert key)
  ;; FIXME: this 'when' is bogus...
  (when desc
    (indent-to 41 1)
    (insert desc))
  (insert "\n"))

(defun package-list-maybe-add (package version status description result)
  (let ((elt (assoc (cons package version) result)))
    (unless elt
      (setq result (cons (list (cons package version) status description)
                         result))))
  result)

(defun package-list-packages-internal ()
  (package-initialize)                  ; FIXME: do this here?
  (with-current-buffer (get-buffer-create "*Packages*")
    (setq buffer-read-only nil)
    (erase-buffer)
    (let ((info-list))
      (mapc (lambda (elt)
              (setq info-list
                    (package-list-maybe-add (car elt)
                                            (package-desc-vers (cdr elt))
                                            ;; FIXME: it turns out to
                                            ;; be tricky to see if
                                            ;; this package is
                                            ;; presently activated.
                                            ;; That is lame!
                                            ""
                                            (package-desc-doc (cdr elt))
                                            info-list)))
            package-alist)
      (mapc (lambda (elt)
              (setq info-list
                    (package-list-maybe-add (car elt)
                                            (package-desc-vers (cdr elt))
                                            "available"
                                            (package-desc-doc (cdr elt))
                                            info-list)))
            package-archive-contents)
      (mapc (lambda (elt)
              (mapc (lambda (inner-elt)
                      (setq info-list
                            (package-list-maybe-add (car elt)
                                                    (package-desc-vers
                                                     (cdr inner-elt))
                                                    "obsolete"
                                                    (package-desc-doc
                                                     (cdr inner-elt))
                                                    info-list)))
                    (cdr elt)))
            package-obsolete-alist)
      (mapc (lambda (elt)
              (package-print-package (car (car elt))
                                     (cdr (car elt))
                                     (car (cdr elt))
                                     (car (cdr (cdr elt)))))
            info-list))
    (sort-lines nil (point-min) (point-max))
    (goto-char (point-min))
    ;; In Emacs 22 we should do like buffer-menu and have real column
    ;; headings.
    (insert "  ")
    (insert "Package")
    (indent-to 20 1)
    (insert "Version")
    (indent-to 30 1)
    (insert "Status")
    (indent-to 41 1)
    (insert "Description")
    (insert "\n")
    (insert "  ")
    (insert "-------")
    (indent-to 20 1)
    (insert "-------")
    (indent-to 30 1)
    (insert "------")
    (indent-to 41 1)
    (insert "-----------")
    (insert "\n")
    (current-buffer)))

(defun package-list-packages ()
  "Display a list of packages.
The list is displayed in a buffer named `*Packages*'."
  (interactive)
  (with-current-buffer (package-list-packages-internal)
    (package-menu-mode)
    ;; It's okay to use pop-to-buffer here.  The package menu buffer
    ;; has keybindings, and the user just typed 'M-x
    ;; package-list-packages', suggesting that they might want to use
    ;; them.
    (pop-to-buffer (current-buffer))))



(eval-when-compile
  (require 'reporter))

(defun package-report-bug ()
  "Submit a bug report for package.el via email."
  (interactive)
  (require 'reporter)
  (reporter-submit-bug-report
   package-el-maintainer
   (concat "package.el " package-el-version)
   '(package-archive-base
     package-archive-version
     package-archive-contents
     package-user-dir
     package-directory-list
     package-alist
     package-activated-list
     package-obsolete-alist)))

(provide 'package)

;;; package.el ends here