annotate lisp/org/org-publish.el @ 104810:86b7fe7d1d8f

2009-09-02 Carsten Dominik <[e-mail]> * org-protocol.el (org-protocol-store-link) (org-protocol-remember, org-protocol-open-source): Add autoloads. * org-compat.el (org-float-time): New function. * org.el (org-clock-update-time-maybe) (org-sort-entries-or-items, org-do-sort) (org-evaluate-time-range, org-time-string-to-seconds) (org-closed-in-range): Use `org-float-time'. * org-timer.el (org-timer-start, org-timer-pause-or-continue) (org-timer-seconds): Use `org-float-time'. * org-clock.el (org-clock-get-clocked-time, org-clock-out) (org-clock-sum, org-dblock-write:clocktable) (org-clocktable-steps): Use `org-float-time'. * org-agenda.el (org-agenda-last-marker-time) (org-agenda-new-marker, org-diary): Use `org-float-time'. * org-compat.el (w32-focus-frame): Declare the w32-focus-frame function. * org-exp.el (org-get-file-contents): Only protect lines that really need it. * org-html.el (require): Require cl for compilation. * org.el: Avoid using `default-major-mode'. * org-plot.el (require): Require CL only at compile time. * org-exp.el (require): Require CL only at compile time. * org-agenda.el (org-agenda-quit): When the agenda window is dedicated, remove other windows before exiting, so that the frame really will be killed. * org-exp.el (org-export-handle-include-files): Reset START and END for each loop cycle. 2009-09-02 Carsten Dominik <[e-mail]> * org.el (org-eval-in-calendar): Use `org-select-frame-set-input-focus'. * org-compat.el (org-select-frame-set-input-focus): New function. * org.el (org-update-statistics-cookies): New function. (org-mode-map): Bind `C-c #' to `org-update-statistics-cookies'. 2009-09-02 Carsten Dominik <[e-mail]> * org-src.el (org-edit-fixed-width-region): Set org-src-mode only after the local variables are set. * org-latex.el (org-export-latex-protect-amp): New function. (org-export-latex-links): Protect link ampersands in tables. * org-exp.el (org-export-select-backend-specific-text): Match in two steps, to avoid regexp problems. * org.el (org-offer-links-in-entry): Improve working with many and duplicate links. * org-agenda.el (org-agenda-show-1): Make more consistent with normal cycling. (org-agenda-cycle-show): Make more consistent with normal cycling. * org-gnus.el (org-gnus-store-link): Restore the linking to a website. 2009-09-02 Bastien Guerry <[e-mail]> * org-latex.el (org-export-latex-first-lines): Bugfix. 2009-09-02 Carsten Dominik <[e-mail]> * org-clock.el (org-clock-modify-effort-estimate): Emit message about new effort. * org.el (org-set-effort): New function. (org-mode-map): New key for effort setting command. * org-agenda.el (org-agenda): Keep window setup when calling agenda from within agenda window. (org-agenda-mode-map): New keys for effort setting commands. (org-agenda-menu): Add effort setting commands to menu. (org-agenda-set-property, org-agenda-set-effort): New functions. * org-latex.el (org-export-latex-tables): Fix `org-table-last-alignment' and `org-table-last-column-widths' if the first column has been removed. 2009-09-02 Carsten Dominik <[e-mail]> * org.el (org-remove-timestamp-with-keyword): Only remove in entry, not in subtree. * org-src.el (org-src-lang-modes): Add abbreviation elisp for emacs lisp. * org.el (org-open-at-point): When on headline, offer all strings in entry. * org-remember.el (org-remember-templates): Documentation fix. * org.el (org-move-subtree-down): Use `org-get-next-sibling' and `org-get-last-sibling' instead of the outline versions of these functions. (org-get-last-sibling): New function. (org-refile): Use `org-get-next-sibling' instead of the outline version of this function. (org-clean-visibility-after-subtree-move): Use `org-get-next-sibling' and `org-get-last-sibling' instead of the outline versions of these functions. 2009-09-02 Carsten Dominik <[e-mail]> * org-agenda.el (org-prepare-agenda): When creating a new frame for the agenda, make the window dedicated. * org-agenda.el (org-agenda-mode-map): New keys for time motion. * org-table.el (org-table-align): Change the order of reinsertion and deletion, to avoid problems with overlays following the table. * org.el (org-parse-time-string): Better error message. (org-show-subtree): Use org-end-of-subtree. * org-macs.el (org-goto-line): New defsubst. * org.el (org-open-file, org-change-tag-in-region) (org-fast-tag-show-exit): Don't use `goto-line'. * org-table.el (org-table-align, org-table-insert-column) (org-table-delete-column, org-table-move-column) (org-table-sort-lines, org-table-copy-region) (org-table-paste-rectangle, org-table-wrap-region) (org-table-get-specials, org-table-rotate-recalc-marks) (org-table-get-range, org-table-recalculate) (org-table-edit-formulas, org-table-fedit-convert-buffer) (org-table-show-reference, org-table-highlight-rectangle): Don't use `goto-line'. * org-src.el (org-edit-src-code, org-edit-fixed-width-region) (org-edit-src-exit): Don't use `goto-line'. * org-macs.el (org-preserve-lc): Don't use `goto-line'. * org-list.el (org-renumber-ordered-list, org-fix-bullet-type): Don't use `goto-line'. * org-exp.el (org-export-number-lines): Don't use `goto-line'. * org-colview.el (org-columns, org-columns-redo) (org-agenda-columns): Don't use `goto-line'. * org-colview-xemacs.el (org-columns, org-agenda-columns): Don't use `goto-line'. * org-agenda.el (org-agenda-mode): Force visual line motion off. (org-agenda-add-entry-text-maxlines): Improve docstring. (org-agenda-start-with-entry-text-mode): New option. (org-agenda-entry-text-maxlines): New option. (org-agenda-entry-text-mode): New variable. (org-agenda-mode): Set initial value of `org-agenda-entry-text-mode'. (org-agenda-mode-map): Add the `E' key. (org-agenda-menu): Add entry text mode to the menu. (org-agenda-get-some-entry-text): Fix line count bug. (org-finalize-agenda): Apply entry text mode if appropriate. (org-agenda-entry-text-show-here): New function. (org-agenda-entry-text-show): New function. (org-agenda-entry-text-hide): New function. (org-agenda-view-mode-dispatch): Add entry text mode to the view key menu. (org-agenda-entry-text-mode): New command. (org-agenda-set-mode-name): Add entry text mode to the mode line string. (org-agenda-undo, org-agenda-get-restriction-and-command) (org-agenda-get-some-entry-text, org-agenda-redo): Don't use `goto-line'. 2009-09-02 Bernt Hansen <[e-mail]> * org-clock.el (org-notify): Bugfix. 2009-09-02 Carsten Dominik <[e-mail]> * org-agenda.el (org-agenda-open-link): Handle multiple links and check for after-string. * org-gnus.el (org-gnus-store-link): Simplify. * org.el (org-latex-regexps): Don't add extra empty lines for display formulas. 2009-09-02 Carsten Dominik <[e-mail]> * org-agenda.el (org-agenda-get-some-entry-text): New function. (org-agenda-add-entry-text): Use `org-agenda-get-some-entry-text'. * org.el (org-cycle-separator-lines): Update docstring. (org-cycle-show-empty-lines): Handle negative values for `org-cycle-show-empty-lines'. * org-exp.el (org-export-protect-sub-super): New function. (org-export-normalize-links): Protect the url of plain links from supscript and superscript processing. * org-remember.el (org-remember-escaped-%): New function. (org-remember-apply-template): Use `org-remember-escaped-%' to detect escaped % signs. 2009-09-02 Bastien Guerry <[e-mail]> * org-timer.el (org-timer-set-timer): Use `org-notify' and play a sound when showing the notification. * org-clock.el (org-notify): New function. (org-clock-notify-once-if-expired): Use `org-notify'. * org-gnus.el (org-gnus-store-link): Handle `gnus-summary-mode' and `gnus-article-mode' separately. (gnus-summary-article-header): Fix the declare-function. 2009-09-02 Carsten Dominik <[e-mail]> * org-exp.el (org-export-format-source-code-or-example): Translate language. * org-src.el (org-src-lang-modes): New variable (org-edit-src-code): Translate language. * org-exp.el (org-export-format-source-code-or-example): Deal wit the new structure of the `org-export-latex-listings-langs' variable. * org-latex.el (org-export-latex-listings-langs): Change structure of the variable from plist to alist. 2009-09-02 Carsten Dominik <[e-mail]> * org.el (org-in-commented-line): New function. 2009-09-02 Carsten Dominik <[e-mail]> * org.el (org-hide-block-toggle): Make folded blocks searchable. 2009-09-02 Friedrich Delgado Friedrichs <[e-mail]> (tiny change) * org.el (org-flag-drawer): More useful error. 2009-09-02 Carsten Dominik <[e-mail]> * org-remember.el (org-remember-apply-template): Use org-icompleting-read. * org-publish.el (org-publish): Use org-icompleting-read. * org-colview.el (org-columns-edit-value, org-columns-new) (org-insert-columns-dblock): Use org-icompleting-read. * org-colview-xemacs.el (org-columns-edit-value) (org-columns-new, org-insert-columns-dblock): Use org-icompleting-read. * org-attach.el (org-attach-delete-one, org-attach-open): Use org-icompleting-read. 2009-09-02 Carsten Dominik <[e-mail]> * org.el (org-hierarchical-todo-statistics): Improve docstring. (org-version): Return the version text. (org-org-menu): Add a menu entry for the new bug reporter. (org-submit-bug-report): New command. * org-list.el (org-hierarchical-checkbox-statistics): Improve docstring. * org.el (org-emphasis-regexp-components): Add "`" to set of pre-emphasis characters. * org-latex.el (org-export-latex-classes): Always include the soul package. (org-export-latex-emphasis-alist): Use \st for strikethough. * org-exp-blocks.el (org-export-blocks-preprocess): Use `indent-code-rigidly' to indent. * org-agenda.el (org-agenda-get-restriction-and-command): Remove properties only if MATCH really is a string. 2009-09-02 Carsten Dominik <[e-mail]> * org-latex.el (org-export-latex-packages-alist): Fix customization type. * org.el (org-create-formula-image): Also use `org-export-latex-packages-alist'. * org-html.el (org-export-as-html): Fix bug in footnote regexp. (org-export-as-html): Format footnotes correctly. 2009-09-02 Carsten Dominik <[e-mail]> * org.el (org-fast-tag-selection): Avoid text properties on tags in the alist. * org-agenda.el (org-agenda-get-restriction-and-command): Avoid text properties on the match element. 2009-09-02 Carsten Dominik <[e-mail]> * org.el (org-set-regexps-and-options): Make sure the list of done keywords is not invalid. * org-exp.el (org-export-interpolate-newlines): New function. 2009-09-02 Carsten Dominik <[e-mail]> * org.el (org-format-latex): Avoid nested overlays. * org-latex.el (org-export-latex-listings-langs): Add a few more languages. * org-exp.el (org-export-preprocess-apply-macros): Make sure to ignore newlines and space before the first macro argument. * org-latex.el (org-export-latex-tables): Remove save-excursion around `org-table-align'. 2009-09-02 Carsten Dominik <[e-mail]> * org.el (org-export-html-special-string-regexps): Definition moved into org.el * org-exp.el (org-export-preprocess-apply-macros): Allow newlines in macro calls. 2009-09-02 Carsten Dominik <[e-mail]> * org-latex.el (org-export-latex-listings) (org-export-latex-listings-langs): New options. * org-exp.el (org-export-format-source-code-or-example): Use listing package if requested by the user. 2009-09-02 Bastien Guerry <[e-mail]> * org.el (org-iswitchb): Fix bug when aborting the `org-iswitchb' command before actually switching to a buffer. 2009-09-02 Carsten Dominik <[e-mail]> * org-exp.el (org-get-file-contents): Only quote org lines when the markup is src or example. * org-agenda.el (org-agenda-skip-scheduled-if-deadline-is-shown): New option (org-agenda-get-day-entries): Remember deadline results and pass them on into the function getting the scheduling information. (org-agenda-get-scheduled): Accept deadline results as parameters and maybe skip some entries. (org-agenda-skip-scheduled-if-deadline-is-shown): New option. * org.el (org-insert-heading): When respecting content, do not convert current line to headline. * org-clock.el (org-clock-save-markers-for-cut-and-paste): Also cheeeeeck the hd marker (org-clock-in): Also set the hd marker. (org-clock-out): Also set the hd marker. (org-clock-cancel): Reset markers. * org.el (org-clock-hd-marker): New marker. * org-faces.el (org-agenda-clocking): New face. * org-agenda.el (org-agenda-mark-clocking-task): New function. (org-finalize-agenda): call `org-agenda-mark-clocking-task'. * org.el (org-modules): Add org-track.el. * org-agenda.el (org-agenda-bulk-marked-p): New function. (org-agenda-bulk-mark, org-agenda-bulk-unmark): Use `org-agenda-bulk-marked-p'. (org-agenda-bulk-toggle): New command. 2009-09-02 Carsten Dominik <[e-mail]> * org.el (org-move-subtree-down): Hide subtree if it was folded, not just the body. * org-remember.el (org-remember-finalize): Avoid buffer-modified messages.

;;; org-publish.el --- publish related org-mode files as a website
;; Copyright (C) 2006, 2007, 2008, 2009  Free Software Foundation, Inc.

;; Author: David O'Toole <[e-mail]>
;; Maintainer: Carsten Dominik <carsten DOT dominik AT gmail DOT com>
;; Keywords: hypermedia, outlines, wp
;; Version: 6.30c

;; This file is part of GNU Emacs.
;;
;; GNU Emacs is free software: you can redistribute it and/or modify
;; it under the terms of the GNU General Public License as published by
;; the Free Software Foundation, either version 3 of the License, or
;; (at your option) any later version.

;; GNU Emacs is distributed in the hope that it will be useful,
;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;; GNU General Public License for more details.

;; You should have received a copy of the GNU General Public License
;; along with GNU Emacs.  If not, see <http://www.gnu.org/licenses/>.

;;; Commentary:

;; This program allow configurable publishing of related sets of
;; Org-mode files as a complete website.
;;
;; org-publish.el can do the following:
;;
;; + Publish all one's org-files to HTML or PDF
;; + Upload HTML, images, attachments and other files to a web server
;; + Exclude selected private pages from publishing
;; + Publish a clickable index of pages
;; + Manage local timestamps for publishing only changed files
;; + Accept plugin functions to extend range of publishable content
;;
;; Documentation for publishing is in the manual.

;;; Code:

(eval-when-compile
  (require 'cl))
(require 'org)
(require 'org-exp)

(eval-and-compile
  (unless (fboundp 'declare-function)
    (defmacro declare-function (fn file &optional arglist fileonly))))

(defgroup org-publish nil
  "Options for publishing a set of Org-mode and related files."
  :tag "Org Publishing"
  :group 'org)

(defcustom org-publish-project-alist nil
  "Association list to control publishing behavior.
Each element of the alist is a publishing 'project.'  The CAR of
each element is a string, uniquely identifying the project.  The
CDR of each element is in one of the following forms:

  (:property value :property value ... )

OR,

  (:components (\"project-1\" \"project-2\" ...))

When the CDR of an element of org-publish-project-alist is in
this second form, the elements of the list after :components are
taken to be components of the project, which group together files
requiring different publishing options.  When you publish such a
project with \\[org-publish], the components all publish.

When a property is given a value in org-publish-project-alist, its
setting overrides the value of the corresponding user variable
\(if any) during publishing.  However, options set within a file
override everything.

Most properties are optional, but some should always be set:

  :base-directory        Directory containing publishing source files
  :base-extension        Extension (without the dot!) of source files.
                         This can be a regular expression.
  :publishing-directory  Directory (possibly remote) where output
                         files will be published

The :exclude property may be used to prevent certain files from
being published.  Its value may be a string or regexp matching
file names you don't want to be published.

The :include property may be used to include extra files.  Its
value may be a list of filenames to include. The filenames are
considered relative to the base directory.

When both :include and :exclude properties are given values, the
exclusion step happens first.

One special property controls which back-end function to use for
publishing files in the project.  This can be used to extend the
set of file types publishable by org-publish, as well as the set
of output formats.

  :publishing-function     Function to publish file.  The default is
                           `org-publish-org-to-html', but other
                           values are possible.  May also be a
                           list of functions, in which case
                           each function in the list is invoked
                           in turn.

Another property allows you to insert code that prepares a
project for publishing.  For example, you could call GNU Make on a
certain makefile, to ensure published files are built up to date.

  :preparation-function   Function to be called before publishing
                          this project.
  :completion-function    Function to be called after publishing
                          this project.

Some properties control details of the Org publishing process,
and are equivalent to the corresponding user variables listed in
the right column.  See the documentation for those variables to
learn more about their use and default values.

  :language              `org-export-default-language'
  :headline-levels       `org-export-headline-levels'
  :section-numbers       `org-export-with-section-numbers'
  :table-of-contents     `org-export-with-toc'
  :emphasize             `org-export-with-emphasize'
  :sub-superscript       `org-export-with-sub-superscripts'
  :TeX-macros            `org-export-with-TeX-macros'
  :fixed-width           `org-export-with-fixed-width'
  :tables                `org-export-with-tables'
  :table-auto-headline   `org-export-highlight-first-table-line'
  :style                 `org-export-html-style'
  :convert-org-links     `org-export-html-link-org-files-as-html'
  :inline-images         `org-export-html-inline-images'
  :expand-quoted-html    `org-export-html-expand'
  :timestamp             `org-export-html-with-timestamp'
  :publishing-directory  `org-export-publishing-directory'
  :preamble              `org-export-html-preamble'
  :postamble             `org-export-html-postamble'
  :auto-preamble         `org-export-html-auto-preamble'
  :auto-postamble        `org-export-html-auto-postamble'
  :author                `user-full-name'
  :email                 `user-mail-address'

The following properties may be used to control publishing of an
index of files or summary page for a given project.

  :auto-index            Whether to publish an index during
                         `org-publish-current-project' or `org-publish-all'.
  :index-filename        Filename for output of index.  Defaults
                         to 'sitemap.org' (which becomes 'sitemap.html').
  :index-title           Title of index page.  Defaults to name of file.
  :index-function        Plugin function to use for generation of index.
                         Defaults to `org-publish-org-index', which
                         generates a plain list of links to all files
                         in the project.
  :index-style           Can be `list' (index is just an itemized list
                         of the titles of the files involved) or
                         `tree' (the directory structure of the source
                         files is reflected in the index).  Defaults to
                         `tree'."
  :group 'org-publish
  :type 'alist)

(defcustom org-publish-use-timestamps-flag t
  "When non-nil, use timestamp checking to publish only changed files.
When nil, do no timestamp checking and always publish all files."
  :group 'org-publish
  :type 'boolean)

(defcustom org-publish-timestamp-directory (convert-standard-filename
					    "~/.org-timestamps/")
  "Name of directory in which to store publishing timestamps."
  :group 'org-publish
  :type 'directory)

(defcustom org-publish-list-skipped-files t
  "Non-nil means, show message about files *not* published."
  :group 'org-publish
  :type 'boolean)

(defcustom org-publish-before-export-hook nil
  "Hook run before export on the Org file.
The hook may modify the file in arbitrary ways before publishing happens.
The orgiginal version of the buffer will be restored after publishing."
  :group 'org-publish
  :type 'hook)

(defcustom org-publish-after-export-hook nil
  "Hook run after export on the exported buffer.
Any changes made by this hook will be saved."
  :group 'org-publish
  :type 'hook)

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; Timestamp-related functions

(defun org-publish-timestamp-filename (filename &optional pub-dir pub-func)
  "Return path to timestamp file for filename FILENAME."
  (setq filename (concat filename "::" (or pub-dir "") "::"
			 (format "%s" (or pub-func ""))))
  (concat (file-name-as-directory org-publish-timestamp-directory)
	  "X" (if (fboundp 'sha1) (sha1 filename) (md5 filename))))

(defun org-publish-needed-p (filename &optional pub-dir pub-func true-pub-dir)
  "Return `t' if FILENAME should be published in PUB-DIR using PUB-FUNC.
TRUE-PUB-DIR is there the file will truely end up.  Currently we are not using
this - maybe it can eventually be used to check if the file is present at
the target location, and how old it is.  Right ow we cannot do this, because
we do not know under what file name the file will be stored - the publishing
function can still decide about that independently."
  (let ((rtn
	 (if org-publish-use-timestamps-flag
	     (if (file-exists-p org-publish-timestamp-directory)
		 ;; first handle possible wrong timestamp directory
		 (if (not (file-directory-p org-publish-timestamp-directory))
		     (error "Org publish timestamp: %s is not a directory"
			    org-publish-timestamp-directory)
		   ;; there is a timestamp, check if FILENAME is newer
		   (file-newer-than-file-p
		    filename (org-publish-timestamp-filename
			      filename pub-dir pub-func)))
	       (make-directory org-publish-timestamp-directory)
	       t)
	   ;; don't use timestamps, always return t
	   t)))
    (if rtn
	(message "Publishing file %s using `%s'" filename pub-func)
      (when org-publish-list-skipped-files
	(message   "Skipping unmodified file %s" filename)))
    rtn))

(defun org-publish-update-timestamp (filename &optional pub-dir pub-func)
  "Update publishing timestamp for file FILENAME.
If there is no timestamp, create one."
  (let ((timestamp-file (org-publish-timestamp-filename
			 filename pub-dir pub-func))
	newly-created-timestamp)
    (if (not (file-exists-p timestamp-file))
	;; create timestamp file if needed
	(with-temp-buffer
	  (make-directory (file-name-directory timestamp-file) t)
	  (write-file timestamp-file)
	  (setq newly-created-timestamp t)))
    ;; Emacs 21 doesn't have `set-file-times'
    (if (and (fboundp 'set-file-times)
	     (not newly-created-timestamp))
	(set-file-times timestamp-file)
      (call-process "touch" nil 0 nil (expand-file-name timestamp-file)))))

(defun org-publish-remove-all-timestamps ()
  "Remove all files in the timstamp directory."
  (let ((dir org-publish-timestamp-directory)
	files)
    (when (and (file-exists-p dir)
	       (file-directory-p dir))
      (mapc 'delete-file (directory-files dir 'full "[^.]\\'")))))


;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; Mapping files to project names

(defvar org-publish-files-alist nil
  "Alist of files and their parent projects.
Each element of this alist is of the form:

  (file-name . project-name)")

(defvar org-publish-initial-buffer nil
  "The buffer `org-publish' has been called from.")
(defvar org-publish-temp-files nil
  "Temporary list of files to be published.")

(defun org-publish-initialize-files-alist (&optional refresh)
  "Set `org-publish-files-alist' if it is not set.
Also set it if the optional argument REFRESH is non-nil."
  (interactive "P")
  (when (or refresh (not org-publish-files-alist))
    (setq org-publish-files-alist
	  (org-publish-get-files org-publish-project-alist))))

(defun org-publish-validate-link (link &optional directory)
  "Check if LINK points to a file in the current project."
  (assoc (expand-file-name link directory) org-publish-files-alist))

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; Compatibility aliases

;; Delete-dups is not in Emacs <22
(if (fboundp 'delete-dups)
    (defalias 'org-publish-delete-dups 'delete-dups)
  (defun org-publish-delete-dups (list)
    "Destructively remove `equal' duplicates from LIST.
Store the result in LIST and return it.  LIST must be a proper list.
Of several `equal' occurrences of an element in LIST, the first
one is kept.

This is a compatibility function for Emacsen without `delete-dups'."
    ;; Code from `subr.el' in Emacs 22:
    (let ((tail list))
      (while tail
	(setcdr tail (delete (car tail) (cdr tail)))
	(setq tail (cdr tail))))
    list))

(declare-function org-publish-delete-dups "org-publish" (list))

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; Getting project information out of org-publish-project-alist

(defun org-publish-get-files (projects-alist &optional no-exclusion)
  "Return the list of all publishable files for PROJECTS-ALIST.
If NO-EXCLUSION is non-nil, don't exclude files."
  (let (all-files)
    ;; add all projects
    (mapc
     (lambda(p)
       (let* ((exclude (plist-get (cdr p) :exclude))
	      (files (and p (org-publish-get-base-files p exclude))))
	 ;; add all files from this project
	 (mapc (lambda(f)
		 (add-to-list 'all-files
			      (cons (expand-file-name f) (car p))))
	       files)))
     (org-publish-expand-projects projects-alist))
    all-files))

(defun org-publish-expand-projects (projects-alist)
  "Expand projects in PROJECTS-ALIST.
This splices all the components into the list."
  (let ((rest projects-alist) rtn p components)
    (while (setq p (pop rest))
      (if (setq components (plist-get (cdr p) :components))
	  (setq rest (append
		      (mapcar (lambda (x) (assoc x org-publish-project-alist))
			      components)
		      rest))
	(push p rtn)))
    (nreverse (org-publish-delete-dups (delq nil rtn)))))

(defun org-publish-get-base-files-1 (base-dir &optional recurse match skip-file skip-dir)
  "Set `org-publish-temp-files' with files from BASE-DIR directory.
If RECURSE is non-nil, check BASE-DIR recursively.  If MATCH is
non-nil, restrict this list to the files matching the regexp
MATCH.  If SKIP-FILE is non-nil, skip file matching the regexp
SKIP-FILE.  If SKIP-DIR is non-nil, don't check directories
matching the regexp SKIP-DIR when recursing through BASE-DIR."
  (mapc (lambda (f)
	  (let ((fd-p (file-directory-p f))
		(fnd (file-name-nondirectory f)))
	    (if (and fd-p recurse
		     (not (string-match "^\\.+$" fnd))
		     (if skip-dir (not (string-match skip-dir fnd)) t))
		(org-publish-get-base-files-1 f recurse match skip-file skip-dir)
	      (unless (or fd-p ;; this is a directory
			  (and skip-file (string-match skip-file fnd))
			  (not (file-exists-p (file-truename f)))
			  (not (string-match match fnd)))
		(pushnew f org-publish-temp-files)))))
	(directory-files base-dir t (unless recurse match))))

(defun org-publish-get-base-files (project &optional exclude-regexp)
  "Return a list of all files in PROJECT.
If EXCLUDE-REGEXP is set, this will be used to filter out
matching filenames."
  (let* ((project-plist (cdr project))
	 (base-dir (file-name-as-directory
		    (plist-get project-plist :base-directory)))
	 (include-list (plist-get project-plist :include))
	 (recurse (plist-get project-plist :recursive))
	 (extension (or (plist-get project-plist :base-extension) "org"))
	 (match (if (eq extension 'any)
                    "^[^\\.]"
		  (concat "^[^\\.].*\\.\\(" extension "\\)$"))))
    (setq org-publish-temp-files nil)
    (org-publish-get-base-files-1 base-dir recurse match
				  ;; FIXME distinguish exclude regexp
				  ;; for skip-file and skip-dir?
				  exclude-regexp exclude-regexp)
    (mapc (lambda (f)
	    (pushnew
	     (expand-file-name (concat base-dir f))
	     org-publish-temp-files))
	  include-list)
    org-publish-temp-files))

(defun org-publish-get-project-from-filename (filename &optional up)
  "Return the project FILENAME belongs."
  (let* ((project-name (cdr (assoc (expand-file-name filename)
				   org-publish-files-alist))))
    (when up
      (dolist (prj org-publish-project-alist)
	(if (member project-name (plist-get (cdr prj) :components))
	    (setq project-name (car prj)))))
    (assoc project-name org-publish-project-alist)))

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; Pluggable publishing back-end functions

(defun org-publish-org-to (format plist filename pub-dir)
  "Publish an org file to FORMAT.
PLIST is the property list for the given project.
FILENAME is the filename of the org file to be published.
PUB-DIR is the publishing directory."
  (require 'org)
  (unless (file-exists-p pub-dir)
    (make-directory pub-dir t))
  (let ((visiting (find-buffer-visiting filename)))
    (save-excursion
      (switch-to-buffer (or visiting (find-file filename)))
      (let* ((plist (cons :buffer-will-be-killed (cons t plist)))
	     (init-buf (current-buffer))
	     (init-point (point))
	     (init-buf-string (buffer-string))
	     export-buf-or-file)
	;; run hooks before exporting
	(run-hooks 'org-publish-before-export-hook)
	;; export the possibly modified buffer
	(setq export-buf-or-file
	      (funcall (intern (concat "org-export-as-" format))
		       (plist-get plist :headline-levels)
		       nil plist nil nil pub-dir))
	(when (and (bufferp export-buf-or-file)
		   (buffer-live-p export-buf-or-file))
	  (set-buffer export-buf-or-file)
	  ;; run hooks after export and save export
	  (and (run-hooks 'org-publish-after-export-hook)
	       (if (buffer-modified-p) (save-buffer)))
	  (kill-buffer export-buf-or-file))
	;; maybe restore buffer's content
	(set-buffer init-buf)
	(when (buffer-modified-p init-buf)
	  (erase-buffer)
	  (insert init-buf-string)
	  (save-buffer)
	  (goto-char init-point))
	(unless visiting
	  (kill-buffer init-buf))))))

(defun org-publish-org-to-latex (plist filename pub-dir)
  "Publish an org file to LaTeX.
See `org-publish-org-to' to the list of arguments."
  (org-publish-org-to "latex" plist filename pub-dir))

(defun org-publish-org-to-pdf (plist filename pub-dir)
  "Publish an org file to PDF (via LaTeX).
See `org-publish-org-to' to the list of arguments."
  (org-publish-org-to "pdf" plist filename pub-dir))

(defun org-publish-org-to-html (plist filename pub-dir)
  "Publish an org file to HTML.
See `org-publish-org-to' to the list of arguments."
  (org-publish-org-to "html" plist filename pub-dir))

(defun org-publish-org-to-org (plist filename pub-dir)
  "Publish an org file to HTML.
See `org-publish-org-to' to the list of arguments."
  (org-publish-org-to "org" plist filename pub-dir))

(defun org-publish-attachment (plist filename pub-dir)
  "Publish a file with no transformation of any kind.
See `org-publish-org-to' to the list of arguments."
  ;; make sure eshell/cp code is loaded
  (unless (file-directory-p pub-dir)
    (make-directory pub-dir t))
  (or (equal (expand-file-name (file-name-directory filename))
	     (file-name-as-directory (expand-file-name pub-dir)))
      (copy-file filename
		 (expand-file-name (file-name-nondirectory filename) pub-dir)
		 t)))

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; Publishing files, sets of files, and indices

(defun org-publish-file (filename &optional project)
  "Publish file FILENAME from PROJECT."
  (let* ((project
	  (or project
	      (or (org-publish-get-project-from-filename filename)
		  (if (y-or-n-p
		       (format "%s is not in a project.  Re-read the list of projects files? "
			       (abbreviate-file-name filename)))
		      ;; If requested, re-initialize the list of projects files
		      (progn (org-publish-initialize-files-alist t)
			     (or (org-publish-get-project-from-filename filename)
				 (error "File %s not part of any known project"
					(abbreviate-file-name filename))))
		    (error "Can't publish file outside of a project")))))
	 (project-plist (cdr project))
	 (ftname (file-truename filename))
	 (publishing-function
	  (or (plist-get project-plist :publishing-function)
	      'org-publish-org-to-html))
	 (base-dir (file-name-as-directory
		    (file-truename (plist-get project-plist :base-directory))))
	 (pub-dir (file-name-as-directory
		   (file-truename (plist-get project-plist :publishing-directory))))
	 tmp-pub-dir)
    (setq tmp-pub-dir
	  (file-name-directory
	   (concat pub-dir
		   (and (string-match (regexp-quote base-dir) ftname)
			(substring ftname (match-end 0))))))
    (if (listp publishing-function)
	;; allow chain of publishing functions
	(mapc (lambda (f)
		(when (org-publish-needed-p filename pub-dir f tmp-pub-dir)
		  (funcall f project-plist filename tmp-pub-dir)
		  (org-publish-update-timestamp filename pub-dir f)))
	      publishing-function)
      (when (org-publish-needed-p filename pub-dir publishing-function
				  tmp-pub-dir)
	(funcall publishing-function project-plist filename tmp-pub-dir)
	(org-publish-update-timestamp
	 filename pub-dir publishing-function)))))

(defun org-publish-projects (projects)
  "Publish all files belonging to the PROJECTS alist.
If :auto-index is set, publish the index too."
  (mapc
   (lambda (project)
     (let*
	 ((project-plist (cdr project))
	  (exclude-regexp (plist-get project-plist :exclude))
	  (index-p (plist-get project-plist :auto-index))
	  (index-filename (or (plist-get project-plist :index-filename)
			      "sitemap.org"))
	  (index-function (or (plist-get project-plist :index-function)
			      'org-publish-org-index))
	  (preparation-function (plist-get project-plist :preparation-function))
	  (completion-function (plist-get project-plist :completion-function))
	  (files (org-publish-get-base-files project exclude-regexp)) file)
       (when preparation-function (funcall preparation-function))
       (if index-p (funcall index-function project index-filename))
       (while (setq file (pop files))
	 (org-publish-file file project))
       (when completion-function (funcall completion-function))))
   (org-publish-expand-projects projects)))

(defun org-publish-org-index (project &optional index-filename)
  "Create an index of pages in set defined by PROJECT.
Optionally set the filename of the index with INDEX-FILENAME.
Default for INDEX-FILENAME is 'sitemap.org'."
  (let* ((project-plist (cdr project))
	 (dir (file-name-as-directory
	       (plist-get project-plist :base-directory)))
	 (localdir (file-name-directory dir))
	 (indent-str (make-string 2 ?\ ))
	 (exclude-regexp (plist-get project-plist :exclude))
	 (files (nreverse (org-publish-get-base-files project exclude-regexp)))
	 (index-filename (concat dir (or index-filename "sitemap.org")))
	 (index-title (or (plist-get project-plist :index-title)
			  (concat "Index for project " (car project))))
	 (index-style (or (plist-get project-plist :index-style)
			  'tree))
	 (visiting (find-buffer-visiting index-filename))
	 (ifn (file-name-nondirectory index-filename))
	 file index-buffer)
    (with-current-buffer (setq index-buffer
			       (or visiting (find-file index-filename)))
      (erase-buffer)
      (insert (concat "#+TITLE: " index-title "\n\n"))
      (while (setq file (pop files))
	(let ((fn (file-name-nondirectory file))
	      (link (file-relative-name file dir))
	      (oldlocal localdir))
	  ;; index shouldn't index itself
	  (unless (equal (file-truename index-filename)
			 (file-truename file))
	    (if (eq index-style 'list)
		(message "Generating list-style index for %s" index-title)
	      (message "Generating tree-style index for %s" index-title)
	      (setq localdir (concat (file-name-as-directory dir)
				     (file-name-directory link)))
	      (unless (string= localdir oldlocal)
		(if (string= localdir dir)
		    (setq indent-str (make-string 2 ?\ ))
		  (let ((subdirs
			 (split-string
			  (directory-file-name
			   (file-name-directory
			    (file-relative-name localdir dir))) "/"))
			(subdir "")
			(old-subdirs (split-string
				      (file-relative-name oldlocal dir) "/")))
		    (setq indent-str (make-string 2 ?\ ))
		    (while (string= (car old-subdirs) (car subdirs))
		      (setq indent-str (concat indent-str (make-string 2 ?\ )))
		      (pop old-subdirs)
		      (pop subdirs))
		    (dolist (d subdirs)
		      (setq subdir (concat subdir d "/"))
		      (insert (concat indent-str " + " d "\n"))
		      (setq indent-str (make-string
					(+ (length indent-str) 2) ?\ )))))))
	    ;; This is common to 'flat and 'tree
	    (insert (concat indent-str " + [[file:" link "]["
			    (org-publish-find-title file)
			    "]]\n")))))
      (save-buffer))
    (or visiting (kill-buffer index-buffer))))

(defun org-publish-find-title (file)
  "Find the title of file in project."
  (let* ((visiting (find-buffer-visiting file))
	 (buffer (or visiting (find-file-noselect file)))
	 title)
    (save-excursion
      (set-buffer buffer)
      (let* ((opt-plist (org-combine-plists (org-default-export-plist)
					    (org-infile-export-plist))))
	(setq title
	      (or (plist-get opt-plist :title)
		  (and (not
			(plist-get opt-plist :skip-before-1st-heading))
		       (org-export-grab-title-from-buffer))
		  (file-name-nondirectory (file-name-sans-extension file))))))
    (unless visiting
      (kill-buffer buffer))
    title))

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; Interactive publishing functions

;;;###autoload
(defalias 'org-publish-project 'org-publish)

;;;###autoload
(defun org-publish (project &optional force)
  "Publish PROJECT."
  (interactive
   (list
    (assoc (org-icompleting-read
	    "Publish project: "
	    org-publish-project-alist nil t)
	   org-publish-project-alist)
    current-prefix-arg))
  (setq org-publish-initial-buffer (current-buffer))
  (save-window-excursion
    (let* ((org-publish-use-timestamps-flag
	    (if force nil org-publish-use-timestamps-flag)))
      (org-publish-projects (list project)))))

;;;###autoload
(defun org-publish-all (&optional force)
  "Publish all projects.
With prefix argument, remove all files in the timestamp
directory and force publishing all files."
  (interactive "P")
  (when force
    (org-publish-remove-all-timestamps))
  (org-publish-initialize-files-alist)
  (save-window-excursion
    (let ((org-publish-use-timestamps-flag
	   (if force nil org-publish-use-timestamps-flag)))
      (org-publish-projects org-publish-project-alist))))

;;;###autoload
(defun org-publish-current-file (&optional force)
  "Publish the current file.
With prefix argument, force publish the file."
  (interactive "P")
  (org-publish-initialize-files-alist)
  (save-window-excursion
    (let ((org-publish-use-timestamps-flag
	   (if force nil org-publish-use-timestamps-flag)))
      (org-publish-file (buffer-file-name)))))

;;;###autoload
(defun org-publish-current-project (&optional force)
  "Publish the project associated with the current file.
With a prefix argument, force publishing of all files in
the project."
  (interactive "P")
  (org-publish-initialize-files-alist)
  (save-window-excursion
    (let ((project (org-publish-get-project-from-filename (buffer-file-name) 'up))
	  (org-publish-use-timestamps-flag
	   (if force nil org-publish-use-timestamps-flag)))
      (if (not project)
	  (error "File %s is not part of any known project" (buffer-file-name)))
      (org-publish project))))

(provide 'org-publish)


;; arch-tag: 72807f3c-8af0-4a6b-8dca-c3376eb25adb

;;; org-publish.el ends here