annotate lisp/gnus/mm-bodies.el @ 110523:a5ad4f188e19

Synch Semantic to CEDET 1.0. Move CEDET ChangeLog entries to new file lisp/cedet/ChangeLog. * semantic.el (semantic-version): Update to 2.0. (semantic-mode-map): Add "," and "m" bindings. (navigate-menu): Update. * semantic/symref.el (semantic-symref-calculate-rootdir): New function. (semantic-symref-detect-symref-tool): Use it. * semantic/symref/grep.el (semantic-symref-grep-shell): New var. (semantic-symref-perform-search): Use it. Calculate root dir with semantic-symref-calculate-rootdir. (semantic-symref-derive-find-filepatterns): Improve error message. * semantic/symref/list.el (semantic-symref-results-mode-map): New bindings. (semantic-symref-auto-expand-results): New option. (semantic-symref-results-dump): Obey auto-expand. (semantic-symref-list-expand-all, semantic-symref-regexp) (semantic-symref-list-contract-all) (semantic-symref-list-map-open-hits) (semantic-symref-list-update-open-hits) (semantic-symref-list-create-macro-on-open-hit) (semantic-symref-list-call-macro-on-open-hits): New functions. (semantic-symref-list-menu-entries) (semantic-symref-list-menu): New vars. (semantic-symref-list-map-open-hits): Move cursor to beginning of match before calling the mapped function. * semantic/doc.el (semantic-documentation-comment-preceeding-tag): Do nothing if the mode doesn't provide comment-start-skip. * semantic/scope.el (semantic-analyze-scope-nested-tags-default): Strip duplicates. (semantic-analyze-scoped-inherited-tag-map): Take the tag we are looking for as part of the scoped tags list. * semantic/html.el (semantic-default-html-setup): Add senator-step-at-tag-classes. * semantic/decorate/include.el (semantic-decoration-on-unknown-includes): Change light bgcolor. (semantic-decoration-on-includes-highlight-default): Check that the include tag has a postion. * semantic/complete.el (semantic-collector-local-members): (semantic-complete-read-tag-local-members) (semantic-complete-jump-local-members): New class and functions. (semantic-complete-self-insert): Save excursion before completing. * semantic/analyze/complete.el (semantic-analyze-possible-completions-default): If no completions are found, return the raw by-name-only completion list. Add FLAGS arguments. Add support for 'no-tc (type constraint) and 'no-unique, or no stripping duplicates. (semantic-analyze-possible-completions-default): Add FLAGS arg. * semantic/util-modes.el (semantic-stickyfunc-show-only-functions-p): New option. (semantic-stickyfunc-fetch-stickyline): Don't show stickytext for the very first line in a buffer. * semantic/util.el (semantic-hack-search) (semantic-recursive-find-nonterminal-by-name) (semantic-current-tag-interactive): Deleted. (semantic-describe-buffer): Fix expand-nonterminal. Add lex-syntax-mods, type relation separator char, and command separation char. (semantic-sanity-check): Only message if called interactively. * semantic/tag.el (semantic-tag-deep-copy-one-tag): Copy the :filename property and the tag position. * semantic/lex-spp.el (semantic-lex-spp-lex-text-string): Add recursion limit. * semantic/imenu.el (semantic-imenu-bucketize-type-members): Make this buffer local, not the obsoleted variable. * semantic/idle.el: Add breadcrumbs support. (semantic-idle-summary-current-symbol-info-default) (semantic-idle-tag-highlight) (semantic-idle-completion-list-default): Use semanticdb-without-unloaded-file-searches for speed, and to conform to the controls that specify if the idle timer is supposed to be parsing unparsed includes. (semantic-idle-symbol-highlight-face) (semantic-idle-symbol-maybe-highlight): Rename from *-summary-*. Callers changed. (semantic-idle-work-parse-neighboring-files-flag): Default to nil. (semantic-idle-work-update-headers-flag): New var. (semantic-idle-work-for-one-buffer): Use it. (semantic-idle-local-symbol-highlight): Rename from semantic-idle-tag-highlight. (semantic-idle-truncate-long-summaries): New option. * semantic/ia.el (semantic-ia-cache) (semantic-ia-get-completions): Deleted. Callers changed. (semantic-ia-show-variants): New command. (semantic-ia-show-doc): If doc is empty, don't make a temp buffer. (semantic-ia-show-summary): If there isn't anything to show, say so. * semantic/grammar.el (semantic-grammar-create-package): Save the buffer even in batch mode. * semantic/fw.el (semanticdb-without-unloaded-file-searches): New macro. * semantic/dep.el (semantic-dependency-find-file-on-path): Fix case dereferencing ede-object when it is a list. * semantic/db-typecache.el (semanticdb-expand-nested-tag) (semanticdb-typecache-faux-namespace): New functions. (semanticdb-typecache-file-tags) (semanticdb-typecache-merge-streams): Use them. (semanticdb-typecache-file-tags): When deriving tags from a file, give the mode a chance to monkey with the tag copy. (semanticdb-typecache-find-default): Wrap find in save-excursion. (semanticdb-typecache-find-by-name-helper): Merge found names down. * semantic/db-global.el (semanticdb-enable-gnu-global-in-buffer): Don't show messages if GNU Global is not available and we don't want to throw an error. * semantic/db-find.el (semanticdb-find-result-nth-in-buffer): When trying to normalize the tag to a buffer, don't error if set-buffer method doesn't exist. * semantic/db-file.el (semanticdb-save-db): Simplify msg. * semantic/db.el (semanticdb-refresh-table): If forcing a refresh on a file not in a buffer, use semantic-find-file-noselect and delete the buffer after use. (semanticdb-current-database-list): When calculating root via hooks, force it through true-filename and skip the list of possible roots. * semantic/ctxt.el (semantic-ctxt-imported-packages): New. * semantic/analyze/debug.el (semantic-analyzer-debug-insert-tag): Reset standard output to current buffer. (semantic-analyzer-debug-global-symbol) (semantic-analyzer-debug-missing-innertype): Change "prefix" to "symbol" in messages. * semantic/analyze/refs.el: (semantic-analyze-refs-impl) (semantic-analyze-refs-proto): When calculating value, make sure the found tag is 'similar' to the originating tag. (semantic--analyze-refs-find-tags-with-parent): Attempt to identify matches via imported symbols of parents. (semantic--analyze-refs-full-lookup-with-parents): Do a deep search during the brute search. * semantic/analyze.el (semantic-analyze-find-tag-sequence-default): Be robust to calculated scopes being nil. * semantic/bovine/c.el (semantic-c-describe-environment): Add project macro symbol array. (semantic-c-parse-lexical-token): Add recursion limit. (semantic-ctxt-imported-packages, semanticdb-expand-nested-tag): New overrides. (semantic-expand-c-tag-namelist): Split a full type from a typedef out to its own tag. (semantic-expand-c-tag-namelist): Do not split out a typedef'd inline type if it is an anonymous type. (semantic-c-reconstitute-token): Use the optional initializers as a clue that some function is probably a constructor. When defining the type of these constructors, split the parent name, and use only the class part, if applicable. * semantic/bovine/c-by.el: * semantic/wisent/python-wy.el: Regenerate.

;;; mm-bodies.el --- Functions for decoding MIME things

;; Copyright (C) 1998, 1999, 2000, 2001, 2002, 2003, 2004,
;;   2005, 2006, 2007, 2008, 2009, 2010 Free Software Foundation, Inc.

;; Author: Lars Magne Ingebrigtsen <[e-mail]>
;;	MORIOKA Tomohiko <[e-mail]>
;; This file is part of GNU Emacs.

;; GNU Emacs is free software: you can redistribute it and/or modify
;; it under the terms of the GNU General Public License as published by
;; the Free Software Foundation, either version 3 of the License, or
;; (at your option) any later version.

;; GNU Emacs is distributed in the hope that it will be useful,
;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;; GNU General Public License for more details.

;; You should have received a copy of the GNU General Public License
;; along with GNU Emacs.  If not, see <http://www.gnu.org/licenses/>.

;;; Commentary:

;;; Code:

;; For Emacs < 22.2.
(eval-and-compile
  (unless (fboundp 'declare-function) (defmacro declare-function (&rest r))))

(require 'mm-util)
(require 'rfc2047)
(require 'mm-encode)

(defvar mm-uu-yenc-decode-function)
(defvar mm-uu-decode-function)
(defvar mm-uu-binhex-decode-function)

;; 8bit treatment gets any char except: 0x32 - 0x7f, LF, TAB, BEL,
;; BS, vertical TAB, form feed, and ^_
;;
;; Note that CR is *not* included, as that would allow a non-paired CR
;; in the body contrary to RFC 2822:
;;
;;   - CR and LF MUST only occur together as CRLF; they MUST NOT
;;     appear independently in the body.

(defvar mm-7bit-chars "\x20-\x7f\n\t\x7\x8\xb\xc\x1f")

(defcustom mm-body-charset-encoding-alist
  '((iso-2022-jp . 7bit)
    (iso-2022-jp-2 . 7bit)
    ;; We MUST encode UTF-16 because it can contain \0's which is
    ;; known to break servers.
    ;; Note: UTF-16 variants are invalid for text parts [RFC 2781],
    ;; so this can't happen :-/.
    ;; PPS: Yes, it can happen if the user specifies UTF-16 in the MML
    ;; markup. - jh.
    (utf-16 . base64)
    (utf-16be . base64)
    (utf-16le . base64))
  "Alist of MIME charsets to encodings.
Valid encodings are `7bit', `8bit', `quoted-printable' and `base64'."
  :type '(repeat (cons (symbol :tag "charset")
		       (choice :tag "encoding"
			       (const 7bit)
			       (const 8bit)
			       (const quoted-printable)
			       (const base64))))
  :group 'mime)

(autoload 'message-options-get "message")
(declare-function message-options-set "message" (symbol value))

(defun mm-encode-body (&optional charset)
  "Encode a body.
Should be called narrowed to the body that is to be encoded.
If there is more than one non-ASCII MULE charset in the body, then the
list of MULE charsets found is returned.
If CHARSET is non-nil, it is used as the MIME charset to encode the body.
If successful, the MIME charset is returned.
If no encoding was done, nil is returned."
  (if (not (mm-multibyte-p))
      ;; In the non-Mule case, we search for non-ASCII chars and
      ;; return the value of `mail-parse-charset' if any are found.
      (or charset
	  (save-excursion
	    (goto-char (point-min))
	    (if (re-search-forward "[^\x0-\x7f]" nil t)
		(or mail-parse-charset
		    (message-options-get 'mm-encody-body-charset)
		    (message-options-set
		     'mm-encody-body-charset
		     (mm-read-coding-system "Charset used in the article: ")))
	      ;; The logic in `mml-generate-mime-1' confirms that it's OK
	      ;; to return nil here.
	      nil)))
    (save-excursion
      (if charset
	  (progn
	    (mm-encode-coding-region (point-min) (point-max)
				     (mm-charset-to-coding-system charset))
	    charset)
	(goto-char (point-min))
	(let ((charsets (mm-find-mime-charset-region (point-min) (point-max)
						     mm-hack-charsets)))
	  (cond
	   ;; No encoding.
	   ((null charsets)
	    nil)
	   ;; Too many charsets.
	   ((> (length charsets) 1)
	    charsets)
	   ;; We encode.
	   (t
	    (prog1
		(setq charset (car charsets))
	      (mm-encode-coding-region (point-min) (point-max)
				       (mm-charset-to-coding-system charset))))
	   ))))))

(defun mm-long-lines-p (length)
  "Say whether any of the lines in the buffer is longer than LENGTH."
  (save-excursion
    (goto-char (point-min))
    (end-of-line)
    (while (and (not (eobp))
		(not (> (current-column) length)))
      (forward-line 1)
      (end-of-line))
    (and (> (current-column) length)
	 (current-column))))

(defvar message-posting-charset)

(defun mm-body-encoding (charset &optional encoding)
  "Do Content-Transfer-Encoding and return the encoding of the current buffer."
  (when (stringp encoding)
    (setq encoding (intern (downcase encoding))))
  (let ((bits (mm-body-7-or-8))
	(longp (mm-long-lines-p 1000)))
    (require 'message)
    (cond
     ((and (not longp)
	   (not (and mm-use-ultra-safe-encoding
		     (or (save-excursion (re-search-forward " $" nil t))
			 (save-excursion (re-search-forward "^From " nil t)))))
	   (eq bits '7bit))
      bits)
     ((and (not mm-use-ultra-safe-encoding)
	   (not longp)
	   (not (cdr (assq charset mm-body-charset-encoding-alist)))
	   (or (eq t (cdr message-posting-charset))
	       (memq charset (cdr message-posting-charset))
	       (eq charset mail-parse-charset)))
      bits)
     (t
      (let ((encoding (or encoding
			  (cdr (assq charset mm-body-charset-encoding-alist))
			  (mm-qp-or-base64))))
	(when mm-use-ultra-safe-encoding
	  (setq encoding (mm-safer-encoding encoding)))
	(mm-encode-content-transfer-encoding encoding "text/plain")
	encoding)))))

(defun mm-body-7-or-8 ()
  "Say whether the body is 7bit or 8bit."
  (if (save-excursion
	(goto-char (point-min))
	(skip-chars-forward mm-7bit-chars)
	(eobp))
      '7bit
    '8bit))

;;;
;;; Functions for decoding
;;;

(defun mm-decode-content-transfer-encoding (encoding &optional type)
  "Decodes buffer encoded with ENCODING, returning success status.
If TYPE is `text/plain' CRLF->LF translation may occur."
  (prog1
      (condition-case error
	  (cond
	   ((eq encoding 'quoted-printable)
	    (quoted-printable-decode-region (point-min) (point-max))
	    t)
	   ((eq encoding 'base64)
	    (base64-decode-region
	     (point-min)
	     ;; Some mailers insert whitespace
	     ;; junk at the end which
	     ;; base64-decode-region dislikes.
	     ;; Also remove possible junk which could
	     ;; have been added by mailing list software.
	     (save-excursion
	       (goto-char (point-min))
	       (while (re-search-forward "^[\t ]*\r?\n" nil t)
		 (delete-region (match-beginning 0) (match-end 0)))
	       (goto-char (point-max))
	       (when (re-search-backward "^[A-Za-z0-9+/]+=*[\t ]*$" nil t)
		 (forward-line))
	       (point))))
	   ((memq encoding '(nil 7bit 8bit binary))
	    ;; Do nothing.
	    t)
	   ((memq encoding '(x-uuencode x-uue))
	    (require 'mm-uu)
	    (funcall mm-uu-decode-function (point-min) (point-max))
	    t)
	   ((eq encoding 'x-binhex)
	    (require 'mm-uu)
	    (funcall mm-uu-binhex-decode-function (point-min) (point-max))
	    t)
	   ((eq encoding 'x-yenc)
	    (require 'mm-uu)
	    (funcall mm-uu-yenc-decode-function (point-min) (point-max))
	    )
	   ((functionp encoding)
	    (funcall encoding (point-min) (point-max))
	    t)
	   (t
	    (message "Unknown encoding %s; defaulting to 8bit" encoding)))
	(error
	 (message "Error while decoding: %s" error)
	 nil))
    (when (and
	   type
	   (memq encoding '(base64 x-uuencode x-uue x-binhex x-yenc))
	   (string-match "\\`text/" type))
      (goto-char (point-min))
      (while (search-forward "\r\n" nil t)
	(replace-match "\n" t t)))))

(defun mm-decode-body (charset &optional encoding type)
  "Decode the current article that has been encoded with ENCODING to CHARSET.
ENCODING is a MIME content transfer encoding.
CHARSET is the MIME charset with which to decode the data after transfer
decoding.  If it is nil, default to `mail-parse-charset'."
  (when (stringp charset)
    (setq charset (intern (downcase charset))))
  (when (or (not charset)
	    (eq 'gnus-all mail-parse-ignored-charsets)
	    (memq 'gnus-all mail-parse-ignored-charsets)
	    (memq charset mail-parse-ignored-charsets))
    (setq charset mail-parse-charset))
  (save-excursion
    (when encoding
      (mm-decode-content-transfer-encoding encoding type))
    (when (and (featurep 'mule) ;; Fixme: Wrong test for unibyte session.
	       (not (eq charset 'gnus-decoded)))
      (let ((coding-system (mm-charset-to-coding-system
			    ;; Allow overwrite using
			    ;; `mm-charset-override-alist'.
			    charset nil t)))
	(if (and (not coding-system)
		 (listp mail-parse-ignored-charsets)
		 (memq 'gnus-unknown mail-parse-ignored-charsets))
	    (setq coding-system
		  (mm-charset-to-coding-system mail-parse-charset)))
	(when (and charset coding-system
		   ;; buffer-file-coding-system
		   ;;Article buffer is nil coding system
		   ;;in XEmacs
		   (mm-multibyte-p)
		   (or (not (eq coding-system 'ascii))
		       (setq coding-system mail-parse-charset)))
	  (mm-decode-coding-region (point-min) (point-max)
				   coding-system))
	(setq buffer-file-coding-system
	      (if (boundp 'last-coding-system-used)
		  (symbol-value 'last-coding-system-used)
		coding-system))))))

(defun mm-decode-string (string charset)
  "Decode STRING with CHARSET."
  (when (stringp charset)
    (setq charset (intern (downcase charset))))
  (when (or (not charset)
	    (eq 'gnus-all mail-parse-ignored-charsets)
	    (memq 'gnus-all mail-parse-ignored-charsets)
	    (memq charset mail-parse-ignored-charsets))
    (setq charset mail-parse-charset))
  (or
   (when (featurep 'mule)
     (let ((coding-system (mm-charset-to-coding-system
			   charset
			   ;; Allow overwrite using
			   ;; `mm-charset-override-alist'.
			   nil t)))
       (if (and (not coding-system)
		(listp mail-parse-ignored-charsets)
		(memq 'gnus-unknown mail-parse-ignored-charsets))
	   (setq coding-system
		 (mm-charset-to-coding-system mail-parse-charset)))
       (when (and charset coding-system
		  (mm-multibyte-p)
		  (or (not (eq coding-system 'ascii))
		      (setq coding-system mail-parse-charset)))
	 (mm-decode-coding-string string coding-system))))
   string))

(provide 'mm-bodies)

;; arch-tag: 41104bb6-4443-4ca9-8d5c-ff87ecf27d8d
;;; mm-bodies.el ends here