# HG changeset patch # User Katsumi Yamaoka # Date 1283648902 0 # Node ID 48695a2e29d9bb72964b49037b18c317754b5cd4 # Parent 1acd79f80f508b531c301867a9ae33933ba96c73 gnus-start.el: White space clean up; mail-source.el (mail-source-fetch-pop): Use streaming pop3 retrieval; pop3.el (pop3-streaming-movemail): Respect pop3-leave-mail-on-server; pop3.el (pop3-logon): Fix up unbound variable typo; mail-source.el (mail-source-delete-crash-box): Only check the incoming files for deletion once per day to save a lot of file accesses. diff -r 1acd79f80f50 -r 48695a2e29d9 lisp/gnus/ChangeLog --- a/lisp/gnus/ChangeLog Sun Sep 05 00:56:31 2010 +0000 +++ b/lisp/gnus/ChangeLog Sun Sep 05 01:08:22 2010 +0000 @@ -1,5 +1,25 @@ 2010-09-04 Lars Magne Ingebrigtsen + * mail-source.el (mail-source-delete-crash-box): Only check the + incoming files for deletion once per day to save a lot of file + accesses. + + * pop3.el (pop3-logon): Fix up unbound variable typo. + + * mail-source.el (pop3-streaming-movemail): Autoload. + + * pop3.el (pop3-streaming-movemail): Respect + pop3-leave-mail-on-server. + + * mail-source.el (mail-source-fetch-pop): Use streaming pop3 + retrieval. + + * pop3.el (pop3-process-filter): Removed unused function. + (pop3-streaming-movemail, pop3-send-streaming-command) + (pop3-wait-for-messages, pop3-write-to-file) + (pop3-number-of-responses): New functions for streaming pop3 + retrieval. + * gnus-start.el (gnus-get-unread-articles): Protect against groups that come from no known methods. (gnus-make-hashtable-from-newsrc-alist): Remove duplicates from .newsrc diff -r 1acd79f80f50 -r 48695a2e29d9 lisp/gnus/gnus-start.el --- a/lisp/gnus/gnus-start.el Sun Sep 05 00:56:31 2010 +0000 +++ b/lisp/gnus/gnus-start.el Sun Sep 05 01:08:22 2010 +0000 @@ -3184,5 +3184,3 @@ (provide 'gnus-start) ;;; gnus-start.el ends here - - diff -r 1acd79f80f50 -r 48695a2e29d9 lisp/gnus/mail-source.el --- a/lisp/gnus/mail-source.el Sun Sep 05 00:56:31 2010 +0000 +++ b/lisp/gnus/mail-source.el Sun Sep 05 01:08:22 2010 +0000 @@ -34,7 +34,7 @@ (require 'cl) (require 'imap)) (autoload 'auth-source-user-or-password "auth-source") -(autoload 'pop3-movemail "pop3") +(autoload 'pop3-streaming-movemail "pop3") (autoload 'pop3-get-message-count "pop3") (autoload 'nnheader-cancel-timer "nnheader") (require 'mm-util) @@ -624,11 +624,20 @@ 0) (funcall callback mail-source-crash-box info))) +(defvar mail-source-incoming-last-checked-time nil) + (defun mail-source-delete-crash-box () (when (file-exists-p mail-source-crash-box) ;; Delete or move the incoming mail out of the way. (if (eq mail-source-delete-incoming t) (delete-file mail-source-crash-box) + ;; Don't check for old incoming files more than once per day to + ;; save a lot of file accesses. + (when (or (null mail-source-incoming-last-checked-time) + (> (time-to-seconds + (time-since mail-source-incoming-last-checked-time)) + (* 24 60 60))) + (setq mail-source-incoming-last-checked-time (current-time))) (let ((incoming (mm-make-temp-file (expand-file-name @@ -825,9 +834,11 @@ (if (eq authentication 'apop) 'apop 'pass)) (pop3-stream-type stream)) (if (or debug-on-quit debug-on-error) - (save-excursion (pop3-movemail mail-source-crash-box)) + (save-excursion (pop3-streaming-movemail + mail-source-crash-box)) (condition-case err - (save-excursion (pop3-movemail mail-source-crash-box)) + (save-excursion (pop3-streaming-movemail + mail-source-crash-box)) (error ;; We nix out the password in case the error ;; was because of a wrong password being given. diff -r 1acd79f80f50 -r 48695a2e29d9 lisp/gnus/pop3.el --- a/lisp/gnus/pop3.el Sun Sep 05 00:56:31 2010 +0000 +++ b/lisp/gnus/pop3.el Sun Sep 05 01:08:22 2010 +0000 @@ -128,15 +128,90 @@ (truncate pop3-read-timeout)) 1000)))))) -(defun pop3-movemail (&optional crashbox) - "Transfer contents of a maildrop to the specified CRASHBOX." - (or crashbox (setq crashbox (expand-file-name "~/.crashbox"))) +(defun pop3-streaming-movemail (file) + "Transfer contents of a maildrop to the specified FILE. +Use streaming commands." (let* ((process (pop3-open-server pop3-mailhost pop3-port)) - (crashbuf (get-buffer-create " *pop3-retr*")) - (n 1) - message-count - message-sizes - (pop3-password pop3-password)) + message-count message-total-size) + (pop3-logon process) + (with-current-buffer (process-buffer process) + (let ((size (pop3-stat process))) + (setq message-count (car size) + message-total-size (cadr size))) + (when (plusp message-count) + (pop3-send-streaming-command + process "RETR" message-count message-total-size) + (pop3-write-to-file file) + (unless pop3-leave-mail-on-server + (pop3-send-streaming-command + process "DELE" message-count nil)) + (pop3-quit process))))) + +(defun pop3-send-streaming-command (process command count total-size) + (erase-buffer) + (let ((i 1)) + (while (>= (1+ count) i) + (process-send-string process (format "%s %d\r\n" command i)) + ;; Only do 100 messages at a time to avoid pipe stalls. + (when (zerop (% i 100)) + (pop3-wait-for-messages process i total-size)) + (incf i))) + (pop3-wait-for-messages process count total-size)) + +(defun pop3-wait-for-messages (process count total-size) + (while (< (pop3-number-of-responses total-size) count) + (when total-size + (message "pop3 retrieved %dKB (%d%%)" + (truncate (/ (buffer-size) 1000)) + (truncate (* (/ (* (buffer-size) 1.0) + total-size) 100)))) + (nnheader-accept-process-output process))) + +(defun pop3-write-to-file (file) + (let ((pop-buffer (current-buffer)) + (start (point-min)) + beg end + temp-buffer) + (with-temp-buffer + (setq temp-buffer (current-buffer)) + (with-current-buffer pop-buffer + (goto-char (point-min)) + (while (re-search-forward "^\\+OK" nil t) + (forward-line 1) + (setq beg (point)) + (when (re-search-forward "^\\.\r?\n" nil t) + (setq start (point)) + (forward-line -1) + (setq end (point))) + (with-current-buffer temp-buffer + (goto-char (point-max)) + (let ((hstart (point))) + (insert-buffer-substring pop-buffer beg end) + (pop3-clean-region hstart (point)) + (goto-char (point-max)) + (pop3-munge-message-separator hstart (point)) + (goto-char (point-max)))))) + (let ((coding-system-for-write 'binary)) + (goto-char (point-min)) + ;; Check whether something inserted a newline at the start and + ;; delete it. + (when (eolp) + (delete-char 1)) + (write-region (point-min) (point-max) file))))) + +(defun pop3-number-of-responses (endp) + (let ((responses 0)) + (save-excursion + (goto-char (point-min)) + (while (or (and (re-search-forward "^\\+OK " nil t) + (or (not endp) + (re-search-forward "^\\.\r?\n" nil t))) + (re-search-forward "^-ERR " nil t)) + (incf responses))) + responses)) + +(defun pop3-logon (process) + (let ((pop3-password pop3-password)) ;; for debugging only (if pop3-debug (switch-to-buffer (process-buffer process))) ;; query for password @@ -148,10 +223,19 @@ ((equal 'pass pop3-authentication-scheme) (pop3-user process pop3-maildrop) (pop3-pass process)) - (t (error "Invalid POP3 authentication scheme"))) + (t (error "Invalid POP3 authentication scheme"))))) + +(defun pop3-movemail (&optional crashbox) + "Transfer contents of a maildrop to the specified CRASHBOX." + (or crashbox (setq crashbox (expand-file-name "~/.crashbox"))) + (let* ((process (pop3-open-server pop3-mailhost pop3-port)) + (crashbuf (get-buffer-create " *pop3-retr*")) + (n 1) + message-count + message-sizes) + (pop3-logon process) (setq message-count (car (pop3-stat process))) - (when (and pop3-display-message-size-flag - (> message-count 0)) + (when (> message-count 0) (setq message-sizes (pop3-list process))) (unwind-protect (while (<= n message-count) @@ -277,16 +361,11 @@ (setq pop3-timestamp (substring response (or (string-match "<" response) 0) (+ 1 (or (string-match ">" response) -1))))) + (set-process-query-on-exit-flag process nil) process))) ;; Support functions -(defun pop3-process-filter (process output) - (save-excursion - (set-buffer (process-buffer process)) - (goto-char (point-max)) - (insert output))) - (defun pop3-send-command (process command) (set-buffer (process-buffer process)) (goto-char (point-max)) @@ -403,10 +482,7 @@ nil (goto-char (point-max)) (insert "\n")) - (narrow-to-region (point) (point-max)) - (let ((size (- (point-max) (point-min)))) - (goto-char (point-min)) - (widen) + (let ((size (- (point-max) (point)))) (forward-line -1) (insert (format "Content-Length: %s\n" size))) )))))