21006
|
1 @c -*-texinfo-*-
|
|
2 @c This is part of the GNU Emacs Lisp Reference Manual.
|
27189
|
3 @c Copyright (C) 1998, 1999 Free Software Foundation, Inc.
|
21006
|
4 @c See the file elisp.texi for copying conditions.
|
|
5 @setfilename ../info/characters
|
|
6 @node Non-ASCII Characters, Searching and Matching, Text, Top
|
27374
|
7 @chapter Non-@sc{ascii} Characters
|
21006
|
8 @cindex multibyte characters
|
27374
|
9 @cindex non-@sc{ascii} characters
|
21006
|
10
|
25751
|
11 This chapter covers the special issues relating to non-@sc{ascii}
|
21006
|
12 characters and how they are stored in strings and buffers.
|
|
13
|
|
14 @menu
|
28635
|
15 * Text Representations:: Unibyte and multibyte representations
|
|
16 * Converting Representations:: Converting unibyte to multibyte and vice versa.
|
|
17 * Selecting a Representation:: Treating a byte sequence as unibyte or multi.
|
|
18 * Character Codes:: How unibyte and multibyte relate to
|
|
19 codes of individual characters.
|
|
20 * Character Sets:: The space of possible characters codes
|
|
21 is divided into various character sets.
|
|
22 * Chars and Bytes:: More information about multibyte encodings.
|
|
23 * Splitting Characters:: Converting a character to its byte sequence.
|
|
24 * Scanning Charsets:: Which character sets are used in a buffer?
|
|
25 * Translation of Characters:: Translation tables are used for conversion.
|
|
26 * Coding Systems:: Coding systems are conversions for saving files.
|
|
27 * Input Methods:: Input methods allow users to enter various
|
40834
|
28 non-ASCII characters without special keyboards.
|
28635
|
29 * Locales:: Interacting with the POSIX locale.
|
21006
|
30 @end menu
|
|
31
|
|
32 @node Text Representations
|
|
33 @section Text Representations
|
|
34 @cindex text representations
|
|
35
|
|
36 Emacs has two @dfn{text representations}---two ways to represent text
|
|
37 in a string or buffer. These are called @dfn{unibyte} and
|
|
38 @dfn{multibyte}. Each string, and each buffer, uses one of these two
|
|
39 representations. For most purposes, you can ignore the issue of
|
|
40 representations, because Emacs converts text between them as
|
|
41 appropriate. Occasionally in Lisp programming you will need to pay
|
|
42 attention to the difference.
|
|
43
|
|
44 @cindex unibyte text
|
|
45 In unibyte representation, each character occupies one byte and
|
|
46 therefore the possible character codes range from 0 to 255. Codes 0
|
25751
|
47 through 127 are @sc{ascii} characters; the codes from 128 through 255
|
|
48 are used for one non-@sc{ascii} character set (you can choose which
|
21682
|
49 character set by setting the variable @code{nonascii-insert-offset}).
|
21006
|
50
|
|
51 @cindex leading code
|
|
52 @cindex multibyte text
|
22252
|
53 @cindex trailing codes
|
21006
|
54 In multibyte representation, a character may occupy more than one
|
|
55 byte, and as a result, the full range of Emacs character codes can be
|
|
56 stored. The first byte of a multibyte character is always in the range
|
|
57 128 through 159 (octal 0200 through 0237). These values are called
|
22138
|
58 @dfn{leading codes}. The second and subsequent bytes of a multibyte
|
|
59 character are always in the range 160 through 255 (octal 0240 through
|
22252
|
60 0377); these values are @dfn{trailing codes}.
|
21006
|
61
|
28877
|
62 Some sequences of bytes are not valid in multibyte text: for example,
|
32523
|
63 a single isolated byte in the range 128 through 159 is not allowed. But
|
|
64 character codes 128 through 159 can appear in multibyte text,
|
|
65 represented as two-byte sequences. All the character codes 128 through
|
|
66 255 are possible (though slightly abnormal) in multibyte text; they
|
28877
|
67 appear in multibyte buffers and strings when you do explicit encoding
|
|
68 and decoding (@pxref{Explicit Encoding}).
|
24951
|
69
|
21006
|
70 In a buffer, the buffer-local value of the variable
|
|
71 @code{enable-multibyte-characters} specifies the representation used.
|
24952
|
72 The representation for a string is determined and recorded in the string
|
|
73 when the string is constructed.
|
21006
|
74
|
22138
|
75 @defvar enable-multibyte-characters
|
21006
|
76 This variable specifies the current buffer's text representation.
|
|
77 If it is non-@code{nil}, the buffer contains multibyte text; otherwise,
|
|
78 it contains unibyte text.
|
|
79
|
21682
|
80 You cannot set this variable directly; instead, use the function
|
|
81 @code{set-buffer-multibyte} to change a buffer's representation.
|
21006
|
82 @end defvar
|
|
83
|
22138
|
84 @defvar default-enable-multibyte-characters
|
|
85 This variable's value is entirely equivalent to @code{(default-value
|
21006
|
86 'enable-multibyte-characters)}, and setting this variable changes that
|
22138
|
87 default value. Setting the local binding of
|
|
88 @code{enable-multibyte-characters} in a specific buffer is not allowed,
|
|
89 but changing the default value is supported, and it is a reasonable
|
|
90 thing to do, because it has no effect on existing buffers.
|
21006
|
91
|
|
92 The @samp{--unibyte} command line option does its job by setting the
|
|
93 default value to @code{nil} early in startup.
|
|
94 @end defvar
|
|
95
|
24951
|
96 @defun position-bytes position
|
|
97 @tindex position-bytes
|
|
98 Return the byte-position corresponding to buffer position @var{position}
|
|
99 in the current buffer.
|
|
100 @end defun
|
|
101
|
|
102 @defun byte-to-position byte-position
|
|
103 @tindex byte-to-position
|
|
104 Return the buffer position corresponding to byte-position
|
|
105 @var{byte-position} in the current buffer.
|
|
106 @end defun
|
|
107
|
22138
|
108 @defun multibyte-string-p string
|
24951
|
109 Return @code{t} if @var{string} is a multibyte string.
|
21006
|
110 @end defun
|
|
111
|
|
112 @node Converting Representations
|
|
113 @section Converting Text Representations
|
|
114
|
|
115 Emacs can convert unibyte text to multibyte; it can also convert
|
|
116 multibyte text to unibyte, though this conversion loses information. In
|
|
117 general these conversions happen when inserting text into a buffer, or
|
|
118 when putting text from several strings together in one string. You can
|
|
119 also explicitly convert a string's contents to either representation.
|
|
120
|
|
121 Emacs chooses the representation for a string based on the text that
|
|
122 it is constructed from. The general rule is to convert unibyte text to
|
|
123 multibyte text when combining it with other multibyte text, because the
|
|
124 multibyte representation is more general and can hold whatever
|
|
125 characters the unibyte text has.
|
|
126
|
|
127 When inserting text into a buffer, Emacs converts the text to the
|
|
128 buffer's representation, as specified by
|
|
129 @code{enable-multibyte-characters} in that buffer. In particular, when
|
|
130 you insert multibyte text into a unibyte buffer, Emacs converts the text
|
|
131 to unibyte, even though this conversion cannot in general preserve all
|
|
132 the characters that might be in the multibyte text. The other natural
|
|
133 alternative, to convert the buffer contents to multibyte, is not
|
|
134 acceptable because the buffer's representation is a choice made by the
|
21682
|
135 user that cannot be overridden automatically.
|
21006
|
136
|
25751
|
137 Converting unibyte text to multibyte text leaves @sc{ascii} characters
|
32523
|
138 unchanged, and likewise character codes 128 through 159. It converts
|
|
139 the non-@sc{ascii} codes 160 through 255 by adding the value
|
|
140 @code{nonascii-insert-offset} to each character code. By setting this
|
|
141 variable, you specify which character set the unibyte characters
|
|
142 correspond to (@pxref{Character Sets}). For example, if
|
|
143 @code{nonascii-insert-offset} is 2048, which is @code{(- (make-char
|
|
144 'latin-iso8859-1) 128)}, then the unibyte non-@sc{ascii} characters
|
|
145 correspond to Latin 1. If it is 2688, which is @code{(- (make-char
|
|
146 'greek-iso8859-7) 128)}, then they correspond to Greek letters.
|
21006
|
147
|
25751
|
148 Converting multibyte text to unibyte is simpler: it discards all but
|
|
149 the low 8 bits of each character code. If @code{nonascii-insert-offset}
|
|
150 has a reasonable value, corresponding to the beginning of some character
|
|
151 set, this conversion is the inverse of the other: converting unibyte
|
|
152 text to multibyte and back to unibyte reproduces the original unibyte
|
|
153 text.
|
21006
|
154
|
22138
|
155 @defvar nonascii-insert-offset
|
25751
|
156 This variable specifies the amount to add to a non-@sc{ascii} character
|
21006
|
157 when converting unibyte text to multibyte. It also applies when
|
22138
|
158 @code{self-insert-command} inserts a character in the unibyte
|
29339
|
159 non-@sc{ascii} range, 128 through 255. However, the functions
|
29265
|
160 @code{insert} and @code{insert-char} do not perform this conversion.
|
21006
|
161
|
|
162 The right value to use to select character set @var{cs} is @code{(-
|
22138
|
163 (make-char @var{cs}) 128)}. If the value of
|
21006
|
164 @code{nonascii-insert-offset} is zero, then conversion actually uses the
|
|
165 value for the Latin 1 character set, rather than zero.
|
|
166 @end defvar
|
|
167
|
22138
|
168 @defvar nonascii-translation-table
|
21006
|
169 This variable provides a more general alternative to
|
|
170 @code{nonascii-insert-offset}. You can use it to specify independently
|
|
171 how to translate each code in the range of 128 through 255 into a
|
29265
|
172 multibyte character. The value should be a char-table, or @code{nil}.
|
21682
|
173 If this is non-@code{nil}, it overrides @code{nonascii-insert-offset}.
|
21006
|
174 @end defvar
|
|
175
|
22138
|
176 @defun string-make-unibyte string
|
21006
|
177 This function converts the text of @var{string} to unibyte
|
22252
|
178 representation, if it isn't already, and returns the result. If
|
45652
|
179 @var{string} is a unibyte string, it is returned unchanged. Multibyte
|
|
180 character codes are converted to unibyte according to
|
|
181 @code{nonascii-translation-table} or, if that is @code{nil}, using
|
|
182 @code{nonascii-insert-offset}. If the lookup in the translation table
|
|
183 fails, this function takes just the low 8 bits of each character.
|
21006
|
184 @end defun
|
|
185
|
22138
|
186 @defun string-make-multibyte string
|
21006
|
187 This function converts the text of @var{string} to multibyte
|
22252
|
188 representation, if it isn't already, and returns the result. If
|
21682
|
189 @var{string} is a multibyte string, it is returned unchanged.
|
33912
|
190 The function @code{unibyte-char-to-multibyte} is used to convert
|
|
191 each unibyte character to a multibyte character.
|
21006
|
192 @end defun
|
|
193
|
|
194 @node Selecting a Representation
|
|
195 @section Selecting a Representation
|
|
196
|
|
197 Sometimes it is useful to examine an existing buffer or string as
|
|
198 multibyte when it was unibyte, or vice versa.
|
|
199
|
22138
|
200 @defun set-buffer-multibyte multibyte
|
21006
|
201 Set the representation type of the current buffer. If @var{multibyte}
|
|
202 is non-@code{nil}, the buffer becomes multibyte. If @var{multibyte}
|
|
203 is @code{nil}, the buffer becomes unibyte.
|
|
204
|
|
205 This function leaves the buffer contents unchanged when viewed as a
|
|
206 sequence of bytes. As a consequence, it can change the contents viewed
|
|
207 as characters; a sequence of two bytes which is treated as one character
|
|
208 in multibyte representation will count as two characters in unibyte
|
29265
|
209 representation. Character codes 128 through 159 are an exception. They
|
|
210 are represented by one byte in a unibyte buffer, but when the buffer is
|
|
211 set to multibyte, they are converted to two-byte sequences, and vice
|
|
212 versa.
|
21006
|
213
|
|
214 This function sets @code{enable-multibyte-characters} to record which
|
|
215 representation is in use. It also adjusts various data in the buffer
|
21682
|
216 (including overlays, text properties and markers) so that they cover the
|
|
217 same text as they did before.
|
24951
|
218
|
|
219 You cannot use @code{set-buffer-multibyte} on an indirect buffer,
|
|
220 because indirect buffers always inherit the representation of the
|
|
221 base buffer.
|
21006
|
222 @end defun
|
|
223
|
22138
|
224 @defun string-as-unibyte string
|
21006
|
225 This function returns a string with the same bytes as @var{string} but
|
|
226 treating each byte as a character. This means that the value may have
|
|
227 more characters than @var{string} has.
|
|
228
|
24951
|
229 If @var{string} is already a unibyte string, then the value is
|
33912
|
230 @var{string} itself. Otherwise it is a newly created string, with no
|
|
231 text properties. If @var{string} is multibyte, any characters it
|
|
232 contains of charset @var{eight-bit-control} or @var{eight-bit-graphic}
|
|
233 are converted to the corresponding single byte.
|
21006
|
234 @end defun
|
|
235
|
22138
|
236 @defun string-as-multibyte string
|
21006
|
237 This function returns a string with the same bytes as @var{string} but
|
|
238 treating each multibyte sequence as one character. This means that the
|
|
239 value may have fewer characters than @var{string} has.
|
|
240
|
24951
|
241 If @var{string} is already a multibyte string, then the value is
|
33912
|
242 @var{string} itself. Otherwise it is a newly created string, with no
|
|
243 text properties. If @var{string} is unibyte and contains any individual
|
|
244 8-bit bytes (i.e.@: not part of a multibyte form), they are converted to
|
|
245 the corresponding multibyte character of charset @var{eight-bit-control}
|
|
246 or @var{eight-bit-graphic}.
|
21006
|
247 @end defun
|
|
248
|
|
249 @node Character Codes
|
|
250 @section Character Codes
|
|
251 @cindex character codes
|
|
252
|
|
253 The unibyte and multibyte text representations use different character
|
|
254 codes. The valid character codes for unibyte representation range from
|
|
255 0 to 255---the values that can fit in one byte. The valid character
|
|
256 codes for multibyte representation range from 0 to 524287, but not all
|
28877
|
257 values in that range are valid. The values 128 through 255 are not
|
32523
|
258 entirely proper in multibyte text, but they can occur if you do explicit
|
28877
|
259 encoding and decoding (@pxref{Explicit Encoding}). Some other character
|
|
260 codes cannot occur at all in multibyte text. Only the @sc{ascii} codes
|
32523
|
261 0 through 127 are completely legitimate in both representations.
|
21006
|
262
|
29265
|
263 @defun char-valid-p charcode &optional genericp
|
21006
|
264 This returns @code{t} if @var{charcode} is valid for either one of the two
|
|
265 text representations.
|
|
266
|
|
267 @example
|
|
268 (char-valid-p 65)
|
|
269 @result{} t
|
|
270 (char-valid-p 256)
|
|
271 @result{} nil
|
|
272 (char-valid-p 2248)
|
|
273 @result{} t
|
|
274 @end example
|
29265
|
275
|
|
276 If the optional argument @var{genericp} is non-nil, this function
|
|
277 returns @code{t} if @var{charcode} is a generic character
|
29339
|
278 (@pxref{Splitting Characters}).
|
21006
|
279 @end defun
|
|
280
|
|
281 @node Character Sets
|
|
282 @section Character Sets
|
|
283 @cindex character sets
|
|
284
|
|
285 Emacs classifies characters into various @dfn{character sets}, each of
|
|
286 which has a name which is a symbol. Each character belongs to one and
|
|
287 only one character set.
|
|
288
|
|
289 In general, there is one character set for each distinct script. For
|
|
290 example, @code{latin-iso8859-1} is one character set,
|
|
291 @code{greek-iso8859-7} is another, and @code{ascii} is another. An
|
21682
|
292 Emacs character set can hold at most 9025 characters; therefore, in some
|
|
293 cases, characters that would logically be grouped together are split
|
22138
|
294 into several character sets. For example, one set of Chinese
|
|
295 characters, generally known as Big 5, is divided into two Emacs
|
|
296 character sets, @code{chinese-big5-1} and @code{chinese-big5-2}.
|
21006
|
297
|
28900
|
298 @sc{ascii} characters are in character set @code{ascii}. The
|
|
299 non-@sc{ascii} characters 128 through 159 are in character set
|
|
300 @code{eight-bit-control}, and codes 160 through 255 are in character set
|
|
301 @code{eight-bit-graphic}.
|
|
302
|
22138
|
303 @defun charsetp object
|
25751
|
304 Returns @code{t} if @var{object} is a symbol that names a character set,
|
21006
|
305 @code{nil} otherwise.
|
|
306 @end defun
|
|
307
|
22138
|
308 @defun charset-list
|
21006
|
309 This function returns a list of all defined character set names.
|
|
310 @end defun
|
|
311
|
22138
|
312 @defun char-charset character
|
24951
|
313 This function returns the name of the character set that @var{character}
|
|
314 belongs to.
|
21006
|
315 @end defun
|
|
316
|
25751
|
317 @defun charset-plist charset
|
|
318 @tindex charset-plist
|
|
319 This function returns the charset property list of the character set
|
|
320 @var{charset}. Although @var{charset} is a symbol, this is not the same
|
|
321 as the property list of that symbol. Charset properties are used for
|
29265
|
322 special purposes within Emacs; for example,
|
|
323 @code{preferred-coding-system} helps determine which coding system to
|
|
324 use to encode characters in a charset.
|
25751
|
325 @end defun
|
|
326
|
21006
|
327 @node Chars and Bytes
|
|
328 @section Characters and Bytes
|
|
329 @cindex bytes and characters
|
|
330
|
22138
|
331 @cindex introduction sequence
|
|
332 @cindex dimension (of character set)
|
21006
|
333 In multibyte representation, each character occupies one or more
|
22138
|
334 bytes. Each character set has an @dfn{introduction sequence}, which is
|
25751
|
335 normally one or two bytes long. (Exception: the @sc{ascii} character
|
29265
|
336 set and the @sc{eight-bit-graphic} character set have a zero-length
|
|
337 introduction sequence.) The introduction sequence is the beginning of
|
|
338 the byte sequence for any character in the character set. The rest of
|
|
339 the character's bytes distinguish it from the other characters in the
|
|
340 same character set. Depending on the character set, there are either
|
|
341 one or two distinguishing bytes; the number of such bytes is called the
|
|
342 @dfn{dimension} of the character set.
|
22138
|
343
|
|
344 @defun charset-dimension charset
|
24951
|
345 This function returns the dimension of @var{charset}; at present, the
|
|
346 dimension is always 1 or 2.
|
|
347 @end defun
|
|
348
|
|
349 @defun charset-bytes charset
|
|
350 @tindex charset-bytes
|
|
351 This function returns the number of bytes used to represent a character
|
|
352 in character set @var{charset}.
|
22138
|
353 @end defun
|
|
354
|
|
355 This is the simplest way to determine the byte length of a character
|
|
356 set's introduction sequence:
|
|
357
|
|
358 @example
|
24951
|
359 (- (charset-bytes @var{charset})
|
22138
|
360 (charset-dimension @var{charset}))
|
|
361 @end example
|
|
362
|
|
363 @node Splitting Characters
|
|
364 @section Splitting Characters
|
|
365
|
|
366 The functions in this section convert between characters and the byte
|
|
367 values used to represent them. For most purposes, there is no need to
|
|
368 be concerned with the sequence of bytes used to represent a character,
|
21682
|
369 because Emacs translates automatically when necessary.
|
21006
|
370
|
22138
|
371 @defun split-char character
|
21006
|
372 Return a list containing the name of the character set of
|
22138
|
373 @var{character}, followed by one or two byte values (integers) which
|
|
374 identify @var{character} within that character set. The number of byte
|
|
375 values is the character set's dimension.
|
21006
|
376
|
|
377 @example
|
|
378 (split-char 2248)
|
|
379 @result{} (latin-iso8859-1 72)
|
|
380 (split-char 65)
|
|
381 @result{} (ascii 65)
|
29265
|
382 (split-char 128)
|
|
383 @result{} (eight-bit-control 128)
|
21006
|
384 @end example
|
|
385 @end defun
|
|
386
|
34811
|
387 @defun make-char charset &optional code1 code2
|
|
388 This function returns the character in character set @var{charset} whose
|
|
389 position codes are @var{code1} and @var{code2}. This is roughly the
|
|
390 inverse of @code{split-char}. Normally, you should specify either one
|
|
391 or both of @var{code1} and @var{code2} according to the dimension of
|
|
392 @var{charset}. For example,
|
21006
|
393
|
|
394 @example
|
|
395 (make-char 'latin-iso8859-1 72)
|
|
396 @result{} 2248
|
|
397 @end example
|
|
398 @end defun
|
|
399
|
22138
|
400 @cindex generic characters
|
|
401 If you call @code{make-char} with no @var{byte-values}, the result is
|
|
402 a @dfn{generic character} which stands for @var{charset}. A generic
|
|
403 character is an integer, but it is @emph{not} valid for insertion in the
|
|
404 buffer as a character. It can be used in @code{char-table-range} to
|
|
405 refer to the whole character set (@pxref{Char-Tables}).
|
|
406 @code{char-valid-p} returns @code{nil} for generic characters.
|
|
407 For example:
|
|
408
|
|
409 @example
|
|
410 (make-char 'latin-iso8859-1)
|
|
411 @result{} 2176
|
|
412 (char-valid-p 2176)
|
|
413 @result{} nil
|
29265
|
414 (char-valid-p 2176 t)
|
|
415 @result{} t
|
22138
|
416 (split-char 2176)
|
|
417 @result{} (latin-iso8859-1 0)
|
|
418 @end example
|
|
419
|
29265
|
420 The character sets @sc{ascii}, @sc{eight-bit-control}, and
|
34811
|
421 @sc{eight-bit-graphic} don't have corresponding generic characters. If
|
|
422 @var{charset} is one of them and you don't supply @var{code1},
|
|
423 @code{make-char} returns the character code corresponding to the
|
|
424 smallest code in @var{charset}.
|
29265
|
425
|
22138
|
426 @node Scanning Charsets
|
|
427 @section Scanning for Character Sets
|
|
428
|
|
429 Sometimes it is useful to find out which character sets appear in a
|
|
430 part of a buffer or a string. One use for this is in determining which
|
|
431 coding systems (@pxref{Coding Systems}) are capable of representing all
|
|
432 of the text in question.
|
|
433
|
|
434 @defun find-charset-region beg end &optional translation
|
|
435 This function returns a list of the character sets that appear in the
|
|
436 current buffer between positions @var{beg} and @var{end}.
|
|
437
|
|
438 The optional argument @var{translation} specifies a translation table to
|
|
439 be used in scanning the text (@pxref{Translation of Characters}). If it
|
|
440 is non-@code{nil}, then each character in the region is translated
|
|
441 through this table, and the value returned describes the translated
|
|
442 characters instead of the characters actually in the buffer.
|
28887
|
443 @end defun
|
22138
|
444
|
|
445 @defun find-charset-string string &optional translation
|
24951
|
446 This function returns a list of the character sets that appear in the
|
|
447 string @var{string}. It is just like @code{find-charset-region}, except
|
|
448 that it applies to the contents of @var{string} instead of part of the
|
|
449 current buffer.
|
22138
|
450 @end defun
|
|
451
|
|
452 @node Translation of Characters
|
|
453 @section Translation of Characters
|
|
454 @cindex character translation tables
|
|
455 @cindex translation tables
|
|
456
|
|
457 A @dfn{translation table} specifies a mapping of characters
|
|
458 into characters. These tables are used in encoding and decoding, and
|
|
459 for other purposes. Some coding systems specify their own particular
|
|
460 translation tables; there are also default translation tables which
|
|
461 apply to all other coding systems.
|
|
462
|
25751
|
463 @defun make-translation-table &rest translations
|
|
464 This function returns a translation table based on the argument
|
35752
|
465 @var{translations}. Each element of @var{translations} should be a
|
|
466 list of elements of the form @code{(@var{from} . @var{to})}; this says
|
|
467 to translate the character @var{from} into @var{to}.
|
22138
|
468
|
35493
|
469 The arguments and the forms in each argument are processed in order,
|
|
470 and if a previous form already translates @var{to} to some other
|
|
471 character, say @var{to-alt}, @var{from} is also translated to
|
|
472 @var{to-alt}.
|
|
473
|
22138
|
474 You can also map one whole character set into another character set with
|
|
475 the same dimension. To do this, you specify a generic character (which
|
|
476 designates a character set) for @var{from} (@pxref{Splitting Characters}).
|
|
477 In this case, @var{to} should also be a generic character, for another
|
|
478 character set of the same dimension. Then the translation table
|
|
479 translates each character of @var{from}'s character set into the
|
|
480 corresponding character of @var{to}'s character set.
|
|
481 @end defun
|
|
482
|
|
483 In decoding, the translation table's translations are applied to the
|
|
484 characters that result from ordinary decoding. If a coding system has
|
|
485 property @code{character-translation-table-for-decode}, that specifies
|
|
486 the translation table to use. Otherwise, if
|
23433
|
487 @code{standard-translation-table-for-decode} is non-@code{nil}, decoding
|
|
488 uses that table.
|
22138
|
489
|
|
490 In encoding, the translation table's translations are applied to the
|
|
491 characters in the buffer, and the result of translation is actually
|
|
492 encoded. If a coding system has property
|
|
493 @code{character-translation-table-for-encode}, that specifies the
|
|
494 translation table to use. Otherwise the variable
|
23433
|
495 @code{standard-translation-table-for-encode} specifies the translation
|
|
496 table.
|
22138
|
497
|
23433
|
498 @defvar standard-translation-table-for-decode
|
22138
|
499 This is the default translation table for decoding, for
|
|
500 coding systems that don't specify any other translation table.
|
|
501 @end defvar
|
|
502
|
23433
|
503 @defvar standard-translation-table-for-encode
|
22138
|
504 This is the default translation table for encoding, for
|
|
505 coding systems that don't specify any other translation table.
|
|
506 @end defvar
|
|
507
|
21006
|
508 @node Coding Systems
|
|
509 @section Coding Systems
|
|
510
|
|
511 @cindex coding system
|
|
512 When Emacs reads or writes a file, and when Emacs sends text to a
|
|
513 subprocess or receives text from a subprocess, it normally performs
|
|
514 character code conversion and end-of-line conversion as specified
|
|
515 by a particular @dfn{coding system}.
|
|
516
|
25751
|
517 How to define a coding system is an arcane matter, and is not
|
|
518 documented here.
|
24951
|
519
|
22138
|
520 @menu
|
28635
|
521 * Coding System Basics:: Basic concepts.
|
|
522 * Encoding and I/O:: How file I/O functions handle coding systems.
|
|
523 * Lisp and Coding Systems:: Functions to operate on coding system names.
|
|
524 * User-Chosen Coding Systems:: Asking the user to choose a coding system.
|
|
525 * Default Coding Systems:: Controlling the default choices.
|
|
526 * Specifying Coding Systems:: Requesting a particular coding system
|
|
527 for a single file operation.
|
|
528 * Explicit Encoding:: Encoding or decoding text without doing I/O.
|
|
529 * Terminal I/O Encoding:: Use of encoding for terminal I/O.
|
|
530 * MS-DOS File Types:: How DOS "text" and "binary" files
|
|
531 relate to coding systems.
|
22138
|
532 @end menu
|
|
533
|
|
534 @node Coding System Basics
|
|
535 @subsection Basic Concepts of Coding Systems
|
|
536
|
21006
|
537 @cindex character code conversion
|
|
538 @dfn{Character code conversion} involves conversion between the encoding
|
|
539 used inside Emacs and some other encoding. Emacs supports many
|
|
540 different encodings, in that it can convert to and from them. For
|
|
541 example, it can convert text to or from encodings such as Latin 1, Latin
|
|
542 2, Latin 3, Latin 4, Latin 5, and several variants of ISO 2022. In some
|
|
543 cases, Emacs supports several alternative encodings for the same
|
|
544 characters; for example, there are three coding systems for the Cyrillic
|
|
545 (Russian) alphabet: ISO, Alternativnyj, and KOI8.
|
|
546
|
|
547 Most coding systems specify a particular character code for
|
25751
|
548 conversion, but some of them leave the choice unspecified---to be chosen
|
|
549 heuristically for each file, based on the data.
|
21006
|
550
|
21682
|
551 @cindex end of line conversion
|
|
552 @dfn{End of line conversion} handles three different conventions used
|
|
553 on various systems for representing end of line in files. The Unix
|
|
554 convention is to use the linefeed character (also called newline). The
|
25751
|
555 DOS convention is to use a carriage-return and a linefeed at the end of
|
|
556 a line. The Mac convention is to use just carriage-return.
|
21682
|
557
|
21006
|
558 @cindex base coding system
|
|
559 @cindex variant coding system
|
|
560 @dfn{Base coding systems} such as @code{latin-1} leave the end-of-line
|
|
561 conversion unspecified, to be chosen based on the data. @dfn{Variant
|
|
562 coding systems} such as @code{latin-1-unix}, @code{latin-1-dos} and
|
|
563 @code{latin-1-mac} specify the end-of-line conversion explicitly as
|
22138
|
564 well. Most base coding systems have three corresponding variants whose
|
21006
|
565 names are formed by adding @samp{-unix}, @samp{-dos} and @samp{-mac}.
|
|
566
|
22138
|
567 The coding system @code{raw-text} is special in that it prevents
|
|
568 character code conversion, and causes the buffer visited with that
|
|
569 coding system to be a unibyte buffer. It does not specify the
|
|
570 end-of-line conversion, allowing that to be determined as usual by the
|
|
571 data, and has the usual three variants which specify the end-of-line
|
|
572 conversion. @code{no-conversion} is equivalent to @code{raw-text-unix}:
|
|
573 it specifies no conversion of either character codes or end-of-line.
|
|
574
|
|
575 The coding system @code{emacs-mule} specifies that the data is
|
|
576 represented in the internal Emacs encoding. This is like
|
|
577 @code{raw-text} in that no code conversion happens, but different in
|
|
578 that the result is multibyte data.
|
|
579
|
|
580 @defun coding-system-get coding-system property
|
|
581 This function returns the specified property of the coding system
|
|
582 @var{coding-system}. Most coding system properties exist for internal
|
|
583 purposes, but one that you might find useful is @code{mime-charset}.
|
|
584 That property's value is the name used in MIME for the character coding
|
|
585 which this coding system can read and write. Examples:
|
|
586
|
|
587 @example
|
|
588 (coding-system-get 'iso-latin-1 'mime-charset)
|
|
589 @result{} iso-8859-1
|
|
590 (coding-system-get 'iso-2022-cn 'mime-charset)
|
|
591 @result{} iso-2022-cn
|
|
592 (coding-system-get 'cyrillic-koi8 'mime-charset)
|
|
593 @result{} koi8-r
|
|
594 @end example
|
|
595
|
|
596 The value of the @code{mime-charset} property is also defined
|
|
597 as an alias for the coding system.
|
|
598 @end defun
|
|
599
|
|
600 @node Encoding and I/O
|
|
601 @subsection Encoding and I/O
|
|
602
|
22252
|
603 The principal purpose of coding systems is for use in reading and
|
22138
|
604 writing files. The function @code{insert-file-contents} uses
|
|
605 a coding system for decoding the file data, and @code{write-region}
|
|
606 uses one to encode the buffer contents.
|
|
607
|
|
608 You can specify the coding system to use either explicitly
|
|
609 (@pxref{Specifying Coding Systems}), or implicitly using the defaulting
|
|
610 mechanism (@pxref{Default Coding Systems}). But these methods may not
|
|
611 completely specify what to do. For example, they may choose a coding
|
|
612 system such as @code{undefined} which leaves the character code
|
|
613 conversion to be determined from the data. In these cases, the I/O
|
|
614 operation finishes the job of choosing a coding system. Very often
|
|
615 you will want to find out afterwards which coding system was chosen.
|
|
616
|
|
617 @defvar buffer-file-coding-system
|
|
618 This variable records the coding system that was used for visiting the
|
|
619 current buffer. It is used for saving the buffer, and for writing part
|
43632
faa7540b3866
(Encoding and I/O): Mention select-safe-coding-system in the documentation
Eli Zaretskii <eliz@gnu.org>
diff
changeset
|
620 of the buffer with @code{write-region}. If the text to be written
|
faa7540b3866
(Encoding and I/O): Mention select-safe-coding-system in the documentation
Eli Zaretskii <eliz@gnu.org>
diff
changeset
|
621 cannot be safely encoded using the coding system specified by this
|
faa7540b3866
(Encoding and I/O): Mention select-safe-coding-system in the documentation
Eli Zaretskii <eliz@gnu.org>
diff
changeset
|
622 variable, these operations select an alternative encoding by calling
|
faa7540b3866
(Encoding and I/O): Mention select-safe-coding-system in the documentation
Eli Zaretskii <eliz@gnu.org>
diff
changeset
|
623 the function @code{select-safe-coding-system} (@pxref{User-Chosen
|
faa7540b3866
(Encoding and I/O): Mention select-safe-coding-system in the documentation
Eli Zaretskii <eliz@gnu.org>
diff
changeset
|
624 Coding Systems}). If selecting a different encoding requires to ask
|
faa7540b3866
(Encoding and I/O): Mention select-safe-coding-system in the documentation
Eli Zaretskii <eliz@gnu.org>
diff
changeset
|
625 the user to specify a coding system, @code{buffer-file-coding-system}
|
faa7540b3866
(Encoding and I/O): Mention select-safe-coding-system in the documentation
Eli Zaretskii <eliz@gnu.org>
diff
changeset
|
626 is updated to the newly selected coding system.
|
24951
|
627
|
43632
faa7540b3866
(Encoding and I/O): Mention select-safe-coding-system in the documentation
Eli Zaretskii <eliz@gnu.org>
diff
changeset
|
628 @code{buffer-file-coding-system} does @emph{not} affect sending text
|
24951
|
629 to a subprocess.
|
22138
|
630 @end defvar
|
|
631
|
|
632 @defvar save-buffer-coding-system
|
29265
|
633 This variable specifies the coding system for saving the buffer (by
|
|
634 overriding @code{buffer-file-coding-system}). Note that it is not used
|
|
635 for @code{write-region}.
|
25751
|
636
|
|
637 When a command to save the buffer starts out to use
|
29265
|
638 @code{buffer-file-coding-system} (or @code{save-buffer-coding-system}),
|
|
639 and that coding system cannot handle
|
25751
|
640 the actual text in the buffer, the command asks the user to choose
|
43632
faa7540b3866
(Encoding and I/O): Mention select-safe-coding-system in the documentation
Eli Zaretskii <eliz@gnu.org>
diff
changeset
|
641 another coding system (by calling @code{select-safe-coding-system}).
|
faa7540b3866
(Encoding and I/O): Mention select-safe-coding-system in the documentation
Eli Zaretskii <eliz@gnu.org>
diff
changeset
|
642 After that happens, the command also updates
|
faa7540b3866
(Encoding and I/O): Mention select-safe-coding-system in the documentation
Eli Zaretskii <eliz@gnu.org>
diff
changeset
|
643 @code{buffer-file-coding-system} to represent the coding system that
|
faa7540b3866
(Encoding and I/O): Mention select-safe-coding-system in the documentation
Eli Zaretskii <eliz@gnu.org>
diff
changeset
|
644 the user specified.
|
22138
|
645 @end defvar
|
|
646
|
|
647 @defvar last-coding-system-used
|
|
648 I/O operations for files and subprocesses set this variable to the
|
|
649 coding system name that was used. The explicit encoding and decoding
|
|
650 functions (@pxref{Explicit Encoding}) set it too.
|
|
651
|
|
652 @strong{Warning:} Since receiving subprocess output sets this variable,
|
25751
|
653 it can change whenever Emacs waits; therefore, you should copy the
|
|
654 value shortly after the function call that stores the value you are
|
22138
|
655 interested in.
|
|
656 @end defvar
|
|
657
|
23110
|
658 The variable @code{selection-coding-system} specifies how to encode
|
|
659 selections for the window system. @xref{Window System Selections}.
|
|
660
|
21682
|
661 @node Lisp and Coding Systems
|
|
662 @subsection Coding Systems in Lisp
|
|
663
|
25751
|
664 Here are the Lisp facilities for working with coding systems:
|
21006
|
665
|
22138
|
666 @defun coding-system-list &optional base-only
|
21006
|
667 This function returns a list of all coding system names (symbols). If
|
|
668 @var{base-only} is non-@code{nil}, the value includes only the
|
29265
|
669 base coding systems. Otherwise, it includes alias and variant coding
|
|
670 systems as well.
|
21006
|
671 @end defun
|
|
672
|
22138
|
673 @defun coding-system-p object
|
21006
|
674 This function returns @code{t} if @var{object} is a coding system
|
|
675 name.
|
|
676 @end defun
|
|
677
|
22138
|
678 @defun check-coding-system coding-system
|
21006
|
679 This function checks the validity of @var{coding-system}.
|
|
680 If that is valid, it returns @var{coding-system}.
|
|
681 Otherwise it signals an error with condition @code{coding-system-error}.
|
|
682 @end defun
|
|
683
|
22138
|
684 @defun coding-system-change-eol-conversion coding-system eol-type
|
|
685 This function returns a coding system which is like @var{coding-system}
|
22252
|
686 except for its eol conversion, which is specified by @code{eol-type}.
|
22138
|
687 @var{eol-type} should be @code{unix}, @code{dos}, @code{mac}, or
|
|
688 @code{nil}. If it is @code{nil}, the returned coding system determines
|
|
689 the end-of-line conversion from the data.
|
|
690 @end defun
|
21682
|
691
|
22138
|
692 @defun coding-system-change-text-conversion eol-coding text-coding
|
|
693 This function returns a coding system which uses the end-of-line
|
|
694 conversion of @var{eol-coding}, and the text conversion of
|
|
695 @var{text-coding}. If @var{text-coding} is @code{nil}, it returns
|
|
696 @code{undecided}, or one of its variants according to @var{eol-coding}.
|
21682
|
697 @end defun
|
|
698
|
22138
|
699 @defun find-coding-systems-region from to
|
|
700 This function returns a list of coding systems that could be used to
|
|
701 encode a text between @var{from} and @var{to}. All coding systems in
|
|
702 the list can safely encode any multibyte characters in that portion of
|
|
703 the text.
|
|
704
|
|
705 If the text contains no multibyte characters, the function returns the
|
|
706 list @code{(undecided)}.
|
|
707 @end defun
|
|
708
|
|
709 @defun find-coding-systems-string string
|
|
710 This function returns a list of coding systems that could be used to
|
|
711 encode the text of @var{string}. All coding systems in the list can
|
|
712 safely encode any multibyte characters in @var{string}. If the text
|
|
713 contains no multibyte characters, this returns the list
|
|
714 @code{(undecided)}.
|
|
715 @end defun
|
|
716
|
|
717 @defun find-coding-systems-for-charsets charsets
|
|
718 This function returns a list of coding systems that could be used to
|
|
719 encode all the character sets in the list @var{charsets}.
|
|
720 @end defun
|
|
721
|
|
722 @defun detect-coding-region start end &optional highest
|
21006
|
723 This function chooses a plausible coding system for decoding the text
|
28877
|
724 from @var{start} to @var{end}. This text should be a byte sequence
|
21682
|
725 (@pxref{Explicit Encoding}).
|
21006
|
726
|
22138
|
727 Normally this function returns a list of coding systems that could
|
21006
|
728 handle decoding the text that was scanned. They are listed in order of
|
22138
|
729 decreasing priority. But if @var{highest} is non-@code{nil}, then the
|
|
730 return value is just one coding system, the one that is highest in
|
|
731 priority.
|
|
732
|
25751
|
733 If the region contains only @sc{ascii} characters, the value
|
22138
|
734 is @code{undecided} or @code{(undecided)}.
|
21006
|
735 @end defun
|
|
736
|
22138
|
737 @defun detect-coding-string string highest
|
21006
|
738 This function is like @code{detect-coding-region} except that it
|
|
739 operates on the contents of @var{string} instead of bytes in the buffer.
|
|
740 @end defun
|
|
741
|
22252
|
742 @xref{Process Information}, for how to examine or set the coding
|
|
743 systems used for I/O to a subprocess.
|
|
744
|
|
745 @node User-Chosen Coding Systems
|
|
746 @subsection User-Chosen Coding Systems
|
|
747
|
43632
faa7540b3866
(Encoding and I/O): Mention select-safe-coding-system in the documentation
Eli Zaretskii <eliz@gnu.org>
diff
changeset
|
748 @cindex select safe coding system
|
39204
|
749 @defun select-safe-coding-system from to &optional default-coding-system accept-default-p
|
|
750 This function selects a coding system for encoding specified text,
|
|
751 asking the user to choose if necessary. Normally the specified text
|
|
752 is the text in the current buffer between @var{from} and @var{to},
|
|
753 defaulting to the whole buffer if they are @code{nil}. If @var{from}
|
40855
|
754 is a string, the string specifies the text to encode, and @var{to} is
|
|
755 ignored.
|
39204
|
756
|
|
757 If @var{default-coding-system} is non-@code{nil}, that is the first
|
|
758 coding system to try; if that can handle the text,
|
|
759 @code{select-safe-coding-system} returns that coding system. It can
|
|
760 also be a list of coding systems; then the function tries each of them
|
|
761 one by one. After trying all of them, it next tries the user's most
|
|
762 preferred coding system (@pxref{Recognize Coding,
|
|
763 prefer-coding-system, the description of @code{prefer-coding-system},
|
|
764 emacs, GNU Emacs Manual}), and after that the current buffer's value
|
|
765 of @code{buffer-file-coding-system} (if it is not @code{undecided}).
|
22252
|
766
|
39204
|
767 If one of those coding systems can safely encode all the specified
|
|
768 text, @code{select-safe-coding-system} chooses it and returns it.
|
|
769 Otherwise, it asks the user to choose from a list of coding systems
|
|
770 which can encode all the text, and returns the user's choice.
|
22252
|
771
|
39204
|
772 The optional argument @var{accept-default-p}, if non-@code{nil},
|
|
773 should be a function to determine whether the coding system selected
|
|
774 without user interaction is acceptable. If this function returns
|
|
775 @code{nil}, the silently selected coding system is rejected, and the
|
|
776 user is asked to select a coding system from a list of possible
|
|
777 candidates.
|
22252
|
778
|
39204
|
779 @vindex select-safe-coding-system-accept-default-p
|
|
780 If the variable @code{select-safe-coding-system-accept-default-p} is
|
|
781 non-@code{nil}, its value overrides the value of
|
|
782 @var{accept-default-p}.
|
22252
|
783 @end defun
|
|
784
|
22138
|
785 Here are two functions you can use to let the user specify a coding
|
|
786 system, with completion. @xref{Completion}.
|
|
787
|
|
788 @defun read-coding-system prompt &optional default
|
|
789 This function reads a coding system using the minibuffer, prompting with
|
|
790 string @var{prompt}, and returns the coding system name as a symbol. If
|
|
791 the user enters null input, @var{default} specifies which coding system
|
|
792 to return. It should be a symbol or a string.
|
|
793 @end defun
|
|
794
|
|
795 @defun read-non-nil-coding-system prompt
|
|
796 This function reads a coding system using the minibuffer, prompting with
|
|
797 string @var{prompt}, and returns the coding system name as a symbol. If
|
|
798 the user tries to enter null input, it asks the user to try again.
|
|
799 @xref{Coding Systems}.
|
|
800 @end defun
|
|
801
|
|
802 @node Default Coding Systems
|
|
803 @subsection Default Coding Systems
|
|
804
|
|
805 This section describes variables that specify the default coding
|
|
806 system for certain files or when running certain subprograms, and the
|
22252
|
807 function that I/O operations use to access them.
|
22138
|
808
|
|
809 The idea of these variables is that you set them once and for all to the
|
|
810 defaults you want, and then do not change them again. To specify a
|
|
811 particular coding system for a particular operation in a Lisp program,
|
|
812 don't change these variables; instead, override them using
|
|
813 @code{coding-system-for-read} and @code{coding-system-for-write}
|
|
814 (@pxref{Specifying Coding Systems}).
|
|
815
|
39204
|
816 @defvar auto-coding-regexp-alist
|
|
817 This variable is an alist of text patterns and corresponding coding
|
|
818 systems. Each element has the form @code{(@var{regexp}
|
|
819 . @var{coding-system})}; a file whose first few kilobytes match
|
|
820 @var{regexp} is decoded with @var{coding-system} when its contents are
|
|
821 read into a buffer. The settings in this alist take priority over
|
|
822 @code{coding:} tags in the files and the contents of
|
|
823 @code{file-coding-system-alist} (see below). The default value is set
|
|
824 so that Emacs automatically recognizes mail files in Babyl format and
|
|
825 reads them with no code conversions.
|
|
826 @end defvar
|
|
827
|
22138
|
828 @defvar file-coding-system-alist
|
|
829 This variable is an alist that specifies the coding systems to use for
|
|
830 reading and writing particular files. Each element has the form
|
|
831 @code{(@var{pattern} . @var{coding})}, where @var{pattern} is a regular
|
|
832 expression that matches certain file names. The element applies to file
|
|
833 names that match @var{pattern}.
|
|
834
|
22252
|
835 The @sc{cdr} of the element, @var{coding}, should be either a coding
|
25751
|
836 system, a cons cell containing two coding systems, or a function name (a
|
|
837 symbol with a function definition). If @var{coding} is a coding system,
|
|
838 that coding system is used for both reading the file and writing it. If
|
|
839 @var{coding} is a cons cell containing two coding systems, its @sc{car}
|
|
840 specifies the coding system for decoding, and its @sc{cdr} specifies the
|
|
841 coding system for encoding.
|
22138
|
842
|
25751
|
843 If @var{coding} is a function name, the function must return a coding
|
22138
|
844 system or a cons cell containing two coding systems. This value is used
|
|
845 as described above.
|
|
846 @end defvar
|
|
847
|
|
848 @defvar process-coding-system-alist
|
|
849 This variable is an alist specifying which coding systems to use for a
|
|
850 subprocess, depending on which program is running in the subprocess. It
|
|
851 works like @code{file-coding-system-alist}, except that @var{pattern} is
|
|
852 matched against the program name used to start the subprocess. The coding
|
|
853 system or systems specified in this alist are used to initialize the
|
|
854 coding systems used for I/O to the subprocess, but you can specify
|
|
855 other coding systems later using @code{set-process-coding-system}.
|
|
856 @end defvar
|
|
857
|
25751
|
858 @strong{Warning:} Coding systems such as @code{undecided}, which
|
|
859 determine the coding system from the data, do not work entirely reliably
|
22252
|
860 with asynchronous subprocess output. This is because Emacs handles
|
22138
|
861 asynchronous subprocess output in batches, as it arrives. If the coding
|
|
862 system leaves the character code conversion unspecified, or leaves the
|
|
863 end-of-line conversion unspecified, Emacs must try to detect the proper
|
|
864 conversion from one batch at a time, and this does not always work.
|
|
865
|
|
866 Therefore, with an asynchronous subprocess, if at all possible, use a
|
|
867 coding system which determines both the character code conversion and
|
|
868 the end of line conversion---that is, one like @code{latin-1-unix},
|
|
869 rather than @code{undecided} or @code{latin-1}.
|
|
870
|
|
871 @defvar network-coding-system-alist
|
|
872 This variable is an alist that specifies the coding system to use for
|
|
873 network streams. It works much like @code{file-coding-system-alist},
|
|
874 with the difference that the @var{pattern} in an element may be either a
|
|
875 port number or a regular expression. If it is a regular expression, it
|
|
876 is matched against the network service name used to open the network
|
|
877 stream.
|
|
878 @end defvar
|
|
879
|
|
880 @defvar default-process-coding-system
|
|
881 This variable specifies the coding systems to use for subprocess (and
|
|
882 network stream) input and output, when nothing else specifies what to
|
|
883 do.
|
|
884
|
|
885 The value should be a cons cell of the form @code{(@var{input-coding}
|
|
886 . @var{output-coding})}. Here @var{input-coding} applies to input from
|
|
887 the subprocess, and @var{output-coding} applies to output to it.
|
|
888 @end defvar
|
|
889
|
21006
|
890 @defun find-operation-coding-system operation &rest arguments
|
|
891 This function returns the coding system to use (by default) for
|
|
892 performing @var{operation} with @var{arguments}. The value has this
|
|
893 form:
|
|
894
|
|
895 @example
|
|
896 (@var{decoding-system} @var{encoding-system})
|
|
897 @end example
|
|
898
|
|
899 The first element, @var{decoding-system}, is the coding system to use
|
|
900 for decoding (in case @var{operation} does decoding), and
|
|
901 @var{encoding-system} is the coding system for encoding (in case
|
|
902 @var{operation} does encoding).
|
|
903
|
25751
|
904 The argument @var{operation} should be a symbol, one of
|
21006
|
905 @code{insert-file-contents}, @code{write-region}, @code{call-process},
|
|
906 @code{call-process-region}, @code{start-process}, or
|
25751
|
907 @code{open-network-stream}. These are the names of the Emacs I/O primitives
|
|
908 that can do coding system conversion.
|
21006
|
909
|
|
910 The remaining arguments should be the same arguments that might be given
|
25751
|
911 to that I/O primitive. Depending on the primitive, one of those
|
21006
|
912 arguments is selected as the @dfn{target}. For example, if
|
|
913 @var{operation} does file I/O, whichever argument specifies the file
|
|
914 name is the target. For subprocess primitives, the process name is the
|
|
915 target. For @code{open-network-stream}, the target is the service name
|
|
916 or port number.
|
|
917
|
|
918 This function looks up the target in @code{file-coding-system-alist},
|
|
919 @code{process-coding-system-alist}, or
|
|
920 @code{network-coding-system-alist}, depending on @var{operation}.
|
|
921 @xref{Default Coding Systems}.
|
|
922 @end defun
|
|
923
|
|
924 @node Specifying Coding Systems
|
22138
|
925 @subsection Specifying a Coding System for One Operation
|
21006
|
926
|
|
927 You can specify the coding system for a specific operation by binding
|
|
928 the variables @code{coding-system-for-read} and/or
|
|
929 @code{coding-system-for-write}.
|
|
930
|
22138
|
931 @defvar coding-system-for-read
|
21006
|
932 If this variable is non-@code{nil}, it specifies the coding system to
|
|
933 use for reading a file, or for input from a synchronous subprocess.
|
|
934
|
|
935 It also applies to any asynchronous subprocess or network stream, but in
|
|
936 a different way: the value of @code{coding-system-for-read} when you
|
|
937 start the subprocess or open the network stream specifies the input
|
|
938 decoding method for that subprocess or network stream. It remains in
|
|
939 use for that subprocess or network stream unless and until overridden.
|
|
940
|
|
941 The right way to use this variable is to bind it with @code{let} for a
|
|
942 specific I/O operation. Its global value is normally @code{nil}, and
|
|
943 you should not globally set it to any other value. Here is an example
|
|
944 of the right way to use the variable:
|
|
945
|
|
946 @example
|
|
947 ;; @r{Read the file with no character code conversion.}
|
21682
|
948 ;; @r{Assume @sc{crlf} represents end-of-line.}
|
21006
|
949 (let ((coding-system-for-write 'emacs-mule-dos))
|
|
950 (insert-file-contents filename))
|
|
951 @end example
|
|
952
|
|
953 When its value is non-@code{nil}, @code{coding-system-for-read} takes
|
22138
|
954 precedence over all other methods of specifying a coding system to use for
|
21006
|
955 input, including @code{file-coding-system-alist},
|
|
956 @code{process-coding-system-alist} and
|
|
957 @code{network-coding-system-alist}.
|
|
958 @end defvar
|
|
959
|
22138
|
960 @defvar coding-system-for-write
|
21006
|
961 This works much like @code{coding-system-for-read}, except that it
|
|
962 applies to output rather than input. It affects writing to files,
|
24951
|
963 as well as sending output to subprocesses and net connections.
|
21006
|
964
|
|
965 When a single operation does both input and output, as do
|
|
966 @code{call-process-region} and @code{start-process}, both
|
|
967 @code{coding-system-for-read} and @code{coding-system-for-write}
|
|
968 affect it.
|
|
969 @end defvar
|
|
970
|
22138
|
971 @defvar inhibit-eol-conversion
|
21006
|
972 When this variable is non-@code{nil}, no end-of-line conversion is done,
|
|
973 no matter which coding system is specified. This applies to all the
|
|
974 Emacs I/O and subprocess primitives, and to the explicit encoding and
|
|
975 decoding functions (@pxref{Explicit Encoding}).
|
|
976 @end defvar
|
|
977
|
|
978 @node Explicit Encoding
|
22138
|
979 @subsection Explicit Encoding and Decoding
|
21006
|
980 @cindex encoding text
|
|
981 @cindex decoding text
|
|
982
|
|
983 All the operations that transfer text in and out of Emacs have the
|
|
984 ability to use a coding system to encode or decode the text.
|
|
985 You can also explicitly encode and decode text using the functions
|
|
986 in this section.
|
|
987
|
|
988 The result of encoding, and the input to decoding, are not ordinary
|
28877
|
989 text. They logically consist of a series of byte values; that is, a
|
|
990 series of characters whose codes are in the range 0 through 255. In a
|
|
991 multibyte buffer or string, character codes 128 through 159 are
|
|
992 represented by multibyte sequences, but this is invisible to Lisp
|
|
993 programs.
|
21006
|
994
|
28877
|
995 The usual way to read a file into a buffer as a sequence of bytes, so
|
|
996 you can decode the contents explicitly, is with
|
|
997 @code{insert-file-contents-literally} (@pxref{Reading from Files});
|
|
998 alternatively, specify a non-@code{nil} @var{rawfile} argument when
|
|
999 visiting a file with @code{find-file-noselect}. These methods result in
|
|
1000 a unibyte buffer.
|
24951
|
1001
|
28877
|
1002 The usual way to use the byte sequence that results from explicitly
|
|
1003 encoding text is to copy it to a file or process---for example, to write
|
|
1004 it with @code{write-region} (@pxref{Writing to Files}), and suppress
|
|
1005 encoding by binding @code{coding-system-for-write} to
|
|
1006 @code{no-conversion}.
|
24951
|
1007
|
|
1008 Here are the functions to perform explicit encoding or decoding. The
|
28877
|
1009 decoding functions produce sequences of bytes; the encoding functions
|
|
1010 are meant to operate on sequences of bytes. All of these functions
|
|
1011 discard text properties.
|
22252
|
1012
|
22138
|
1013 @defun encode-coding-region start end coding-system
|
21006
|
1014 This function encodes the text from @var{start} to @var{end} according
|
21682
|
1015 to coding system @var{coding-system}. The encoded text replaces the
|
28877
|
1016 original text in the buffer. The result of encoding is logically a
|
|
1017 sequence of bytes, but the buffer remains multibyte if it was multibyte
|
|
1018 before.
|
21006
|
1019 @end defun
|
|
1020
|
22138
|
1021 @defun encode-coding-string string coding-system
|
21006
|
1022 This function encodes the text in @var{string} according to coding
|
|
1023 system @var{coding-system}. It returns a new string containing the
|
28877
|
1024 encoded text. The result of encoding is a unibyte string.
|
21006
|
1025 @end defun
|
|
1026
|
22138
|
1027 @defun decode-coding-region start end coding-system
|
21006
|
1028 This function decodes the text from @var{start} to @var{end} according
|
|
1029 to coding system @var{coding-system}. The decoded text replaces the
|
|
1030 original text in the buffer. To make explicit decoding useful, the text
|
28877
|
1031 before decoding ought to be a sequence of byte values, but both
|
|
1032 multibyte and unibyte buffers are acceptable.
|
21006
|
1033 @end defun
|
|
1034
|
22138
|
1035 @defun decode-coding-string string coding-system
|
21006
|
1036 This function decodes the text in @var{string} according to coding
|
|
1037 system @var{coding-system}. It returns a new string containing the
|
|
1038 decoded text. To make explicit decoding useful, the contents of
|
28877
|
1039 @var{string} ought to be a sequence of byte values, but a multibyte
|
|
1040 string is acceptable.
|
21006
|
1041 @end defun
|
21682
|
1042
|
22138
|
1043 @node Terminal I/O Encoding
|
|
1044 @subsection Terminal I/O Encoding
|
|
1045
|
|
1046 Emacs can decode keyboard input using a coding system, and encode
|
23110
|
1047 terminal output. This is useful for terminals that transmit or display
|
|
1048 text using a particular encoding such as Latin-1. Emacs does not set
|
|
1049 @code{last-coding-system-used} for encoding or decoding for the
|
|
1050 terminal.
|
22138
|
1051
|
|
1052 @defun keyboard-coding-system
|
|
1053 This function returns the coding system that is in use for decoding
|
|
1054 keyboard input---or @code{nil} if no coding system is to be used.
|
|
1055 @end defun
|
|
1056
|
|
1057 @defun set-keyboard-coding-system coding-system
|
|
1058 This function specifies @var{coding-system} as the coding system to
|
|
1059 use for decoding keyboard input. If @var{coding-system} is @code{nil},
|
|
1060 that means do not decode keyboard input.
|
|
1061 @end defun
|
|
1062
|
|
1063 @defun terminal-coding-system
|
|
1064 This function returns the coding system that is in use for encoding
|
|
1065 terminal output---or @code{nil} for no encoding.
|
|
1066 @end defun
|
|
1067
|
|
1068 @defun set-terminal-coding-system coding-system
|
|
1069 This function specifies @var{coding-system} as the coding system to use
|
|
1070 for encoding terminal output. If @var{coding-system} is @code{nil},
|
|
1071 that means do not encode terminal output.
|
|
1072 @end defun
|
|
1073
|
21682
|
1074 @node MS-DOS File Types
|
22138
|
1075 @subsection MS-DOS File Types
|
21682
|
1076 @cindex DOS file types
|
|
1077 @cindex MS-DOS file types
|
|
1078 @cindex Windows file types
|
|
1079 @cindex file types on MS-DOS and Windows
|
|
1080 @cindex text files and binary files
|
|
1081 @cindex binary files and text files
|
|
1082
|
25751
|
1083 On MS-DOS and Microsoft Windows, Emacs guesses the appropriate
|
|
1084 end-of-line conversion for a file by looking at the file's name. This
|
28877
|
1085 feature classifies files as @dfn{text files} and @dfn{binary files}. By
|
25751
|
1086 ``binary file'' we mean a file of literal byte values that are not
|
|
1087 necessarily meant to be characters; Emacs does no end-of-line conversion
|
|
1088 and no character code conversion for them. On the other hand, the bytes
|
|
1089 in a text file are intended to represent characters; when you create a
|
|
1090 new file whose name implies that it is a text file, Emacs uses DOS
|
|
1091 end-of-line conversion.
|
21682
|
1092
|
|
1093 @defvar buffer-file-type
|
|
1094 This variable, automatically buffer-local in each buffer, records the
|
22138
|
1095 file type of the buffer's visited file. When a buffer does not specify
|
|
1096 a coding system with @code{buffer-file-coding-system}, this variable is
|
|
1097 used to determine which coding system to use when writing the contents
|
|
1098 of the buffer. It should be @code{nil} for text, @code{t} for binary.
|
|
1099 If it is @code{t}, the coding system is @code{no-conversion}.
|
|
1100 Otherwise, @code{undecided-dos} is used.
|
|
1101
|
|
1102 Normally this variable is set by visiting a file; it is set to
|
|
1103 @code{nil} if the file was visited without any actual conversion.
|
21682
|
1104 @end defvar
|
|
1105
|
|
1106 @defopt file-name-buffer-file-type-alist
|
|
1107 This variable holds an alist for recognizing text and binary files.
|
|
1108 Each element has the form (@var{regexp} . @var{type}), where
|
|
1109 @var{regexp} is matched against the file name, and @var{type} may be
|
|
1110 @code{nil} for text, @code{t} for binary, or a function to call to
|
|
1111 compute which. If it is a function, then it is called with a single
|
|
1112 argument (the file name) and should return @code{t} or @code{nil}.
|
|
1113
|
25751
|
1114 When running on MS-DOS or MS-Windows, Emacs checks this alist to decide
|
21682
|
1115 which coding system to use when reading a file. For a text file,
|
|
1116 @code{undecided-dos} is used. For a binary file, @code{no-conversion}
|
|
1117 is used.
|
|
1118
|
|
1119 If no element in this alist matches a given file name, then
|
|
1120 @code{default-buffer-file-type} says how to treat the file.
|
|
1121 @end defopt
|
|
1122
|
|
1123 @defopt default-buffer-file-type
|
|
1124 This variable says how to handle files for which
|
|
1125 @code{file-name-buffer-file-type-alist} says nothing about the type.
|
|
1126
|
|
1127 If this variable is non-@code{nil}, then these files are treated as
|
22138
|
1128 binary: the coding system @code{no-conversion} is used. Otherwise,
|
|
1129 nothing special is done for them---the coding system is deduced solely
|
|
1130 from the file contents, in the usual Emacs fashion.
|
21682
|
1131 @end defopt
|
|
1132
|
22138
|
1133 @node Input Methods
|
|
1134 @section Input Methods
|
|
1135 @cindex input methods
|
|
1136
|
25751
|
1137 @dfn{Input methods} provide convenient ways of entering non-@sc{ascii}
|
22138
|
1138 characters from the keyboard. Unlike coding systems, which translate
|
25751
|
1139 non-@sc{ascii} characters to and from encodings meant to be read by
|
22138
|
1140 programs, input methods provide human-friendly commands. (@xref{Input
|
|
1141 Methods,,, emacs, The GNU Emacs Manual}, for information on how users
|
|
1142 use input methods to enter text.) How to define input methods is not
|
|
1143 yet documented in this manual, but here we describe how to use them.
|
21682
|
1144
|
22138
|
1145 Each input method has a name, which is currently a string;
|
|
1146 in the future, symbols may also be usable as input method names.
|
21682
|
1147
|
22138
|
1148 @defvar current-input-method
|
|
1149 This variable holds the name of the input method now active in the
|
|
1150 current buffer. (It automatically becomes local in each buffer when set
|
|
1151 in any fashion.) It is @code{nil} if no input method is active in the
|
|
1152 buffer now.
|
|
1153 @end defvar
|
|
1154
|
|
1155 @defvar default-input-method
|
|
1156 This variable holds the default input method for commands that choose an
|
|
1157 input method. Unlike @code{current-input-method}, this variable is
|
|
1158 normally global.
|
21682
|
1159 @end defvar
|
|
1160
|
22138
|
1161 @defun set-input-method input-method
|
|
1162 This function activates input method @var{input-method} for the current
|
|
1163 buffer. It also sets @code{default-input-method} to @var{input-method}.
|
|
1164 If @var{input-method} is @code{nil}, this function deactivates any input
|
|
1165 method for the current buffer.
|
|
1166 @end defun
|
|
1167
|
|
1168 @defun read-input-method-name prompt &optional default inhibit-null
|
|
1169 This function reads an input method name with the minibuffer, prompting
|
|
1170 with @var{prompt}. If @var{default} is non-@code{nil}, that is returned
|
|
1171 by default, if the user enters empty input. However, if
|
|
1172 @var{inhibit-null} is non-@code{nil}, empty input signals an error.
|
|
1173
|
|
1174 The returned value is a string.
|
|
1175 @end defun
|
|
1176
|
|
1177 @defvar input-method-alist
|
|
1178 This variable defines all the supported input methods.
|
|
1179 Each element defines one input method, and should have the form:
|
|
1180
|
|
1181 @example
|
22252
|
1182 (@var{input-method} @var{language-env} @var{activate-func}
|
|
1183 @var{title} @var{description} @var{args}...)
|
22138
|
1184 @end example
|
|
1185
|
22252
|
1186 Here @var{input-method} is the input method name, a string;
|
|
1187 @var{language-env} is another string, the name of the language
|
|
1188 environment this input method is recommended for. (That serves only for
|
|
1189 documentation purposes.)
|
22138
|
1190
|
|
1191 @var{activate-func} is a function to call to activate this method. The
|
|
1192 @var{args}, if any, are passed as arguments to @var{activate-func}. All
|
|
1193 told, the arguments to @var{activate-func} are @var{input-method} and
|
|
1194 the @var{args}.
|
28877
|
1195
|
|
1196 @var{title} is a string to display in the mode line while this method is
|
|
1197 active. @var{description} is a string describing this method and what
|
|
1198 it is good for.
|
22252
|
1199 @end defvar
|
22138
|
1200
|
23110
|
1201 The fundamental interface to input methods is through the
|
|
1202 variable @code{input-method-function}. @xref{Reading One Event}.
|
26696
|
1203
|
|
1204 @node Locales
|
|
1205 @section Locales
|
|
1206 @cindex locale
|
|
1207
|
|
1208 POSIX defines a concept of ``locales'' which control which language
|
|
1209 to use in language-related features. These Emacs variables control
|
|
1210 how Emacs interacts with these features.
|
|
1211
|
|
1212 @defvar locale-coding-system
|
|
1213 @tindex locale-coding-system
|
43634
f55024232f5d
(Locales): locale-coding-system is used for decoding keyboard input on X.
Eli Zaretskii <eliz@gnu.org>
diff
changeset
|
1214 @cindex keyboard input decoding on X
|
26696
|
1215 This variable specifies the coding system to use for decoding system
|
43634
f55024232f5d
(Locales): locale-coding-system is used for decoding keyboard input on X.
Eli Zaretskii <eliz@gnu.org>
diff
changeset
|
1216 error messages and---on X Window system only---keyboard input, for
|
f55024232f5d
(Locales): locale-coding-system is used for decoding keyboard input on X.
Eli Zaretskii <eliz@gnu.org>
diff
changeset
|
1217 encoding the format argument to @code{format-time-string}, and for
|
f55024232f5d
(Locales): locale-coding-system is used for decoding keyboard input on X.
Eli Zaretskii <eliz@gnu.org>
diff
changeset
|
1218 decoding the return value of @code{format-time-string}.
|
26696
|
1219 @end defvar
|
|
1220
|
|
1221 @defvar system-messages-locale
|
|
1222 @tindex system-messages-locale
|
|
1223 This variable specifies the locale to use for generating system error
|
|
1224 messages. Changing the locale can cause messages to come out in a
|
27362
|
1225 different language or in a different orthography. If the variable is
|
26696
|
1226 @code{nil}, the locale is specified by environment variables in the
|
|
1227 usual POSIX fashion.
|
|
1228 @end defvar
|
|
1229
|
|
1230 @defvar system-time-locale
|
|
1231 @tindex system-time-locale
|
|
1232 This variable specifies the locale to use for formatting time values.
|
|
1233 Changing the locale can cause messages to appear according to the
|
|
1234 conventions of a different language. If the variable is @code{nil}, the
|
|
1235 locale is specified by environment variables in the usual POSIX fashion.
|
|
1236 @end defvar
|
28877
|
1237
|