view README.unicode @ 89061:9a9b54d06f3d

* regex.c (RE_TARGET_MULTIBYTE_P): New macro. (GET_CHAR_BEFORE_2): Check target_multibyte, not multibyte. If that is zero, convert an eight-bit char to multibyte. (MAKE_CHAR_MULTIBYTE, CHAR_LEADING_CODE): New dummy new macros for non-emacs case. (PATFETCH): Convert an eight-bit char to multibyte. (HANDLE_UNIBYTE_RANGE): New macro. (regex_compile): Setup the compiled pattern for multibyte chars even if the given regex string is unibyte. Use PATFETCH_RAW instead of PATFETCH in many places. To handle `charset' specification of unibyte, call HANDLE_UNIBYTE_RANGE. Use bitmap only for ASCII chars. (analyse_first) <exactn>: Simplified because the compiled pattern is multibyte. <charset_not>: Setup fastmap from bitmap only for ASCII chars. <charset>: Use CHAR_LEADING_CODE to get leading codes. <categoryspec>: If multibyte, setup fastmap only for ASCII chars here. (re_compile_fastmap) [emacs]: Call analyse_first with the arg multibyte always 1. (re_search_2) In emacs, set the locale variable multibyte to 1, otherwise to 0. New local variable target_multibyte. Check it to decide the multibyteness of STR1 and STR2. If target_multibyte is zero, convert unibyte chars to multibyte before translating and checking fastmap. (TARGET_CHAR_AND_LENGTH): New macro. (re_match_2_internal): In emacs, set the locale variable multibyte to 1, otherwise to 0. New local variable target_multibyte. Check it to decide the multibyteness of STR1 and STR2. Use TARGET_CHAR_AND_LENGTH to fetch a character from D. <charset, charset_not>: If multibyte is nonzero, check fastmap only for ASCII chars. Call bcmp_translate with target_multibyte, not with multibyte. <begline>: Declare the local variable C as `unsigned'. (bcmp_translate): Change the last arg name to target_multibyte.

                                                                   -*-text-*-

Problems, fixmes and other issues in the emacs-unicode branch
-------------------------------------------------------------

Notes by fx to record various things of variable importance.  handa
needs to check them -- don't take too seriously, especially with
regard to completeness.

_Do take seriously that you don't want this branch unless you're
actually working on it; you risk your data by actually using it._  If
you just want to edit Unicode and/or unify iso-8859 et al, see the
existing support and the extra stuff at
<URL:ftp://dlpx1.dl.ac.uk/fx/emacs/Mule>, mostly now in the CVS trunk.
(Editing support is mostly orthogonal to the internal representation.)

 * SINGLE_BYTE_CHAR_P returns true for Latin-1 characters, which has
   undesirable effects.

 * Rationalize character syntax and its relationship to the Unicode
   database.  Specifically, the latin-N.el files aren't consistent for
   common characters (and obviously have redundancies except in
   unibyte mode).

 * Fontset handling and customization needs work.  We want to relate
   fonts to scripts, probably based on the Unicode blocks.  The
   presence of small-repertoire 10646-encoded fonts in XFree 4 is a
   pain, not currently worked round.

	With the change on 2002-07-26, multiple fonts can be
	specified in a fontset for a specific range of characters.
	Each range can also be specified by script.  Before using
	ISO10646 fonts, Emacs checks their repertories to avoid such
	fonts that don't have a glyph for a specific character.

 * Work is also needed on charset and coding system priorities.

 * The relevant bits of latin1-disp.el need porting (and probably
   re-naming/updating).  See also cyril-util.el.

 * Quail files need more work now the encoding is irrelevant.

 * What to do with the old coding categories stuff?

 * Syntax for symbols &c in characters.el needs looking at.

 * The preferred-coding-system property of charsets should probably be
   junked unless it can be made more useful now.

 * find-coding-systems-for-charsets needs re-writing or removing.

 * find-multibyte-characters needs looking at.

 * Implement Korean cp949/UHC and any other important missing
   charsets.

 * Check up on definitions of tcvn and alternativnj.

 * Lazy-load tables for unify-charset somehow?

	Actually, Emacs clear out all charset maps and unify-map just
	before dumping, and their are loaded again on demand the
	dumped emacs.  But, those maps (char tables) generated while
	temacs is running can't be get rid of from the dumped emacs.

 * Translation tables for {en,de}code currently aren't supported.

 * Defining CCL coding systems currently doesn't work.

 * iso-2022 charsets get unified on i/o.

 * Revisit locale processing: look at treating the language and
   charset parts separately.  (Language should affect things like
   speling and calendar, but that's not a Unicode issue.)

 * Handle Unicode combining characters usefully, e.g. diacritics, and
   handle more scripts specifically (à la Devanagari).  There are
   issues with canonicalization.

 * Bidi is a separate issue with no support currently.

 * DTRT with X keysyms.  We should get the right unicode for a given
   keysym, not decode raw bytes in some ill-defined coding system.
   (fx has some data on keysyms v. unicodes.)

 * We need tabular input methods, e.g. for maths symbols.  (Not
   specific to Unicode.)

 * Need multibyte text in menus, e.g. for the above.  (Not specific to
   Unicode.)

 * Still can't have case pairs which have different byte lengths --
   can that be fixed for Turkish, at least?

 * There's currently no support for Unicode normalization.

 * Populate char-width-table correctly for Unicode chanaracters and
   worry about what happens when double-width charsets covering
   non-CJK characters are unified.

 * Emacs 20/21 .elc files are currently not loadable.  It may or may
   not be possible to do this properly.

	With the change on 2002-07-24, elc files generated by Emacs
	20.3 and later are correctly loaded (including those
	containing multibyte characters and compressed).  But, elc
	files generated by 20.2 and the primer are still not loadable.
	Is it really worth working on it?

 * Encoding issues in babyl files/rmail need sorting out.

 * Gnus still needs some attention, and we need to get changes
   accepted by Gnus maintainers...

 * You can grep the code for lots of fixmes.