Mercurial > emacs
annotate admin/charsets/mapconv @ 111453:5f5f21c6fca1
ChangeLog fix.
author | Glenn Morris <rgm@gnu.org> |
---|---|
date | Tue, 09 Nov 2010 19:43:44 -0800 |
parents | 1d1d5d9bd884 |
children | 376148b31b5e |
rev | line source |
---|---|
88123 | 1 #!/bin/sh |
94832
eb2d9dfc8486
Switch to recommended form of GPLv3 permissions notice.
Glenn Morris <rgm@gnu.org>
parents:
91418
diff
changeset
|
2 |
106815 | 3 # Copyright (C) 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010 |
88123 | 4 # National Institute of Advanced Industrial Science and Technology (AIST) |
5 # Registration Number H13PRO009 | |
94832
eb2d9dfc8486
Switch to recommended form of GPLv3 permissions notice.
Glenn Morris <rgm@gnu.org>
parents:
91418
diff
changeset
|
6 |
88123 | 7 # This file is part of GNU Emacs. |
94832
eb2d9dfc8486
Switch to recommended form of GPLv3 permissions notice.
Glenn Morris <rgm@gnu.org>
parents:
91418
diff
changeset
|
8 |
eb2d9dfc8486
Switch to recommended form of GPLv3 permissions notice.
Glenn Morris <rgm@gnu.org>
parents:
91418
diff
changeset
|
9 # GNU Emacs is free software: you can redistribute it and/or modify |
88123 | 10 # it under the terms of the GNU General Public License as published by |
94832
eb2d9dfc8486
Switch to recommended form of GPLv3 permissions notice.
Glenn Morris <rgm@gnu.org>
parents:
91418
diff
changeset
|
11 # the Free Software Foundation, either version 3 of the License, or |
eb2d9dfc8486
Switch to recommended form of GPLv3 permissions notice.
Glenn Morris <rgm@gnu.org>
parents:
91418
diff
changeset
|
12 # (at your option) any later version. |
eb2d9dfc8486
Switch to recommended form of GPLv3 permissions notice.
Glenn Morris <rgm@gnu.org>
parents:
91418
diff
changeset
|
13 |
88123 | 14 # GNU Emacs is distributed in the hope that it will be useful, |
15 # but WITHOUT ANY WARRANTY; without even the implied warranty of | |
16 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | |
17 # GNU General Public License for more details. | |
94832
eb2d9dfc8486
Switch to recommended form of GPLv3 permissions notice.
Glenn Morris <rgm@gnu.org>
parents:
91418
diff
changeset
|
18 |
88123 | 19 # You should have received a copy of the GNU General Public License |
94832
eb2d9dfc8486
Switch to recommended form of GPLv3 permissions notice.
Glenn Morris <rgm@gnu.org>
parents:
91418
diff
changeset
|
20 # along with GNU Emacs. If not, see <http://www.gnu.org/licenses/>. |
88123 | 21 |
94832
eb2d9dfc8486
Switch to recommended form of GPLv3 permissions notice.
Glenn Morris <rgm@gnu.org>
parents:
91418
diff
changeset
|
22 # Commentary: |
eb2d9dfc8486
Switch to recommended form of GPLv3 permissions notice.
Glenn Morris <rgm@gnu.org>
parents:
91418
diff
changeset
|
23 |
88123 | 24 # Convert charset map of various format into this: |
25 # 0xXX 0xYYYY | |
26 # where, | |
27 # XX is a code point of the charset in hexa-decimal, | |
28 # YYYY is the corresponding Unicode character code in hexa-decimal. | |
29 # Arguments are: | |
30 # $1: source map file | |
31 # $2: address pattern for sed (optionally with substitution command) | |
32 # $3: format of source map file | |
103578
63a1307441f9
For the UNICODE format files, do reverse sort
Kenichi Handa <handa@m17n.org>
parents:
103389
diff
changeset
|
33 # GLIBC-1 GLIBC-2 GLIBC-2-7 CZYBORRA IANA UNICODE UNICODE2 YASUOKA |
88123 | 34 # $4: awk script |
35 | |
103389
24af3ac0cac1
Mostly re-written to handle "gzip"ed input files.
Kenichi Handa <handa@m17n.org>
parents:
100971
diff
changeset
|
36 FILE="admin/charsets/$1" |
24af3ac0cac1
Mostly re-written to handle "gzip"ed input files.
Kenichi Handa <handa@m17n.org>
parents:
100971
diff
changeset
|
37 BASE=`basename $1 .gz` |
88123 | 38 |
39 case "$3" in | |
40 GLIBC*) | |
103389
24af3ac0cac1
Mostly re-written to handle "gzip"ed input files.
Kenichi Handa <handa@m17n.org>
parents:
100971
diff
changeset
|
41 FILE="$BASE in localedate/charmaps of glibc"; |
24af3ac0cac1
Mostly re-written to handle "gzip"ed input files.
Kenichi Handa <handa@m17n.org>
parents:
100971
diff
changeset
|
42 SOURCE="";; |
88123 | 43 CZYBORRA) |
103389
24af3ac0cac1
Mostly re-written to handle "gzip"ed input files.
Kenichi Handa <handa@m17n.org>
parents:
100971
diff
changeset
|
44 BASE="$BASE.gz"; |
88123 | 45 SOURCE="http://czyborra.com/charsets/${BASE}";; |
46 IANA) | |
47 SOURCE="http://www.iana.org/assignments/charset-reg/${BASE}";; | |
48 UNICODE) | |
103389
24af3ac0cac1
Mostly re-written to handle "gzip"ed input files.
Kenichi Handa <handa@m17n.org>
parents:
100971
diff
changeset
|
49 SOURCE="http://www.unicode.org/Public/MAPPINGS/VENDORS/ADOBE/${BASE}";; |
89742 | 50 UNICODE2) |
103389
24af3ac0cac1
Mostly re-written to handle "gzip"ed input files.
Kenichi Handa <handa@m17n.org>
parents:
100971
diff
changeset
|
51 SOURCE="http://www.unicode.org/Public/MAPPINGS/VENDORS/MICSFT/WINDOWS/${BASE}";; |
88123 | 52 YASUOKA) |
103389
24af3ac0cac1
Mostly re-written to handle "gzip"ed input files.
Kenichi Handa <handa@m17n.org>
parents:
100971
diff
changeset
|
53 BASE="$BASE.Z"; |
24af3ac0cac1
Mostly re-written to handle "gzip"ed input files.
Kenichi Handa <handa@m17n.org>
parents:
100971
diff
changeset
|
54 SOURCE="http://kanji.zinbun.kyoto-u.ac.jp/~yasuoka/ftp/CJKtable/${BASE}";; |
89815
59bbae51ad73
Add code for handling KANJI-DATABASE format.
Kenichi Handa <handa@m17n.org>
parents:
89746
diff
changeset
|
55 KANJI-DATABASE) |
103389
24af3ac0cac1
Mostly re-written to handle "gzip"ed input files.
Kenichi Handa <handa@m17n.org>
parents:
100971
diff
changeset
|
56 SOURCE="http://kanji-database.cvs.sourceforge.net/viewvc/*checkout*/kanji-database/kanji-database/data/cns2ucsdkw.txt?revision=1.4";; |
88123 | 57 *) |
58 echo "Unknown file type: $3"; | |
59 exit 1;; | |
60 esac | |
61 | |
103389
24af3ac0cac1
Mostly re-written to handle "gzip"ed input files.
Kenichi Handa <handa@m17n.org>
parents:
100971
diff
changeset
|
62 if [ -n "$SOURCE" ] ; then |
24af3ac0cac1
Mostly re-written to handle "gzip"ed input files.
Kenichi Handa <handa@m17n.org>
parents:
100971
diff
changeset
|
63 echo "# Generated from $FILE which is a copy of"; |
24af3ac0cac1
Mostly re-written to handle "gzip"ed input files.
Kenichi Handa <handa@m17n.org>
parents:
100971
diff
changeset
|
64 echo "# $SOURCE" |
24af3ac0cac1
Mostly re-written to handle "gzip"ed input files.
Kenichi Handa <handa@m17n.org>
parents:
100971
diff
changeset
|
65 else |
24af3ac0cac1
Mostly re-written to handle "gzip"ed input files.
Kenichi Handa <handa@m17n.org>
parents:
100971
diff
changeset
|
66 echo "# Generated from $FILE" |
24af3ac0cac1
Mostly re-written to handle "gzip"ed input files.
Kenichi Handa <handa@m17n.org>
parents:
100971
diff
changeset
|
67 fi |
24af3ac0cac1
Mostly re-written to handle "gzip"ed input files.
Kenichi Handa <handa@m17n.org>
parents:
100971
diff
changeset
|
68 |
88123 | 69 |
70 if [ -n "$4" ] ; then | |
71 if [ -f "$4" ] ; then | |
72 AWKPROG="gawk -f $4" | |
73 else | |
74 echo "Awk program does not exist: $4" | |
75 exit 1 | |
76 fi | |
77 else | |
78 AWKPROG=cat | |
79 fi | |
80 | |
103389
24af3ac0cac1
Mostly re-written to handle "gzip"ed input files.
Kenichi Handa <handa@m17n.org>
parents:
100971
diff
changeset
|
81 if [ "$3" = "GLIBC-1" ] ; then |
88123 | 82 # Source format is: |
83 # <UYYYY> /xXX | |
103389
24af3ac0cac1
Mostly re-written to handle "gzip"ed input files.
Kenichi Handa <handa@m17n.org>
parents:
100971
diff
changeset
|
84 zcat $1 | sed -n -e "$2 p" \ |
88123 | 85 | sed -e 's,<U\([^>]*\)>[ ]*/x\(..\).*,0x\2 0x\1,' \ |
86 | sort | ${AWKPROG} | |
103389
24af3ac0cac1
Mostly re-written to handle "gzip"ed input files.
Kenichi Handa <handa@m17n.org>
parents:
100971
diff
changeset
|
87 elif [ "$3" = "GLIBC-2" ] ; then |
88123 | 88 # Source format is: |
89 # <UYYYY> /xXX/xZZ | |
103389
24af3ac0cac1
Mostly re-written to handle "gzip"ed input files.
Kenichi Handa <handa@m17n.org>
parents:
100971
diff
changeset
|
90 zcat $1 | sed -n -e "$2 p" \ |
88123 | 91 | sed -e 's,<U\([^>]*\)>[ ]*/x\(..\)/x\(..\).*,0x\2\3 0x\1,' \ |
92 | sort | ${AWKPROG} | |
103389
24af3ac0cac1
Mostly re-written to handle "gzip"ed input files.
Kenichi Handa <handa@m17n.org>
parents:
100971
diff
changeset
|
93 elif [ "$3" = "GLIBC-2-7" ] ; then |
88123 | 94 # Source format is: |
95 # <UYYYY> /xXX/xZZ | |
96 # We must drop MSBs of XX and ZZ | |
103389
24af3ac0cac1
Mostly re-written to handle "gzip"ed input files.
Kenichi Handa <handa@m17n.org>
parents:
100971
diff
changeset
|
97 zcat $1 | sed -n -e "$2 p" \ |
88123 | 98 | sed -e 's/xa/x2/g' -e 's/xb/x3/g' -e 's/xc/x4/g' \ |
99 -e 's/xd/x5/g' -e 's/xe/x6/g' -e 's/xf/x7/g' \ | |
100 -e 's,<U\([^>]*\)>[ ]*/x\(..\)/x\(..\).*,0x\2\3 0x\1,' \ | |
101 | tee temp \ | |
102 | sort | ${AWKPROG} | |
103389
24af3ac0cac1
Mostly re-written to handle "gzip"ed input files.
Kenichi Handa <handa@m17n.org>
parents:
100971
diff
changeset
|
103 elif [ "$3" = "CZYBORRA" ] ; then |
88123 | 104 # Source format is: |
105 # =XX U+YYYY | |
106 zcat $1 | sed -n -e "$2 p" \ | |
107 | sed -e 's/=\(..\)[^U]*U+\([0-9A-F]*\).*/0x\1 0x\2/' \ | |
108 | sort | ${AWKPROG} | |
103389
24af3ac0cac1
Mostly re-written to handle "gzip"ed input files.
Kenichi Handa <handa@m17n.org>
parents:
100971
diff
changeset
|
109 elif [ "$3" = "IANA" ] ; then |
88123 | 110 # Source format is: |
111 # 0xXX 0xYYYY | |
103389
24af3ac0cac1
Mostly re-written to handle "gzip"ed input files.
Kenichi Handa <handa@m17n.org>
parents:
100971
diff
changeset
|
112 zcat $1 | sed -n -e "$2 p" \ |
88123 | 113 | sed -e 's/\(0x[0-9A-Fa-f]*\)[^0]*\(0x[0-9A-Fa-f]*\).*/\1 \2/' \ |
114 | sort | ${AWKPROG} | |
103389
24af3ac0cac1
Mostly re-written to handle "gzip"ed input files.
Kenichi Handa <handa@m17n.org>
parents:
100971
diff
changeset
|
115 elif [ "$3" = "UNICODE" ] ; then |
88123 | 116 # Source format is: |
117 # YYYY XX | |
103578
63a1307441f9
For the UNICODE format files, do reverse sort
Kenichi Handa <handa@m17n.org>
parents:
103389
diff
changeset
|
118 # We perform reverse sort to prefer the first one in the |
63a1307441f9
For the UNICODE format files, do reverse sort
Kenichi Handa <handa@m17n.org>
parents:
103389
diff
changeset
|
119 # duplicated mappings (e.g. 0x20->U+0020, 0x20->U+00A0). |
103389
24af3ac0cac1
Mostly re-written to handle "gzip"ed input files.
Kenichi Handa <handa@m17n.org>
parents:
100971
diff
changeset
|
120 zcat $1 | sed -n -e "$2 p" \ |
88123 | 121 | sed -e 's/\([0-9A-F]*\)[^0-9A-F]*\([0-9A-F]*\).*/0x\2 0x\1/' \ |
103578
63a1307441f9
For the UNICODE format files, do reverse sort
Kenichi Handa <handa@m17n.org>
parents:
103389
diff
changeset
|
122 | sort -r |
103389
24af3ac0cac1
Mostly re-written to handle "gzip"ed input files.
Kenichi Handa <handa@m17n.org>
parents:
100971
diff
changeset
|
123 elif [ "$3" = "UNICODE2" ] ; then |
89742 | 124 # Source format is: |
125 # 0xXXXX 0xYYYY # ... | |
103389
24af3ac0cac1
Mostly re-written to handle "gzip"ed input files.
Kenichi Handa <handa@m17n.org>
parents:
100971
diff
changeset
|
126 zcat $1 | sed -n -e "$2 p" \ |
89742 | 127 | sed -e 's/\([0-9A-Fx]*\)[^0]*\([0-9A-Fx]*\).*/\1 \2/' \ |
89746
b7afe7c870d3
In UNICODE2 case, sort by 4th field after
Kenichi Handa <handa@m17n.org>
parents:
89742
diff
changeset
|
128 | ${AWKPROG} | sort -n -k 4,4 |
103389
24af3ac0cac1
Mostly re-written to handle "gzip"ed input files.
Kenichi Handa <handa@m17n.org>
parents:
100971
diff
changeset
|
129 elif [ "$3" = "YASUOKA" ] ; then |
88123 | 130 # Source format is: |
131 # YYYY 0-XXXX (XXXX is a Kuten code) | |
103389
24af3ac0cac1
Mostly re-written to handle "gzip"ed input files.
Kenichi Handa <handa@m17n.org>
parents:
100971
diff
changeset
|
132 zcat $1 | sed -n -e "$2 p" \ |
88123 | 133 | sed -e 's/\([0-9A-F]*\)[^0]*0-\([0-9]*\).*/0x\2 0x\1/' \ |
134 | sort | ${AWKPROG} | |
103389
24af3ac0cac1
Mostly re-written to handle "gzip"ed input files.
Kenichi Handa <handa@m17n.org>
parents:
100971
diff
changeset
|
135 elif [ "$3" = "KANJI-DATABASE" ] ; then |
89815
59bbae51ad73
Add code for handling KANJI-DATABASE format.
Kenichi Handa <handa@m17n.org>
parents:
89746
diff
changeset
|
136 # Source format is: |
59bbae51ad73
Add code for handling KANJI-DATABASE format.
Kenichi Handa <handa@m17n.org>
parents:
89746
diff
changeset
|
137 # C?-XXXX U+YYYYY ..... |
103389
24af3ac0cac1
Mostly re-written to handle "gzip"ed input files.
Kenichi Handa <handa@m17n.org>
parents:
100971
diff
changeset
|
138 zcat $1 | sed -n -e "$2 p" \ |
89815
59bbae51ad73
Add code for handling KANJI-DATABASE format.
Kenichi Handa <handa@m17n.org>
parents:
89746
diff
changeset
|
139 | sed -e 's/...\(....\) U+\([0-9A-F]*\).*/0x\1 0x\2/' \ |
59bbae51ad73
Add code for handling KANJI-DATABASE format.
Kenichi Handa <handa@m17n.org>
parents:
89746
diff
changeset
|
140 | sort | ${AWKPROG} |
88123 | 141 else |
103389
24af3ac0cac1
Mostly re-written to handle "gzip"ed input files.
Kenichi Handa <handa@m17n.org>
parents:
100971
diff
changeset
|
142 echo "Invalid arguments: $3" |
88123 | 143 exit 1 |
144 fi | |
89916
e0e4e6a0599f
Changes from arch/CVS synchronization
Miles Bader <miles@gnu.org>
parents:
89823
diff
changeset
|
145 |
e0e4e6a0599f
Changes from arch/CVS synchronization
Miles Bader <miles@gnu.org>
parents:
89823
diff
changeset
|
146 # arch-tag: c33acb47-7eb6-4872-b871-15e1447e8f0e |