Mercurial > emacs
annotate lisp/cedet/semantic/lex.el @ 104413:6524f06f3a75
cedet/semantic/lex.el: New file.
author | Chong Yidong <cyd@stupidchicken.com> |
---|---|
date | Fri, 28 Aug 2009 15:02:24 +0000 |
parents | |
children | c13af98da4d6 |
rev | line source |
---|---|
104413
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1 ;;; semantic-lex.el --- Lexical Analyzer builder |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
3 ;;; Copyright (C) 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
4 ;;; 2007, 2008, 2009 Free Software Foundation, Inc. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
5 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
6 ;; Author: Eric M. Ludlam <zappo@gnu.org> |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
7 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
8 ;; This file is part of GNU Emacs. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
9 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
10 ;; GNU Emacs is free software: you can redistribute it and/or modify |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
11 ;; it under the terms of the GNU General Public License as published by |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
12 ;; the Free Software Foundation, either version 3 of the License, or |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
13 ;; (at your option) any later version. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
14 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
15 ;; GNU Emacs is distributed in the hope that it will be useful, |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
16 ;; but WITHOUT ANY WARRANTY; without even the implied warranty of |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
17 ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
18 ;; GNU General Public License for more details. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
19 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
20 ;; You should have received a copy of the GNU General Public License |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
21 ;; along with GNU Emacs. If not, see <http://www.gnu.org/licenses/>. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
22 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
23 ;;; Commentary: |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
24 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
25 ;; This file handles the creation of lexical analyzers for different |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
26 ;; languages in Emacs Lisp. The purpose of a lexical analyzer is to |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
27 ;; convert a buffer into a list of lexical tokens. Each token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
28 ;; contains the token class (such as 'number, 'symbol, 'IF, etc) and |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
29 ;; the location in the buffer it was found. Optionally, a token also |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
30 ;; contains a string representing what is at the designated buffer |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
31 ;; location. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
32 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
33 ;; Tokens are pushed onto a token stream, which is basically a list of |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
34 ;; all the lexical tokens from the analyzed region. The token stream |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
35 ;; is then handed to the grammar which parsers the file. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
36 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
37 ;;; How it works |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
38 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
39 ;; Each analyzer specifies a condition and forms. These conditions |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
40 ;; and forms are assembled into a function by `define-lex' that does |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
41 ;; the lexical analysis. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
42 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
43 ;; In the lexical analyzer created with `define-lex', each condition |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
44 ;; is tested for a given point. When the conditin is true, the forms |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
45 ;; run. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
46 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
47 ;; The forms can push a lexical token onto the token stream. The |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
48 ;; analyzer forms also must move the current analyzer point. If the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
49 ;; analyzer point is moved without pushing a token, then tne matched |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
50 ;; syntax is effectively ignored, or skipped. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
51 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
52 ;; Thus, starting at the beginning of a region to be analyzed, each |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
53 ;; condition is tested. One will match, and a lexical token might be |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
54 ;; pushed, and the point is moved to the end of the lexical token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
55 ;; identified. At the new position, the process occurs again until |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
56 ;; the end of the specified region is reached. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
57 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
58 ;;; How to use semantic-lex |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
59 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
60 ;; To create a lexer for a language, use the `define-lex' macro. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
61 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
62 ;; The `define-lex' macro accepts a list of lexical analyzers. Each |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
63 ;; analyzer is created with `define-lex-analyzer', or one of the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
64 ;; derivitive macros. A single analyzer defines a regular expression |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
65 ;; to match text in a buffer, and a short segment of code to create |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
66 ;; one lexical token. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
67 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
68 ;; Each analyzer has a NAME, DOC, a CONDITION, and possibly some |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
69 ;; FORMS. The NAME is the name used in `define-lex'. The DOC |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
70 ;; describes what the analyzer should do. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
71 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
72 ;; The CONDITION evaluates the text at the current point in the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
73 ;; current buffer. If CONDITION is true, then the FORMS will be |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
74 ;; executed. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
75 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
76 ;; The purpose of the FORMS is to push new lexical tokens onto the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
77 ;; list of tokens for the current buffer, and to move point after the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
78 ;; matched text. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
79 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
80 ;; Some macros for creating one analyzer are: |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
81 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
82 ;; define-lex-analyzer - A generic analyzer associating any style of |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
83 ;; condition to forms. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
84 ;; define-lex-regex-analyzer - Matches a regular expression. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
85 ;; define-lex-simple-regex-analyzer - Matches a regular expressions, |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
86 ;; and pushes the match. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
87 ;; define-lex-block-analyzer - Matches list syntax, and defines |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
88 ;; handles open/close delimiters. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
89 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
90 ;; These macros are used by the grammar compiler when lexical |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
91 ;; information is specified in a grammar: |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
92 ;; define-lex- * -type-analyzer - Matches syntax specified in |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
93 ;; a grammar, and pushes one token for it. The * would |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
94 ;; be `sexp' for things like lists or strings, and |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
95 ;; `string' for things that need to match some special |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
96 ;; string, such as "\\." where a literal match is needed. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
97 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
98 ;;; Lexical Tables |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
99 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
100 ;; There are tables of different symbols managed in semantic-lex.el. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
101 ;; They are: |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
102 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
103 ;; Lexical keyword table - A Table of symbols declared in a grammar |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
104 ;; file with the %keyword declaration. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
105 ;; Keywords are used by `semantic-lex-symbol-or-keyword' |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
106 ;; to create lexical tokens based on the keyword. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
107 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
108 ;; Lexical type table - A table of symbols declared in a grammer |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
109 ;; file with the %type declaration. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
110 ;; The grammar compiler uses the type table to create new |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
111 ;; lexical analyzers. These analyzers are then used to when |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
112 ;; a new lexical analyzer is made for a language. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
113 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
114 ;;; Lexical Types |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
115 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
116 ;; A lexical type defines a kind of lexical analyzer that will be |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
117 ;; automatically generated from a grammar file based on some |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
118 ;; predetermined attributes. For now these two attributes are |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
119 ;; recognized : |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
120 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
121 ;; * matchdatatype : define the kind of lexical analyzer. That is : |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
122 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
123 ;; - regexp : define a regexp analyzer (see |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
124 ;; `define-lex-regex-type-analyzer') |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
125 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
126 ;; - string : define a string analyzer (see |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
127 ;; `define-lex-string-type-analyzer') |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
128 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
129 ;; - block : define a block type analyzer (see |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
130 ;; `define-lex-block-type-analyzer') |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
131 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
132 ;; - sexp : define a sexp analyzer (see |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
133 ;; `define-lex-sexp-type-analyzer') |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
134 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
135 ;; - keyword : define a keyword analyzer (see |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
136 ;; `define-lex-keyword-type-analyzer') |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
137 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
138 ;; * syntax : define the syntax that matches a syntactic |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
139 ;; expression. When syntax is matched the corresponding type |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
140 ;; analyzer is entered and the resulting match data will be |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
141 ;; interpreted based on the kind of analyzer (see matchdatatype |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
142 ;; above). |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
143 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
144 ;; The following lexical types are predefined : |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
145 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
146 ;; +-------------+---------------+--------------------------------+ |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
147 ;; | type | matchdatatype | syntax | |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
148 ;; +-------------+---------------+--------------------------------+ |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
149 ;; | punctuation | string | "\\(\\s.\\|\\s$\\|\\s'\\)+" | |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
150 ;; | keyword | keyword | "\\(\\sw\\|\\s_\\)+" | |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
151 ;; | symbol | regexp | "\\(\\sw\\|\\s_\\)+" | |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
152 ;; | string | sexp | "\\s\"" | |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
153 ;; | number | regexp | semantic-lex-number-expression | |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
154 ;; | block | block | "\\s(\\|\\s)" | |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
155 ;; +-------------+---------------+--------------------------------+ |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
156 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
157 ;; In a grammar you must use a %type expression to automatically generate |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
158 ;; the corresponding analyzers of that type. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
159 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
160 ;; Here is an example to auto-generate punctuation analyzers |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
161 ;; with 'matchdatatype and 'syntax predefined (see table above) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
162 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
163 ;; %type <punctuation> ;; will auto-generate this kind of analyzers |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
164 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
165 ;; It is equivalent to write : |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
166 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
167 ;; %type <punctuation> syntax "\\(\\s.\\|\\s$\\|\\s'\\)+" matchdatatype string |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
168 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
169 ;; ;; Some punctuations based on the type defines above |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
170 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
171 ;; %token <punctuation> NOT "!" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
172 ;; %token <punctuation> NOTEQ "!=" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
173 ;; %token <punctuation> MOD "%" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
174 ;; %token <punctuation> MODEQ "%=" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
175 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
176 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
177 ;;; On the Semantic 1.x lexer |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
178 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
179 ;; In semantic 1.x, the lexical analyzer was an all purpose routine. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
180 ;; To boost efficiency, the analyzer is now a series of routines that |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
181 ;; are constructed at build time into a single routine. This will |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
182 ;; eliminate unneeded if statements to speed the lexer. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
183 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
184 (require 'semantic/fw) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
185 ;;; Code: |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
186 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
187 ;;; Compatibility |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
188 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
189 (eval-and-compile |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
190 (if (not (fboundp 'with-syntax-table)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
191 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
192 ;; Copied from Emacs 21 for compatibility with released Emacses. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
193 (defmacro with-syntax-table (table &rest body) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
194 "With syntax table of current buffer set to a copy of TABLE, evaluate BODY. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
195 The syntax table of the current buffer is saved, BODY is evaluated, and the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
196 saved table is restored, even in case of an abnormal exit. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
197 Value is what BODY returns." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
198 (let ((old-table (make-symbol "table")) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
199 (old-buffer (make-symbol "buffer"))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
200 `(let ((,old-table (syntax-table)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
201 (,old-buffer (current-buffer))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
202 (unwind-protect |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
203 (progn |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
204 (set-syntax-table (copy-syntax-table ,table)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
205 ,@body) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
206 (save-current-buffer |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
207 (set-buffer ,old-buffer) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
208 (set-syntax-table ,old-table)))))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
209 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
210 )) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
211 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
212 ;;; Semantic 2.x lexical analysis |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
213 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
214 (defun semantic-lex-map-symbols (fun table &optional property) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
215 "Call function FUN on every symbol in TABLE. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
216 If optional PROPERTY is non-nil, call FUN only on every symbol which |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
217 as a PROPERTY value. FUN receives a symbol as argument." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
218 (if (arrayp table) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
219 (mapatoms |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
220 #'(lambda (symbol) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
221 (if (or (null property) (get symbol property)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
222 (funcall fun symbol))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
223 table))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
224 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
225 ;;; Lexical keyword table handling. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
226 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
227 ;; These keywords are keywords defined for using in a grammar with the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
228 ;; %keyword declaration, and are not keywords used in Emacs Lisp. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
229 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
230 (defvar semantic-flex-keywords-obarray nil |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
231 "Buffer local keyword obarray for the lexical analyzer. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
232 These keywords are matched explicitly, and converted into special symbols.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
233 (make-variable-buffer-local 'semantic-flex-keywords-obarray) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
234 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
235 (defmacro semantic-lex-keyword-invalid (name) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
236 "Signal that NAME is an invalid keyword name." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
237 `(signal 'wrong-type-argument '(semantic-lex-keyword-p ,name))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
238 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
239 (defsubst semantic-lex-keyword-symbol (name) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
240 "Return keyword symbol with NAME or nil if not found." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
241 (and (arrayp semantic-flex-keywords-obarray) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
242 (stringp name) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
243 (intern-soft name semantic-flex-keywords-obarray))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
244 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
245 (defsubst semantic-lex-keyword-p (name) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
246 "Return non-nil if a keyword with NAME exists in the keyword table. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
247 Return nil otherwise." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
248 (and (setq name (semantic-lex-keyword-symbol name)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
249 (symbol-value name))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
250 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
251 (defsubst semantic-lex-keyword-set (name value) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
252 "Set value of keyword with NAME to VALUE and return VALUE." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
253 (set (intern name semantic-flex-keywords-obarray) value)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
254 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
255 (defsubst semantic-lex-keyword-value (name) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
256 "Return value of keyword with NAME. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
257 Signal an error if a keyword with NAME does not exist." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
258 (let ((keyword (semantic-lex-keyword-symbol name))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
259 (if keyword |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
260 (symbol-value keyword) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
261 (semantic-lex-keyword-invalid name)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
262 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
263 (defsubst semantic-lex-keyword-put (name property value) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
264 "For keyword with NAME, set its PROPERTY to VALUE." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
265 (let ((keyword (semantic-lex-keyword-symbol name))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
266 (if keyword |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
267 (put keyword property value) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
268 (semantic-lex-keyword-invalid name)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
269 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
270 (defsubst semantic-lex-keyword-get (name property) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
271 "For keyword with NAME, return its PROPERTY value." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
272 (let ((keyword (semantic-lex-keyword-symbol name))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
273 (if keyword |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
274 (get keyword property) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
275 (semantic-lex-keyword-invalid name)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
276 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
277 (defun semantic-lex-make-keyword-table (specs &optional propspecs) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
278 "Convert keyword SPECS into an obarray and return it. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
279 SPECS must be a list of (NAME . TOKSYM) elements, where: |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
280 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
281 NAME is the name of the keyword symbol to define. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
282 TOKSYM is the lexical token symbol of that keyword. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
283 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
284 If optional argument PROPSPECS is non nil, then interpret it, and |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
285 apply those properties. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
286 PROPSPECS must be a list of (NAME PROPERTY VALUE) elements." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
287 ;; Create the symbol hash table |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
288 (let ((semantic-flex-keywords-obarray (make-vector 13 0)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
289 spec) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
290 ;; fill it with stuff |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
291 (while specs |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
292 (setq spec (car specs) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
293 specs (cdr specs)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
294 (semantic-lex-keyword-set (car spec) (cdr spec))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
295 ;; Apply all properties |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
296 (while propspecs |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
297 (setq spec (car propspecs) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
298 propspecs (cdr propspecs)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
299 (semantic-lex-keyword-put (car spec) (nth 1 spec) (nth 2 spec))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
300 semantic-flex-keywords-obarray)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
301 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
302 (defsubst semantic-lex-map-keywords (fun &optional property) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
303 "Call function FUN on every lexical keyword. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
304 If optional PROPERTY is non-nil, call FUN only on every keyword which |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
305 as a PROPERTY value. FUN receives a lexical keyword as argument." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
306 (semantic-lex-map-symbols |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
307 fun semantic-flex-keywords-obarray property)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
308 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
309 (defun semantic-lex-keywords (&optional property) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
310 "Return a list of lexical keywords. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
311 If optional PROPERTY is non-nil, return only keywords which have a |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
312 PROPERTY set." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
313 (let (keywords) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
314 (semantic-lex-map-keywords |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
315 #'(lambda (symbol) (setq keywords (cons symbol keywords))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
316 property) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
317 keywords)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
318 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
319 ;;; Type table handling. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
320 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
321 ;; The lexical type table manages types that occur in a grammar file |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
322 ;; with the %type declaration. Types represent different syntaxes. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
323 ;; See code for `semantic-lex-preset-default-types' for the classic |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
324 ;; types of syntax. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
325 (defvar semantic-lex-types-obarray nil |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
326 "Buffer local types obarray for the lexical analyzer.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
327 (make-variable-buffer-local 'semantic-lex-types-obarray) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
328 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
329 (defmacro semantic-lex-type-invalid (type) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
330 "Signal that TYPE is an invalid lexical type name." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
331 `(signal 'wrong-type-argument '(semantic-lex-type-p ,type))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
332 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
333 (defsubst semantic-lex-type-symbol (type) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
334 "Return symbol with TYPE or nil if not found." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
335 (and (arrayp semantic-lex-types-obarray) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
336 (stringp type) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
337 (intern-soft type semantic-lex-types-obarray))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
338 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
339 (defsubst semantic-lex-type-p (type) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
340 "Return non-nil if a symbol with TYPE name exists." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
341 (and (setq type (semantic-lex-type-symbol type)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
342 (symbol-value type))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
343 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
344 (defsubst semantic-lex-type-set (type value) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
345 "Set value of symbol with TYPE name to VALUE and return VALUE." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
346 (set (intern type semantic-lex-types-obarray) value)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
347 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
348 (defsubst semantic-lex-type-value (type &optional noerror) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
349 "Return value of symbol with TYPE name. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
350 If optional argument NOERROR is non-nil return nil if a symbol with |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
351 TYPE name does not exist. Otherwise signal an error." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
352 (let ((sym (semantic-lex-type-symbol type))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
353 (if sym |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
354 (symbol-value sym) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
355 (unless noerror |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
356 (semantic-lex-type-invalid type))))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
357 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
358 (defsubst semantic-lex-type-put (type property value &optional add) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
359 "For symbol with TYPE name, set its PROPERTY to VALUE. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
360 If optional argument ADD is non-nil, create a new symbol with TYPE |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
361 name if it does not already exist. Otherwise signal an error." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
362 (let ((sym (semantic-lex-type-symbol type))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
363 (unless sym |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
364 (or add (semantic-lex-type-invalid type)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
365 (semantic-lex-type-set type nil) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
366 (setq sym (semantic-lex-type-symbol type))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
367 (put sym property value))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
368 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
369 (defsubst semantic-lex-type-get (type property &optional noerror) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
370 "For symbol with TYPE name, return its PROPERTY value. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
371 If optional argument NOERROR is non-nil return nil if a symbol with |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
372 TYPE name does not exist. Otherwise signal an error." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
373 (let ((sym (semantic-lex-type-symbol type))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
374 (if sym |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
375 (get sym property) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
376 (unless noerror |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
377 (semantic-lex-type-invalid type))))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
378 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
379 (defun semantic-lex-preset-default-types () |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
380 "Install useful default properties for well known types." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
381 (semantic-lex-type-put "punctuation" 'matchdatatype 'string t) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
382 (semantic-lex-type-put "punctuation" 'syntax "\\(\\s.\\|\\s$\\|\\s'\\)+") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
383 (semantic-lex-type-put "keyword" 'matchdatatype 'keyword t) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
384 (semantic-lex-type-put "keyword" 'syntax "\\(\\sw\\|\\s_\\)+") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
385 (semantic-lex-type-put "symbol" 'matchdatatype 'regexp t) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
386 (semantic-lex-type-put "symbol" 'syntax "\\(\\sw\\|\\s_\\)+") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
387 (semantic-lex-type-put "string" 'matchdatatype 'sexp t) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
388 (semantic-lex-type-put "string" 'syntax "\\s\"") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
389 (semantic-lex-type-put "number" 'matchdatatype 'regexp t) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
390 (semantic-lex-type-put "number" 'syntax 'semantic-lex-number-expression) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
391 (semantic-lex-type-put "block" 'matchdatatype 'block t) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
392 (semantic-lex-type-put "block" 'syntax "\\s(\\|\\s)") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
393 ) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
394 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
395 (defun semantic-lex-make-type-table (specs &optional propspecs) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
396 "Convert type SPECS into an obarray and return it. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
397 SPECS must be a list of (TYPE . TOKENS) elements, where: |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
398 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
399 TYPE is the name of the type symbol to define. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
400 TOKENS is an list of (TOKSYM . MATCHER) elements, where: |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
401 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
402 TOKSYM is any lexical token symbol. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
403 MATCHER is a string or regexp a text must match to be a such |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
404 lexical token. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
405 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
406 If optional argument PROPSPECS is non nil, then interpret it, and |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
407 apply those properties. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
408 PROPSPECS must be a list of (TYPE PROPERTY VALUE)." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
409 ;; Create the symbol hash table |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
410 (let* ((semantic-lex-types-obarray (make-vector 13 0)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
411 spec type tokens token alist default) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
412 ;; fill it with stuff |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
413 (while specs |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
414 (setq spec (car specs) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
415 specs (cdr specs) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
416 type (car spec) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
417 tokens (cdr spec) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
418 default nil |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
419 alist nil) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
420 (while tokens |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
421 (setq token (car tokens) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
422 tokens (cdr tokens)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
423 (if (cdr token) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
424 (setq alist (cons token alist)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
425 (setq token (car token)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
426 (if default |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
427 (message |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
428 "*Warning* default value of <%s> tokens changed to %S, was %S" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
429 type default token)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
430 (setq default token))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
431 ;; Ensure the default matching spec is the first one. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
432 (semantic-lex-type-set type (cons default (nreverse alist)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
433 ;; Install useful default types & properties |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
434 (semantic-lex-preset-default-types) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
435 ;; Apply all properties |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
436 (while propspecs |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
437 (setq spec (car propspecs) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
438 propspecs (cdr propspecs)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
439 ;; Create the type if necessary. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
440 (semantic-lex-type-put (car spec) (nth 1 spec) (nth 2 spec) t)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
441 semantic-lex-types-obarray)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
442 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
443 (defsubst semantic-lex-map-types (fun &optional property) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
444 "Call function FUN on every lexical type. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
445 If optional PROPERTY is non-nil, call FUN only on every type symbol |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
446 which as a PROPERTY value. FUN receives a type symbol as argument." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
447 (semantic-lex-map-symbols |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
448 fun semantic-lex-types-obarray property)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
449 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
450 (defun semantic-lex-types (&optional property) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
451 "Return a list of lexical type symbols. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
452 If optional PROPERTY is non-nil, return only type symbols which have |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
453 PROPERTY set." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
454 (let (types) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
455 (semantic-lex-map-types |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
456 #'(lambda (symbol) (setq types (cons symbol types))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
457 property) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
458 types)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
459 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
460 ;;; Lexical Analyzer framework settings |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
461 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
462 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
463 (defvar semantic-lex-analyzer 'semantic-flex |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
464 "The lexical analyzer used for a given buffer. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
465 See `semantic-lex' for documentation. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
466 For compatibility with Semantic 1.x it defaults to `semantic-flex'.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
467 (make-variable-buffer-local 'semantic-lex-analyzer) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
468 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
469 (defvar semantic-lex-tokens |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
470 '( |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
471 (bol) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
472 (charquote) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
473 (close-paren) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
474 (comment) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
475 (newline) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
476 (open-paren) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
477 (punctuation) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
478 (semantic-list) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
479 (string) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
480 (symbol) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
481 (whitespace) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
482 ) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
483 "An alist of of semantic token types. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
484 As of December 2001 (semantic 1.4beta13), this variable is not used in |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
485 any code. The only use is to refer to the doc-string from elsewhere. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
486 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
487 The key to this alist is the symbol representing token type that |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
488 \\[semantic-flex] returns. These are |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
489 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
490 - bol: Empty string matching a beginning of line. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
491 This token is produced with |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
492 `semantic-lex-beginning-of-line'. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
493 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
494 - charquote: String sequences that match `\\s\\+' regexp. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
495 This token is produced with `semantic-lex-charquote'. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
496 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
497 - close-paren: Characters that match `\\s)' regexp. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
498 These are typically `)', `}', `]', etc. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
499 This token is produced with |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
500 `semantic-lex-close-paren'. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
501 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
502 - comment: A comment chunk. These token types are not |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
503 produced by default. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
504 This token is produced with `semantic-lex-comments'. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
505 Comments are ignored with `semantic-lex-ignore-comments'. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
506 Comments are treated as whitespace with |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
507 `semantic-lex-comments-as-whitespace'. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
508 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
509 - newline Characters matching `\\s-*\\(\n\\|\\s>\\)' regexp. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
510 This token is produced with `semantic-lex-newline'. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
511 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
512 - open-paren: Characters that match `\\s(' regexp. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
513 These are typically `(', `{', `[', etc. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
514 If `semantic-lex-paren-or-list' is used, |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
515 then `open-paren' is not usually generated unless |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
516 the `depth' argument to \\[semantic-lex] is |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
517 greater than 0. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
518 This token is always produced if the analyzer |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
519 `semantic-lex-open-paren' is used. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
520 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
521 - punctuation: Characters matching `{\\(\\s.\\|\\s$\\|\\s'\\)' |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
522 regexp. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
523 This token is produced with `semantic-lex-punctuation'. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
524 Always specify this analyzer after the comment |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
525 analyzer. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
526 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
527 - semantic-list: String delimited by matching parenthesis, braces, |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
528 etc. that the lexer skipped over, because the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
529 `depth' parameter to \\[semantic-flex] was not high |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
530 enough. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
531 This token is produced with `semantic-lex-paren-or-list'. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
532 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
533 - string: Quoted strings, i.e., string sequences that start |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
534 and end with characters matching `\\s\"' |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
535 regexp. The lexer relies on @code{forward-sexp} to |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
536 find the matching end. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
537 This token is produced with `semantic-lex-string'. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
538 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
539 - symbol: String sequences that match `\\(\\sw\\|\\s_\\)+' |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
540 regexp. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
541 This token is produced with |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
542 `semantic-lex-symbol-or-keyword'. Always add this analyzer |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
543 after `semantic-lex-number', or other analyzers that |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
544 match its regular expression. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
545 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
546 - whitespace: Characters that match `\\s-+' regexp. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
547 This token is produced with `semantic-lex-whitespace'.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
548 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
549 (defvar semantic-lex-syntax-modifications nil |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
550 "Changes to the syntax table for this buffer. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
551 These changes are active only while the buffer is being flexed. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
552 This is a list where each element has the form: |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
553 (CHAR CLASS) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
554 CHAR is the char passed to `modify-syntax-entry', |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
555 and CLASS is the string also passed to `modify-syntax-entry' to define |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
556 what syntax class CHAR has.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
557 (make-variable-buffer-local 'semantic-lex-syntax-modifications) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
558 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
559 (defvar semantic-lex-syntax-table nil |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
560 "Syntax table used by lexical analysis. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
561 See also `semantic-lex-syntax-modifications'.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
562 (make-variable-buffer-local 'semantic-lex-syntax-table) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
563 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
564 (defvar semantic-lex-comment-regex nil |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
565 "Regular expression for identifying comment start during lexical analysis. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
566 This may be automatically set when semantic initializes in a mode, but |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
567 may need to be overriden for some special languages.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
568 (make-variable-buffer-local 'semantic-lex-comment-regex) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
569 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
570 (defvar semantic-lex-number-expression |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
571 ;; This expression was written by David Ponce for Java, and copied |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
572 ;; here for C and any other similar language. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
573 (eval-when-compile |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
574 (concat "\\(" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
575 "\\<[0-9]+[.][0-9]+\\([eE][-+]?[0-9]+\\)?[fFdD]?\\>" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
576 "\\|" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
577 "\\<[0-9]+[.][eE][-+]?[0-9]+[fFdD]?\\>" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
578 "\\|" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
579 "\\<[0-9]+[.][fFdD]\\>" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
580 "\\|" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
581 "\\<[0-9]+[.]" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
582 "\\|" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
583 "[.][0-9]+\\([eE][-+]?[0-9]+\\)?[fFdD]?\\>" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
584 "\\|" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
585 "\\<[0-9]+[eE][-+]?[0-9]+[fFdD]?\\>" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
586 "\\|" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
587 "\\<0[xX][0-9a-fA-F]+[lL]?\\>" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
588 "\\|" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
589 "\\<[0-9]+[lLfFdD]?\\>" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
590 "\\)" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
591 )) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
592 "Regular expression for matching a number. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
593 If this value is nil, no number extraction is done during lex. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
594 This expression tries to match C and Java like numbers. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
595 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
596 DECIMAL_LITERAL: |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
597 [1-9][0-9]* |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
598 ; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
599 HEX_LITERAL: |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
600 0[xX][0-9a-fA-F]+ |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
601 ; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
602 OCTAL_LITERAL: |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
603 0[0-7]* |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
604 ; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
605 INTEGER_LITERAL: |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
606 <DECIMAL_LITERAL>[lL]? |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
607 | <HEX_LITERAL>[lL]? |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
608 | <OCTAL_LITERAL>[lL]? |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
609 ; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
610 EXPONENT: |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
611 [eE][+-]?[09]+ |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
612 ; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
613 FLOATING_POINT_LITERAL: |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
614 [0-9]+[.][0-9]*<EXPONENT>?[fFdD]? |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
615 | [.][0-9]+<EXPONENT>?[fFdD]? |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
616 | [0-9]+<EXPONENT>[fFdD]? |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
617 | [0-9]+<EXPONENT>?[fFdD] |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
618 ;") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
619 (make-variable-buffer-local 'semantic-lex-number-expression) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
620 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
621 (defvar semantic-lex-depth 0 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
622 "Default lexing depth. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
623 This specifies how many lists to create tokens in.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
624 (make-variable-buffer-local 'semantic-lex-depth) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
625 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
626 (defvar semantic-lex-unterminated-syntax-end-function |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
627 (lambda (syntax syntax-start lex-end) lex-end) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
628 "Function called when unterminated syntax is encountered. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
629 This should be set to one function. That function should take three |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
630 parameters. The SYNTAX, or type of syntax which is unterminated. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
631 SYNTAX-START where the broken syntax begins. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
632 LEX-END is where the lexical analysis was asked to end. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
633 This function can be used for languages that can intelligently fix up |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
634 broken syntax, or the exit lexical analysis via `throw' or `signal' |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
635 when finding unterminated syntax.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
636 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
637 ;;; Interactive testing commands |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
638 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
639 (defun semantic-lex-test (arg) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
640 "Test the semantic lexer in the current buffer. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
641 If universal argument ARG, then try the whole buffer." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
642 (interactive "P") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
643 (let* ((start (current-time)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
644 (result (semantic-lex |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
645 (if arg (point-min) (point)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
646 (point-max))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
647 (end (current-time))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
648 (message "Elapsed Time: %.2f seconds." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
649 (semantic-elapsed-time start end)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
650 (pop-to-buffer "*Lexer Output*") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
651 (require 'pp) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
652 (erase-buffer) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
653 (insert (pp-to-string result)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
654 (goto-char (point-min)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
655 )) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
656 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
657 (defun semantic-lex-test-full-depth (arg) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
658 "Test the semantic lexer in the current buffer parsing through lists. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
659 Usually the lexer parses |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
660 If universal argument ARG, then try the whole buffer." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
661 (interactive "P") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
662 (let* ((start (current-time)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
663 (result (semantic-lex |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
664 (if arg (point-min) (point)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
665 (point-max) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
666 100)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
667 (end (current-time))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
668 (message "Elapsed Time: %.2f seconds." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
669 (semantic-elapsed-time start end)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
670 (pop-to-buffer "*Lexer Output*") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
671 (require 'pp) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
672 (erase-buffer) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
673 (insert (pp-to-string result)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
674 (goto-char (point-min)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
675 )) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
676 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
677 (defun semantic-lex-test-region (beg end) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
678 "Test the semantic lexer in the current buffer. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
679 Analyze the area between BEG and END." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
680 (interactive "r") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
681 (let ((result (semantic-lex beg end))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
682 (pop-to-buffer "*Lexer Output*") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
683 (require 'pp) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
684 (erase-buffer) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
685 (insert (pp-to-string result)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
686 (goto-char (point-min)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
687 )) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
688 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
689 (defvar semantic-lex-debug nil |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
690 "When non-nil, debug the local lexical analyzer.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
691 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
692 (defun semantic-lex-debug (arg) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
693 "Debug the semantic lexer in the current buffer. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
694 Argument ARG specifies of the analyze the whole buffer, or start at point. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
695 While engaged, each token identified by the lexer will be highlighted |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
696 in the target buffer A description of the current token will be |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
697 displayed in the minibuffer. Press SPC to move to the next lexical token." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
698 (interactive "P") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
699 (require 'semantic/debug) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
700 (let ((semantic-lex-debug t)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
701 (semantic-lex-test arg))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
702 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
703 (defun semantic-lex-highlight-token (token) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
704 "Highlight the lexical TOKEN. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
705 TOKEN is a lexical token with a START And END position. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
706 Return the overlay." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
707 (let ((o (semantic-make-overlay (semantic-lex-token-start token) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
708 (semantic-lex-token-end token)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
709 (semantic-overlay-put o 'face 'highlight) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
710 o)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
711 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
712 (defsubst semantic-lex-debug-break (token) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
713 "Break during lexical analysis at TOKEN." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
714 (when semantic-lex-debug |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
715 (let ((o nil)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
716 (unwind-protect |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
717 (progn |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
718 (when token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
719 (setq o (semantic-lex-highlight-token token))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
720 (semantic-read-event |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
721 (format "%S :: SPC - continue" token)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
722 ) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
723 (when o |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
724 (semantic-overlay-delete o)))))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
725 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
726 ;;; Lexical analyzer creation |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
727 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
728 ;; Code for creating a lex function from lists of analyzers. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
729 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
730 ;; A lexical analyzer is created from a list of individual analyzers. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
731 ;; Each individual analyzer specifies a single match, and code that |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
732 ;; goes with it. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
733 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
734 ;; Creation of an analyzer assembles these analyzers into a new function |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
735 ;; with the behaviors of all the individual analyzers. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
736 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
737 (defmacro semantic-lex-one-token (analyzers) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
738 "Calculate one token from the current buffer at point. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
739 Uses locally bound variables from `define-lex'. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
740 Argument ANALYZERS is the list of analyzers being used." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
741 (cons 'cond (mapcar #'symbol-value analyzers))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
742 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
743 (defvar semantic-lex-end-point nil |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
744 "The end point as tracked through lexical functions.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
745 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
746 (defvar semantic-lex-current-depth nil |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
747 "The current depth as tracked through lexical functions.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
748 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
749 (defvar semantic-lex-maximum-depth nil |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
750 "The maximum depth of parenthisis as tracked through lexical functions.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
751 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
752 (defvar semantic-lex-token-stream nil |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
753 "The current token stream we are collecting.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
754 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
755 (defvar semantic-lex-analysis-bounds nil |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
756 "The bounds of the current analysis.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
757 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
758 (defvar semantic-lex-block-streams nil |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
759 "Streams of tokens inside collapsed blocks. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
760 This is an alist of (ANCHOR . STREAM) elements where ANCHOR is the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
761 start position of the block, and STREAM is the list of tokens in that |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
762 block.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
763 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
764 (defvar semantic-lex-reset-hooks nil |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
765 "List of hooks major-modes use to reset lexical analyzers. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
766 Hooks are called with START and END values for the current lexical pass. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
767 Should be set with `add-hook'specifying a LOCAL option.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
768 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
769 ;; Stack of nested blocks. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
770 (defvar semantic-lex-block-stack nil) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
771 ;;(defvar semantic-lex-timeout 5 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
772 ;; "*Number of sections of lexing before giving up.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
773 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
774 (defmacro define-lex (name doc &rest analyzers) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
775 "Create a new lexical analyzer with NAME. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
776 DOC is a documentation string describing this analyzer. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
777 ANALYZERS are small code snippets of analyzers to use when |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
778 building the new NAMED analyzer. Only use analyzers which |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
779 are written to be used in `define-lex'. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
780 Each analyzer should be an analyzer created with `define-lex-analyzer'. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
781 Note: The order in which analyzers are listed is important. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
782 If two analyzers can match the same text, it is important to order the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
783 analyzers so that the one you want to match first occurs first. For |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
784 example, it is good to put a numbe analyzer in front of a symbol |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
785 analyzer which might mistake a number for as a symbol." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
786 `(defun ,name (start end &optional depth length) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
787 ,(concat doc "\nSee `semantic-lex' for more information.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
788 ;; Make sure the state of block parsing starts over. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
789 (setq semantic-lex-block-streams nil) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
790 ;; Allow specialty reset items. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
791 (run-hook-with-args 'semantic-lex-reset-hooks start end) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
792 ;; Lexing state. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
793 (let* (;(starttime (current-time)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
794 (starting-position (point)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
795 (semantic-lex-token-stream nil) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
796 (semantic-lex-block-stack nil) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
797 (tmp-start start) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
798 (semantic-lex-end-point start) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
799 (semantic-lex-current-depth 0) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
800 ;; Use the default depth when not specified. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
801 (semantic-lex-maximum-depth |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
802 (or depth semantic-lex-depth)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
803 ;; Bounds needed for unterminated syntax |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
804 (semantic-lex-analysis-bounds (cons start end)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
805 ;; This entry prevents text properties from |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
806 ;; confusing our lexical analysis. See Emacs 22 (CVS) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
807 ;; version of C++ mode with template hack text properties. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
808 (parse-sexp-lookup-properties nil) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
809 ) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
810 ;; Maybe REMOVE THIS LATER. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
811 ;; Trying to find incremental parser bug. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
812 (when (> end (point-max)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
813 (error ,(format "%s: end (%%d) > point-max (%%d)" name) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
814 end (point-max))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
815 (with-syntax-table semantic-lex-syntax-table |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
816 (goto-char start) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
817 (while (and (< (point) end) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
818 (or (not length) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
819 (<= (length semantic-lex-token-stream) length))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
820 (semantic-lex-one-token ,analyzers) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
821 (when (eq semantic-lex-end-point tmp-start) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
822 (error ,(format "%s: endless loop at %%d, after %%S" name) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
823 tmp-start (car semantic-lex-token-stream))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
824 (setq tmp-start semantic-lex-end-point) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
825 (goto-char semantic-lex-end-point) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
826 ;;(when (> (semantic-elapsed-time starttime (current-time)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
827 ;; semantic-lex-timeout) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
828 ;; (error "Timeout during lex at char %d" (point))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
829 (semantic-throw-on-input 'lex) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
830 (semantic-lex-debug-break (car semantic-lex-token-stream)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
831 )) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
832 ;; Check that there is no unterminated block. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
833 (when semantic-lex-block-stack |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
834 (let* ((last (pop semantic-lex-block-stack)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
835 (blk last)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
836 (while blk |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
837 (message |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
838 ,(format "%s: `%%s' block from %%S is unterminated" name) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
839 (car blk) (cadr blk)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
840 (setq blk (pop semantic-lex-block-stack))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
841 (semantic-lex-unterminated-syntax-detected (car last)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
842 ;; Return to where we started. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
843 ;; Do not wrap in protective stuff so that if there is an error |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
844 ;; thrown, the user knows where. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
845 (goto-char starting-position) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
846 ;; Return the token stream |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
847 (nreverse semantic-lex-token-stream)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
848 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
849 ;;; Collapsed block tokens delimited by any tokens. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
850 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
851 (defun semantic-lex-start-block (syntax) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
852 "Mark the last read token as the beginning of a SYNTAX block." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
853 (if (or (not semantic-lex-maximum-depth) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
854 (< semantic-lex-current-depth semantic-lex-maximum-depth)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
855 (setq semantic-lex-current-depth (1+ semantic-lex-current-depth)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
856 (push (list syntax (car semantic-lex-token-stream)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
857 semantic-lex-block-stack))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
858 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
859 (defun semantic-lex-end-block (syntax) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
860 "Process the end of a previously marked SYNTAX block. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
861 That is, collapse the tokens inside that block, including the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
862 beginning and end of block tokens, into a high level block token of |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
863 class SYNTAX. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
864 The token at beginning of block is the one marked by a previous call |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
865 to `semantic-lex-start-block'. The current token is the end of block. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
866 The collapsed tokens are saved in `semantic-lex-block-streams'." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
867 (if (null semantic-lex-block-stack) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
868 (setq semantic-lex-current-depth (1- semantic-lex-current-depth)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
869 (let* ((stream semantic-lex-token-stream) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
870 (blk (pop semantic-lex-block-stack)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
871 (bstream (cdr blk)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
872 (first (car bstream)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
873 (last (pop stream)) ;; The current token mark the EOBLK |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
874 tok) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
875 (if (not (eq (car blk) syntax)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
876 ;; SYNTAX doesn't match the syntax of the current block in |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
877 ;; the stack. So we encountered the end of the SYNTAX block |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
878 ;; before the end of the current one in the stack which is |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
879 ;; signaled unterminated. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
880 (semantic-lex-unterminated-syntax-detected (car blk)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
881 ;; Move tokens found inside the block from the main stream |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
882 ;; into a separate block stream. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
883 (while (and stream (not (eq (setq tok (pop stream)) first))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
884 (push tok bstream)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
885 ;; The token marked as beginning of block was not encountered. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
886 ;; This should not happen! |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
887 (or (eq tok first) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
888 (error "Token %S not found at beginning of block `%s'" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
889 first syntax)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
890 ;; Save the block stream for future reuse, to avoid to redo |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
891 ;; the lexical analysis of the block content! |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
892 ;; Anchor the block stream with its start position, so we can |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
893 ;; use: (cdr (assq start semantic-lex-block-streams)) to |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
894 ;; quickly retrieve the lexical stream associated to a block. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
895 (setcar blk (semantic-lex-token-start first)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
896 (setcdr blk (nreverse bstream)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
897 (push blk semantic-lex-block-streams) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
898 ;; In the main stream, replace the tokens inside the block by |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
899 ;; a high level block token of class SYNTAX. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
900 (setq semantic-lex-token-stream stream) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
901 (semantic-lex-push-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
902 (semantic-lex-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
903 syntax (car blk) (semantic-lex-token-end last))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
904 )))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
905 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
906 ;;; Lexical token API |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
907 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
908 ;; Functions for accessing parts of a token. Use these functions |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
909 ;; instead of accessing the list structure directly because the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
910 ;; contents of the lexical may change. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
911 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
912 (defmacro semantic-lex-token (symbol start end &optional str) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
913 "Create a lexical token. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
914 SYMBOL is a symbol representing the class of syntax found. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
915 START and END define the bounds of the token in the current buffer. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
916 Optional STR is the string for the token iff the the bounds |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
917 in the buffer do not cover the string they represent. (As from |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
918 macro expansion.)" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
919 ;; This if statement checks the existance of a STR argument at |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
920 ;; compile time, where STR is some symbol or constant. If the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
921 ;; variable STr (runtime) is nil, this will make an incorrect decision. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
922 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
923 ;; It is like this to maintain the original speed of the compiled |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
924 ;; code. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
925 (if str |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
926 `(cons ,symbol (cons ,str (cons ,start ,end))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
927 `(cons ,symbol (cons ,start ,end)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
928 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
929 (defun semantic-lex-token-p (thing) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
930 "Return non-nil if THING is a semantic lex token. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
931 This is an exhaustively robust check." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
932 (and (consp thing) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
933 (symbolp (car thing)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
934 (or (and (numberp (nth 1 thing)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
935 (numberp (nthcdr 2 thing))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
936 (and (stringp (nth 1 thing)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
937 (numberp (nth 2 thing)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
938 (numberp (nthcdr 3 thing))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
939 )) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
940 ) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
941 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
942 (defun semantic-lex-token-with-text-p (thing) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
943 "Return non-nil if THING is a semantic lex token. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
944 This is an exhaustively robust check." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
945 (and (consp thing) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
946 (symbolp (car thing)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
947 (= (length thing) 4) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
948 (stringp (nth 1 thing)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
949 (numberp (nth 2 thing)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
950 (numberp (nth 3 thing))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
951 ) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
952 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
953 (defun semantic-lex-token-without-text-p (thing) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
954 "Return non-nil if THING is a semantic lex token. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
955 This is an exhaustively robust check." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
956 (and (consp thing) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
957 (symbolp (car thing)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
958 (= (length thing) 3) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
959 (numberp (nth 1 thing)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
960 (numberp (nth 2 thing))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
961 ) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
962 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
963 (defun semantic-lex-expand-block-specs (specs) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
964 "Expand block specifications SPECS into a Lisp form. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
965 SPECS is a list of (BLOCK BEGIN END) elements where BLOCK, BEGIN, and |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
966 END are token class symbols that indicate to produce one collapsed |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
967 BLOCK token from tokens found between BEGIN and END ones. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
968 BLOCK must be a non-nil symbol, and at least one of the BEGIN or END |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
969 symbols must be non-nil too. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
970 When BEGIN is non-nil, generate a call to `semantic-lex-start-block' |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
971 when a BEGIN token class is encountered. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
972 When END is non-nil, generate a call to `semantic-lex-end-block' when |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
973 an END token class is encountered." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
974 (let ((class (make-symbol "class")) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
975 (form nil)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
976 (dolist (spec specs) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
977 (when (car spec) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
978 (when (nth 1 spec) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
979 (push `((eq ',(nth 1 spec) ,class) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
980 (semantic-lex-start-block ',(car spec))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
981 form)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
982 (when (nth 2 spec) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
983 (push `((eq ',(nth 2 spec) ,class) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
984 (semantic-lex-end-block ',(car spec))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
985 form)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
986 (when form |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
987 `((let ((,class (semantic-lex-token-class |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
988 (car semantic-lex-token-stream)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
989 (cond ,@(nreverse form)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
990 ))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
991 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
992 (defmacro semantic-lex-push-token (token &rest blockspecs) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
993 "Push TOKEN in the lexical analyzer token stream. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
994 Return the lexical analysis current end point. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
995 If optional arguments BLOCKSPECS is non-nil, it specifies to process |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
996 collapsed block tokens. See `semantic-lex-expand-block-specs' for |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
997 more details. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
998 This macro should only be called within the bounds of |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
999 `define-lex-analyzer'. It changes the values of the lexical analyzer |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1000 variables `token-stream' and `semantic-lex-end-point'. If you need to |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1001 move `semantic-lex-end-point' somewhere else, just modify this |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1002 variable after calling `semantic-lex-push-token'." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1003 `(progn |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1004 (push ,token semantic-lex-token-stream) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1005 ,@(semantic-lex-expand-block-specs blockspecs) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1006 (setq semantic-lex-end-point |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1007 (semantic-lex-token-end (car semantic-lex-token-stream))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1008 )) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1009 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1010 (defsubst semantic-lex-token-class (token) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1011 "Fetch the class of the lexical token TOKEN. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1012 See also the function `semantic-lex-token'." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1013 (car token)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1014 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1015 (defsubst semantic-lex-token-bounds (token) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1016 "Fetch the start and end locations of the lexical token TOKEN. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1017 Return a pair (START . END)." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1018 (if (not (numberp (car (cdr token)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1019 (cdr (cdr token)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1020 (cdr token))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1021 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1022 (defsubst semantic-lex-token-start (token) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1023 "Fetch the start position of the lexical token TOKEN. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1024 See also the function `semantic-lex-token'." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1025 (car (semantic-lex-token-bounds token))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1026 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1027 (defsubst semantic-lex-token-end (token) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1028 "Fetch the end position of the lexical token TOKEN. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1029 See also the function `semantic-lex-token'." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1030 (cdr (semantic-lex-token-bounds token))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1031 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1032 (defsubst semantic-lex-token-text (token) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1033 "Fetch the text associated with the lexical token TOKEN. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1034 See also the function `semantic-lex-token'." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1035 (if (stringp (car (cdr token))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1036 (car (cdr token)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1037 (buffer-substring-no-properties |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1038 (semantic-lex-token-start token) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1039 (semantic-lex-token-end token)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1040 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1041 (defun semantic-lex-init () |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1042 "Initialize any lexical state for this buffer." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1043 (unless semantic-lex-comment-regex |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1044 (setq semantic-lex-comment-regex |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1045 (if comment-start-skip |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1046 (concat "\\(\\s<\\|" comment-start-skip "\\)") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1047 "\\(\\s<\\)"))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1048 ;; Setup the lexer syntax-table |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1049 (setq semantic-lex-syntax-table (copy-syntax-table (syntax-table))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1050 (dolist (mod semantic-lex-syntax-modifications) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1051 (modify-syntax-entry |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1052 (car mod) (nth 1 mod) semantic-lex-syntax-table))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1053 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1054 (define-overloadable-function semantic-lex (start end &optional depth length) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1055 "Lexically analyze text in the current buffer between START and END. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1056 Optional argument DEPTH indicates at what level to scan over entire |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1057 lists. The last argument, LENGTH specifies that `semantic-lex' |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1058 should only return LENGTH tokens. The return value is a token stream. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1059 Each element is a list, such of the form |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1060 (symbol start-expression . end-expression) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1061 where SYMBOL denotes the token type. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1062 See `semantic-lex-tokens' variable for details on token types. END |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1063 does not mark the end of the text scanned, only the end of the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1064 beginning of text scanned. Thus, if a string extends past END, the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1065 end of the return token will be larger than END. To truly restrict |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1066 scanning, use `narrow-to-region'." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1067 (funcall semantic-lex-analyzer start end depth length)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1068 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1069 (defsubst semantic-lex-buffer (&optional depth) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1070 "Lex the current buffer. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1071 Optional argument DEPTH is the depth to scan into lists." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1072 (semantic-lex (point-min) (point-max) depth)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1073 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1074 (defsubst semantic-lex-list (semlist depth) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1075 "Lex the body of SEMLIST to DEPTH." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1076 (semantic-lex (semantic-lex-token-start semlist) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1077 (semantic-lex-token-end semlist) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1078 depth)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1079 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1080 ;;; Analyzer creation macros |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1081 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1082 ;; An individual analyzer is a condition and code that goes with it. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1083 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1084 ;; Created analyzers become variables with the code associated with them |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1085 ;; as the symbol value. These analyzers are assembled into a lexer |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1086 ;; to create new lexical analyzers. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1087 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1088 (defsubst semantic-lex-unterminated-syntax-detected (syntax) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1089 "Inside a lexical analyzer, use this when unterminated syntax was found. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1090 Argument SYNTAX indicates the type of syntax that is unterminated. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1091 The job of this function is to move (point) to a new logical location |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1092 so that analysis can continue, if possible." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1093 (goto-char |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1094 (funcall semantic-lex-unterminated-syntax-end-function |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1095 syntax |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1096 (car semantic-lex-analysis-bounds) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1097 (cdr semantic-lex-analysis-bounds) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1098 )) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1099 (setq semantic-lex-end-point (point))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1100 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1101 (defcustom semantic-lex-debug-analyzers nil |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1102 "Non nil means to debug analyzers with syntax protection. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1103 Only in effect if `debug-on-error' is also non-nil." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1104 :group 'semantic |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1105 :type 'boolean) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1106 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1107 (defmacro semantic-lex-unterminated-syntax-protection (syntax &rest forms) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1108 "For SYNTAX, execute FORMS with protection for unterminated syntax. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1109 If FORMS throws an error, treat this as a syntax problem, and |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1110 execute the unterminated syntax code. FORMS should return a position. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1111 Irreguardless of an error, the cursor should be moved to the end of |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1112 the desired syntax, and a position returned. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1113 If `debug-on-error' is set, errors are not caught, so that you can |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1114 debug them. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1115 Avoid using a large FORMS since it is duplicated." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1116 `(if (and debug-on-error semantic-lex-debug-analyzers) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1117 (progn ,@forms) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1118 (condition-case nil |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1119 (progn ,@forms) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1120 (error |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1121 (semantic-lex-unterminated-syntax-detected ,syntax))))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1122 (put 'semantic-lex-unterminated-syntax-protection |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1123 'lisp-indent-function 1) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1124 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1125 (defmacro define-lex-analyzer (name doc condition &rest forms) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1126 "Create a single lexical analyzer NAME with DOC. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1127 When an analyzer is called, the current buffer and point are |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1128 positioned in a buffer at the location to be analyzed. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1129 CONDITION is an expression which returns t if FORMS should be run. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1130 Within the bounds of CONDITION and FORMS, the use of backquote |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1131 can be used to evaluate expressions at compile time. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1132 While forms are running, the following variables will be locally bound: |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1133 `semantic-lex-analysis-bounds' - The bounds of the current analysis. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1134 of the form (START . END) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1135 `semantic-lex-maximum-depth' - The maximum depth of semantic-list |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1136 for the current analysis. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1137 `semantic-lex-current-depth' - The current depth of `semantic-list' that has |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1138 been decended. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1139 `semantic-lex-end-point' - End Point after match. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1140 Analyzers should set this to a buffer location if their |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1141 match string does not represent the end of the matched text. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1142 `semantic-lex-token-stream' - The token list being collected. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1143 Add new lexical tokens to this list. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1144 Proper action in FORMS is to move the value of `semantic-lex-end-point' to |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1145 after the location of the analyzed entry, and to add any discovered tokens |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1146 at the beginning of `semantic-lex-token-stream'. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1147 This can be done by using `semantic-lex-push-token'." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1148 `(eval-and-compile |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1149 (defvar ,name nil ,doc) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1150 (defun ,name nil) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1151 ;; Do this part separately so that re-evaluation rebuilds this code. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1152 (setq ,name '(,condition ,@forms)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1153 ;; Build a single lexical analyzer function, so the doc for |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1154 ;; function help is automatically provided, and perhaps the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1155 ;; function could be useful for testing and debugging one |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1156 ;; analyzer. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1157 (fset ',name (lambda () ,doc |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1158 (let ((semantic-lex-token-stream nil) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1159 (semantic-lex-end-point (point)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1160 (semantic-lex-analysis-bounds |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1161 (cons (point) (point-max))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1162 (semantic-lex-current-depth 0) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1163 (semantic-lex-maximum-depth |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1164 semantic-lex-depth) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1165 ) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1166 (when ,condition ,@forms) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1167 semantic-lex-token-stream))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1168 )) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1169 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1170 (defmacro define-lex-regex-analyzer (name doc regexp &rest forms) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1171 "Create a lexical analyzer with NAME and DOC that will match REGEXP. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1172 FORMS are evaluated upon a successful match. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1173 See `define-lex-analyzer' for more about analyzers." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1174 `(define-lex-analyzer ,name |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1175 ,doc |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1176 (looking-at ,regexp) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1177 ,@forms |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1178 )) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1179 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1180 (defmacro define-lex-simple-regex-analyzer (name doc regexp toksym |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1181 &optional index |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1182 &rest forms) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1183 "Create a lexical analyzer with NAME and DOC that match REGEXP. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1184 TOKSYM is the symbol to use when creating a semantic lexical token. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1185 INDEX is the index into the match that defines the bounds of the token. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1186 Index should be a plain integer, and not specified in the macro as an |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1187 expression. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1188 FORMS are evaluated upon a successful match BEFORE the new token is |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1189 created. It is valid to ignore FORMS. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1190 See `define-lex-analyzer' for more about analyzers." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1191 `(define-lex-analyzer ,name |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1192 ,doc |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1193 (looking-at ,regexp) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1194 ,@forms |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1195 (semantic-lex-push-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1196 (semantic-lex-token ,toksym |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1197 (match-beginning ,(or index 0)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1198 (match-end ,(or index 0)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1199 )) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1200 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1201 (defmacro define-lex-block-analyzer (name doc spec1 &rest specs) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1202 "Create a lexical analyzer NAME for paired delimiters blocks. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1203 It detects a paired delimiters block or the corresponding open or |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1204 close delimiter depending on the value of the variable |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1205 `semantic-lex-current-depth'. DOC is the documentation string of the lexical |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1206 analyzer. SPEC1 and SPECS specify the token symbols and open, close |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1207 delimiters used. Each SPEC has the form: |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1208 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1209 \(BLOCK-SYM (OPEN-DELIM OPEN-SYM) (CLOSE-DELIM CLOSE-SYM)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1210 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1211 where BLOCK-SYM is the symbol returned in a block token. OPEN-DELIM |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1212 and CLOSE-DELIM are respectively the open and close delimiters |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1213 identifying a block. OPEN-SYM and CLOSE-SYM are respectively the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1214 symbols returned in open and close tokens." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1215 (let ((specs (cons spec1 specs)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1216 spec open olist clist) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1217 (while specs |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1218 (setq spec (car specs) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1219 specs (cdr specs) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1220 open (nth 1 spec) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1221 ;; build alist ((OPEN-DELIM OPEN-SYM BLOCK-SYM) ...) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1222 olist (cons (list (car open) (cadr open) (car spec)) olist) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1223 ;; build alist ((CLOSE-DELIM CLOSE-SYM) ...) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1224 clist (cons (nth 2 spec) clist))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1225 `(define-lex-analyzer ,name |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1226 ,doc |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1227 (and |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1228 (looking-at "\\(\\s(\\|\\s)\\)") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1229 (let ((text (match-string 0)) match) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1230 (cond |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1231 ((setq match (assoc text ',olist)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1232 (if (or (not semantic-lex-maximum-depth) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1233 (< semantic-lex-current-depth semantic-lex-maximum-depth)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1234 (progn |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1235 (setq semantic-lex-current-depth (1+ semantic-lex-current-depth)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1236 (semantic-lex-push-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1237 (semantic-lex-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1238 (nth 1 match) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1239 (match-beginning 0) (match-end 0)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1240 (semantic-lex-push-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1241 (semantic-lex-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1242 (nth 2 match) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1243 (match-beginning 0) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1244 (save-excursion |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1245 (semantic-lex-unterminated-syntax-protection (nth 2 match) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1246 (forward-list 1) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1247 (point))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1248 )) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1249 )) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1250 ((setq match (assoc text ',clist)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1251 (setq semantic-lex-current-depth (1- semantic-lex-current-depth)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1252 (semantic-lex-push-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1253 (semantic-lex-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1254 (nth 1 match) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1255 (match-beginning 0) (match-end 0))))))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1256 ))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1257 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1258 ;;; Analyzers |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1259 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1260 ;; Pre-defined common analyzers. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1261 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1262 (define-lex-analyzer semantic-lex-default-action |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1263 "The default action when no other lexical actions match text. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1264 This action will just throw an error." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1265 t |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1266 (error "Unmatched Text during Lexical Analysis")) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1267 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1268 (define-lex-analyzer semantic-lex-beginning-of-line |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1269 "Detect and create a beginning of line token (BOL)." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1270 (and (bolp) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1271 ;; Just insert a (bol N . N) token in the token stream, |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1272 ;; without moving the point. N is the point at the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1273 ;; beginning of line. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1274 (semantic-lex-push-token (semantic-lex-token 'bol (point) (point))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1275 nil) ;; CONTINUE |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1276 ;; We identify and add the BOL token onto the stream, but since |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1277 ;; semantic-lex-end-point doesn't move, we always fail CONDITION, and have no |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1278 ;; FORMS body. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1279 nil) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1280 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1281 (define-lex-simple-regex-analyzer semantic-lex-newline |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1282 "Detect and create newline tokens." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1283 "\\s-*\\(\n\\|\\s>\\)" 'newline 1) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1284 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1285 (define-lex-regex-analyzer semantic-lex-newline-as-whitespace |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1286 "Detect and create newline tokens. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1287 Use this ONLY if newlines are not whitespace characters (such as when |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1288 they are comment end characters) AND when you want whitespace tokens." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1289 "\\s-*\\(\n\\|\\s>\\)" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1290 ;; Language wants whitespaces. Create a token for it. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1291 (if (eq (semantic-lex-token-class (car semantic-lex-token-stream)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1292 'whitespace) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1293 ;; Merge whitespace tokens together if they are adjacent. Two |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1294 ;; whitespace tokens may be sperated by a comment which is not in |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1295 ;; the token stream. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1296 (setcdr (semantic-lex-token-bounds (car semantic-lex-token-stream)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1297 (match-end 0)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1298 (semantic-lex-push-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1299 (semantic-lex-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1300 'whitespace (match-beginning 0) (match-end 0))))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1301 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1302 (define-lex-regex-analyzer semantic-lex-ignore-newline |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1303 "Detect and ignore newline tokens. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1304 Use this ONLY if newlines are not whitespace characters (such as when |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1305 they are comment end characters)." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1306 "\\s-*\\(\n\\|\\s>\\)" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1307 (setq semantic-lex-end-point (match-end 0))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1308 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1309 (define-lex-regex-analyzer semantic-lex-whitespace |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1310 "Detect and create whitespace tokens." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1311 ;; catch whitespace when needed |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1312 "\\s-+" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1313 ;; Language wants whitespaces. Create a token for it. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1314 (if (eq (semantic-lex-token-class (car semantic-lex-token-stream)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1315 'whitespace) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1316 ;; Merge whitespace tokens together if they are adjacent. Two |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1317 ;; whitespace tokens may be sperated by a comment which is not in |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1318 ;; the token stream. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1319 (progn |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1320 (setq semantic-lex-end-point (match-end 0)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1321 (setcdr (semantic-lex-token-bounds (car semantic-lex-token-stream)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1322 semantic-lex-end-point)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1323 (semantic-lex-push-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1324 (semantic-lex-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1325 'whitespace (match-beginning 0) (match-end 0))))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1326 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1327 (define-lex-regex-analyzer semantic-lex-ignore-whitespace |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1328 "Detect and skip over whitespace tokens." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1329 ;; catch whitespace when needed |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1330 "\\s-+" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1331 ;; Skip over the detected whitespace, do not create a token for it. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1332 (setq semantic-lex-end-point (match-end 0))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1333 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1334 (define-lex-simple-regex-analyzer semantic-lex-number |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1335 "Detect and create number tokens. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1336 See `semantic-lex-number-expression' for details on matching numbers, |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1337 and number formats." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1338 semantic-lex-number-expression 'number) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1339 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1340 (define-lex-regex-analyzer semantic-lex-symbol-or-keyword |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1341 "Detect and create symbol and keyword tokens." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1342 "\\(\\sw\\|\\s_\\)+" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1343 (semantic-lex-push-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1344 (semantic-lex-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1345 (or (semantic-lex-keyword-p (match-string 0)) 'symbol) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1346 (match-beginning 0) (match-end 0)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1347 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1348 (define-lex-simple-regex-analyzer semantic-lex-charquote |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1349 "Detect and create charquote tokens." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1350 ;; Character quoting characters (ie, \n as newline) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1351 "\\s\\+" 'charquote) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1352 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1353 (define-lex-simple-regex-analyzer semantic-lex-punctuation |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1354 "Detect and create punctuation tokens." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1355 "\\(\\s.\\|\\s$\\|\\s'\\)" 'punctuation) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1356 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1357 (define-lex-analyzer semantic-lex-punctuation-type |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1358 "Detect and create a punctuation type token. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1359 Recognized punctuations are defined in the current table of lexical |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1360 types, as the value of the `punctuation' token type." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1361 (and (looking-at "\\(\\s.\\|\\s$\\|\\s'\\)+") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1362 (let* ((key (match-string 0)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1363 (pos (match-beginning 0)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1364 (end (match-end 0)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1365 (len (- end pos)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1366 (lst (semantic-lex-type-value "punctuation" t)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1367 (def (car lst)) ;; default lexical symbol or nil |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1368 (lst (cdr lst)) ;; alist of (LEX-SYM . PUNCT-STRING) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1369 (elt nil)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1370 (if lst |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1371 ;; Starting with the longest one, search if the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1372 ;; punctuation string is defined for this language. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1373 (while (and (> len 0) (not (setq elt (rassoc key lst)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1374 (setq len (1- len) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1375 key (substring key 0 len)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1376 (if elt ;; Return the punctuation token found |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1377 (semantic-lex-push-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1378 (semantic-lex-token (car elt) pos (+ pos len))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1379 (if def ;; Return a default generic token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1380 (semantic-lex-push-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1381 (semantic-lex-token def pos end)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1382 ;; Nothing match |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1383 ))))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1384 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1385 (define-lex-regex-analyzer semantic-lex-paren-or-list |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1386 "Detect open parenthesis. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1387 Return either a paren token or a semantic list token depending on |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1388 `semantic-lex-current-depth'." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1389 "\\s(" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1390 (if (or (not semantic-lex-maximum-depth) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1391 (< semantic-lex-current-depth semantic-lex-maximum-depth)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1392 (progn |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1393 (setq semantic-lex-current-depth (1+ semantic-lex-current-depth)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1394 (semantic-lex-push-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1395 (semantic-lex-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1396 'open-paren (match-beginning 0) (match-end 0)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1397 (semantic-lex-push-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1398 (semantic-lex-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1399 'semantic-list (match-beginning 0) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1400 (save-excursion |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1401 (semantic-lex-unterminated-syntax-protection 'semantic-list |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1402 (forward-list 1) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1403 (point)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1404 ))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1405 )) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1406 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1407 (define-lex-simple-regex-analyzer semantic-lex-open-paren |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1408 "Detect and create an open parenthisis token." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1409 "\\s(" 'open-paren 0 (setq semantic-lex-current-depth (1+ semantic-lex-current-depth))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1410 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1411 (define-lex-simple-regex-analyzer semantic-lex-close-paren |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1412 "Detect and create a close paren token." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1413 "\\s)" 'close-paren 0 (setq semantic-lex-current-depth (1- semantic-lex-current-depth))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1414 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1415 (define-lex-regex-analyzer semantic-lex-string |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1416 "Detect and create a string token." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1417 "\\s\"" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1418 ;; Zing to the end of this string. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1419 (semantic-lex-push-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1420 (semantic-lex-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1421 'string (point) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1422 (save-excursion |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1423 (semantic-lex-unterminated-syntax-protection 'string |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1424 (forward-sexp 1) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1425 (point)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1426 )))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1427 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1428 (define-lex-regex-analyzer semantic-lex-comments |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1429 "Detect and create a comment token." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1430 semantic-lex-comment-regex |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1431 (save-excursion |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1432 (forward-comment 1) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1433 ;; Generate newline token if enabled |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1434 (if (bolp) (backward-char 1)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1435 (setq semantic-lex-end-point (point)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1436 ;; Language wants comments or want them as whitespaces, |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1437 ;; link them together. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1438 (if (eq (semantic-lex-token-class (car semantic-lex-token-stream)) 'comment) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1439 (setcdr (semantic-lex-token-bounds (car semantic-lex-token-stream)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1440 semantic-lex-end-point) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1441 (semantic-lex-push-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1442 (semantic-lex-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1443 'comment (match-beginning 0) semantic-lex-end-point))))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1444 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1445 (define-lex-regex-analyzer semantic-lex-comments-as-whitespace |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1446 "Detect comments and create a whitespace token." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1447 semantic-lex-comment-regex |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1448 (save-excursion |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1449 (forward-comment 1) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1450 ;; Generate newline token if enabled |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1451 (if (bolp) (backward-char 1)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1452 (setq semantic-lex-end-point (point)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1453 ;; Language wants comments or want them as whitespaces, |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1454 ;; link them together. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1455 (if (eq (semantic-lex-token-class (car semantic-lex-token-stream)) 'whitespace) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1456 (setcdr (semantic-lex-token-bounds (car semantic-lex-token-stream)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1457 semantic-lex-end-point) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1458 (semantic-lex-push-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1459 (semantic-lex-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1460 'whitespace (match-beginning 0) semantic-lex-end-point))))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1461 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1462 (define-lex-regex-analyzer semantic-lex-ignore-comments |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1463 "Detect and create a comment token." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1464 semantic-lex-comment-regex |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1465 (let ((comment-start-point (point))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1466 (forward-comment 1) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1467 (if (eq (point) comment-start-point) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1468 ;; In this case our start-skip string failed |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1469 ;; to work properly. Lets try and move over |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1470 ;; whatever white space we matched to begin |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1471 ;; with. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1472 (skip-syntax-forward "-.'" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1473 (save-excursion |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1474 (end-of-line) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1475 (point))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1476 ;; We may need to back up so newlines or whitespace is generated. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1477 (if (bolp) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1478 (backward-char 1))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1479 (if (eq (point) comment-start-point) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1480 (error "Strange comment syntax prevents lexical analysis")) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1481 (setq semantic-lex-end-point (point)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1482 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1483 ;;; Comment lexer |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1484 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1485 ;; Predefined lexers that could be used instead of creating new |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1486 ;; analyers. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1487 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1488 (define-lex semantic-comment-lexer |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1489 "A simple lexical analyzer that handles comments. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1490 This lexer will only return comment tokens. It is the default lexer |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1491 used by `semantic-find-doc-snarf-comment' to snarf up the comment at |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1492 point." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1493 semantic-lex-ignore-whitespace |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1494 semantic-lex-ignore-newline |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1495 semantic-lex-comments |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1496 semantic-lex-default-action) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1497 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1498 ;;; Test Lexer |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1499 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1500 (define-lex semantic-simple-lexer |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1501 "A simple lexical analyzer that handles simple buffers. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1502 This lexer ignores comments and whitespace, and will return |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1503 syntax as specified by the syntax table." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1504 semantic-lex-ignore-whitespace |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1505 semantic-lex-ignore-newline |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1506 semantic-lex-number |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1507 semantic-lex-symbol-or-keyword |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1508 semantic-lex-charquote |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1509 semantic-lex-paren-or-list |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1510 semantic-lex-close-paren |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1511 semantic-lex-string |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1512 semantic-lex-ignore-comments |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1513 semantic-lex-punctuation |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1514 semantic-lex-default-action) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1515 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1516 ;;; Analyzers generated from grammar. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1517 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1518 ;; Some analyzers are hand written. Analyzers created with these |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1519 ;; functions are generated from the grammar files. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1520 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1521 (defmacro define-lex-keyword-type-analyzer (name doc syntax) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1522 "Define a keyword type analyzer NAME with DOC string. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1523 SYNTAX is the regexp that matches a keyword syntactic expression." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1524 (let ((key (make-symbol "key"))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1525 `(define-lex-analyzer ,name |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1526 ,doc |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1527 (and (looking-at ,syntax) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1528 (let ((,key (semantic-lex-keyword-p (match-string 0)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1529 (when ,key |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1530 (semantic-lex-push-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1531 (semantic-lex-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1532 ,key (match-beginning 0) (match-end 0))))))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1533 )) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1534 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1535 (defmacro define-lex-sexp-type-analyzer (name doc syntax token) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1536 "Define a sexp type analyzer NAME with DOC string. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1537 SYNTAX is the regexp that matches the beginning of the s-expression. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1538 TOKEN is the lexical token returned when SYNTAX matches." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1539 `(define-lex-regex-analyzer ,name |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1540 ,doc |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1541 ,syntax |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1542 (semantic-lex-push-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1543 (semantic-lex-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1544 ,token (point) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1545 (save-excursion |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1546 (semantic-lex-unterminated-syntax-protection ,token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1547 (forward-sexp 1) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1548 (point)))))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1549 ) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1550 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1551 (defmacro define-lex-regex-type-analyzer (name doc syntax matches default) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1552 "Define a regexp type analyzer NAME with DOC string. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1553 SYNTAX is the regexp that matches a syntactic expression. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1554 MATCHES is an alist of lexical elements used to refine the syntactic |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1555 expression. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1556 DEFAULT is the default lexical token returned when no MATCHES." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1557 (if matches |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1558 (let* ((val (make-symbol "val")) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1559 (lst (make-symbol "lst")) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1560 (elt (make-symbol "elt")) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1561 (pos (make-symbol "pos")) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1562 (end (make-symbol "end"))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1563 `(define-lex-analyzer ,name |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1564 ,doc |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1565 (and (looking-at ,syntax) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1566 (let* ((,val (match-string 0)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1567 (,pos (match-beginning 0)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1568 (,end (match-end 0)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1569 (,lst ,matches) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1570 ,elt) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1571 (while (and ,lst (not ,elt)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1572 (if (string-match (cdar ,lst) ,val) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1573 (setq ,elt (caar ,lst)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1574 (setq ,lst (cdr ,lst)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1575 (semantic-lex-push-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1576 (semantic-lex-token (or ,elt ,default) ,pos ,end)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1577 )) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1578 `(define-lex-simple-regex-analyzer ,name |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1579 ,doc |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1580 ,syntax ,default) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1581 )) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1582 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1583 (defmacro define-lex-string-type-analyzer (name doc syntax matches default) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1584 "Define a string type analyzer NAME with DOC string. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1585 SYNTAX is the regexp that matches a syntactic expression. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1586 MATCHES is an alist of lexical elements used to refine the syntactic |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1587 expression. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1588 DEFAULT is the default lexical token returned when no MATCHES." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1589 (if matches |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1590 (let* ((val (make-symbol "val")) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1591 (lst (make-symbol "lst")) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1592 (elt (make-symbol "elt")) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1593 (pos (make-symbol "pos")) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1594 (end (make-symbol "end")) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1595 (len (make-symbol "len"))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1596 `(define-lex-analyzer ,name |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1597 ,doc |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1598 (and (looking-at ,syntax) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1599 (let* ((,val (match-string 0)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1600 (,pos (match-beginning 0)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1601 (,end (match-end 0)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1602 (,len (- ,end ,pos)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1603 (,lst ,matches) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1604 ,elt) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1605 ;; Starting with the longest one, search if a lexical |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1606 ;; value match a token defined for this language. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1607 (while (and (> ,len 0) (not (setq ,elt (rassoc ,val ,lst)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1608 (setq ,len (1- ,len) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1609 ,val (substring ,val 0 ,len))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1610 (when ,elt ;; Adjust token end position. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1611 (setq ,elt (car ,elt) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1612 ,end (+ ,pos ,len))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1613 (semantic-lex-push-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1614 (semantic-lex-token (or ,elt ,default) ,pos ,end)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1615 )) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1616 `(define-lex-simple-regex-analyzer ,name |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1617 ,doc |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1618 ,syntax ,default) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1619 )) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1620 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1621 (defmacro define-lex-block-type-analyzer (name doc syntax matches) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1622 "Define a block type analyzer NAME with DOC string. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1623 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1624 SYNTAX is the regexp that matches block delimiters, typically the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1625 open (`\\\\s(') and close (`\\\\s)') parenthesis syntax classes. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1626 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1627 MATCHES is a pair (OPEN-SPECS . CLOSE-SPECS) that defines blocks. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1628 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1629 OPEN-SPECS is a list of (OPEN-DELIM OPEN-TOKEN BLOCK-TOKEN) elements |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1630 where: |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1631 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1632 OPEN-DELIM is a string: the block open delimiter character. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1633 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1634 OPEN-TOKEN is the lexical token class associated to the OPEN-DELIM |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1635 delimiter. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1636 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1637 BLOCK-TOKEN is the lexical token class associated to the block |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1638 that starts at the OPEN-DELIM delimiter. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1639 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1640 CLOSE-SPECS is a list of (CLOSE-DELIM CLOSE-TOKEN) elements where: |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1641 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1642 CLOSE-DELIM is a string: the block end delimiter character. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1643 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1644 CLOSE-TOKEN is the lexical token class associated to the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1645 CLOSE-DELIM delimiter. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1646 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1647 Each element in OPEN-SPECS must have a corresponding element in |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1648 CLOSE-SPECS. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1649 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1650 The lexer will return a BLOCK-TOKEN token when the value of |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1651 `semantic-lex-current-depth' is greater than or equal to the maximum |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1652 depth of parenthesis tracking (see also the function `semantic-lex'). |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1653 Otherwise it will return OPEN-TOKEN and CLOSE-TOKEN tokens. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1654 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1655 TO DO: Put the following in the developer's guide and just put a |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1656 reference here. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1657 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1658 In the grammar: |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1659 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1660 The value of a block token must be a string that contains a readable |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1661 sexp of the form: |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1662 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1663 \"(OPEN-TOKEN CLOSE-TOKEN)\" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1664 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1665 OPEN-TOKEN and CLOSE-TOKEN represent the block delimiters, and must be |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1666 lexical tokens of respectively `open-paren' and `close-paren' types. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1667 Their value is the corresponding delimiter character as a string. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1668 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1669 Here is a small example to analyze a parenthesis block: |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1670 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1671 %token <block> PAREN_BLOCK \"(LPAREN RPAREN)\" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1672 %token <open-paren> LPAREN \"(\" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1673 %token <close-paren> RPAREN \")\" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1674 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1675 When the lexer encounters the open-paren delimiter \"(\": |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1676 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1677 - If the maximum depth of parenthesis tracking is not reached (that |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1678 is, current depth < max depth), it returns a (LPAREN start . end) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1679 token, then continue analysis inside the block. Later, when the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1680 corresponding close-paren delimiter \")\" will be encountered, it |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1681 will return a (RPAREN start . end) token. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1682 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1683 - If the maximum depth of parenthesis tracking is reached (current |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1684 depth >= max depth), it returns the whole parenthesis block as |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1685 a (PAREN_BLOCK start . end) token." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1686 (let* ((val (make-symbol "val")) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1687 (lst (make-symbol "lst")) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1688 (elt (make-symbol "elt"))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1689 `(define-lex-analyzer ,name |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1690 ,doc |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1691 (and |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1692 (looking-at ,syntax) ;; "\\(\\s(\\|\\s)\\)" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1693 (let ((,val (match-string 0)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1694 (,lst ,matches) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1695 ,elt) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1696 (cond |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1697 ((setq ,elt (assoc ,val (car ,lst))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1698 (if (or (not semantic-lex-maximum-depth) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1699 (< semantic-lex-current-depth semantic-lex-maximum-depth)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1700 (progn |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1701 (setq semantic-lex-current-depth (1+ semantic-lex-current-depth)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1702 (semantic-lex-push-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1703 (semantic-lex-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1704 (nth 1 ,elt) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1705 (match-beginning 0) (match-end 0)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1706 (semantic-lex-push-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1707 (semantic-lex-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1708 (nth 2 ,elt) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1709 (match-beginning 0) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1710 (save-excursion |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1711 (semantic-lex-unterminated-syntax-protection (nth 2 ,elt) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1712 (forward-list 1) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1713 (point))))))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1714 ((setq ,elt (assoc ,val (cdr ,lst))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1715 (setq semantic-lex-current-depth (1- semantic-lex-current-depth)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1716 (semantic-lex-push-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1717 (semantic-lex-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1718 (nth 1 ,elt) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1719 (match-beginning 0) (match-end 0)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1720 )))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1721 )) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1722 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1723 ;;; Lexical Safety |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1724 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1725 ;; The semantic lexers, unlike other lexers, can throw errors on |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1726 ;; unbalanced syntax. Since editing is all about changeging test |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1727 ;; we need to provide a convenient way to protect against syntactic |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1728 ;; inequalities. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1729 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1730 (defmacro semantic-lex-catch-errors (symbol &rest forms) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1731 "Using SYMBOL, execute FORMS catching lexical errors. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1732 If FORMS results in a call to the parser that throws a lexical error, |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1733 the error will be caught here without the buffer's cache being thrown |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1734 out of date. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1735 If there is an error, the syntax that failed is returned. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1736 If there is no error, then the last value of FORMS is returned." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1737 (let ((ret (make-symbol "ret")) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1738 (syntax (make-symbol "syntax")) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1739 (start (make-symbol "start")) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1740 (end (make-symbol "end"))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1741 `(let* ((semantic-lex-unterminated-syntax-end-function |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1742 (lambda (,syntax ,start ,end) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1743 (throw ',symbol ,syntax))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1744 ;; Delete the below when semantic-flex is fully retired. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1745 (semantic-flex-unterminated-syntax-end-function |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1746 semantic-lex-unterminated-syntax-end-function) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1747 (,ret (catch ',symbol |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1748 (save-excursion |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1749 ,@forms |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1750 nil)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1751 ;; Great Sadness. Assume that FORMS execute within the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1752 ;; confines of the current buffer only! Mark this thing |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1753 ;; unparseable iff the special symbol was thrown. This |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1754 ;; will prevent future calls from parsing, but will allow |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1755 ;; then to still return the cache. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1756 (when ,ret |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1757 ;; Leave this message off. If an APP using this fcn wants |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1758 ;; a message, they can do it themselves. This cleans up |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1759 ;; problems with the idle scheduler obscuring useful data. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1760 ;;(message "Buffer not currently parsable (%S)." ,ret) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1761 (semantic-parse-tree-unparseable)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1762 ,ret))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1763 (put 'semantic-lex-catch-errors 'lisp-indent-function 1) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1764 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1765 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1766 ;;; Interfacing with edebug |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1767 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1768 (add-hook |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1769 'edebug-setup-hook |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1770 #'(lambda () |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1771 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1772 (def-edebug-spec define-lex |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1773 (&define name stringp (&rest symbolp)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1774 ) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1775 (def-edebug-spec define-lex-analyzer |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1776 (&define name stringp form def-body) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1777 ) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1778 (def-edebug-spec define-lex-regex-analyzer |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1779 (&define name stringp form def-body) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1780 ) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1781 (def-edebug-spec define-lex-simple-regex-analyzer |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1782 (&define name stringp form symbolp [ &optional form ] def-body) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1783 ) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1784 (def-edebug-spec define-lex-block-analyzer |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1785 (&define name stringp form (&rest form)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1786 ) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1787 (def-edebug-spec semantic-lex-catch-errors |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1788 (symbolp def-body) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1789 ) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1790 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1791 )) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1792 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1793 ;;; Compatibility with Semantic 1.x lexical analysis |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1794 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1795 ;; NOTE: DELETE THIS SOMEDAY SOON |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1796 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1797 (semantic-alias-obsolete 'semantic-flex-start 'semantic-lex-token-start) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1798 (semantic-alias-obsolete 'semantic-flex-end 'semantic-lex-token-end) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1799 (semantic-alias-obsolete 'semantic-flex-text 'semantic-lex-token-text) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1800 (semantic-alias-obsolete 'semantic-flex-make-keyword-table 'semantic-lex-make-keyword-table) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1801 (semantic-alias-obsolete 'semantic-flex-keyword-p 'semantic-lex-keyword-p) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1802 (semantic-alias-obsolete 'semantic-flex-keyword-put 'semantic-lex-keyword-put) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1803 (semantic-alias-obsolete 'semantic-flex-keyword-get 'semantic-lex-keyword-get) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1804 (semantic-alias-obsolete 'semantic-flex-map-keywords 'semantic-lex-map-keywords) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1805 (semantic-alias-obsolete 'semantic-flex-keywords 'semantic-lex-keywords) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1806 (semantic-alias-obsolete 'semantic-flex-buffer 'semantic-lex-buffer) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1807 (semantic-alias-obsolete 'semantic-flex-list 'semantic-lex-list) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1808 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1809 ;; This simple scanner uses the syntax table to generate a stream of |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1810 ;; simple tokens of the form: |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1811 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1812 ;; (SYMBOL START . END) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1813 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1814 ;; Where symbol is the type of thing it is. START and END mark that |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1815 ;; objects boundary. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1816 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1817 (defvar semantic-flex-tokens semantic-lex-tokens |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1818 "An alist of of semantic token types. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1819 See variable `semantic-lex-tokens'.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1820 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1821 (defvar semantic-flex-unterminated-syntax-end-function |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1822 (lambda (syntax syntax-start flex-end) flex-end) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1823 "Function called when unterminated syntax is encountered. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1824 This should be set to one function. That function should take three |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1825 parameters. The SYNTAX, or type of syntax which is unterminated. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1826 SYNTAX-START where the broken syntax begins. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1827 FLEX-END is where the lexical analysis was asked to end. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1828 This function can be used for languages that can intelligently fix up |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1829 broken syntax, or the exit lexical analysis via `throw' or `signal' |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1830 when finding unterminated syntax.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1831 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1832 (defvar semantic-flex-extensions nil |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1833 "Buffer local extensions to the lexical analyzer. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1834 This should contain an alist with a key of a regex and a data element of |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1835 a function. The function should both move point, and return a lexical |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1836 token of the form: |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1837 ( TYPE START . END) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1838 nil is also a valid return value. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1839 TYPE can be any type of symbol, as long as it doesn't occur as a |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1840 nonterminal in the language definition.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1841 (make-variable-buffer-local 'semantic-flex-extensions) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1842 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1843 (defvar semantic-flex-syntax-modifications nil |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1844 "Changes to the syntax table for this buffer. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1845 These changes are active only while the buffer is being flexed. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1846 This is a list where each element has the form: |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1847 (CHAR CLASS) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1848 CHAR is the char passed to `modify-syntax-entry', |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1849 and CLASS is the string also passed to `modify-syntax-entry' to define |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1850 what syntax class CHAR has.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1851 (make-variable-buffer-local 'semantic-flex-syntax-modifications) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1852 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1853 (defvar semantic-ignore-comments t |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1854 "Default comment handling. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1855 t means to strip comments when flexing. Nil means to keep comments |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1856 as part of the token stream.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1857 (make-variable-buffer-local 'semantic-ignore-comments) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1858 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1859 (defvar semantic-flex-enable-newlines nil |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1860 "When flexing, report 'newlines as syntactic elements. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1861 Useful for languages where the newline is a special case terminator. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1862 Only set this on a per mode basis, not globally.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1863 (make-variable-buffer-local 'semantic-flex-enable-newlines) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1864 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1865 (defvar semantic-flex-enable-whitespace nil |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1866 "When flexing, report 'whitespace as syntactic elements. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1867 Useful for languages where the syntax is whitespace dependent. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1868 Only set this on a per mode basis, not globally.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1869 (make-variable-buffer-local 'semantic-flex-enable-whitespace) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1870 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1871 (defvar semantic-flex-enable-bol nil |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1872 "When flexing, report beginning of lines as syntactic elements. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1873 Useful for languages like python which are indentation sensitive. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1874 Only set this on a per mode basis, not globally.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1875 (make-variable-buffer-local 'semantic-flex-enable-bol) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1876 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1877 (defvar semantic-number-expression semantic-lex-number-expression |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1878 "See variable `semantic-lex-number-expression'.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1879 (make-variable-buffer-local 'semantic-number-expression) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1880 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1881 (defvar semantic-flex-depth 0 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1882 "Default flexing depth. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1883 This specifies how many lists to create tokens in.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1884 (make-variable-buffer-local 'semantic-flex-depth) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1885 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1886 (defun semantic-flex (start end &optional depth length) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1887 "Using the syntax table, do something roughly equivalent to flex. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1888 Semantically check between START and END. Optional argument DEPTH |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1889 indicates at what level to scan over entire lists. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1890 The return value is a token stream. Each element is a list, such of |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1891 the form (symbol start-expression . end-expression) where SYMBOL |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1892 denotes the token type. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1893 See `semantic-flex-tokens' variable for details on token types. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1894 END does not mark the end of the text scanned, only the end of the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1895 beginning of text scanned. Thus, if a string extends past END, the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1896 end of the return token will be larger than END. To truly restrict |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1897 scanning, use `narrow-to-region'. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1898 The last argument, LENGTH specifies that `semantic-flex' should only |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1899 return LENGTH tokens." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1900 (message "`semantic-flex' is an obsolete function. Use `define-lex' to create lexers.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1901 (if (not semantic-flex-keywords-obarray) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1902 (setq semantic-flex-keywords-obarray [ nil ])) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1903 (let ((ts nil) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1904 (pos (point)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1905 (ep nil) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1906 (curdepth 0) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1907 (cs (if comment-start-skip |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1908 (concat "\\(\\s<\\|" comment-start-skip "\\)") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1909 (concat "\\(\\s<\\)"))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1910 (newsyntax (copy-syntax-table (syntax-table))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1911 (mods semantic-flex-syntax-modifications) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1912 ;; Use the default depth if it is not specified. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1913 (depth (or depth semantic-flex-depth))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1914 ;; Update the syntax table |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1915 (while mods |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1916 (modify-syntax-entry (car (car mods)) (car (cdr (car mods))) newsyntax) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1917 (setq mods (cdr mods))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1918 (with-syntax-table newsyntax |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1919 (goto-char start) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1920 (while (and (< (point) end) (or (not length) (<= (length ts) length))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1921 (cond |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1922 ;; catch beginning of lines when needed. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1923 ;; Must be done before catching any other tokens! |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1924 ((and semantic-flex-enable-bol |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1925 (bolp) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1926 ;; Just insert a (bol N . N) token in the token stream, |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1927 ;; without moving the point. N is the point at the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1928 ;; beginning of line. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1929 (setq ts (cons (cons 'bol (cons (point) (point))) ts)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1930 nil)) ;; CONTINUE |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1931 ;; special extensions, includes whitespace, nl, etc. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1932 ((and semantic-flex-extensions |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1933 (let ((fe semantic-flex-extensions) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1934 (r nil)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1935 (while fe |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1936 (if (looking-at (car (car fe))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1937 (setq ts (cons (funcall (cdr (car fe))) ts) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1938 r t |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1939 fe nil |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1940 ep (point))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1941 (setq fe (cdr fe))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1942 (if (and r (not (car ts))) (setq ts (cdr ts))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1943 r))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1944 ;; catch newlines when needed |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1945 ((looking-at "\\s-*\\(\n\\|\\s>\\)") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1946 (if semantic-flex-enable-newlines |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1947 (setq ep (match-end 1) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1948 ts (cons (cons 'newline |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1949 (cons (match-beginning 1) ep)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1950 ts)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1951 ;; catch whitespace when needed |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1952 ((looking-at "\\s-+") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1953 (if semantic-flex-enable-whitespace |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1954 ;; Language wants whitespaces, link them together. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1955 (if (eq (car (car ts)) 'whitespace) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1956 (setcdr (cdr (car ts)) (match-end 0)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1957 (setq ts (cons (cons 'whitespace |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1958 (cons (match-beginning 0) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1959 (match-end 0))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1960 ts))))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1961 ;; numbers |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1962 ((and semantic-number-expression |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1963 (looking-at semantic-number-expression)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1964 (setq ts (cons (cons 'number |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1965 (cons (match-beginning 0) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1966 (match-end 0))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1967 ts))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1968 ;; symbols |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1969 ((looking-at "\\(\\sw\\|\\s_\\)+") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1970 (setq ts (cons (cons |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1971 ;; Get info on if this is a keyword or not |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1972 (or (semantic-flex-keyword-p (match-string 0)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1973 'symbol) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1974 (cons (match-beginning 0) (match-end 0))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1975 ts))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1976 ;; Character quoting characters (ie, \n as newline) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1977 ((looking-at "\\s\\+") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1978 (setq ts (cons (cons 'charquote |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1979 (cons (match-beginning 0) (match-end 0))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1980 ts))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1981 ;; Open parens, or semantic-lists. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1982 ((looking-at "\\s(") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1983 (if (or (not depth) (< curdepth depth)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1984 (progn |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1985 (setq curdepth (1+ curdepth)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1986 (setq ts (cons (cons 'open-paren |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1987 (cons (match-beginning 0) (match-end 0))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1988 ts))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1989 (setq ts (cons |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1990 (cons 'semantic-list |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1991 (cons (match-beginning 0) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1992 (save-excursion |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1993 (condition-case nil |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1994 (forward-list 1) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1995 ;; This case makes flex robust |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1996 ;; to broken lists. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1997 (error |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1998 (goto-char |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1999 (funcall |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2000 semantic-flex-unterminated-syntax-end-function |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2001 'semantic-list |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2002 start end)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2003 (setq ep (point))))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2004 ts)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2005 ;; Close parens |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2006 ((looking-at "\\s)") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2007 (setq ts (cons (cons 'close-paren |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2008 (cons (match-beginning 0) (match-end 0))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2009 ts)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2010 (setq curdepth (1- curdepth))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2011 ;; String initiators |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2012 ((looking-at "\\s\"") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2013 ;; Zing to the end of this string. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2014 (setq ts (cons (cons 'string |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2015 (cons (match-beginning 0) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2016 (save-excursion |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2017 (condition-case nil |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2018 (forward-sexp 1) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2019 ;; This case makes flex |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2020 ;; robust to broken strings. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2021 (error |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2022 (goto-char |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2023 (funcall |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2024 semantic-flex-unterminated-syntax-end-function |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2025 'string |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2026 start end)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2027 (setq ep (point))))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2028 ts))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2029 ;; comments |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2030 ((looking-at cs) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2031 (if (and semantic-ignore-comments |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2032 (not semantic-flex-enable-whitespace)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2033 ;; If the language doesn't deal with comments nor |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2034 ;; whitespaces, ignore them here. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2035 (let ((comment-start-point (point))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2036 (forward-comment 1) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2037 (if (eq (point) comment-start-point) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2038 ;; In this case our start-skip string failed |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2039 ;; to work properly. Lets try and move over |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2040 ;; whatever white space we matched to begin |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2041 ;; with. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2042 (skip-syntax-forward "-.'" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2043 (save-excursion |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2044 (end-of-line) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2045 (point))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2046 ;;(forward-comment 1) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2047 ;; Generate newline token if enabled |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2048 (if (and semantic-flex-enable-newlines |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2049 (bolp)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2050 (backward-char 1))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2051 (if (eq (point) comment-start-point) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2052 (error "Strange comment syntax prevents lexical analysis")) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2053 (setq ep (point))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2054 (let ((tk (if semantic-ignore-comments 'whitespace 'comment))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2055 (save-excursion |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2056 (forward-comment 1) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2057 ;; Generate newline token if enabled |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2058 (if (and semantic-flex-enable-newlines |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2059 (bolp)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2060 (backward-char 1)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2061 (setq ep (point))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2062 ;; Language wants comments or want them as whitespaces, |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2063 ;; link them together. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2064 (if (eq (car (car ts)) tk) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2065 (setcdr (cdr (car ts)) ep) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2066 (setq ts (cons (cons tk (cons (match-beginning 0) ep)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2067 ts)))))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2068 ;; punctuation |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2069 ((looking-at "\\(\\s.\\|\\s$\\|\\s'\\)") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2070 (setq ts (cons (cons 'punctuation |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2071 (cons (match-beginning 0) (match-end 0))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2072 ts))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2073 ;; unknown token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2074 (t |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2075 (error "What is that?"))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2076 (goto-char (or ep (match-end 0))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2077 (setq ep nil))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2078 ;; maybe catch the last beginning of line when needed |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2079 (and semantic-flex-enable-bol |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2080 (= (point) end) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2081 (bolp) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2082 (setq ts (cons (cons 'bol (cons (point) (point))) ts))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2083 (goto-char pos) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2084 ;;(message "Flexing muscles...done") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2085 (nreverse ts))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2086 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2087 (provide 'semantic/lex) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2088 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2089 ;;; semantic-lex.el ends here |