Mercurial > emacs
annotate lisp/cedet/semantic/lex.el @ 104447:273e528a9f9b
* emacs-lisp/autoload.el (generated-autoload-load-name): New var.
(autoload-generate-file-autoloads): Use it.
(make-autoload): Handle define-overloadable-function.
* emacs-lisp/lisp-mode.el (define-overloadable-function): Add
doc-string-elt property for define-overloadable-function.
lisp/cedet/semantic/analyze.el: Add generated-autoload-load-name var.
lisp/cedet/semantic/ctxt.el: Add generated-autoload-load-name var.
lisp/cedet/semantic/db.el: Add generated-autoload-load-name var.
lisp/cedet/semantic/db-file.el: Add generated-autoload-load-name var.
lisp/cedet/semantic/db-mode.el: Add generated-autoload-load-name var.
lisp/cedet/semantic/db-typecache.el: Add generated-autoload-load-name var.
lisp/cedet/semantic/doc.el: Add generated-autoload-load-name var.
lisp/cedet/semantic/find.el: Add generated-autoload-load-name var.
lisp/cedet/semantic/format.el: Add generated-autoload-load-name var.
lisp/cedet/semantic/lex.el: Add generated-autoload-load-name var.
lisp/cedet/semantic/sb.el: Require semantic/format.
lisp/cedet/semantic/sort.el: Add generated-autoload-load-name var.
lisp/cedet/semantic/tag.el: Add generated-autoload-load-name var.
lisp/cedet/semantic/tag-file.el: Add generated-autoload-load-name var.
lisp/cedet/semantic/tag-ls.el: Add generated-autoload-load-name var.
author | Chong Yidong <cyd@stupidchicken.com> |
---|---|
date | Sat, 05 Sep 2009 01:00:36 +0000 |
parents | df08b7ab0ba0 |
children | a4e1a12c8b97 |
rev | line source |
---|---|
104416
c13af98da4d6
Fix renamed filenames in file headers.
Chong Yidong <cyd@stupidchicken.com>
parents:
104413
diff
changeset
|
1 ;;; lex.el --- Lexical Analyzer builder |
104413
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
3 ;;; Copyright (C) 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
4 ;;; 2007, 2008, 2009 Free Software Foundation, Inc. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
5 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
6 ;; Author: Eric M. Ludlam <zappo@gnu.org> |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
7 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
8 ;; This file is part of GNU Emacs. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
9 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
10 ;; GNU Emacs is free software: you can redistribute it and/or modify |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
11 ;; it under the terms of the GNU General Public License as published by |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
12 ;; the Free Software Foundation, either version 3 of the License, or |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
13 ;; (at your option) any later version. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
14 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
15 ;; GNU Emacs is distributed in the hope that it will be useful, |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
16 ;; but WITHOUT ANY WARRANTY; without even the implied warranty of |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
17 ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
18 ;; GNU General Public License for more details. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
19 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
20 ;; You should have received a copy of the GNU General Public License |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
21 ;; along with GNU Emacs. If not, see <http://www.gnu.org/licenses/>. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
22 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
23 ;;; Commentary: |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
24 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
25 ;; This file handles the creation of lexical analyzers for different |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
26 ;; languages in Emacs Lisp. The purpose of a lexical analyzer is to |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
27 ;; convert a buffer into a list of lexical tokens. Each token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
28 ;; contains the token class (such as 'number, 'symbol, 'IF, etc) and |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
29 ;; the location in the buffer it was found. Optionally, a token also |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
30 ;; contains a string representing what is at the designated buffer |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
31 ;; location. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
32 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
33 ;; Tokens are pushed onto a token stream, which is basically a list of |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
34 ;; all the lexical tokens from the analyzed region. The token stream |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
35 ;; is then handed to the grammar which parsers the file. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
36 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
37 ;;; How it works |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
38 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
39 ;; Each analyzer specifies a condition and forms. These conditions |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
40 ;; and forms are assembled into a function by `define-lex' that does |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
41 ;; the lexical analysis. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
42 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
43 ;; In the lexical analyzer created with `define-lex', each condition |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
44 ;; is tested for a given point. When the conditin is true, the forms |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
45 ;; run. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
46 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
47 ;; The forms can push a lexical token onto the token stream. The |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
48 ;; analyzer forms also must move the current analyzer point. If the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
49 ;; analyzer point is moved without pushing a token, then tne matched |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
50 ;; syntax is effectively ignored, or skipped. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
51 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
52 ;; Thus, starting at the beginning of a region to be analyzed, each |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
53 ;; condition is tested. One will match, and a lexical token might be |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
54 ;; pushed, and the point is moved to the end of the lexical token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
55 ;; identified. At the new position, the process occurs again until |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
56 ;; the end of the specified region is reached. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
57 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
58 ;;; How to use semantic-lex |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
59 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
60 ;; To create a lexer for a language, use the `define-lex' macro. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
61 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
62 ;; The `define-lex' macro accepts a list of lexical analyzers. Each |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
63 ;; analyzer is created with `define-lex-analyzer', or one of the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
64 ;; derivitive macros. A single analyzer defines a regular expression |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
65 ;; to match text in a buffer, and a short segment of code to create |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
66 ;; one lexical token. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
67 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
68 ;; Each analyzer has a NAME, DOC, a CONDITION, and possibly some |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
69 ;; FORMS. The NAME is the name used in `define-lex'. The DOC |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
70 ;; describes what the analyzer should do. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
71 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
72 ;; The CONDITION evaluates the text at the current point in the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
73 ;; current buffer. If CONDITION is true, then the FORMS will be |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
74 ;; executed. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
75 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
76 ;; The purpose of the FORMS is to push new lexical tokens onto the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
77 ;; list of tokens for the current buffer, and to move point after the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
78 ;; matched text. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
79 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
80 ;; Some macros for creating one analyzer are: |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
81 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
82 ;; define-lex-analyzer - A generic analyzer associating any style of |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
83 ;; condition to forms. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
84 ;; define-lex-regex-analyzer - Matches a regular expression. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
85 ;; define-lex-simple-regex-analyzer - Matches a regular expressions, |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
86 ;; and pushes the match. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
87 ;; define-lex-block-analyzer - Matches list syntax, and defines |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
88 ;; handles open/close delimiters. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
89 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
90 ;; These macros are used by the grammar compiler when lexical |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
91 ;; information is specified in a grammar: |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
92 ;; define-lex- * -type-analyzer - Matches syntax specified in |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
93 ;; a grammar, and pushes one token for it. The * would |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
94 ;; be `sexp' for things like lists or strings, and |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
95 ;; `string' for things that need to match some special |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
96 ;; string, such as "\\." where a literal match is needed. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
97 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
98 ;;; Lexical Tables |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
99 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
100 ;; There are tables of different symbols managed in semantic-lex.el. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
101 ;; They are: |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
102 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
103 ;; Lexical keyword table - A Table of symbols declared in a grammar |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
104 ;; file with the %keyword declaration. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
105 ;; Keywords are used by `semantic-lex-symbol-or-keyword' |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
106 ;; to create lexical tokens based on the keyword. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
107 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
108 ;; Lexical type table - A table of symbols declared in a grammer |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
109 ;; file with the %type declaration. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
110 ;; The grammar compiler uses the type table to create new |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
111 ;; lexical analyzers. These analyzers are then used to when |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
112 ;; a new lexical analyzer is made for a language. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
113 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
114 ;;; Lexical Types |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
115 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
116 ;; A lexical type defines a kind of lexical analyzer that will be |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
117 ;; automatically generated from a grammar file based on some |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
118 ;; predetermined attributes. For now these two attributes are |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
119 ;; recognized : |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
120 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
121 ;; * matchdatatype : define the kind of lexical analyzer. That is : |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
122 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
123 ;; - regexp : define a regexp analyzer (see |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
124 ;; `define-lex-regex-type-analyzer') |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
125 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
126 ;; - string : define a string analyzer (see |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
127 ;; `define-lex-string-type-analyzer') |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
128 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
129 ;; - block : define a block type analyzer (see |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
130 ;; `define-lex-block-type-analyzer') |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
131 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
132 ;; - sexp : define a sexp analyzer (see |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
133 ;; `define-lex-sexp-type-analyzer') |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
134 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
135 ;; - keyword : define a keyword analyzer (see |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
136 ;; `define-lex-keyword-type-analyzer') |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
137 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
138 ;; * syntax : define the syntax that matches a syntactic |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
139 ;; expression. When syntax is matched the corresponding type |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
140 ;; analyzer is entered and the resulting match data will be |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
141 ;; interpreted based on the kind of analyzer (see matchdatatype |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
142 ;; above). |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
143 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
144 ;; The following lexical types are predefined : |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
145 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
146 ;; +-------------+---------------+--------------------------------+ |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
147 ;; | type | matchdatatype | syntax | |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
148 ;; +-------------+---------------+--------------------------------+ |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
149 ;; | punctuation | string | "\\(\\s.\\|\\s$\\|\\s'\\)+" | |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
150 ;; | keyword | keyword | "\\(\\sw\\|\\s_\\)+" | |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
151 ;; | symbol | regexp | "\\(\\sw\\|\\s_\\)+" | |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
152 ;; | string | sexp | "\\s\"" | |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
153 ;; | number | regexp | semantic-lex-number-expression | |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
154 ;; | block | block | "\\s(\\|\\s)" | |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
155 ;; +-------------+---------------+--------------------------------+ |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
156 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
157 ;; In a grammar you must use a %type expression to automatically generate |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
158 ;; the corresponding analyzers of that type. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
159 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
160 ;; Here is an example to auto-generate punctuation analyzers |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
161 ;; with 'matchdatatype and 'syntax predefined (see table above) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
162 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
163 ;; %type <punctuation> ;; will auto-generate this kind of analyzers |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
164 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
165 ;; It is equivalent to write : |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
166 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
167 ;; %type <punctuation> syntax "\\(\\s.\\|\\s$\\|\\s'\\)+" matchdatatype string |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
168 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
169 ;; ;; Some punctuations based on the type defines above |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
170 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
171 ;; %token <punctuation> NOT "!" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
172 ;; %token <punctuation> NOTEQ "!=" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
173 ;; %token <punctuation> MOD "%" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
174 ;; %token <punctuation> MODEQ "%=" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
175 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
176 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
177 ;;; On the Semantic 1.x lexer |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
178 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
179 ;; In semantic 1.x, the lexical analyzer was an all purpose routine. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
180 ;; To boost efficiency, the analyzer is now a series of routines that |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
181 ;; are constructed at build time into a single routine. This will |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
182 ;; eliminate unneeded if statements to speed the lexer. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
183 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
184 (require 'semantic/fw) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
185 ;;; Code: |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
186 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
187 ;;; Compatibility |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
188 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
189 (eval-and-compile |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
190 (if (not (fboundp 'with-syntax-table)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
191 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
192 ;; Copied from Emacs 21 for compatibility with released Emacses. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
193 (defmacro with-syntax-table (table &rest body) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
194 "With syntax table of current buffer set to a copy of TABLE, evaluate BODY. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
195 The syntax table of the current buffer is saved, BODY is evaluated, and the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
196 saved table is restored, even in case of an abnormal exit. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
197 Value is what BODY returns." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
198 (let ((old-table (make-symbol "table")) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
199 (old-buffer (make-symbol "buffer"))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
200 `(let ((,old-table (syntax-table)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
201 (,old-buffer (current-buffer))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
202 (unwind-protect |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
203 (progn |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
204 (set-syntax-table (copy-syntax-table ,table)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
205 ,@body) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
206 (save-current-buffer |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
207 (set-buffer ,old-buffer) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
208 (set-syntax-table ,old-table)))))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
209 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
210 )) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
211 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
212 ;;; Semantic 2.x lexical analysis |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
213 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
214 (defun semantic-lex-map-symbols (fun table &optional property) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
215 "Call function FUN on every symbol in TABLE. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
216 If optional PROPERTY is non-nil, call FUN only on every symbol which |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
217 as a PROPERTY value. FUN receives a symbol as argument." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
218 (if (arrayp table) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
219 (mapatoms |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
220 #'(lambda (symbol) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
221 (if (or (null property) (get symbol property)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
222 (funcall fun symbol))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
223 table))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
224 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
225 ;;; Lexical keyword table handling. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
226 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
227 ;; These keywords are keywords defined for using in a grammar with the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
228 ;; %keyword declaration, and are not keywords used in Emacs Lisp. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
229 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
230 (defvar semantic-flex-keywords-obarray nil |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
231 "Buffer local keyword obarray for the lexical analyzer. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
232 These keywords are matched explicitly, and converted into special symbols.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
233 (make-variable-buffer-local 'semantic-flex-keywords-obarray) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
234 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
235 (defmacro semantic-lex-keyword-invalid (name) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
236 "Signal that NAME is an invalid keyword name." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
237 `(signal 'wrong-type-argument '(semantic-lex-keyword-p ,name))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
238 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
239 (defsubst semantic-lex-keyword-symbol (name) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
240 "Return keyword symbol with NAME or nil if not found." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
241 (and (arrayp semantic-flex-keywords-obarray) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
242 (stringp name) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
243 (intern-soft name semantic-flex-keywords-obarray))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
244 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
245 (defsubst semantic-lex-keyword-p (name) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
246 "Return non-nil if a keyword with NAME exists in the keyword table. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
247 Return nil otherwise." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
248 (and (setq name (semantic-lex-keyword-symbol name)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
249 (symbol-value name))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
250 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
251 (defsubst semantic-lex-keyword-set (name value) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
252 "Set value of keyword with NAME to VALUE and return VALUE." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
253 (set (intern name semantic-flex-keywords-obarray) value)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
254 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
255 (defsubst semantic-lex-keyword-value (name) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
256 "Return value of keyword with NAME. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
257 Signal an error if a keyword with NAME does not exist." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
258 (let ((keyword (semantic-lex-keyword-symbol name))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
259 (if keyword |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
260 (symbol-value keyword) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
261 (semantic-lex-keyword-invalid name)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
262 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
263 (defsubst semantic-lex-keyword-put (name property value) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
264 "For keyword with NAME, set its PROPERTY to VALUE." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
265 (let ((keyword (semantic-lex-keyword-symbol name))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
266 (if keyword |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
267 (put keyword property value) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
268 (semantic-lex-keyword-invalid name)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
269 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
270 (defsubst semantic-lex-keyword-get (name property) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
271 "For keyword with NAME, return its PROPERTY value." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
272 (let ((keyword (semantic-lex-keyword-symbol name))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
273 (if keyword |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
274 (get keyword property) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
275 (semantic-lex-keyword-invalid name)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
276 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
277 (defun semantic-lex-make-keyword-table (specs &optional propspecs) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
278 "Convert keyword SPECS into an obarray and return it. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
279 SPECS must be a list of (NAME . TOKSYM) elements, where: |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
280 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
281 NAME is the name of the keyword symbol to define. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
282 TOKSYM is the lexical token symbol of that keyword. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
283 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
284 If optional argument PROPSPECS is non nil, then interpret it, and |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
285 apply those properties. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
286 PROPSPECS must be a list of (NAME PROPERTY VALUE) elements." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
287 ;; Create the symbol hash table |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
288 (let ((semantic-flex-keywords-obarray (make-vector 13 0)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
289 spec) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
290 ;; fill it with stuff |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
291 (while specs |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
292 (setq spec (car specs) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
293 specs (cdr specs)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
294 (semantic-lex-keyword-set (car spec) (cdr spec))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
295 ;; Apply all properties |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
296 (while propspecs |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
297 (setq spec (car propspecs) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
298 propspecs (cdr propspecs)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
299 (semantic-lex-keyword-put (car spec) (nth 1 spec) (nth 2 spec))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
300 semantic-flex-keywords-obarray)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
301 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
302 (defsubst semantic-lex-map-keywords (fun &optional property) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
303 "Call function FUN on every lexical keyword. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
304 If optional PROPERTY is non-nil, call FUN only on every keyword which |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
305 as a PROPERTY value. FUN receives a lexical keyword as argument." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
306 (semantic-lex-map-symbols |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
307 fun semantic-flex-keywords-obarray property)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
308 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
309 (defun semantic-lex-keywords (&optional property) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
310 "Return a list of lexical keywords. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
311 If optional PROPERTY is non-nil, return only keywords which have a |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
312 PROPERTY set." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
313 (let (keywords) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
314 (semantic-lex-map-keywords |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
315 #'(lambda (symbol) (setq keywords (cons symbol keywords))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
316 property) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
317 keywords)) |
104419
b1ac14799f78
cedet/semantic/analyze.el, cedet/semantic/complete.el,
Chong Yidong <cyd@stupidchicken.com>
parents:
104416
diff
changeset
|
318 |
b1ac14799f78
cedet/semantic/analyze.el, cedet/semantic/complete.el,
Chong Yidong <cyd@stupidchicken.com>
parents:
104416
diff
changeset
|
319 ;;; Inline functions: |
b1ac14799f78
cedet/semantic/analyze.el, cedet/semantic/complete.el,
Chong Yidong <cyd@stupidchicken.com>
parents:
104416
diff
changeset
|
320 |
b1ac14799f78
cedet/semantic/analyze.el, cedet/semantic/complete.el,
Chong Yidong <cyd@stupidchicken.com>
parents:
104416
diff
changeset
|
321 (defvar semantic-lex-unterminated-syntax-end-function) |
b1ac14799f78
cedet/semantic/analyze.el, cedet/semantic/complete.el,
Chong Yidong <cyd@stupidchicken.com>
parents:
104416
diff
changeset
|
322 (defvar semantic-lex-analysis-bounds) |
b1ac14799f78
cedet/semantic/analyze.el, cedet/semantic/complete.el,
Chong Yidong <cyd@stupidchicken.com>
parents:
104416
diff
changeset
|
323 (defvar semantic-lex-end-point) |
b1ac14799f78
cedet/semantic/analyze.el, cedet/semantic/complete.el,
Chong Yidong <cyd@stupidchicken.com>
parents:
104416
diff
changeset
|
324 |
b1ac14799f78
cedet/semantic/analyze.el, cedet/semantic/complete.el,
Chong Yidong <cyd@stupidchicken.com>
parents:
104416
diff
changeset
|
325 (defsubst semantic-lex-token-bounds (token) |
b1ac14799f78
cedet/semantic/analyze.el, cedet/semantic/complete.el,
Chong Yidong <cyd@stupidchicken.com>
parents:
104416
diff
changeset
|
326 "Fetch the start and end locations of the lexical token TOKEN. |
b1ac14799f78
cedet/semantic/analyze.el, cedet/semantic/complete.el,
Chong Yidong <cyd@stupidchicken.com>
parents:
104416
diff
changeset
|
327 Return a pair (START . END)." |
b1ac14799f78
cedet/semantic/analyze.el, cedet/semantic/complete.el,
Chong Yidong <cyd@stupidchicken.com>
parents:
104416
diff
changeset
|
328 (if (not (numberp (car (cdr token)))) |
b1ac14799f78
cedet/semantic/analyze.el, cedet/semantic/complete.el,
Chong Yidong <cyd@stupidchicken.com>
parents:
104416
diff
changeset
|
329 (cdr (cdr token)) |
b1ac14799f78
cedet/semantic/analyze.el, cedet/semantic/complete.el,
Chong Yidong <cyd@stupidchicken.com>
parents:
104416
diff
changeset
|
330 (cdr token))) |
b1ac14799f78
cedet/semantic/analyze.el, cedet/semantic/complete.el,
Chong Yidong <cyd@stupidchicken.com>
parents:
104416
diff
changeset
|
331 |
b1ac14799f78
cedet/semantic/analyze.el, cedet/semantic/complete.el,
Chong Yidong <cyd@stupidchicken.com>
parents:
104416
diff
changeset
|
332 (defsubst semantic-lex-token-start (token) |
b1ac14799f78
cedet/semantic/analyze.el, cedet/semantic/complete.el,
Chong Yidong <cyd@stupidchicken.com>
parents:
104416
diff
changeset
|
333 "Fetch the start position of the lexical token TOKEN. |
b1ac14799f78
cedet/semantic/analyze.el, cedet/semantic/complete.el,
Chong Yidong <cyd@stupidchicken.com>
parents:
104416
diff
changeset
|
334 See also the function `semantic-lex-token'." |
b1ac14799f78
cedet/semantic/analyze.el, cedet/semantic/complete.el,
Chong Yidong <cyd@stupidchicken.com>
parents:
104416
diff
changeset
|
335 (car (semantic-lex-token-bounds token))) |
b1ac14799f78
cedet/semantic/analyze.el, cedet/semantic/complete.el,
Chong Yidong <cyd@stupidchicken.com>
parents:
104416
diff
changeset
|
336 |
b1ac14799f78
cedet/semantic/analyze.el, cedet/semantic/complete.el,
Chong Yidong <cyd@stupidchicken.com>
parents:
104416
diff
changeset
|
337 (defsubst semantic-lex-token-end (token) |
b1ac14799f78
cedet/semantic/analyze.el, cedet/semantic/complete.el,
Chong Yidong <cyd@stupidchicken.com>
parents:
104416
diff
changeset
|
338 "Fetch the end position of the lexical token TOKEN. |
b1ac14799f78
cedet/semantic/analyze.el, cedet/semantic/complete.el,
Chong Yidong <cyd@stupidchicken.com>
parents:
104416
diff
changeset
|
339 See also the function `semantic-lex-token'." |
b1ac14799f78
cedet/semantic/analyze.el, cedet/semantic/complete.el,
Chong Yidong <cyd@stupidchicken.com>
parents:
104416
diff
changeset
|
340 (cdr (semantic-lex-token-bounds token))) |
b1ac14799f78
cedet/semantic/analyze.el, cedet/semantic/complete.el,
Chong Yidong <cyd@stupidchicken.com>
parents:
104416
diff
changeset
|
341 |
b1ac14799f78
cedet/semantic/analyze.el, cedet/semantic/complete.el,
Chong Yidong <cyd@stupidchicken.com>
parents:
104416
diff
changeset
|
342 (defsubst semantic-lex-unterminated-syntax-detected (syntax) |
b1ac14799f78
cedet/semantic/analyze.el, cedet/semantic/complete.el,
Chong Yidong <cyd@stupidchicken.com>
parents:
104416
diff
changeset
|
343 "Inside a lexical analyzer, use this when unterminated syntax was found. |
b1ac14799f78
cedet/semantic/analyze.el, cedet/semantic/complete.el,
Chong Yidong <cyd@stupidchicken.com>
parents:
104416
diff
changeset
|
344 Argument SYNTAX indicates the type of syntax that is unterminated. |
b1ac14799f78
cedet/semantic/analyze.el, cedet/semantic/complete.el,
Chong Yidong <cyd@stupidchicken.com>
parents:
104416
diff
changeset
|
345 The job of this function is to move (point) to a new logical location |
b1ac14799f78
cedet/semantic/analyze.el, cedet/semantic/complete.el,
Chong Yidong <cyd@stupidchicken.com>
parents:
104416
diff
changeset
|
346 so that analysis can continue, if possible." |
b1ac14799f78
cedet/semantic/analyze.el, cedet/semantic/complete.el,
Chong Yidong <cyd@stupidchicken.com>
parents:
104416
diff
changeset
|
347 (goto-char |
b1ac14799f78
cedet/semantic/analyze.el, cedet/semantic/complete.el,
Chong Yidong <cyd@stupidchicken.com>
parents:
104416
diff
changeset
|
348 (funcall semantic-lex-unterminated-syntax-end-function |
b1ac14799f78
cedet/semantic/analyze.el, cedet/semantic/complete.el,
Chong Yidong <cyd@stupidchicken.com>
parents:
104416
diff
changeset
|
349 syntax |
b1ac14799f78
cedet/semantic/analyze.el, cedet/semantic/complete.el,
Chong Yidong <cyd@stupidchicken.com>
parents:
104416
diff
changeset
|
350 (car semantic-lex-analysis-bounds) |
b1ac14799f78
cedet/semantic/analyze.el, cedet/semantic/complete.el,
Chong Yidong <cyd@stupidchicken.com>
parents:
104416
diff
changeset
|
351 (cdr semantic-lex-analysis-bounds) |
b1ac14799f78
cedet/semantic/analyze.el, cedet/semantic/complete.el,
Chong Yidong <cyd@stupidchicken.com>
parents:
104416
diff
changeset
|
352 )) |
b1ac14799f78
cedet/semantic/analyze.el, cedet/semantic/complete.el,
Chong Yidong <cyd@stupidchicken.com>
parents:
104416
diff
changeset
|
353 (setq semantic-lex-end-point (point))) |
104413
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
354 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
355 ;;; Type table handling. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
356 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
357 ;; The lexical type table manages types that occur in a grammar file |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
358 ;; with the %type declaration. Types represent different syntaxes. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
359 ;; See code for `semantic-lex-preset-default-types' for the classic |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
360 ;; types of syntax. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
361 (defvar semantic-lex-types-obarray nil |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
362 "Buffer local types obarray for the lexical analyzer.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
363 (make-variable-buffer-local 'semantic-lex-types-obarray) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
364 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
365 (defmacro semantic-lex-type-invalid (type) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
366 "Signal that TYPE is an invalid lexical type name." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
367 `(signal 'wrong-type-argument '(semantic-lex-type-p ,type))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
368 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
369 (defsubst semantic-lex-type-symbol (type) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
370 "Return symbol with TYPE or nil if not found." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
371 (and (arrayp semantic-lex-types-obarray) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
372 (stringp type) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
373 (intern-soft type semantic-lex-types-obarray))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
374 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
375 (defsubst semantic-lex-type-p (type) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
376 "Return non-nil if a symbol with TYPE name exists." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
377 (and (setq type (semantic-lex-type-symbol type)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
378 (symbol-value type))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
379 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
380 (defsubst semantic-lex-type-set (type value) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
381 "Set value of symbol with TYPE name to VALUE and return VALUE." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
382 (set (intern type semantic-lex-types-obarray) value)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
383 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
384 (defsubst semantic-lex-type-value (type &optional noerror) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
385 "Return value of symbol with TYPE name. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
386 If optional argument NOERROR is non-nil return nil if a symbol with |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
387 TYPE name does not exist. Otherwise signal an error." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
388 (let ((sym (semantic-lex-type-symbol type))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
389 (if sym |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
390 (symbol-value sym) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
391 (unless noerror |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
392 (semantic-lex-type-invalid type))))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
393 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
394 (defsubst semantic-lex-type-put (type property value &optional add) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
395 "For symbol with TYPE name, set its PROPERTY to VALUE. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
396 If optional argument ADD is non-nil, create a new symbol with TYPE |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
397 name if it does not already exist. Otherwise signal an error." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
398 (let ((sym (semantic-lex-type-symbol type))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
399 (unless sym |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
400 (or add (semantic-lex-type-invalid type)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
401 (semantic-lex-type-set type nil) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
402 (setq sym (semantic-lex-type-symbol type))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
403 (put sym property value))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
404 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
405 (defsubst semantic-lex-type-get (type property &optional noerror) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
406 "For symbol with TYPE name, return its PROPERTY value. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
407 If optional argument NOERROR is non-nil return nil if a symbol with |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
408 TYPE name does not exist. Otherwise signal an error." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
409 (let ((sym (semantic-lex-type-symbol type))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
410 (if sym |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
411 (get sym property) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
412 (unless noerror |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
413 (semantic-lex-type-invalid type))))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
414 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
415 (defun semantic-lex-preset-default-types () |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
416 "Install useful default properties for well known types." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
417 (semantic-lex-type-put "punctuation" 'matchdatatype 'string t) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
418 (semantic-lex-type-put "punctuation" 'syntax "\\(\\s.\\|\\s$\\|\\s'\\)+") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
419 (semantic-lex-type-put "keyword" 'matchdatatype 'keyword t) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
420 (semantic-lex-type-put "keyword" 'syntax "\\(\\sw\\|\\s_\\)+") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
421 (semantic-lex-type-put "symbol" 'matchdatatype 'regexp t) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
422 (semantic-lex-type-put "symbol" 'syntax "\\(\\sw\\|\\s_\\)+") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
423 (semantic-lex-type-put "string" 'matchdatatype 'sexp t) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
424 (semantic-lex-type-put "string" 'syntax "\\s\"") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
425 (semantic-lex-type-put "number" 'matchdatatype 'regexp t) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
426 (semantic-lex-type-put "number" 'syntax 'semantic-lex-number-expression) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
427 (semantic-lex-type-put "block" 'matchdatatype 'block t) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
428 (semantic-lex-type-put "block" 'syntax "\\s(\\|\\s)") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
429 ) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
430 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
431 (defun semantic-lex-make-type-table (specs &optional propspecs) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
432 "Convert type SPECS into an obarray and return it. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
433 SPECS must be a list of (TYPE . TOKENS) elements, where: |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
434 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
435 TYPE is the name of the type symbol to define. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
436 TOKENS is an list of (TOKSYM . MATCHER) elements, where: |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
437 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
438 TOKSYM is any lexical token symbol. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
439 MATCHER is a string or regexp a text must match to be a such |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
440 lexical token. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
441 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
442 If optional argument PROPSPECS is non nil, then interpret it, and |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
443 apply those properties. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
444 PROPSPECS must be a list of (TYPE PROPERTY VALUE)." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
445 ;; Create the symbol hash table |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
446 (let* ((semantic-lex-types-obarray (make-vector 13 0)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
447 spec type tokens token alist default) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
448 ;; fill it with stuff |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
449 (while specs |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
450 (setq spec (car specs) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
451 specs (cdr specs) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
452 type (car spec) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
453 tokens (cdr spec) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
454 default nil |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
455 alist nil) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
456 (while tokens |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
457 (setq token (car tokens) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
458 tokens (cdr tokens)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
459 (if (cdr token) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
460 (setq alist (cons token alist)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
461 (setq token (car token)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
462 (if default |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
463 (message |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
464 "*Warning* default value of <%s> tokens changed to %S, was %S" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
465 type default token)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
466 (setq default token))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
467 ;; Ensure the default matching spec is the first one. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
468 (semantic-lex-type-set type (cons default (nreverse alist)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
469 ;; Install useful default types & properties |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
470 (semantic-lex-preset-default-types) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
471 ;; Apply all properties |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
472 (while propspecs |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
473 (setq spec (car propspecs) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
474 propspecs (cdr propspecs)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
475 ;; Create the type if necessary. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
476 (semantic-lex-type-put (car spec) (nth 1 spec) (nth 2 spec) t)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
477 semantic-lex-types-obarray)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
478 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
479 (defsubst semantic-lex-map-types (fun &optional property) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
480 "Call function FUN on every lexical type. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
481 If optional PROPERTY is non-nil, call FUN only on every type symbol |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
482 which as a PROPERTY value. FUN receives a type symbol as argument." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
483 (semantic-lex-map-symbols |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
484 fun semantic-lex-types-obarray property)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
485 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
486 (defun semantic-lex-types (&optional property) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
487 "Return a list of lexical type symbols. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
488 If optional PROPERTY is non-nil, return only type symbols which have |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
489 PROPERTY set." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
490 (let (types) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
491 (semantic-lex-map-types |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
492 #'(lambda (symbol) (setq types (cons symbol types))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
493 property) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
494 types)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
495 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
496 ;;; Lexical Analyzer framework settings |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
497 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
498 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
499 (defvar semantic-lex-analyzer 'semantic-flex |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
500 "The lexical analyzer used for a given buffer. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
501 See `semantic-lex' for documentation. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
502 For compatibility with Semantic 1.x it defaults to `semantic-flex'.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
503 (make-variable-buffer-local 'semantic-lex-analyzer) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
504 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
505 (defvar semantic-lex-tokens |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
506 '( |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
507 (bol) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
508 (charquote) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
509 (close-paren) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
510 (comment) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
511 (newline) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
512 (open-paren) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
513 (punctuation) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
514 (semantic-list) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
515 (string) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
516 (symbol) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
517 (whitespace) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
518 ) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
519 "An alist of of semantic token types. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
520 As of December 2001 (semantic 1.4beta13), this variable is not used in |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
521 any code. The only use is to refer to the doc-string from elsewhere. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
522 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
523 The key to this alist is the symbol representing token type that |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
524 \\[semantic-flex] returns. These are |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
525 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
526 - bol: Empty string matching a beginning of line. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
527 This token is produced with |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
528 `semantic-lex-beginning-of-line'. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
529 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
530 - charquote: String sequences that match `\\s\\+' regexp. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
531 This token is produced with `semantic-lex-charquote'. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
532 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
533 - close-paren: Characters that match `\\s)' regexp. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
534 These are typically `)', `}', `]', etc. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
535 This token is produced with |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
536 `semantic-lex-close-paren'. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
537 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
538 - comment: A comment chunk. These token types are not |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
539 produced by default. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
540 This token is produced with `semantic-lex-comments'. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
541 Comments are ignored with `semantic-lex-ignore-comments'. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
542 Comments are treated as whitespace with |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
543 `semantic-lex-comments-as-whitespace'. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
544 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
545 - newline Characters matching `\\s-*\\(\n\\|\\s>\\)' regexp. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
546 This token is produced with `semantic-lex-newline'. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
547 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
548 - open-paren: Characters that match `\\s(' regexp. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
549 These are typically `(', `{', `[', etc. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
550 If `semantic-lex-paren-or-list' is used, |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
551 then `open-paren' is not usually generated unless |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
552 the `depth' argument to \\[semantic-lex] is |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
553 greater than 0. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
554 This token is always produced if the analyzer |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
555 `semantic-lex-open-paren' is used. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
556 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
557 - punctuation: Characters matching `{\\(\\s.\\|\\s$\\|\\s'\\)' |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
558 regexp. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
559 This token is produced with `semantic-lex-punctuation'. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
560 Always specify this analyzer after the comment |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
561 analyzer. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
562 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
563 - semantic-list: String delimited by matching parenthesis, braces, |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
564 etc. that the lexer skipped over, because the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
565 `depth' parameter to \\[semantic-flex] was not high |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
566 enough. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
567 This token is produced with `semantic-lex-paren-or-list'. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
568 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
569 - string: Quoted strings, i.e., string sequences that start |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
570 and end with characters matching `\\s\"' |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
571 regexp. The lexer relies on @code{forward-sexp} to |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
572 find the matching end. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
573 This token is produced with `semantic-lex-string'. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
574 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
575 - symbol: String sequences that match `\\(\\sw\\|\\s_\\)+' |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
576 regexp. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
577 This token is produced with |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
578 `semantic-lex-symbol-or-keyword'. Always add this analyzer |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
579 after `semantic-lex-number', or other analyzers that |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
580 match its regular expression. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
581 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
582 - whitespace: Characters that match `\\s-+' regexp. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
583 This token is produced with `semantic-lex-whitespace'.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
584 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
585 (defvar semantic-lex-syntax-modifications nil |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
586 "Changes to the syntax table for this buffer. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
587 These changes are active only while the buffer is being flexed. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
588 This is a list where each element has the form: |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
589 (CHAR CLASS) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
590 CHAR is the char passed to `modify-syntax-entry', |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
591 and CLASS is the string also passed to `modify-syntax-entry' to define |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
592 what syntax class CHAR has.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
593 (make-variable-buffer-local 'semantic-lex-syntax-modifications) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
594 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
595 (defvar semantic-lex-syntax-table nil |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
596 "Syntax table used by lexical analysis. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
597 See also `semantic-lex-syntax-modifications'.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
598 (make-variable-buffer-local 'semantic-lex-syntax-table) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
599 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
600 (defvar semantic-lex-comment-regex nil |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
601 "Regular expression for identifying comment start during lexical analysis. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
602 This may be automatically set when semantic initializes in a mode, but |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
603 may need to be overriden for some special languages.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
604 (make-variable-buffer-local 'semantic-lex-comment-regex) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
605 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
606 (defvar semantic-lex-number-expression |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
607 ;; This expression was written by David Ponce for Java, and copied |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
608 ;; here for C and any other similar language. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
609 (eval-when-compile |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
610 (concat "\\(" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
611 "\\<[0-9]+[.][0-9]+\\([eE][-+]?[0-9]+\\)?[fFdD]?\\>" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
612 "\\|" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
613 "\\<[0-9]+[.][eE][-+]?[0-9]+[fFdD]?\\>" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
614 "\\|" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
615 "\\<[0-9]+[.][fFdD]\\>" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
616 "\\|" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
617 "\\<[0-9]+[.]" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
618 "\\|" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
619 "[.][0-9]+\\([eE][-+]?[0-9]+\\)?[fFdD]?\\>" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
620 "\\|" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
621 "\\<[0-9]+[eE][-+]?[0-9]+[fFdD]?\\>" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
622 "\\|" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
623 "\\<0[xX][0-9a-fA-F]+[lL]?\\>" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
624 "\\|" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
625 "\\<[0-9]+[lLfFdD]?\\>" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
626 "\\)" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
627 )) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
628 "Regular expression for matching a number. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
629 If this value is nil, no number extraction is done during lex. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
630 This expression tries to match C and Java like numbers. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
631 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
632 DECIMAL_LITERAL: |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
633 [1-9][0-9]* |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
634 ; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
635 HEX_LITERAL: |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
636 0[xX][0-9a-fA-F]+ |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
637 ; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
638 OCTAL_LITERAL: |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
639 0[0-7]* |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
640 ; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
641 INTEGER_LITERAL: |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
642 <DECIMAL_LITERAL>[lL]? |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
643 | <HEX_LITERAL>[lL]? |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
644 | <OCTAL_LITERAL>[lL]? |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
645 ; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
646 EXPONENT: |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
647 [eE][+-]?[09]+ |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
648 ; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
649 FLOATING_POINT_LITERAL: |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
650 [0-9]+[.][0-9]*<EXPONENT>?[fFdD]? |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
651 | [.][0-9]+<EXPONENT>?[fFdD]? |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
652 | [0-9]+<EXPONENT>[fFdD]? |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
653 | [0-9]+<EXPONENT>?[fFdD] |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
654 ;") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
655 (make-variable-buffer-local 'semantic-lex-number-expression) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
656 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
657 (defvar semantic-lex-depth 0 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
658 "Default lexing depth. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
659 This specifies how many lists to create tokens in.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
660 (make-variable-buffer-local 'semantic-lex-depth) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
661 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
662 (defvar semantic-lex-unterminated-syntax-end-function |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
663 (lambda (syntax syntax-start lex-end) lex-end) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
664 "Function called when unterminated syntax is encountered. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
665 This should be set to one function. That function should take three |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
666 parameters. The SYNTAX, or type of syntax which is unterminated. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
667 SYNTAX-START where the broken syntax begins. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
668 LEX-END is where the lexical analysis was asked to end. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
669 This function can be used for languages that can intelligently fix up |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
670 broken syntax, or the exit lexical analysis via `throw' or `signal' |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
671 when finding unterminated syntax.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
672 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
673 ;;; Interactive testing commands |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
674 |
104446
df08b7ab0ba0
lisp/cedet/semantic/analyze.el: Add local vars for autoloading.
Chong Yidong <cyd@stupidchicken.com>
parents:
104419
diff
changeset
|
675 (declare-function semantic-elapsed-time "semantic") |
df08b7ab0ba0
lisp/cedet/semantic/analyze.el: Add local vars for autoloading.
Chong Yidong <cyd@stupidchicken.com>
parents:
104419
diff
changeset
|
676 |
104413
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
677 (defun semantic-lex-test (arg) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
678 "Test the semantic lexer in the current buffer. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
679 If universal argument ARG, then try the whole buffer." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
680 (interactive "P") |
104446
df08b7ab0ba0
lisp/cedet/semantic/analyze.el: Add local vars for autoloading.
Chong Yidong <cyd@stupidchicken.com>
parents:
104419
diff
changeset
|
681 (require 'semantic) |
104413
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
682 (let* ((start (current-time)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
683 (result (semantic-lex |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
684 (if arg (point-min) (point)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
685 (point-max))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
686 (end (current-time))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
687 (message "Elapsed Time: %.2f seconds." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
688 (semantic-elapsed-time start end)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
689 (pop-to-buffer "*Lexer Output*") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
690 (require 'pp) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
691 (erase-buffer) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
692 (insert (pp-to-string result)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
693 (goto-char (point-min)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
694 )) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
695 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
696 (defun semantic-lex-test-full-depth (arg) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
697 "Test the semantic lexer in the current buffer parsing through lists. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
698 Usually the lexer parses |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
699 If universal argument ARG, then try the whole buffer." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
700 (interactive "P") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
701 (let* ((start (current-time)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
702 (result (semantic-lex |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
703 (if arg (point-min) (point)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
704 (point-max) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
705 100)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
706 (end (current-time))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
707 (message "Elapsed Time: %.2f seconds." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
708 (semantic-elapsed-time start end)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
709 (pop-to-buffer "*Lexer Output*") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
710 (require 'pp) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
711 (erase-buffer) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
712 (insert (pp-to-string result)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
713 (goto-char (point-min)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
714 )) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
715 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
716 (defun semantic-lex-test-region (beg end) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
717 "Test the semantic lexer in the current buffer. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
718 Analyze the area between BEG and END." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
719 (interactive "r") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
720 (let ((result (semantic-lex beg end))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
721 (pop-to-buffer "*Lexer Output*") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
722 (require 'pp) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
723 (erase-buffer) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
724 (insert (pp-to-string result)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
725 (goto-char (point-min)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
726 )) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
727 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
728 (defvar semantic-lex-debug nil |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
729 "When non-nil, debug the local lexical analyzer.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
730 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
731 (defun semantic-lex-debug (arg) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
732 "Debug the semantic lexer in the current buffer. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
733 Argument ARG specifies of the analyze the whole buffer, or start at point. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
734 While engaged, each token identified by the lexer will be highlighted |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
735 in the target buffer A description of the current token will be |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
736 displayed in the minibuffer. Press SPC to move to the next lexical token." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
737 (interactive "P") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
738 (require 'semantic/debug) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
739 (let ((semantic-lex-debug t)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
740 (semantic-lex-test arg))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
741 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
742 (defun semantic-lex-highlight-token (token) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
743 "Highlight the lexical TOKEN. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
744 TOKEN is a lexical token with a START And END position. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
745 Return the overlay." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
746 (let ((o (semantic-make-overlay (semantic-lex-token-start token) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
747 (semantic-lex-token-end token)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
748 (semantic-overlay-put o 'face 'highlight) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
749 o)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
750 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
751 (defsubst semantic-lex-debug-break (token) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
752 "Break during lexical analysis at TOKEN." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
753 (when semantic-lex-debug |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
754 (let ((o nil)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
755 (unwind-protect |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
756 (progn |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
757 (when token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
758 (setq o (semantic-lex-highlight-token token))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
759 (semantic-read-event |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
760 (format "%S :: SPC - continue" token)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
761 ) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
762 (when o |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
763 (semantic-overlay-delete o)))))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
764 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
765 ;;; Lexical analyzer creation |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
766 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
767 ;; Code for creating a lex function from lists of analyzers. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
768 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
769 ;; A lexical analyzer is created from a list of individual analyzers. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
770 ;; Each individual analyzer specifies a single match, and code that |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
771 ;; goes with it. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
772 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
773 ;; Creation of an analyzer assembles these analyzers into a new function |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
774 ;; with the behaviors of all the individual analyzers. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
775 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
776 (defmacro semantic-lex-one-token (analyzers) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
777 "Calculate one token from the current buffer at point. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
778 Uses locally bound variables from `define-lex'. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
779 Argument ANALYZERS is the list of analyzers being used." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
780 (cons 'cond (mapcar #'symbol-value analyzers))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
781 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
782 (defvar semantic-lex-end-point nil |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
783 "The end point as tracked through lexical functions.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
784 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
785 (defvar semantic-lex-current-depth nil |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
786 "The current depth as tracked through lexical functions.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
787 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
788 (defvar semantic-lex-maximum-depth nil |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
789 "The maximum depth of parenthisis as tracked through lexical functions.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
790 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
791 (defvar semantic-lex-token-stream nil |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
792 "The current token stream we are collecting.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
793 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
794 (defvar semantic-lex-analysis-bounds nil |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
795 "The bounds of the current analysis.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
796 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
797 (defvar semantic-lex-block-streams nil |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
798 "Streams of tokens inside collapsed blocks. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
799 This is an alist of (ANCHOR . STREAM) elements where ANCHOR is the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
800 start position of the block, and STREAM is the list of tokens in that |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
801 block.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
802 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
803 (defvar semantic-lex-reset-hooks nil |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
804 "List of hooks major-modes use to reset lexical analyzers. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
805 Hooks are called with START and END values for the current lexical pass. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
806 Should be set with `add-hook'specifying a LOCAL option.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
807 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
808 ;; Stack of nested blocks. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
809 (defvar semantic-lex-block-stack nil) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
810 ;;(defvar semantic-lex-timeout 5 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
811 ;; "*Number of sections of lexing before giving up.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
812 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
813 (defmacro define-lex (name doc &rest analyzers) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
814 "Create a new lexical analyzer with NAME. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
815 DOC is a documentation string describing this analyzer. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
816 ANALYZERS are small code snippets of analyzers to use when |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
817 building the new NAMED analyzer. Only use analyzers which |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
818 are written to be used in `define-lex'. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
819 Each analyzer should be an analyzer created with `define-lex-analyzer'. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
820 Note: The order in which analyzers are listed is important. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
821 If two analyzers can match the same text, it is important to order the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
822 analyzers so that the one you want to match first occurs first. For |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
823 example, it is good to put a numbe analyzer in front of a symbol |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
824 analyzer which might mistake a number for as a symbol." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
825 `(defun ,name (start end &optional depth length) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
826 ,(concat doc "\nSee `semantic-lex' for more information.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
827 ;; Make sure the state of block parsing starts over. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
828 (setq semantic-lex-block-streams nil) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
829 ;; Allow specialty reset items. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
830 (run-hook-with-args 'semantic-lex-reset-hooks start end) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
831 ;; Lexing state. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
832 (let* (;(starttime (current-time)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
833 (starting-position (point)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
834 (semantic-lex-token-stream nil) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
835 (semantic-lex-block-stack nil) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
836 (tmp-start start) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
837 (semantic-lex-end-point start) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
838 (semantic-lex-current-depth 0) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
839 ;; Use the default depth when not specified. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
840 (semantic-lex-maximum-depth |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
841 (or depth semantic-lex-depth)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
842 ;; Bounds needed for unterminated syntax |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
843 (semantic-lex-analysis-bounds (cons start end)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
844 ;; This entry prevents text properties from |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
845 ;; confusing our lexical analysis. See Emacs 22 (CVS) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
846 ;; version of C++ mode with template hack text properties. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
847 (parse-sexp-lookup-properties nil) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
848 ) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
849 ;; Maybe REMOVE THIS LATER. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
850 ;; Trying to find incremental parser bug. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
851 (when (> end (point-max)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
852 (error ,(format "%s: end (%%d) > point-max (%%d)" name) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
853 end (point-max))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
854 (with-syntax-table semantic-lex-syntax-table |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
855 (goto-char start) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
856 (while (and (< (point) end) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
857 (or (not length) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
858 (<= (length semantic-lex-token-stream) length))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
859 (semantic-lex-one-token ,analyzers) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
860 (when (eq semantic-lex-end-point tmp-start) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
861 (error ,(format "%s: endless loop at %%d, after %%S" name) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
862 tmp-start (car semantic-lex-token-stream))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
863 (setq tmp-start semantic-lex-end-point) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
864 (goto-char semantic-lex-end-point) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
865 ;;(when (> (semantic-elapsed-time starttime (current-time)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
866 ;; semantic-lex-timeout) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
867 ;; (error "Timeout during lex at char %d" (point))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
868 (semantic-throw-on-input 'lex) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
869 (semantic-lex-debug-break (car semantic-lex-token-stream)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
870 )) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
871 ;; Check that there is no unterminated block. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
872 (when semantic-lex-block-stack |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
873 (let* ((last (pop semantic-lex-block-stack)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
874 (blk last)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
875 (while blk |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
876 (message |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
877 ,(format "%s: `%%s' block from %%S is unterminated" name) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
878 (car blk) (cadr blk)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
879 (setq blk (pop semantic-lex-block-stack))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
880 (semantic-lex-unterminated-syntax-detected (car last)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
881 ;; Return to where we started. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
882 ;; Do not wrap in protective stuff so that if there is an error |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
883 ;; thrown, the user knows where. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
884 (goto-char starting-position) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
885 ;; Return the token stream |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
886 (nreverse semantic-lex-token-stream)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
887 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
888 ;;; Collapsed block tokens delimited by any tokens. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
889 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
890 (defun semantic-lex-start-block (syntax) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
891 "Mark the last read token as the beginning of a SYNTAX block." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
892 (if (or (not semantic-lex-maximum-depth) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
893 (< semantic-lex-current-depth semantic-lex-maximum-depth)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
894 (setq semantic-lex-current-depth (1+ semantic-lex-current-depth)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
895 (push (list syntax (car semantic-lex-token-stream)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
896 semantic-lex-block-stack))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
897 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
898 (defun semantic-lex-end-block (syntax) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
899 "Process the end of a previously marked SYNTAX block. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
900 That is, collapse the tokens inside that block, including the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
901 beginning and end of block tokens, into a high level block token of |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
902 class SYNTAX. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
903 The token at beginning of block is the one marked by a previous call |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
904 to `semantic-lex-start-block'. The current token is the end of block. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
905 The collapsed tokens are saved in `semantic-lex-block-streams'." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
906 (if (null semantic-lex-block-stack) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
907 (setq semantic-lex-current-depth (1- semantic-lex-current-depth)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
908 (let* ((stream semantic-lex-token-stream) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
909 (blk (pop semantic-lex-block-stack)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
910 (bstream (cdr blk)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
911 (first (car bstream)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
912 (last (pop stream)) ;; The current token mark the EOBLK |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
913 tok) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
914 (if (not (eq (car blk) syntax)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
915 ;; SYNTAX doesn't match the syntax of the current block in |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
916 ;; the stack. So we encountered the end of the SYNTAX block |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
917 ;; before the end of the current one in the stack which is |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
918 ;; signaled unterminated. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
919 (semantic-lex-unterminated-syntax-detected (car blk)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
920 ;; Move tokens found inside the block from the main stream |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
921 ;; into a separate block stream. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
922 (while (and stream (not (eq (setq tok (pop stream)) first))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
923 (push tok bstream)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
924 ;; The token marked as beginning of block was not encountered. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
925 ;; This should not happen! |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
926 (or (eq tok first) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
927 (error "Token %S not found at beginning of block `%s'" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
928 first syntax)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
929 ;; Save the block stream for future reuse, to avoid to redo |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
930 ;; the lexical analysis of the block content! |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
931 ;; Anchor the block stream with its start position, so we can |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
932 ;; use: (cdr (assq start semantic-lex-block-streams)) to |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
933 ;; quickly retrieve the lexical stream associated to a block. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
934 (setcar blk (semantic-lex-token-start first)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
935 (setcdr blk (nreverse bstream)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
936 (push blk semantic-lex-block-streams) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
937 ;; In the main stream, replace the tokens inside the block by |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
938 ;; a high level block token of class SYNTAX. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
939 (setq semantic-lex-token-stream stream) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
940 (semantic-lex-push-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
941 (semantic-lex-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
942 syntax (car blk) (semantic-lex-token-end last))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
943 )))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
944 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
945 ;;; Lexical token API |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
946 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
947 ;; Functions for accessing parts of a token. Use these functions |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
948 ;; instead of accessing the list structure directly because the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
949 ;; contents of the lexical may change. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
950 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
951 (defmacro semantic-lex-token (symbol start end &optional str) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
952 "Create a lexical token. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
953 SYMBOL is a symbol representing the class of syntax found. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
954 START and END define the bounds of the token in the current buffer. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
955 Optional STR is the string for the token iff the the bounds |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
956 in the buffer do not cover the string they represent. (As from |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
957 macro expansion.)" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
958 ;; This if statement checks the existance of a STR argument at |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
959 ;; compile time, where STR is some symbol or constant. If the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
960 ;; variable STr (runtime) is nil, this will make an incorrect decision. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
961 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
962 ;; It is like this to maintain the original speed of the compiled |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
963 ;; code. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
964 (if str |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
965 `(cons ,symbol (cons ,str (cons ,start ,end))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
966 `(cons ,symbol (cons ,start ,end)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
967 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
968 (defun semantic-lex-token-p (thing) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
969 "Return non-nil if THING is a semantic lex token. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
970 This is an exhaustively robust check." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
971 (and (consp thing) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
972 (symbolp (car thing)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
973 (or (and (numberp (nth 1 thing)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
974 (numberp (nthcdr 2 thing))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
975 (and (stringp (nth 1 thing)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
976 (numberp (nth 2 thing)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
977 (numberp (nthcdr 3 thing))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
978 )) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
979 ) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
980 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
981 (defun semantic-lex-token-with-text-p (thing) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
982 "Return non-nil if THING is a semantic lex token. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
983 This is an exhaustively robust check." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
984 (and (consp thing) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
985 (symbolp (car thing)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
986 (= (length thing) 4) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
987 (stringp (nth 1 thing)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
988 (numberp (nth 2 thing)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
989 (numberp (nth 3 thing))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
990 ) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
991 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
992 (defun semantic-lex-token-without-text-p (thing) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
993 "Return non-nil if THING is a semantic lex token. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
994 This is an exhaustively robust check." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
995 (and (consp thing) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
996 (symbolp (car thing)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
997 (= (length thing) 3) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
998 (numberp (nth 1 thing)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
999 (numberp (nth 2 thing))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1000 ) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1001 |
104446
df08b7ab0ba0
lisp/cedet/semantic/analyze.el: Add local vars for autoloading.
Chong Yidong <cyd@stupidchicken.com>
parents:
104419
diff
changeset
|
1002 (eval-and-compile |
df08b7ab0ba0
lisp/cedet/semantic/analyze.el: Add local vars for autoloading.
Chong Yidong <cyd@stupidchicken.com>
parents:
104419
diff
changeset
|
1003 |
104413
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1004 (defun semantic-lex-expand-block-specs (specs) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1005 "Expand block specifications SPECS into a Lisp form. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1006 SPECS is a list of (BLOCK BEGIN END) elements where BLOCK, BEGIN, and |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1007 END are token class symbols that indicate to produce one collapsed |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1008 BLOCK token from tokens found between BEGIN and END ones. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1009 BLOCK must be a non-nil symbol, and at least one of the BEGIN or END |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1010 symbols must be non-nil too. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1011 When BEGIN is non-nil, generate a call to `semantic-lex-start-block' |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1012 when a BEGIN token class is encountered. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1013 When END is non-nil, generate a call to `semantic-lex-end-block' when |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1014 an END token class is encountered." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1015 (let ((class (make-symbol "class")) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1016 (form nil)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1017 (dolist (spec specs) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1018 (when (car spec) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1019 (when (nth 1 spec) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1020 (push `((eq ',(nth 1 spec) ,class) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1021 (semantic-lex-start-block ',(car spec))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1022 form)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1023 (when (nth 2 spec) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1024 (push `((eq ',(nth 2 spec) ,class) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1025 (semantic-lex-end-block ',(car spec))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1026 form)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1027 (when form |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1028 `((let ((,class (semantic-lex-token-class |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1029 (car semantic-lex-token-stream)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1030 (cond ,@(nreverse form)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1031 ))) |
104446
df08b7ab0ba0
lisp/cedet/semantic/analyze.el: Add local vars for autoloading.
Chong Yidong <cyd@stupidchicken.com>
parents:
104419
diff
changeset
|
1032 ) |
104413
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1033 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1034 (defmacro semantic-lex-push-token (token &rest blockspecs) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1035 "Push TOKEN in the lexical analyzer token stream. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1036 Return the lexical analysis current end point. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1037 If optional arguments BLOCKSPECS is non-nil, it specifies to process |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1038 collapsed block tokens. See `semantic-lex-expand-block-specs' for |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1039 more details. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1040 This macro should only be called within the bounds of |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1041 `define-lex-analyzer'. It changes the values of the lexical analyzer |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1042 variables `token-stream' and `semantic-lex-end-point'. If you need to |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1043 move `semantic-lex-end-point' somewhere else, just modify this |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1044 variable after calling `semantic-lex-push-token'." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1045 `(progn |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1046 (push ,token semantic-lex-token-stream) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1047 ,@(semantic-lex-expand-block-specs blockspecs) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1048 (setq semantic-lex-end-point |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1049 (semantic-lex-token-end (car semantic-lex-token-stream))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1050 )) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1051 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1052 (defsubst semantic-lex-token-class (token) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1053 "Fetch the class of the lexical token TOKEN. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1054 See also the function `semantic-lex-token'." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1055 (car token)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1056 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1057 (defsubst semantic-lex-token-text (token) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1058 "Fetch the text associated with the lexical token TOKEN. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1059 See also the function `semantic-lex-token'." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1060 (if (stringp (car (cdr token))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1061 (car (cdr token)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1062 (buffer-substring-no-properties |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1063 (semantic-lex-token-start token) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1064 (semantic-lex-token-end token)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1065 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1066 (defun semantic-lex-init () |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1067 "Initialize any lexical state for this buffer." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1068 (unless semantic-lex-comment-regex |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1069 (setq semantic-lex-comment-regex |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1070 (if comment-start-skip |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1071 (concat "\\(\\s<\\|" comment-start-skip "\\)") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1072 "\\(\\s<\\)"))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1073 ;; Setup the lexer syntax-table |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1074 (setq semantic-lex-syntax-table (copy-syntax-table (syntax-table))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1075 (dolist (mod semantic-lex-syntax-modifications) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1076 (modify-syntax-entry |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1077 (car mod) (nth 1 mod) semantic-lex-syntax-table))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1078 |
104446
df08b7ab0ba0
lisp/cedet/semantic/analyze.el: Add local vars for autoloading.
Chong Yidong <cyd@stupidchicken.com>
parents:
104419
diff
changeset
|
1079 ;;;###autoload |
104413
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1080 (define-overloadable-function semantic-lex (start end &optional depth length) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1081 "Lexically analyze text in the current buffer between START and END. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1082 Optional argument DEPTH indicates at what level to scan over entire |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1083 lists. The last argument, LENGTH specifies that `semantic-lex' |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1084 should only return LENGTH tokens. The return value is a token stream. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1085 Each element is a list, such of the form |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1086 (symbol start-expression . end-expression) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1087 where SYMBOL denotes the token type. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1088 See `semantic-lex-tokens' variable for details on token types. END |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1089 does not mark the end of the text scanned, only the end of the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1090 beginning of text scanned. Thus, if a string extends past END, the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1091 end of the return token will be larger than END. To truly restrict |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1092 scanning, use `narrow-to-region'." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1093 (funcall semantic-lex-analyzer start end depth length)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1094 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1095 (defsubst semantic-lex-buffer (&optional depth) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1096 "Lex the current buffer. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1097 Optional argument DEPTH is the depth to scan into lists." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1098 (semantic-lex (point-min) (point-max) depth)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1099 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1100 (defsubst semantic-lex-list (semlist depth) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1101 "Lex the body of SEMLIST to DEPTH." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1102 (semantic-lex (semantic-lex-token-start semlist) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1103 (semantic-lex-token-end semlist) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1104 depth)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1105 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1106 ;;; Analyzer creation macros |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1107 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1108 ;; An individual analyzer is a condition and code that goes with it. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1109 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1110 ;; Created analyzers become variables with the code associated with them |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1111 ;; as the symbol value. These analyzers are assembled into a lexer |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1112 ;; to create new lexical analyzers. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1113 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1114 (defcustom semantic-lex-debug-analyzers nil |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1115 "Non nil means to debug analyzers with syntax protection. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1116 Only in effect if `debug-on-error' is also non-nil." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1117 :group 'semantic |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1118 :type 'boolean) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1119 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1120 (defmacro semantic-lex-unterminated-syntax-protection (syntax &rest forms) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1121 "For SYNTAX, execute FORMS with protection for unterminated syntax. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1122 If FORMS throws an error, treat this as a syntax problem, and |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1123 execute the unterminated syntax code. FORMS should return a position. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1124 Irreguardless of an error, the cursor should be moved to the end of |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1125 the desired syntax, and a position returned. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1126 If `debug-on-error' is set, errors are not caught, so that you can |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1127 debug them. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1128 Avoid using a large FORMS since it is duplicated." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1129 `(if (and debug-on-error semantic-lex-debug-analyzers) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1130 (progn ,@forms) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1131 (condition-case nil |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1132 (progn ,@forms) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1133 (error |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1134 (semantic-lex-unterminated-syntax-detected ,syntax))))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1135 (put 'semantic-lex-unterminated-syntax-protection |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1136 'lisp-indent-function 1) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1137 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1138 (defmacro define-lex-analyzer (name doc condition &rest forms) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1139 "Create a single lexical analyzer NAME with DOC. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1140 When an analyzer is called, the current buffer and point are |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1141 positioned in a buffer at the location to be analyzed. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1142 CONDITION is an expression which returns t if FORMS should be run. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1143 Within the bounds of CONDITION and FORMS, the use of backquote |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1144 can be used to evaluate expressions at compile time. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1145 While forms are running, the following variables will be locally bound: |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1146 `semantic-lex-analysis-bounds' - The bounds of the current analysis. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1147 of the form (START . END) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1148 `semantic-lex-maximum-depth' - The maximum depth of semantic-list |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1149 for the current analysis. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1150 `semantic-lex-current-depth' - The current depth of `semantic-list' that has |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1151 been decended. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1152 `semantic-lex-end-point' - End Point after match. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1153 Analyzers should set this to a buffer location if their |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1154 match string does not represent the end of the matched text. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1155 `semantic-lex-token-stream' - The token list being collected. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1156 Add new lexical tokens to this list. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1157 Proper action in FORMS is to move the value of `semantic-lex-end-point' to |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1158 after the location of the analyzed entry, and to add any discovered tokens |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1159 at the beginning of `semantic-lex-token-stream'. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1160 This can be done by using `semantic-lex-push-token'." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1161 `(eval-and-compile |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1162 (defvar ,name nil ,doc) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1163 (defun ,name nil) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1164 ;; Do this part separately so that re-evaluation rebuilds this code. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1165 (setq ,name '(,condition ,@forms)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1166 ;; Build a single lexical analyzer function, so the doc for |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1167 ;; function help is automatically provided, and perhaps the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1168 ;; function could be useful for testing and debugging one |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1169 ;; analyzer. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1170 (fset ',name (lambda () ,doc |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1171 (let ((semantic-lex-token-stream nil) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1172 (semantic-lex-end-point (point)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1173 (semantic-lex-analysis-bounds |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1174 (cons (point) (point-max))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1175 (semantic-lex-current-depth 0) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1176 (semantic-lex-maximum-depth |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1177 semantic-lex-depth) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1178 ) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1179 (when ,condition ,@forms) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1180 semantic-lex-token-stream))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1181 )) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1182 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1183 (defmacro define-lex-regex-analyzer (name doc regexp &rest forms) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1184 "Create a lexical analyzer with NAME and DOC that will match REGEXP. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1185 FORMS are evaluated upon a successful match. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1186 See `define-lex-analyzer' for more about analyzers." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1187 `(define-lex-analyzer ,name |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1188 ,doc |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1189 (looking-at ,regexp) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1190 ,@forms |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1191 )) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1192 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1193 (defmacro define-lex-simple-regex-analyzer (name doc regexp toksym |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1194 &optional index |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1195 &rest forms) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1196 "Create a lexical analyzer with NAME and DOC that match REGEXP. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1197 TOKSYM is the symbol to use when creating a semantic lexical token. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1198 INDEX is the index into the match that defines the bounds of the token. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1199 Index should be a plain integer, and not specified in the macro as an |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1200 expression. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1201 FORMS are evaluated upon a successful match BEFORE the new token is |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1202 created. It is valid to ignore FORMS. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1203 See `define-lex-analyzer' for more about analyzers." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1204 `(define-lex-analyzer ,name |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1205 ,doc |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1206 (looking-at ,regexp) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1207 ,@forms |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1208 (semantic-lex-push-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1209 (semantic-lex-token ,toksym |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1210 (match-beginning ,(or index 0)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1211 (match-end ,(or index 0)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1212 )) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1213 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1214 (defmacro define-lex-block-analyzer (name doc spec1 &rest specs) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1215 "Create a lexical analyzer NAME for paired delimiters blocks. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1216 It detects a paired delimiters block or the corresponding open or |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1217 close delimiter depending on the value of the variable |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1218 `semantic-lex-current-depth'. DOC is the documentation string of the lexical |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1219 analyzer. SPEC1 and SPECS specify the token symbols and open, close |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1220 delimiters used. Each SPEC has the form: |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1221 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1222 \(BLOCK-SYM (OPEN-DELIM OPEN-SYM) (CLOSE-DELIM CLOSE-SYM)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1223 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1224 where BLOCK-SYM is the symbol returned in a block token. OPEN-DELIM |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1225 and CLOSE-DELIM are respectively the open and close delimiters |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1226 identifying a block. OPEN-SYM and CLOSE-SYM are respectively the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1227 symbols returned in open and close tokens." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1228 (let ((specs (cons spec1 specs)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1229 spec open olist clist) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1230 (while specs |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1231 (setq spec (car specs) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1232 specs (cdr specs) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1233 open (nth 1 spec) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1234 ;; build alist ((OPEN-DELIM OPEN-SYM BLOCK-SYM) ...) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1235 olist (cons (list (car open) (cadr open) (car spec)) olist) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1236 ;; build alist ((CLOSE-DELIM CLOSE-SYM) ...) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1237 clist (cons (nth 2 spec) clist))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1238 `(define-lex-analyzer ,name |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1239 ,doc |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1240 (and |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1241 (looking-at "\\(\\s(\\|\\s)\\)") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1242 (let ((text (match-string 0)) match) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1243 (cond |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1244 ((setq match (assoc text ',olist)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1245 (if (or (not semantic-lex-maximum-depth) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1246 (< semantic-lex-current-depth semantic-lex-maximum-depth)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1247 (progn |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1248 (setq semantic-lex-current-depth (1+ semantic-lex-current-depth)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1249 (semantic-lex-push-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1250 (semantic-lex-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1251 (nth 1 match) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1252 (match-beginning 0) (match-end 0)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1253 (semantic-lex-push-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1254 (semantic-lex-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1255 (nth 2 match) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1256 (match-beginning 0) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1257 (save-excursion |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1258 (semantic-lex-unterminated-syntax-protection (nth 2 match) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1259 (forward-list 1) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1260 (point))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1261 )) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1262 )) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1263 ((setq match (assoc text ',clist)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1264 (setq semantic-lex-current-depth (1- semantic-lex-current-depth)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1265 (semantic-lex-push-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1266 (semantic-lex-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1267 (nth 1 match) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1268 (match-beginning 0) (match-end 0))))))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1269 ))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1270 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1271 ;;; Analyzers |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1272 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1273 ;; Pre-defined common analyzers. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1274 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1275 (define-lex-analyzer semantic-lex-default-action |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1276 "The default action when no other lexical actions match text. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1277 This action will just throw an error." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1278 t |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1279 (error "Unmatched Text during Lexical Analysis")) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1280 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1281 (define-lex-analyzer semantic-lex-beginning-of-line |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1282 "Detect and create a beginning of line token (BOL)." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1283 (and (bolp) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1284 ;; Just insert a (bol N . N) token in the token stream, |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1285 ;; without moving the point. N is the point at the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1286 ;; beginning of line. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1287 (semantic-lex-push-token (semantic-lex-token 'bol (point) (point))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1288 nil) ;; CONTINUE |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1289 ;; We identify and add the BOL token onto the stream, but since |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1290 ;; semantic-lex-end-point doesn't move, we always fail CONDITION, and have no |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1291 ;; FORMS body. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1292 nil) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1293 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1294 (define-lex-simple-regex-analyzer semantic-lex-newline |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1295 "Detect and create newline tokens." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1296 "\\s-*\\(\n\\|\\s>\\)" 'newline 1) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1297 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1298 (define-lex-regex-analyzer semantic-lex-newline-as-whitespace |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1299 "Detect and create newline tokens. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1300 Use this ONLY if newlines are not whitespace characters (such as when |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1301 they are comment end characters) AND when you want whitespace tokens." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1302 "\\s-*\\(\n\\|\\s>\\)" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1303 ;; Language wants whitespaces. Create a token for it. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1304 (if (eq (semantic-lex-token-class (car semantic-lex-token-stream)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1305 'whitespace) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1306 ;; Merge whitespace tokens together if they are adjacent. Two |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1307 ;; whitespace tokens may be sperated by a comment which is not in |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1308 ;; the token stream. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1309 (setcdr (semantic-lex-token-bounds (car semantic-lex-token-stream)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1310 (match-end 0)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1311 (semantic-lex-push-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1312 (semantic-lex-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1313 'whitespace (match-beginning 0) (match-end 0))))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1314 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1315 (define-lex-regex-analyzer semantic-lex-ignore-newline |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1316 "Detect and ignore newline tokens. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1317 Use this ONLY if newlines are not whitespace characters (such as when |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1318 they are comment end characters)." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1319 "\\s-*\\(\n\\|\\s>\\)" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1320 (setq semantic-lex-end-point (match-end 0))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1321 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1322 (define-lex-regex-analyzer semantic-lex-whitespace |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1323 "Detect and create whitespace tokens." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1324 ;; catch whitespace when needed |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1325 "\\s-+" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1326 ;; Language wants whitespaces. Create a token for it. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1327 (if (eq (semantic-lex-token-class (car semantic-lex-token-stream)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1328 'whitespace) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1329 ;; Merge whitespace tokens together if they are adjacent. Two |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1330 ;; whitespace tokens may be sperated by a comment which is not in |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1331 ;; the token stream. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1332 (progn |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1333 (setq semantic-lex-end-point (match-end 0)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1334 (setcdr (semantic-lex-token-bounds (car semantic-lex-token-stream)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1335 semantic-lex-end-point)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1336 (semantic-lex-push-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1337 (semantic-lex-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1338 'whitespace (match-beginning 0) (match-end 0))))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1339 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1340 (define-lex-regex-analyzer semantic-lex-ignore-whitespace |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1341 "Detect and skip over whitespace tokens." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1342 ;; catch whitespace when needed |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1343 "\\s-+" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1344 ;; Skip over the detected whitespace, do not create a token for it. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1345 (setq semantic-lex-end-point (match-end 0))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1346 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1347 (define-lex-simple-regex-analyzer semantic-lex-number |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1348 "Detect and create number tokens. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1349 See `semantic-lex-number-expression' for details on matching numbers, |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1350 and number formats." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1351 semantic-lex-number-expression 'number) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1352 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1353 (define-lex-regex-analyzer semantic-lex-symbol-or-keyword |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1354 "Detect and create symbol and keyword tokens." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1355 "\\(\\sw\\|\\s_\\)+" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1356 (semantic-lex-push-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1357 (semantic-lex-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1358 (or (semantic-lex-keyword-p (match-string 0)) 'symbol) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1359 (match-beginning 0) (match-end 0)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1360 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1361 (define-lex-simple-regex-analyzer semantic-lex-charquote |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1362 "Detect and create charquote tokens." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1363 ;; Character quoting characters (ie, \n as newline) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1364 "\\s\\+" 'charquote) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1365 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1366 (define-lex-simple-regex-analyzer semantic-lex-punctuation |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1367 "Detect and create punctuation tokens." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1368 "\\(\\s.\\|\\s$\\|\\s'\\)" 'punctuation) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1369 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1370 (define-lex-analyzer semantic-lex-punctuation-type |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1371 "Detect and create a punctuation type token. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1372 Recognized punctuations are defined in the current table of lexical |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1373 types, as the value of the `punctuation' token type." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1374 (and (looking-at "\\(\\s.\\|\\s$\\|\\s'\\)+") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1375 (let* ((key (match-string 0)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1376 (pos (match-beginning 0)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1377 (end (match-end 0)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1378 (len (- end pos)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1379 (lst (semantic-lex-type-value "punctuation" t)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1380 (def (car lst)) ;; default lexical symbol or nil |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1381 (lst (cdr lst)) ;; alist of (LEX-SYM . PUNCT-STRING) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1382 (elt nil)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1383 (if lst |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1384 ;; Starting with the longest one, search if the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1385 ;; punctuation string is defined for this language. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1386 (while (and (> len 0) (not (setq elt (rassoc key lst)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1387 (setq len (1- len) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1388 key (substring key 0 len)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1389 (if elt ;; Return the punctuation token found |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1390 (semantic-lex-push-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1391 (semantic-lex-token (car elt) pos (+ pos len))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1392 (if def ;; Return a default generic token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1393 (semantic-lex-push-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1394 (semantic-lex-token def pos end)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1395 ;; Nothing match |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1396 ))))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1397 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1398 (define-lex-regex-analyzer semantic-lex-paren-or-list |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1399 "Detect open parenthesis. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1400 Return either a paren token or a semantic list token depending on |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1401 `semantic-lex-current-depth'." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1402 "\\s(" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1403 (if (or (not semantic-lex-maximum-depth) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1404 (< semantic-lex-current-depth semantic-lex-maximum-depth)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1405 (progn |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1406 (setq semantic-lex-current-depth (1+ semantic-lex-current-depth)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1407 (semantic-lex-push-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1408 (semantic-lex-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1409 'open-paren (match-beginning 0) (match-end 0)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1410 (semantic-lex-push-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1411 (semantic-lex-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1412 'semantic-list (match-beginning 0) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1413 (save-excursion |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1414 (semantic-lex-unterminated-syntax-protection 'semantic-list |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1415 (forward-list 1) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1416 (point)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1417 ))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1418 )) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1419 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1420 (define-lex-simple-regex-analyzer semantic-lex-open-paren |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1421 "Detect and create an open parenthisis token." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1422 "\\s(" 'open-paren 0 (setq semantic-lex-current-depth (1+ semantic-lex-current-depth))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1423 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1424 (define-lex-simple-regex-analyzer semantic-lex-close-paren |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1425 "Detect and create a close paren token." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1426 "\\s)" 'close-paren 0 (setq semantic-lex-current-depth (1- semantic-lex-current-depth))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1427 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1428 (define-lex-regex-analyzer semantic-lex-string |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1429 "Detect and create a string token." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1430 "\\s\"" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1431 ;; Zing to the end of this string. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1432 (semantic-lex-push-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1433 (semantic-lex-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1434 'string (point) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1435 (save-excursion |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1436 (semantic-lex-unterminated-syntax-protection 'string |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1437 (forward-sexp 1) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1438 (point)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1439 )))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1440 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1441 (define-lex-regex-analyzer semantic-lex-comments |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1442 "Detect and create a comment token." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1443 semantic-lex-comment-regex |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1444 (save-excursion |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1445 (forward-comment 1) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1446 ;; Generate newline token if enabled |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1447 (if (bolp) (backward-char 1)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1448 (setq semantic-lex-end-point (point)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1449 ;; Language wants comments or want them as whitespaces, |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1450 ;; link them together. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1451 (if (eq (semantic-lex-token-class (car semantic-lex-token-stream)) 'comment) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1452 (setcdr (semantic-lex-token-bounds (car semantic-lex-token-stream)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1453 semantic-lex-end-point) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1454 (semantic-lex-push-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1455 (semantic-lex-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1456 'comment (match-beginning 0) semantic-lex-end-point))))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1457 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1458 (define-lex-regex-analyzer semantic-lex-comments-as-whitespace |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1459 "Detect comments and create a whitespace token." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1460 semantic-lex-comment-regex |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1461 (save-excursion |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1462 (forward-comment 1) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1463 ;; Generate newline token if enabled |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1464 (if (bolp) (backward-char 1)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1465 (setq semantic-lex-end-point (point)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1466 ;; Language wants comments or want them as whitespaces, |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1467 ;; link them together. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1468 (if (eq (semantic-lex-token-class (car semantic-lex-token-stream)) 'whitespace) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1469 (setcdr (semantic-lex-token-bounds (car semantic-lex-token-stream)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1470 semantic-lex-end-point) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1471 (semantic-lex-push-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1472 (semantic-lex-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1473 'whitespace (match-beginning 0) semantic-lex-end-point))))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1474 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1475 (define-lex-regex-analyzer semantic-lex-ignore-comments |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1476 "Detect and create a comment token." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1477 semantic-lex-comment-regex |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1478 (let ((comment-start-point (point))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1479 (forward-comment 1) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1480 (if (eq (point) comment-start-point) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1481 ;; In this case our start-skip string failed |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1482 ;; to work properly. Lets try and move over |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1483 ;; whatever white space we matched to begin |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1484 ;; with. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1485 (skip-syntax-forward "-.'" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1486 (save-excursion |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1487 (end-of-line) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1488 (point))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1489 ;; We may need to back up so newlines or whitespace is generated. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1490 (if (bolp) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1491 (backward-char 1))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1492 (if (eq (point) comment-start-point) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1493 (error "Strange comment syntax prevents lexical analysis")) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1494 (setq semantic-lex-end-point (point)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1495 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1496 ;;; Comment lexer |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1497 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1498 ;; Predefined lexers that could be used instead of creating new |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1499 ;; analyers. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1500 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1501 (define-lex semantic-comment-lexer |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1502 "A simple lexical analyzer that handles comments. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1503 This lexer will only return comment tokens. It is the default lexer |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1504 used by `semantic-find-doc-snarf-comment' to snarf up the comment at |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1505 point." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1506 semantic-lex-ignore-whitespace |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1507 semantic-lex-ignore-newline |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1508 semantic-lex-comments |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1509 semantic-lex-default-action) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1510 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1511 ;;; Test Lexer |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1512 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1513 (define-lex semantic-simple-lexer |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1514 "A simple lexical analyzer that handles simple buffers. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1515 This lexer ignores comments and whitespace, and will return |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1516 syntax as specified by the syntax table." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1517 semantic-lex-ignore-whitespace |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1518 semantic-lex-ignore-newline |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1519 semantic-lex-number |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1520 semantic-lex-symbol-or-keyword |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1521 semantic-lex-charquote |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1522 semantic-lex-paren-or-list |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1523 semantic-lex-close-paren |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1524 semantic-lex-string |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1525 semantic-lex-ignore-comments |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1526 semantic-lex-punctuation |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1527 semantic-lex-default-action) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1528 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1529 ;;; Analyzers generated from grammar. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1530 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1531 ;; Some analyzers are hand written. Analyzers created with these |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1532 ;; functions are generated from the grammar files. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1533 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1534 (defmacro define-lex-keyword-type-analyzer (name doc syntax) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1535 "Define a keyword type analyzer NAME with DOC string. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1536 SYNTAX is the regexp that matches a keyword syntactic expression." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1537 (let ((key (make-symbol "key"))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1538 `(define-lex-analyzer ,name |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1539 ,doc |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1540 (and (looking-at ,syntax) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1541 (let ((,key (semantic-lex-keyword-p (match-string 0)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1542 (when ,key |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1543 (semantic-lex-push-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1544 (semantic-lex-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1545 ,key (match-beginning 0) (match-end 0))))))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1546 )) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1547 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1548 (defmacro define-lex-sexp-type-analyzer (name doc syntax token) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1549 "Define a sexp type analyzer NAME with DOC string. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1550 SYNTAX is the regexp that matches the beginning of the s-expression. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1551 TOKEN is the lexical token returned when SYNTAX matches." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1552 `(define-lex-regex-analyzer ,name |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1553 ,doc |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1554 ,syntax |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1555 (semantic-lex-push-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1556 (semantic-lex-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1557 ,token (point) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1558 (save-excursion |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1559 (semantic-lex-unterminated-syntax-protection ,token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1560 (forward-sexp 1) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1561 (point)))))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1562 ) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1563 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1564 (defmacro define-lex-regex-type-analyzer (name doc syntax matches default) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1565 "Define a regexp type analyzer NAME with DOC string. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1566 SYNTAX is the regexp that matches a syntactic expression. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1567 MATCHES is an alist of lexical elements used to refine the syntactic |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1568 expression. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1569 DEFAULT is the default lexical token returned when no MATCHES." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1570 (if matches |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1571 (let* ((val (make-symbol "val")) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1572 (lst (make-symbol "lst")) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1573 (elt (make-symbol "elt")) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1574 (pos (make-symbol "pos")) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1575 (end (make-symbol "end"))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1576 `(define-lex-analyzer ,name |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1577 ,doc |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1578 (and (looking-at ,syntax) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1579 (let* ((,val (match-string 0)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1580 (,pos (match-beginning 0)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1581 (,end (match-end 0)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1582 (,lst ,matches) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1583 ,elt) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1584 (while (and ,lst (not ,elt)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1585 (if (string-match (cdar ,lst) ,val) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1586 (setq ,elt (caar ,lst)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1587 (setq ,lst (cdr ,lst)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1588 (semantic-lex-push-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1589 (semantic-lex-token (or ,elt ,default) ,pos ,end)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1590 )) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1591 `(define-lex-simple-regex-analyzer ,name |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1592 ,doc |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1593 ,syntax ,default) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1594 )) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1595 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1596 (defmacro define-lex-string-type-analyzer (name doc syntax matches default) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1597 "Define a string type analyzer NAME with DOC string. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1598 SYNTAX is the regexp that matches a syntactic expression. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1599 MATCHES is an alist of lexical elements used to refine the syntactic |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1600 expression. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1601 DEFAULT is the default lexical token returned when no MATCHES." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1602 (if matches |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1603 (let* ((val (make-symbol "val")) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1604 (lst (make-symbol "lst")) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1605 (elt (make-symbol "elt")) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1606 (pos (make-symbol "pos")) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1607 (end (make-symbol "end")) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1608 (len (make-symbol "len"))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1609 `(define-lex-analyzer ,name |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1610 ,doc |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1611 (and (looking-at ,syntax) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1612 (let* ((,val (match-string 0)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1613 (,pos (match-beginning 0)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1614 (,end (match-end 0)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1615 (,len (- ,end ,pos)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1616 (,lst ,matches) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1617 ,elt) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1618 ;; Starting with the longest one, search if a lexical |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1619 ;; value match a token defined for this language. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1620 (while (and (> ,len 0) (not (setq ,elt (rassoc ,val ,lst)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1621 (setq ,len (1- ,len) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1622 ,val (substring ,val 0 ,len))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1623 (when ,elt ;; Adjust token end position. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1624 (setq ,elt (car ,elt) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1625 ,end (+ ,pos ,len))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1626 (semantic-lex-push-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1627 (semantic-lex-token (or ,elt ,default) ,pos ,end)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1628 )) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1629 `(define-lex-simple-regex-analyzer ,name |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1630 ,doc |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1631 ,syntax ,default) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1632 )) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1633 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1634 (defmacro define-lex-block-type-analyzer (name doc syntax matches) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1635 "Define a block type analyzer NAME with DOC string. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1636 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1637 SYNTAX is the regexp that matches block delimiters, typically the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1638 open (`\\\\s(') and close (`\\\\s)') parenthesis syntax classes. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1639 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1640 MATCHES is a pair (OPEN-SPECS . CLOSE-SPECS) that defines blocks. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1641 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1642 OPEN-SPECS is a list of (OPEN-DELIM OPEN-TOKEN BLOCK-TOKEN) elements |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1643 where: |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1644 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1645 OPEN-DELIM is a string: the block open delimiter character. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1646 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1647 OPEN-TOKEN is the lexical token class associated to the OPEN-DELIM |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1648 delimiter. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1649 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1650 BLOCK-TOKEN is the lexical token class associated to the block |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1651 that starts at the OPEN-DELIM delimiter. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1652 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1653 CLOSE-SPECS is a list of (CLOSE-DELIM CLOSE-TOKEN) elements where: |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1654 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1655 CLOSE-DELIM is a string: the block end delimiter character. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1656 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1657 CLOSE-TOKEN is the lexical token class associated to the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1658 CLOSE-DELIM delimiter. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1659 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1660 Each element in OPEN-SPECS must have a corresponding element in |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1661 CLOSE-SPECS. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1662 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1663 The lexer will return a BLOCK-TOKEN token when the value of |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1664 `semantic-lex-current-depth' is greater than or equal to the maximum |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1665 depth of parenthesis tracking (see also the function `semantic-lex'). |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1666 Otherwise it will return OPEN-TOKEN and CLOSE-TOKEN tokens. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1667 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1668 TO DO: Put the following in the developer's guide and just put a |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1669 reference here. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1670 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1671 In the grammar: |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1672 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1673 The value of a block token must be a string that contains a readable |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1674 sexp of the form: |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1675 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1676 \"(OPEN-TOKEN CLOSE-TOKEN)\" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1677 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1678 OPEN-TOKEN and CLOSE-TOKEN represent the block delimiters, and must be |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1679 lexical tokens of respectively `open-paren' and `close-paren' types. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1680 Their value is the corresponding delimiter character as a string. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1681 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1682 Here is a small example to analyze a parenthesis block: |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1683 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1684 %token <block> PAREN_BLOCK \"(LPAREN RPAREN)\" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1685 %token <open-paren> LPAREN \"(\" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1686 %token <close-paren> RPAREN \")\" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1687 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1688 When the lexer encounters the open-paren delimiter \"(\": |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1689 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1690 - If the maximum depth of parenthesis tracking is not reached (that |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1691 is, current depth < max depth), it returns a (LPAREN start . end) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1692 token, then continue analysis inside the block. Later, when the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1693 corresponding close-paren delimiter \")\" will be encountered, it |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1694 will return a (RPAREN start . end) token. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1695 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1696 - If the maximum depth of parenthesis tracking is reached (current |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1697 depth >= max depth), it returns the whole parenthesis block as |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1698 a (PAREN_BLOCK start . end) token." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1699 (let* ((val (make-symbol "val")) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1700 (lst (make-symbol "lst")) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1701 (elt (make-symbol "elt"))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1702 `(define-lex-analyzer ,name |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1703 ,doc |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1704 (and |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1705 (looking-at ,syntax) ;; "\\(\\s(\\|\\s)\\)" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1706 (let ((,val (match-string 0)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1707 (,lst ,matches) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1708 ,elt) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1709 (cond |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1710 ((setq ,elt (assoc ,val (car ,lst))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1711 (if (or (not semantic-lex-maximum-depth) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1712 (< semantic-lex-current-depth semantic-lex-maximum-depth)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1713 (progn |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1714 (setq semantic-lex-current-depth (1+ semantic-lex-current-depth)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1715 (semantic-lex-push-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1716 (semantic-lex-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1717 (nth 1 ,elt) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1718 (match-beginning 0) (match-end 0)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1719 (semantic-lex-push-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1720 (semantic-lex-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1721 (nth 2 ,elt) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1722 (match-beginning 0) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1723 (save-excursion |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1724 (semantic-lex-unterminated-syntax-protection (nth 2 ,elt) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1725 (forward-list 1) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1726 (point))))))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1727 ((setq ,elt (assoc ,val (cdr ,lst))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1728 (setq semantic-lex-current-depth (1- semantic-lex-current-depth)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1729 (semantic-lex-push-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1730 (semantic-lex-token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1731 (nth 1 ,elt) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1732 (match-beginning 0) (match-end 0)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1733 )))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1734 )) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1735 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1736 ;;; Lexical Safety |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1737 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1738 ;; The semantic lexers, unlike other lexers, can throw errors on |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1739 ;; unbalanced syntax. Since editing is all about changeging test |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1740 ;; we need to provide a convenient way to protect against syntactic |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1741 ;; inequalities. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1742 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1743 (defmacro semantic-lex-catch-errors (symbol &rest forms) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1744 "Using SYMBOL, execute FORMS catching lexical errors. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1745 If FORMS results in a call to the parser that throws a lexical error, |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1746 the error will be caught here without the buffer's cache being thrown |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1747 out of date. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1748 If there is an error, the syntax that failed is returned. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1749 If there is no error, then the last value of FORMS is returned." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1750 (let ((ret (make-symbol "ret")) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1751 (syntax (make-symbol "syntax")) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1752 (start (make-symbol "start")) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1753 (end (make-symbol "end"))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1754 `(let* ((semantic-lex-unterminated-syntax-end-function |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1755 (lambda (,syntax ,start ,end) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1756 (throw ',symbol ,syntax))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1757 ;; Delete the below when semantic-flex is fully retired. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1758 (semantic-flex-unterminated-syntax-end-function |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1759 semantic-lex-unterminated-syntax-end-function) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1760 (,ret (catch ',symbol |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1761 (save-excursion |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1762 ,@forms |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1763 nil)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1764 ;; Great Sadness. Assume that FORMS execute within the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1765 ;; confines of the current buffer only! Mark this thing |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1766 ;; unparseable iff the special symbol was thrown. This |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1767 ;; will prevent future calls from parsing, but will allow |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1768 ;; then to still return the cache. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1769 (when ,ret |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1770 ;; Leave this message off. If an APP using this fcn wants |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1771 ;; a message, they can do it themselves. This cleans up |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1772 ;; problems with the idle scheduler obscuring useful data. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1773 ;;(message "Buffer not currently parsable (%S)." ,ret) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1774 (semantic-parse-tree-unparseable)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1775 ,ret))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1776 (put 'semantic-lex-catch-errors 'lisp-indent-function 1) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1777 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1778 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1779 ;;; Interfacing with edebug |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1780 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1781 (add-hook |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1782 'edebug-setup-hook |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1783 #'(lambda () |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1784 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1785 (def-edebug-spec define-lex |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1786 (&define name stringp (&rest symbolp)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1787 ) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1788 (def-edebug-spec define-lex-analyzer |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1789 (&define name stringp form def-body) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1790 ) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1791 (def-edebug-spec define-lex-regex-analyzer |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1792 (&define name stringp form def-body) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1793 ) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1794 (def-edebug-spec define-lex-simple-regex-analyzer |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1795 (&define name stringp form symbolp [ &optional form ] def-body) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1796 ) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1797 (def-edebug-spec define-lex-block-analyzer |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1798 (&define name stringp form (&rest form)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1799 ) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1800 (def-edebug-spec semantic-lex-catch-errors |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1801 (symbolp def-body) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1802 ) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1803 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1804 )) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1805 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1806 ;;; Compatibility with Semantic 1.x lexical analysis |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1807 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1808 ;; NOTE: DELETE THIS SOMEDAY SOON |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1809 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1810 (semantic-alias-obsolete 'semantic-flex-start 'semantic-lex-token-start) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1811 (semantic-alias-obsolete 'semantic-flex-end 'semantic-lex-token-end) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1812 (semantic-alias-obsolete 'semantic-flex-text 'semantic-lex-token-text) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1813 (semantic-alias-obsolete 'semantic-flex-make-keyword-table 'semantic-lex-make-keyword-table) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1814 (semantic-alias-obsolete 'semantic-flex-keyword-p 'semantic-lex-keyword-p) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1815 (semantic-alias-obsolete 'semantic-flex-keyword-put 'semantic-lex-keyword-put) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1816 (semantic-alias-obsolete 'semantic-flex-keyword-get 'semantic-lex-keyword-get) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1817 (semantic-alias-obsolete 'semantic-flex-map-keywords 'semantic-lex-map-keywords) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1818 (semantic-alias-obsolete 'semantic-flex-keywords 'semantic-lex-keywords) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1819 (semantic-alias-obsolete 'semantic-flex-buffer 'semantic-lex-buffer) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1820 (semantic-alias-obsolete 'semantic-flex-list 'semantic-lex-list) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1821 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1822 ;; This simple scanner uses the syntax table to generate a stream of |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1823 ;; simple tokens of the form: |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1824 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1825 ;; (SYMBOL START . END) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1826 ;; |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1827 ;; Where symbol is the type of thing it is. START and END mark that |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1828 ;; objects boundary. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1829 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1830 (defvar semantic-flex-tokens semantic-lex-tokens |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1831 "An alist of of semantic token types. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1832 See variable `semantic-lex-tokens'.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1833 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1834 (defvar semantic-flex-unterminated-syntax-end-function |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1835 (lambda (syntax syntax-start flex-end) flex-end) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1836 "Function called when unterminated syntax is encountered. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1837 This should be set to one function. That function should take three |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1838 parameters. The SYNTAX, or type of syntax which is unterminated. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1839 SYNTAX-START where the broken syntax begins. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1840 FLEX-END is where the lexical analysis was asked to end. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1841 This function can be used for languages that can intelligently fix up |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1842 broken syntax, or the exit lexical analysis via `throw' or `signal' |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1843 when finding unterminated syntax.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1844 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1845 (defvar semantic-flex-extensions nil |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1846 "Buffer local extensions to the lexical analyzer. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1847 This should contain an alist with a key of a regex and a data element of |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1848 a function. The function should both move point, and return a lexical |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1849 token of the form: |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1850 ( TYPE START . END) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1851 nil is also a valid return value. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1852 TYPE can be any type of symbol, as long as it doesn't occur as a |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1853 nonterminal in the language definition.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1854 (make-variable-buffer-local 'semantic-flex-extensions) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1855 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1856 (defvar semantic-flex-syntax-modifications nil |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1857 "Changes to the syntax table for this buffer. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1858 These changes are active only while the buffer is being flexed. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1859 This is a list where each element has the form: |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1860 (CHAR CLASS) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1861 CHAR is the char passed to `modify-syntax-entry', |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1862 and CLASS is the string also passed to `modify-syntax-entry' to define |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1863 what syntax class CHAR has.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1864 (make-variable-buffer-local 'semantic-flex-syntax-modifications) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1865 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1866 (defvar semantic-ignore-comments t |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1867 "Default comment handling. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1868 t means to strip comments when flexing. Nil means to keep comments |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1869 as part of the token stream.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1870 (make-variable-buffer-local 'semantic-ignore-comments) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1871 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1872 (defvar semantic-flex-enable-newlines nil |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1873 "When flexing, report 'newlines as syntactic elements. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1874 Useful for languages where the newline is a special case terminator. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1875 Only set this on a per mode basis, not globally.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1876 (make-variable-buffer-local 'semantic-flex-enable-newlines) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1877 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1878 (defvar semantic-flex-enable-whitespace nil |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1879 "When flexing, report 'whitespace as syntactic elements. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1880 Useful for languages where the syntax is whitespace dependent. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1881 Only set this on a per mode basis, not globally.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1882 (make-variable-buffer-local 'semantic-flex-enable-whitespace) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1883 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1884 (defvar semantic-flex-enable-bol nil |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1885 "When flexing, report beginning of lines as syntactic elements. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1886 Useful for languages like python which are indentation sensitive. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1887 Only set this on a per mode basis, not globally.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1888 (make-variable-buffer-local 'semantic-flex-enable-bol) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1889 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1890 (defvar semantic-number-expression semantic-lex-number-expression |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1891 "See variable `semantic-lex-number-expression'.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1892 (make-variable-buffer-local 'semantic-number-expression) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1893 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1894 (defvar semantic-flex-depth 0 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1895 "Default flexing depth. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1896 This specifies how many lists to create tokens in.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1897 (make-variable-buffer-local 'semantic-flex-depth) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1898 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1899 (defun semantic-flex (start end &optional depth length) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1900 "Using the syntax table, do something roughly equivalent to flex. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1901 Semantically check between START and END. Optional argument DEPTH |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1902 indicates at what level to scan over entire lists. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1903 The return value is a token stream. Each element is a list, such of |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1904 the form (symbol start-expression . end-expression) where SYMBOL |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1905 denotes the token type. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1906 See `semantic-flex-tokens' variable for details on token types. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1907 END does not mark the end of the text scanned, only the end of the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1908 beginning of text scanned. Thus, if a string extends past END, the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1909 end of the return token will be larger than END. To truly restrict |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1910 scanning, use `narrow-to-region'. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1911 The last argument, LENGTH specifies that `semantic-flex' should only |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1912 return LENGTH tokens." |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1913 (message "`semantic-flex' is an obsolete function. Use `define-lex' to create lexers.") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1914 (if (not semantic-flex-keywords-obarray) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1915 (setq semantic-flex-keywords-obarray [ nil ])) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1916 (let ((ts nil) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1917 (pos (point)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1918 (ep nil) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1919 (curdepth 0) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1920 (cs (if comment-start-skip |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1921 (concat "\\(\\s<\\|" comment-start-skip "\\)") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1922 (concat "\\(\\s<\\)"))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1923 (newsyntax (copy-syntax-table (syntax-table))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1924 (mods semantic-flex-syntax-modifications) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1925 ;; Use the default depth if it is not specified. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1926 (depth (or depth semantic-flex-depth))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1927 ;; Update the syntax table |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1928 (while mods |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1929 (modify-syntax-entry (car (car mods)) (car (cdr (car mods))) newsyntax) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1930 (setq mods (cdr mods))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1931 (with-syntax-table newsyntax |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1932 (goto-char start) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1933 (while (and (< (point) end) (or (not length) (<= (length ts) length))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1934 (cond |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1935 ;; catch beginning of lines when needed. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1936 ;; Must be done before catching any other tokens! |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1937 ((and semantic-flex-enable-bol |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1938 (bolp) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1939 ;; Just insert a (bol N . N) token in the token stream, |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1940 ;; without moving the point. N is the point at the |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1941 ;; beginning of line. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1942 (setq ts (cons (cons 'bol (cons (point) (point))) ts)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1943 nil)) ;; CONTINUE |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1944 ;; special extensions, includes whitespace, nl, etc. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1945 ((and semantic-flex-extensions |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1946 (let ((fe semantic-flex-extensions) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1947 (r nil)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1948 (while fe |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1949 (if (looking-at (car (car fe))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1950 (setq ts (cons (funcall (cdr (car fe))) ts) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1951 r t |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1952 fe nil |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1953 ep (point))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1954 (setq fe (cdr fe))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1955 (if (and r (not (car ts))) (setq ts (cdr ts))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1956 r))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1957 ;; catch newlines when needed |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1958 ((looking-at "\\s-*\\(\n\\|\\s>\\)") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1959 (if semantic-flex-enable-newlines |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1960 (setq ep (match-end 1) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1961 ts (cons (cons 'newline |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1962 (cons (match-beginning 1) ep)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1963 ts)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1964 ;; catch whitespace when needed |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1965 ((looking-at "\\s-+") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1966 (if semantic-flex-enable-whitespace |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1967 ;; Language wants whitespaces, link them together. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1968 (if (eq (car (car ts)) 'whitespace) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1969 (setcdr (cdr (car ts)) (match-end 0)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1970 (setq ts (cons (cons 'whitespace |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1971 (cons (match-beginning 0) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1972 (match-end 0))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1973 ts))))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1974 ;; numbers |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1975 ((and semantic-number-expression |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1976 (looking-at semantic-number-expression)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1977 (setq ts (cons (cons 'number |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1978 (cons (match-beginning 0) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1979 (match-end 0))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1980 ts))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1981 ;; symbols |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1982 ((looking-at "\\(\\sw\\|\\s_\\)+") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1983 (setq ts (cons (cons |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1984 ;; Get info on if this is a keyword or not |
104446
df08b7ab0ba0
lisp/cedet/semantic/analyze.el: Add local vars for autoloading.
Chong Yidong <cyd@stupidchicken.com>
parents:
104419
diff
changeset
|
1985 (or (semantic-lex-keyword-p (match-string 0)) |
104413
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1986 'symbol) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1987 (cons (match-beginning 0) (match-end 0))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1988 ts))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1989 ;; Character quoting characters (ie, \n as newline) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1990 ((looking-at "\\s\\+") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1991 (setq ts (cons (cons 'charquote |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1992 (cons (match-beginning 0) (match-end 0))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1993 ts))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1994 ;; Open parens, or semantic-lists. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1995 ((looking-at "\\s(") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1996 (if (or (not depth) (< curdepth depth)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1997 (progn |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1998 (setq curdepth (1+ curdepth)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
1999 (setq ts (cons (cons 'open-paren |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2000 (cons (match-beginning 0) (match-end 0))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2001 ts))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2002 (setq ts (cons |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2003 (cons 'semantic-list |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2004 (cons (match-beginning 0) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2005 (save-excursion |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2006 (condition-case nil |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2007 (forward-list 1) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2008 ;; This case makes flex robust |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2009 ;; to broken lists. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2010 (error |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2011 (goto-char |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2012 (funcall |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2013 semantic-flex-unterminated-syntax-end-function |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2014 'semantic-list |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2015 start end)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2016 (setq ep (point))))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2017 ts)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2018 ;; Close parens |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2019 ((looking-at "\\s)") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2020 (setq ts (cons (cons 'close-paren |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2021 (cons (match-beginning 0) (match-end 0))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2022 ts)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2023 (setq curdepth (1- curdepth))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2024 ;; String initiators |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2025 ((looking-at "\\s\"") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2026 ;; Zing to the end of this string. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2027 (setq ts (cons (cons 'string |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2028 (cons (match-beginning 0) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2029 (save-excursion |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2030 (condition-case nil |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2031 (forward-sexp 1) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2032 ;; This case makes flex |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2033 ;; robust to broken strings. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2034 (error |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2035 (goto-char |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2036 (funcall |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2037 semantic-flex-unterminated-syntax-end-function |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2038 'string |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2039 start end)))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2040 (setq ep (point))))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2041 ts))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2042 ;; comments |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2043 ((looking-at cs) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2044 (if (and semantic-ignore-comments |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2045 (not semantic-flex-enable-whitespace)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2046 ;; If the language doesn't deal with comments nor |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2047 ;; whitespaces, ignore them here. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2048 (let ((comment-start-point (point))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2049 (forward-comment 1) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2050 (if (eq (point) comment-start-point) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2051 ;; In this case our start-skip string failed |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2052 ;; to work properly. Lets try and move over |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2053 ;; whatever white space we matched to begin |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2054 ;; with. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2055 (skip-syntax-forward "-.'" |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2056 (save-excursion |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2057 (end-of-line) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2058 (point))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2059 ;;(forward-comment 1) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2060 ;; Generate newline token if enabled |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2061 (if (and semantic-flex-enable-newlines |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2062 (bolp)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2063 (backward-char 1))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2064 (if (eq (point) comment-start-point) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2065 (error "Strange comment syntax prevents lexical analysis")) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2066 (setq ep (point))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2067 (let ((tk (if semantic-ignore-comments 'whitespace 'comment))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2068 (save-excursion |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2069 (forward-comment 1) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2070 ;; Generate newline token if enabled |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2071 (if (and semantic-flex-enable-newlines |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2072 (bolp)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2073 (backward-char 1)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2074 (setq ep (point))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2075 ;; Language wants comments or want them as whitespaces, |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2076 ;; link them together. |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2077 (if (eq (car (car ts)) tk) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2078 (setcdr (cdr (car ts)) ep) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2079 (setq ts (cons (cons tk (cons (match-beginning 0) ep)) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2080 ts)))))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2081 ;; punctuation |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2082 ((looking-at "\\(\\s.\\|\\s$\\|\\s'\\)") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2083 (setq ts (cons (cons 'punctuation |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2084 (cons (match-beginning 0) (match-end 0))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2085 ts))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2086 ;; unknown token |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2087 (t |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2088 (error "What is that?"))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2089 (goto-char (or ep (match-end 0))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2090 (setq ep nil))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2091 ;; maybe catch the last beginning of line when needed |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2092 (and semantic-flex-enable-bol |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2093 (= (point) end) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2094 (bolp) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2095 (setq ts (cons (cons 'bol (cons (point) (point))) ts))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2096 (goto-char pos) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2097 ;;(message "Flexing muscles...done") |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2098 (nreverse ts))) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2099 |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2100 (provide 'semantic/lex) |
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2101 |
104446
df08b7ab0ba0
lisp/cedet/semantic/analyze.el: Add local vars for autoloading.
Chong Yidong <cyd@stupidchicken.com>
parents:
104419
diff
changeset
|
2102 ;; Local variables: |
df08b7ab0ba0
lisp/cedet/semantic/analyze.el: Add local vars for autoloading.
Chong Yidong <cyd@stupidchicken.com>
parents:
104419
diff
changeset
|
2103 ;; generated-autoload-file: "loaddefs.el" |
df08b7ab0ba0
lisp/cedet/semantic/analyze.el: Add local vars for autoloading.
Chong Yidong <cyd@stupidchicken.com>
parents:
104419
diff
changeset
|
2104 ;; generated-autoload-feature: semantic/loaddefs |
104447
273e528a9f9b
* emacs-lisp/autoload.el (generated-autoload-load-name): New var.
Chong Yidong <cyd@stupidchicken.com>
parents:
104446
diff
changeset
|
2105 ;; generated-autoload-load-name: "semantic/lex" |
104446
df08b7ab0ba0
lisp/cedet/semantic/analyze.el: Add local vars for autoloading.
Chong Yidong <cyd@stupidchicken.com>
parents:
104419
diff
changeset
|
2106 ;; End: |
df08b7ab0ba0
lisp/cedet/semantic/analyze.el: Add local vars for autoloading.
Chong Yidong <cyd@stupidchicken.com>
parents:
104419
diff
changeset
|
2107 |
104413
6524f06f3a75
cedet/semantic/lex.el: New file.
Chong Yidong <cyd@stupidchicken.com>
parents:
diff
changeset
|
2108 ;;; semantic-lex.el ends here |