@c -*-texinfo-*-@c This is part of the GNU Emacs Lisp Reference Manual.@c Copyright (C) 1990, 1991, 1992, 1993, 1994 Free Software Foundation, Inc. @c See the file elisp.texi for copying conditions.@setfilename ../info/syntax@node Syntax Tables, Abbrevs, Searching and Matching, Top@chapter Syntax Tables@cindex parsing@cindex syntax table@cindex text parsing A @dfn{syntax table} specifies the syntactic textual function of eachcharacter. This information is used by the parsing commands, thecomplex movement commands, and others to determine where words, symbols,and other syntactic constructs begin and end. The current syntax tablecontrols the meaning of the word motion functions (@pxref{Word Motion})and the list motion functions (@pxref{List Motion}) as well as thefunctions in this chapter.@menu* Basics: Syntax Basics. Basic concepts of syntax tables.* Desc: Syntax Descriptors. How characters are classified.* Syntax Table Functions:: How to create, examine and alter syntax tables.* Motion and Syntax:: Moving over characters with certain syntaxes.* Parsing Expressions:: Parsing balanced expressions using the syntax table.* Standard Syntax Tables:: Syntax tables used by various major modes.* Syntax Table Internals:: How syntax table information is stored.@end menu@node Syntax Basics@section Syntax Table Concepts@ifinfo A @dfn{syntax table} provides Emacs with the information thatdetermines the syntactic use of each character in a buffer. Thisinformation is used by the parsing commands, the complex movementcommands, and others to determine where words, symbols, and othersyntactic constructs begin and end. The current syntax table controlsthe meaning of the word motion functions (@pxref{Word Motion}) and thelist motion functions (@pxref{List Motion}) as well as the functions inthis chapter.@end ifinfo A syntax table is a vector of 256 elements; it contains one entry foreach of the 256 possible characters in an 8-bit byte. Each element isan integer that encodes the syntax of the character in question. Syntax tables are used only for moving across text, not for the EmacsLisp reader. Emacs Lisp uses built-in syntactic rules when reading Lispexpressions, and these rules cannot be changed. Each buffer has its own major mode, and each major mode has its ownidea of the syntactic class of various characters. For example, in Lispmode, the character @samp{;} begins a comment, but in C mode, itterminates a statement. To support these variations, Emacs makes thechoice of syntax table local to each buffer. Typically, each majormode has its own syntax table and installs that table in each bufferthat uses that mode. Changing this table alters the syntax in allthose buffers as well as in any buffers subsequently put in that mode.Occasionally several similar modes share one syntax table.@xref{Example Major Modes}, for an example of how to set up a syntaxtable.A syntax table can inherit the data for some characters from thestandard syntax table, while specifying other characters itself. The``inherit'' syntax class means ``inherit this character's syntax fromthe standard syntax table.'' Most major modes' syntax tables inheritthe syntax of character codes 0 through 31 and 128 through 255. This isuseful with character sets such as ISO Latin-1 that have additionalalphabetic characters in the range 128 to 255. Just changing thestandard syntax for these characters affects all major modes.@defun syntax-table-p objectThis function returns @code{t} if @var{object} is a vector of length 256elements. This means that the vector may be a syntax table. However,according to this test, any vector of length 256 is considered to be asyntax table, no matter what its contents.@end defun@node Syntax Descriptors@section Syntax Descriptors@cindex syntax classes This section describes the syntax classes and flags that denote thesyntax of a character, and how they are represented as a @dfn{syntaxdescriptor}, which is a Lisp string that you pass to@code{modify-syntax-entry} to specify the desired syntax. Emacs defines a number of @dfn{syntax classes}. Each syntax tableputs each character into one class. There is no necessary relationshipbetween the class of a character in one syntax table and its class inany other table. Each class is designated by a mnemonic character, which serves as thename of the class when you need to specify a class. Usually thedesignator character is one that is frequently in that class; however,its meaning as a designator is unvarying and independent of what syntaxthat character currently has.@cindex syntax descriptor A syntax descriptor is a Lisp string that specifies a syntax class, amatching character (used only for the parenthesis classes) and flags.The first character is the designator for a syntax class. The secondcharacter is the character to match; if it is unused, put a space there.Then come the characters for any desired flags. If no matchingcharacter or flags are needed, one character is sufficient. For example, the descriptor for the character @samp{*} in C mode is@samp{@w{. 23}} (i.e., punctuation, matching character slot unused,second character of a comment-starter, first character of ancomment-ender), and the entry for @samp{/} is @samp{@w{. 14}} (i.e.,punctuation, matching character slot unused, first character of acomment-starter, second character of a comment-ender).@menu* Syntax Class Table:: Table of syntax classes.* Syntax Flags:: Additional flags each character can have.@end menu@node Syntax Class Table@subsection Table of Syntax Classes Here is a table of syntax classes, the characters that stand for them,their meanings, and examples of their use.@deffn {Syntax class} @w{whitespace character}@dfn{Whitespace characters} (designated with @w{@samp{@ }} or @samp{-})separate symbols and words from each other. Typically, whitespacecharacters have no other syntactic significance, and multiple whitespacecharacters are syntactically equivalent to a single one. Space, tab,newline and formfeed are almost always classified as whitespace.@end deffn@deffn {Syntax class} @w{word constituent}@dfn{Word constituents} (designated with @samp{w}) are parts of normalEnglish words and are typically used in variable and command names inprograms. All upper- and lower-case letters, and the digits, are typicallyword constituents.@end deffn@deffn {Syntax class} @w{symbol constituent}@dfn{Symbol constituents} (designated with @samp{_}) are the extracharacters that are used in variable and command names along with wordconstituents. For example, the symbol constituents class is used inLisp mode to indicate that certain characters may be part of symbolnames even though they are not part of English words. These charactersare @samp{$&*+-_<>}. In standard C, the only non-word-constituentcharacter that is valid in symbols is underscore (@samp{_}).@end deffn@deffn {Syntax class} @w{punctuation character}@dfn{Punctuation characters} (@samp{.}) are those characters that areused as punctuation in English, or are used in some way in a programminglanguage to separate symbols from one another. Most programminglanguage modes, including Emacs Lisp mode, have no characters in thisclass since the few characters that are not symbol or word constituentsall have other uses.@end deffn@deffn {Syntax class} @w{open parenthesis character}@deffnx {Syntax class} @w{close parenthesis character}@cindex parenthesis syntaxOpen and close @dfn{parenthesis characters} are characters used indissimilar pairs to surround sentences or expressions. Such a groupingis begun with an open parenthesis character and terminated with a close.Each open parenthesis character matches a particular close parenthesischaracter, and vice versa. Normally, Emacs indicates momentarily thematching open parenthesis when you insert a close parenthesis.@xref{Blinking}.The class of open parentheses is designated with @samp{(}, and that ofclose parentheses with @samp{)}.In English text, and in C code, the parenthesis pairs are @samp{()},@samp{[]}, and @samp{@{@}}. In Emacs Lisp, the delimiters for lists andvectors (@samp{()} and @samp{[]}) are classified as parenthesischaracters.@end deffn@deffn {Syntax class} @w{string quote}@dfn{String quote characters} (designated with @samp{"}) are used inmany languages, including Lisp and C, to delimit string constants. Thesame string quote character appears at the beginning and the end of astring. Such quoted strings do not nest.The parsing facilities of Emacs consider a string as a single token.The usual syntactic meanings of the characters in the string aresuppressed.The Lisp modes have two string quote characters: double-quote (@samp{"})and vertical bar (@samp{|}). @samp{|} is not used in Emacs Lisp, but itis used in Common Lisp. C also has two string quote characters:double-quote for strings, and single-quote (@samp{'}) for characterconstants.English text has no string quote characters because English is not aprogramming language. Although quotation marks are used in English,we do not want them to turn off the usual syntactic properties ofother characters in the quotation.@end deffn@deffn {Syntax class} @w{escape}An @dfn{escape character} (designated with @samp{\}) starts an escapesequence such as is used in C string and character constants. Thecharacter @samp{\} belongs to this class in both C and Lisp. (In C, itis used thus only inside strings, but it turns out to cause no troubleto treat it this way throughout C code.)Characters in this class count as part of words if@code{words-include-escapes} is non-@code{nil}. @xref{Word Motion}.@end deffn@deffn {Syntax class} @w{character quote}A @dfn{character quote character} (designated with @samp{/}) quotes thefollowing character so that it loses its normal syntactic meaning. Thisdiffers from an escape character in that only the character immediatelyfollowing is ever affected.Characters in this class count as part of words if@code{words-include-escapes} is non-@code{nil}. @xref{Word Motion}.This class is used for backslash in @TeX{} mode.@end deffn@deffn {Syntax class} @w{paired delimiter}@dfn{Paired delimiter characters} (designated with @samp{$}) are likestring quote characters except that the syntactic properties of thecharacters between the delimiters are not suppressed. Only @TeX{} modeuses a paired delimiter presently---the @samp{$} that both enters andleaves math mode.@end deffn@deffn {Syntax class} @w{expression prefix}An @dfn{expression prefix operator} (designated with @samp{'}) is usedfor syntactic operators that are part of an expression if they appearnext to one. These characters in Lisp include the apostrophe, @samp{'}(used for quoting), the comma, @samp{,} (used in macros), and @samp{#}(used in the read syntax for certain data types).@end deffn@deffn {Syntax class} @w{comment starter}@deffnx {Syntax class} @w{comment ender}@cindex comment syntaxThe @dfn{comment starter} and @dfn{comment ender} characters are used invarious languages to delimit comments. These classes are designatedwith @samp{<} and @samp{>}, respectively.English text has no comment characters. In Lisp, the semicolon(@samp{;}) starts a comment and a newline or formfeed ends one.@end deffn@deffn {Syntax class} @w{inherit}This syntax class does not specify a syntax. It says to look in thestandard syntax table to find the syntax of this character. Thedesignator for this syntax code is @samp{@@}.@end deffn@node Syntax Flags@subsection Syntax Flags@cindex syntax flags In addition to the classes, entries for characters in a syntax tablecan include flags. There are six possible flags, represented by thecharacters @samp{1}, @samp{2}, @samp{3}, @samp{4}, @samp{b} and@samp{p}. All the flags except @samp{p} are used to describe multi-charactercomment delimiters. The digit flags indicate that a character can@emph{also} be part of a comment sequence, in addition to the syntacticproperties associated with its character class. The flags areindependent of the class and each other for the sake of characters suchas @samp{*} in C mode, which is a punctuation character, @emph{and} thesecond character of a start-of-comment sequence (@samp{/*}), @emph{and}the first character of an end-of-comment sequence (@samp{*/}).The flags for a character @var{c} are:@itemize @bullet@item@samp{1} means @var{c} is the start of a two-character comment-startsequence.@item@samp{2} means @var{c} is the second character of such a sequence.@item@samp{3} means @var{c} is the start of a two-character comment-endsequence.@item@samp{4} means @var{c} is the second character of such a sequence.@item@c Emacs 19 feature@samp{b} means that @var{c} as a comment delimiter belongs to thealternative ``b'' comment style.Emacs supports two comment styles simultaneously in any one syntaxtable. This is for the sake of C++. Each style of comment syntax hasits own comment-start sequence and its own comment-end sequence. Eachcomment must stick to one style or the other; thus, if it starts withthe comment-start sequence of style ``b'', it must also end with thecomment-end sequence of style ``b''.The two comment-start sequences must begin with the same character; onlythe second character may differ. Mark the second character of the``b''-style comment-start sequence with the @samp{b} flag.A comment-end sequence (one or two characters) applies to the ``b''style if its first character has the @samp{b} flag set; otherwise, itapplies to the ``a'' style.The appropriate comment syntax settings for C++ are as follows:@table @asis@item @samp{/}@samp{124b}@item @samp{*}@samp{23}@item newline@samp{>b}@end tableThis defines four comment-delimiting sequences:@table @asis@item @samp{/*}This is a comment-start sequence for ``a'' style because thesecond character, @samp{*}, does not have the @samp{b} flag.@item @samp{//}This is a comment-start sequence for ``b'' style because the secondcharacter, @samp{/}, does have the @samp{b} flag.@item @samp{*/}This is a comment-end sequence for ``a'' style because the firstcharacter, @samp{*}, does not have the @samp{b} flag@item newlineThis is a comment-end sequence for ``b'' style, because the newlinecharacter has the @samp{b} flag.@end table@item@c Emacs 19 feature@samp{p} identifies an additional ``prefix character'' for Lisp syntax.These characters are treated as whitespace when they appear betweenexpressions. When they appear within an expression, they are handledaccording to their usual syntax codes.The function @code{backward-prefix-chars} moves back over thesecharacters, as well as over characters whose primary syntax class isprefix (@samp{'}). @xref{Motion and Syntax}.@end itemize@node Syntax Table Functions@section Syntax Table Functions In this section we describe functions for creating, accessing andaltering syntax tables.@defun make-syntax-tableThis function creates a new syntax table. Character codes 0 through31 and 128 through 255 are set up to inherit from the standard syntaxtable. The other character codes are set up by copying what thestandard syntax table says about them.Most major mode syntax tables are created in this way.@end defun@defun copy-syntax-table &optional tableThis function constructs a copy of @var{table} and returns it. If@var{table} is not supplied (or is @code{nil}), it returns a copy of thecurrent syntax table. Otherwise, an error is signaled if @var{table} isnot a syntax table.@end defun@deffn Command modify-syntax-entry char syntax-descriptor &optional tableThis function sets the syntax entry for @var{char} according to@var{syntax-descriptor}. The syntax is changed only for @var{table},which defaults to the current buffer's syntax table, and not in anyother syntax table. The argument @var{syntax-descriptor} specifies thedesired syntax; this is a string beginning with a class designatorcharacter, and optionally containing a matching character and flags aswell. @xref{Syntax Descriptors}.This function always returns @code{nil}. The old syntax information inthe table for this character is discarded.An error is signaled if the first character of the syntax descriptor is notone of the twelve syntax class designator characters. An error is alsosignaled if @var{char} is not a character.@example@group@exdent @r{Examples:};; @r{Put the space character in class whitespace.}(modify-syntax-entry ?\ " ") @result{} nil@end group@group;; @r{Make @samp{$} an open parenthesis character,};; @r{with @samp{^} as its matching close.}(modify-syntax-entry ?$ "(^") @result{} nil@end group@group;; @r{Make @samp{^} a close parenthesis character,};; @r{with @samp{$} as its matching open.}(modify-syntax-entry ?^ ")$") @result{} nil@end group@group;; @r{Make @samp{/} a punctuation character,};; @r{the first character of a start-comment sequence,};; @r{and the second character of an end-comment sequence.};; @r{This is used in C mode.}(modify-syntax-entry ?/ ". 14") @result{} nil@end group@end example@end deffn@defun char-syntax characterThis function returns the syntax class of @var{character}, representedby its mnemonic designator character. This @emph{only} returns theclass, not any matching parenthesis or flags.An error is signaled if @var{char} is not a character.The following examples apply to C mode. The first example shows thatthe syntax class of space is whitespace (represented by a space). Thesecond example shows that the syntax of @samp{/} is punctuation. Thisdoes not show the fact that it is also part of comment-start and -endsequences. The third example shows that open parenthesis is in the classof open parentheses. This does not show the fact that it has a matchingcharacter, @samp{)}.@example@group(char-to-string (char-syntax ?\ )) @result{} " "@end group@group(char-to-string (char-syntax ?/)) @result{} "."@end group@group(char-to-string (char-syntax ?\()) @result{} "("@end group@end example@end defun@defun set-syntax-table tableThis function makes @var{table} the syntax table for the current buffer.It returns @var{table}.@end defun@defun syntax-tableThis function returns the current syntax table, which is the table forthe current buffer.@end defun@node Motion and Syntax@section Motion and Syntax This section describes functions for moving across characters incertain syntax classes. None of these functions exists in Emacsversion 18 or earlier.@defun skip-syntax-forward syntaxes &optional limitThis function moves point forward across characters having syntax classesmentioned in @var{syntaxes}. It stops when it encounters the end ofthe buffer, or position @var{limit} (if specified), or a character it isnot supposed to skip.@ignore @c may want to change this.The return value is the distance traveled, which is a nonnegativeinteger.@end ignore@end defun@defun skip-syntax-backward syntaxes &optional limitThis function moves point backward across characters whose syntaxclasses are mentioned in @var{syntaxes}. It stops when it encountersthe beginning of the buffer, or position @var{limit} (if specified), or acharacter it is not supposed to skip.@ignore @c may want to change this.The return value indicates the distance traveled. It is an integer thatis zero or less.@end ignore@end defun@defun backward-prefix-charsThis function moves point backward over any number of characters withexpression prefix syntax. This includes both characters in theexpression prefix syntax class, and characters with the @samp{p} flag.@end defun@node Parsing Expressions@section Parsing Balanced Expressions Here are several functions for parsing and scanning balancedexpressions, also known as @dfn{sexps}, in which parentheses match inpairs. The syntax table controls the interpretation of characters, sothese functions can be used for Lisp expressions when in Lisp mode andfor C expressions when in C mode. @xref{List Motion}, for convenienthigher-level functions for moving over balanced expressions.@defun parse-partial-sexp start limit &optional target-depth stop-before state stop-commentThis function parses a sexp in the current buffer starting at@var{start}, not scanning past @var{limit}. It stops at position@var{limit} or when certain criteria described below are met, and setspoint to the location where parsing stops. It returns a valuedescribing the status of the parse at the point where it stops.If @var{state} is @code{nil}, @var{start} is assumed to be at the toplevel of parenthesis structure, such as the beginning of a functiondefinition. Alternatively, you might wish to resume parsing in themiddle of the structure. To do this, you must provide a @var{state}argument that describes the initial status of parsing.@cindex parenthesis depthIf the third argument @var{target-depth} is non-@code{nil}, parsingstops if the depth in parentheses becomes equal to @var{target-depth}.The depth starts at 0, or at whatever is given in @var{state}.If the fourth argument @var{stop-before} is non-@code{nil}, parsingstops when it comes to any character that starts a sexp. If@var{stop-comment} is non-@code{nil}, parsing stops when it comes to thestart of a comment.@cindex parse stateThe fifth argument @var{state} is an eight-element list of the sameform as the value of this function, described below. The return valueof one call may be used to initialize the state of the parse on anothercall to @code{parse-partial-sexp}.The result is a list of eight elements describing the final state ofthe parse:@enumerate 0@item The depth in parentheses, counting from 0.@item @cindex innermost containing parenthesesThe character position of the start of the innermost parentheticalgrouping containing the stopping point; @code{nil} if none.@item @cindex previous complete subexpressionThe character position of the start of the last complete subexpressionterminated; @code{nil} if none.@item @cindex inside stringNon-@code{nil} if inside a string. More precisely, this is thecharacter that will terminate the string.@item @cindex inside comment@code{t} if inside a comment (of either style).@item @cindex quote character@code{t} if point is just after a quote character.@item The minimum parenthesis depth encountered during this scan.@item@code{t} if inside a comment of style ``b''.@end enumerateElements 0, 3, 4, 5 and 7 are significant in the argument @var{state}.@cindex indenting with parenthesesThis function is most often used to compute indentation for languagesthat have nested parentheses.@end defun@defun scan-lists from count depthThis function scans forward @var{count} balanced parenthetical groupingsfrom character number @var{from}. It returns the character positionwhere the scan stops.If @var{depth} is nonzero, parenthesis depth counting begins from thatvalue. The only candidates for stopping are places where the depth inparentheses becomes zero; @code{scan-lists} counts @var{count} suchplaces and then stops. Thus, a positive value for @var{depth} means goout @var{depth} levels of parenthesis.Scanning ignores comments if @code{parse-sexp-ignore-comments} isnon-@code{nil}.If the scan reaches the beginning or end of the buffer (or itsaccessible portion), and the depth is not zero, an error is signaled.If the depth is zero but the count is not used up, @code{nil} isreturned.@end defun@defun scan-sexps from countThis function scans forward @var{count} sexps from character position@var{from}. It returns the character position where the scan stops.Scanning ignores comments if @code{parse-sexp-ignore-comments} isnon-@code{nil}.If the scan reaches the beginning or end of (the accessible part of) thebuffer in the middle of a parenthetical grouping, an error is signaled.If it reaches the beginning or end between groupings but before count isused up, @code{nil} is returned.@end defun@defvar parse-sexp-ignore-comments@cindex skipping commentsIf the value is non-@code{nil}, then comments are treated aswhitespace by the functions in this section and by @code{forward-sexp}.In older Emacs versions, this feature worked only when the commentterminator is something like @samp{*/}, and appears only to end acomment. In languages where newlines terminate comments, it wasnecessary make this variable @code{nil}, since not every newline is theend of a comment. This limitation no longer exists.@end defvarYou can use @code{forward-comment} to move forward or backward overone comment or several comments.@defun forward-comment countThis function moves point forward across @var{count} comments (backward,if @var{count} is negative). If it finds anything other than a commentor whitespace, it stops, leaving point at the place where it stopped.It also stops after satisfying @var{count}.@end defunTo move forward over all comments and whitespace following point, use@code{(forward-comment (buffer-size))}. @code{(buffer-size)} is a goodargument to use, because the number of comments in the buffer cannotexceed that many.@node Standard Syntax Tables@section Some Standard Syntax Tables Most of the major modes in Emacs have their own syntax tables. Hereare several of them:@defun standard-syntax-tableThis function returns the standard syntax table, which is the syntaxtable used in Fundamental mode.@end defun@defvar text-mode-syntax-tableThe value of this variable is the syntax table used in Text mode.@end defvar@defvar c-mode-syntax-tableThe value of this variable is the syntax table for C-mode buffers.@end defvar@defvar emacs-lisp-mode-syntax-tableThe value of this variable is the syntax table used in Emacs Lisp modeby editing commands. (It has no effect on the Lisp @code{read}function.)@end defvar@node Syntax Table Internals@section Syntax Table Internals@cindex syntax table internals Each element of a syntax table is an integer that encodes the syntaxof one character: the syntax class, possible matching character, andflags. Lisp programs don't usually work with the elements directly; theLisp-level syntax table functions usually work with syntax descriptors(@pxref{Syntax Descriptors}). The low 8 bits of each element of a syntax table indicate thesyntax class.@table @asis@item @i{Integer}@i{Class}@item 0whitespace@item 1punctuation@item 2word@item 3symbol@item 4open parenthesis@item 5close parenthesis@item 6expression prefix@item 7string quote@item 8paired delimiter@item 9escape@item 10character quote@item 11comment-start@item 12comment-end@item 13inherit@end table The next 8 bits are the matching opposite parenthesis (if thecharacter has parenthesis syntax); otherwise, they are not meaningful.The next 6 bits are the flags.