34.3.3 The rx Structured Regexp Notation

As an alternative to the string-based syntax, Emacs provides the structured rx notation based on Lisp S-expressions. This notation is usually easier to read, write and maintain than regexp strings, and can be indented and commented freely. It requires a conversion into string form since that is what regexp functions expect, but that conversion typically takes place during byte-compilation rather than when the Lisp code using the regexp is run.

Here is an rx regexp¹ that matches a block comment in the C programming language:

(rx "/*"                          ; Initial /*
    (zero-or-more
     (or (not (any "*"))          ;  Either non-*,
         (seq "*"                 ;  or * followed by
              (not (any "/")))))  ;  non-/
    (one-or-more "*")             ; At least one star,
    "/")                          ; and the final /

or, using shorter synonyms and written more compactly,

(rx "/*"
    (* (| (not "*")
          (: "*" (not "/"))))
    (+ "*") "/")

In conventional string syntax, it would be written

"/\\*\\(?:[^*]\\|\\*[^/]\\)*\\*+/"

The rx notation is mainly useful in Lisp code; it cannot be used in most interactive situations where a regexp is requested, such as when running query-replace-regexp or in variable customization.

• Rx Constructs		Constructs valid in rx forms.
• Rx Functions		Functions and macros that use rx forms.
• Extending Rx		How to define your own rx forms.

34.3.3.1 Constructs in rx regexps

The various forms in rx regexps are described below. The shorthand rx represents any rx form, and rx… means zero or more rx forms. Where the corresponding string regexp syntax is given, A, B, … are string regexp subexpressions.

Literals

"some-string"

Match the string ‘some-string’ literally. There are no characters with special meaning, unlike in string regexps.

?C

Match the character ‘C’ literally.

Sequence and alternative

(seq rx…)

(sequence rx…)

(: rx…)

(and rx…)

Match the rxs in sequence. Without arguments, the expression matches the empty string.\ Corresponding string regexp: ‘AB…’ (subexpressions in sequence).

(or rx…)

(| rx…)

Match exactly one of the rxs. If all arguments are strings, characters, or or forms so constrained, the longest possible match will always be used. Otherwise, either the longest match or the first (in left-to-right order) will be used. Without arguments, the expression will not match anything at all.\ Corresponding string regexp: ‘A\|B\|…’.

unmatchable

Refuse any match. Equivalent to (or). See regexp-unmatchable.

Repetition

Normally, repetition forms are greedy, in that they attempt to match as many times as possible. Some forms are non-greedy; they try to match as few times as possible (see Non-greedy repetition).

(zero-or-more rx…)

(0+ rx…)

Match the rxs zero or more times. Greedy by default.\ Corresponding string regexp: ‘A*’ (greedy), ‘A*?’ (non-greedy)

(one-or-more rx…)

(1+ rx…)

Match the rxs one or more times. Greedy by default.\ Corresponding string regexp: ‘A+’ (greedy), ‘A+?’ (non-greedy)

(zero-or-one rx…)

(optional rx…)

(opt rx…)

Match the rxs once or an empty string. Greedy by default.\ Corresponding string regexp: ‘A?’ (greedy), ‘A??’ (non-greedy).

(* rx…)

Match the rxs zero or more times. Greedy.\ Corresponding string regexp: ‘A*’

(+ rx…)

Match the rxs one or more times. Greedy.\ Corresponding string regexp: ‘A+’

(? rx…)

Match the rxs once or an empty string. Greedy.\ Corresponding string regexp: ‘A?’

(*? rx…)

Match the rxs zero or more times. Non-greedy.\ Corresponding string regexp: ‘A*?’

(+? rx…)

Match the rxs one or more times. Non-greedy.\ Corresponding string regexp: ‘A+?’

(?? rx…)

Match the rxs or an empty string. Non-greedy.\ Corresponding string regexp: ‘A??’

(= n rx…)

(repeat n rx)

Match the rxs exactly n times.\ Corresponding string regexp: ‘A\{n\}’

(>= n rx…)

Match the rxs n or more times. Greedy.\ Corresponding string regexp: ‘A\{n,\}’

(** n m rx…)

(repeat n m rx…)

Match the rxs at least n but no more than m times. Greedy.\ Corresponding string regexp: ‘A\{n,m\}’

The greediness of some repetition forms can be controlled using the following constructs. However, it is usually better to use the explicit non-greedy forms above when such matching is required.

(minimal-match rx)

Match rx, with zero-or-more, 0+, one-or-more, 1+, zero-or-one, opt and optional using non-greedy matching.

(maximal-match rx)

Match rx, with zero-or-more, 0+, one-or-more, 1+, zero-or-one, opt and optional using greedy matching. This is the default.

Matching single characters

(any set…)

(char set…)

(in set…)

Match a single character from one of the sets. Each set is a character, a string representing the set of its characters, a range or a character class (see below). A range is either a hyphen-separated string like "A-Z", or a cons of characters like (?A . ?Z).

Note that hyphen (-) is special in strings in this construct, since it acts as a range separator. To include a hyphen, add it as a separate character or single-character string.\ Corresponding string regexp: ‘[…]’

(not charspec)

Match a character not included in charspec. charspec can be a character, a single-character string, an any, not, or, intersection, syntax or category form, or a character class. If charspec is an or form, its arguments have the same restrictions as those of intersection; see below.\ Corresponding string regexp: ‘[^…]’, ‘\Scode’, ‘\Ccode’

(intersection charset…)

Match a character included in all of the charsets. Each charset can be a character, a single-character string, an any form without character classes, or an intersection, or or not form whose arguments are also charsets.

not-newline, nonl

Match any character except a newline.\ Corresponding string regexp: ‘.’ (dot)

anychar, anything

Match any character.\ Corresponding string regexp: ‘.\|\n’ (for example)

character class

Match a character from a named character class:

alpha, alphabetic, letter

Match alphabetic characters. More precisely, match characters whose Unicode ‘general-category’ property indicates that they are alphabetic.

alnum, alphanumeric

Match alphabetic characters and digits. More precisely, match characters whose Unicode ‘general-category’ property indicates that they are alphabetic or decimal digits.

digit, numeric, num

Match the digits ‘0’–‘9’.

xdigit, hex-digit, hex

Match the hexadecimal digits ‘0’–‘9’, ‘A’–‘F’ and ‘a’–‘f’.

cntrl, control

Match any character whose code is in the range 0–31.

blank

Match horizontal whitespace. More precisely, match characters whose Unicode ‘general-category’ property indicates that they are spacing separators.

space, whitespace, white

Match any character that has whitespace syntax (see Syntax Class Table).

lower, lower-case

Match anything lower-case, as determined by the current case table. If case-fold-search is non-nil, this also matches any upper-case letter.

upper, upper-case

Match anything upper-case, as determined by the current case table. If case-fold-search is non-nil, this also matches any lower-case letter.

graph, graphic

Match any character except whitespace, ASCII and non-ASCII control characters, surrogates, and codepoints unassigned by Unicode, as indicated by the Unicode ‘general-category’ property.

print, printing

Match whitespace or a character matched by graph.

punct, punctuation

Match any punctuation character. (At present, for multibyte characters, anything that has non-word syntax.)

word, wordchar

Match any character that has word syntax (see Syntax Class Table).

ascii

Match any ASCII character (codes 0–127).

nonascii

Match any non-ASCII character (but not raw bytes).

Corresponding string regexp: ‘[[:class:]]’

(syntax syntax)

Match a character with syntax syntax, being one of the following names:

Syntax name	Syntax character
whitespace	-
punctuation	.
word	w
symbol	_
open-parenthesis	(
close-parenthesis	)
expression-prefix	'
string-quote	"
paired-delimiter	$
escape	\
character-quote	/
comment-start	<
comment-end	>
string-delimiter	\|
comment-delimiter	!

For details, see Syntax Class Table. Please note that (syntax punctuation) is not equivalent to the character class punctuation.\ Corresponding string regexp: ‘\scode’

(category category)

Match a character in category category, which is either one of the names below or its category character.

Category name	Category character
space-for-indent	space
base	.
consonant	0
base-vowel	1
upper-diacritical-mark	2
lower-diacritical-mark	3
tone-mark	4
symbol	5
digit	6
vowel-modifying-diacritical-mark	7
vowel-sign	8
semivowel-lower	9
not-at-end-of-line	<
not-at-beginning-of-line	>
alpha-numeric-two-byte	A
chinese-two-byte	C
greek-two-byte	G
japanese-hiragana-two-byte	H
indian-two-byte	I
japanese-katakana-two-byte	K
strong-left-to-right	L
korean-hangul-two-byte	N
strong-right-to-left	R
cyrillic-two-byte	Y
combining-diacritic	^
ascii	a
arabic	b
chinese	c
ethiopic	e
greek	g
korean	h
indian	i
japanese	j
japanese-katakana	k
latin	l
lao	o
tibetan	q
japanese-roman	r
thai	t
vietnamese	v
hebrew	w
cyrillic	y
can-break	\|

For more information about currently defined categories, run the command M-x describe-categories RET. For how to define new categories, see Categories.\ Corresponding string regexp: ‘\ccode’

Zero-width assertions

These all match the empty string, but only in specific places.

line-start, bol

Match at the beginning of a line.\ Corresponding string regexp: ‘^’

line-end, eol

Match at the end of a line.\ Corresponding string regexp: ‘$’

string-start, bos, buffer-start, bot

Match at the start of the string or buffer being matched against.\ Corresponding string regexp: ‘\`’

string-end, eos, buffer-end, eot

Match at the end of the string or buffer being matched against.\ Corresponding string regexp: ‘\'’

point

Match at point.\ Corresponding string regexp: ‘\=’

word-start, bow

Match at the beginning of a word.\ Corresponding string regexp: ‘\<’

word-end, eow

Match at the end of a word.\ Corresponding string regexp: ‘\>’

word-boundary

Match at the beginning or end of a word.\ Corresponding string regexp: ‘\b’

not-word-boundary

Match anywhere but at the beginning or end of a word.\ Corresponding string regexp: ‘\B’

symbol-start

Match at the beginning of a symbol.\ Corresponding string regexp: ‘\_<’

symbol-end

Match at the end of a symbol.\ Corresponding string regexp: ‘\_>’

Capture groups

(group rx…)

(submatch rx…)

Match the rxs, making the matched text and position accessible in the match data. The first group in a regexp is numbered 1; subsequent groups will be numbered one higher than the previous group.\ Corresponding string regexp: ‘\(…\)’

(group-n n rx…)

(submatch-n n rx…)

Like group, but explicitly assign the group number n. n must be positive.\ Corresponding string regexp: ‘\(?n:…\)’

(backref n)

Match the text previously matched by group number n. n must be in the range 1–9.\ Corresponding string regexp: ‘\n’

Dynamic inclusion

(literal expr)

Match the literal string that is the result from evaluating the Lisp expression expr. The evaluation takes place at call time, in the current lexical environment.

(regexp expr)

(regex expr)

Match the string regexp that is the result from evaluating the Lisp expression expr. The evaluation takes place at call time, in the current lexical environment.

(eval expr)

Match the rx form that is the result from evaluating the Lisp expression expr. The evaluation takes place at macro-expansion time for rx, at call time for rx-to-string, in the current global environment.

34.3.3.2 Functions and macros using rx regexps

macro rx rx-expr…

Translate the rx-exprs to a string regexp, as if they were the body of a (seq …) form. The rx macro expands to a string constant, or, if literal or regexp forms are used, a Lisp expression that evaluates to a string.

function rx-to-string rx-expr \&optional no-group

Translate rx-expr to a string regexp which is returned. If no-group is absent or nil, bracket the result in a non-capturing group, ‘\(?:…\)’, if necessary to ensure that a postfix operator appended to it will apply to the whole expression.

Arguments to literal and regexp forms in rx-expr must be string literals.

The pcase macro can use rx expressions as patterns directly; see rx in pcase.

For mechanisms to add user-defined extensions to the rx notation, see Extending Rx.

34.3.3.3 Defining new rx forms

The rx notation can be extended by defining new symbols and parameterized forms in terms of other rx expressions. This is handy for sharing parts between several regexps, and for making complex ones easier to build and understand by putting them together from smaller pieces.

For example, you could define name to mean (one-or-more letter), and (quoted x) to mean (seq ?' x ?') for any x. These forms could then be used in rx expressions like any other: (rx (quoted name)) would match a nonempty sequence of letters inside single quotes.

The Lisp macros below provide different ways of binding names to definitions. Common to all of them are the following rules:

• Built-in rx forms, like digit and group, cannot be redefined.
• The definitions live in a name space of their own, separate from that of Lisp variables. There is thus no need to attach a suffix like -regexp to names; they cannot collide with anything else.
• Definitions cannot refer to themselves recursively, directly or indirectly. If you find yourself needing this, you want a parser, not a regular expression.
• Definitions are only ever expanded in calls to rx or rx-to-string, not merely by their presence in definition macros. This means that the order of definitions doesn’t matter, even when they refer to each other, and that syntax errors only show up when they are used, not when they are defined.
• User-defined forms are allowed wherever arbitrary rx expressions are expected; for example, in the body of a zero-or-one form, but not inside any or category forms. They are also allowed inside not and intersection forms.

macro rx-define name [arglist] rx-form

Define name globally in all subsequent calls to rx and rx-to-string. If arglist is absent, then name is defined as a plain symbol to be replaced with rx-form. Example:

(rx-define haskell-comment (seq "--" (zero-or-more nonl)))
(rx haskell-comment)
     ⇒ "--.*"

If arglist is present, it must be a list of zero or more argument names, and name is then defined as a parameterized form. When used in an rx expression as (name arg…), each arg will replace the corresponding argument name inside rx-form.

arglist may end in &rest and one final argument name, denoting a rest parameter. The rest parameter will expand to all extra actual argument values not matched by any other parameter in arglist, spliced into rx-form where it occurs. Example:

(rx-define moan (x y &rest r) (seq x (one-or-more y) r "!"))
(rx (moan "MOO" "A" "MEE" "OW"))
     ⇒ "MOOA+MEEOW!"

Since the definition is global, it is recommended to give name a package prefix to avoid name clashes with definitions elsewhere, as is usual when naming non-local variables and functions.

macro rx-let (bindings…) body…

Make the rx definitions in bindings available locally for rx macro invocations in body, which is then evaluated.

Each element of bindings is on the form (name [arglist] rx-form), where the parts have the same meaning as in rx-define above. Example:

(rx-let ((comma-separated (item) (seq item (0+ "," item)))
         (number (1+ digit))
         (numbers (comma-separated number)))
  (re-search-forward (rx "(" numbers ")")))

The definitions are only available during the macro-expansion of body, and are thus not present during execution of compiled code.

rx-let can be used not only inside a function, but also at top level to include global variable and function definitions that need to share a common set of rx forms. Since the names are local inside body, there is no need for any package prefixes. Example:

(rx-let ((phone-number (seq (opt ?+) (1+ (any digit ?-)))))
  (defun find-next-phone-number ()
    (re-search-forward (rx phone-number)))
  (defun phone-number-p (string)
    (string-match-p (rx bos phone-number eos) string)))

The scope of the rx-let bindings is lexical, which means that they are not visible outside body itself, even in functions called from body.

macro rx-let-eval bindings body…

Evaluate bindings to a list of bindings as in rx-let, and evaluate body with those bindings in effect for calls to rx-to-string.

This macro is similar to rx-let, except that the bindings argument is evaluated (and thus needs to be quoted if it is a list literal), and the definitions are substituted at run time, which is required for rx-to-string to work. Example:

(rx-let-eval
    '((ponder (x) (seq "Where have all the " x " gone?")))
  (looking-at (rx-to-string
               '(ponder (or "flowers" "young girls"
                            "left socks")))))

Another difference from rx-let is that the bindings are dynamically scoped, and thus also available in functions called from body. However, they are not visible inside functions defined in body.

---

1. It could be written much simpler with non-greedy operators (how?), but that would make the example less interesting.↩