23.6.9 Multiline Font Lock Constructs

Normally, elements of font-lock-keywords should not match across multiple lines; that doesn’t work reliably, because Font Lock usually scans just part of the buffer, and it can miss a multi-line construct that crosses the line boundary where the scan starts. (The scan normally starts at the beginning of a line.)

Making elements that match multiline constructs work properly has two aspects: correct identification and correct rehighlighting. The first means that Font Lock finds all multiline constructs. The second means that Font Lock will correctly rehighlight all the relevant text when a multiline construct is changed—for example, if some of the text that was previously part of a multiline construct ceases to be part of it. The two aspects are closely related, and often getting one of them to work will appear to make the other also work. However, for reliable results you must attend explicitly to both aspects.

There are three ways to ensure correct identification of multiline constructs:

• Add a function to font-lock-extend-region-functions that does the identification and extends the scan so that the scanned text never starts or ends in the middle of a multiline construct.
• Use the font-lock-fontify-region-function hook similarly to extend the scan so that the scanned text never starts or ends in the middle of a multiline construct.
• Somehow identify the multiline construct right when it gets inserted into the buffer (or at any point after that but before font-lock tries to highlight it), and mark it with a font-lock-multiline which will instruct font-lock not to start or end the scan in the middle of the construct.

There are three ways to do rehighlighting of multiline constructs:

• Place a font-lock-multiline property on the construct. This will rehighlight the whole construct if any part of it is changed. In some cases you can do this automatically by setting the font-lock-multiline variable, which see.
• Make sure jit-lock-contextually is set and rely on it doing its job. This will only rehighlight the part of the construct that follows the actual change, and will do it after a short delay. This only works if the highlighting of the various parts of your multiline construct never depends on text in subsequent lines. Since jit-lock-contextually is activated by default, this can be an attractive solution.
• Place a jit-lock-defer-multiline property on the construct. This works only if jit-lock-contextually is used, and with the same delay before rehighlighting, but like font-lock-multiline, it also handles the case where highlighting depends on subsequent lines.

• Font Lock Multiline		Marking multiline chunks with a text property.
• Region to Refontify		Controlling which region gets refontified after a buffer change.

23.6.9.1 Font Lock Multiline

One way to ensure reliable rehighlighting of multiline Font Lock constructs is to put on them the text property font-lock-multiline. It should be present and non-nil for text that is part of a multiline construct.

When Font Lock is about to highlight a range of text, it first extends the boundaries of the range as necessary so that they do not fall within text marked with the font-lock-multiline property. Then it removes any font-lock-multiline properties from the range, and highlights it. The highlighting specification (mostly font-lock-keywords) must reinstall this property each time, whenever it is appropriate.

Warning: don’t use the font-lock-multiline property on large ranges of text, because that will make rehighlighting slow.

variable font-lock-multiline

If the font-lock-multiline variable is set to t, Font Lock will try to add the font-lock-multiline property automatically on multiline constructs. This is not a universal solution, however, since it slows down Font Lock somewhat. It can miss some multiline constructs, or make the property larger or smaller than necessary.

For elements whose matcher is a function, the function should ensure that submatch 0 covers the whole relevant multiline construct, even if only a small subpart will be highlighted. It is often just as easy to add the font-lock-multiline property by hand.

The font-lock-multiline property is meant to ensure proper refontification; it does not automatically identify new multiline constructs. Identifying them requires that Font Lock mode operate on large enough chunks at a time. This will happen by accident on many cases, which may give the impression that multiline constructs magically work. If you set the font-lock-multiline variable non-nil, this impression will be even stronger, since the highlighting of those constructs which are found will be properly updated from then on. But that does not work reliably.

To find multiline constructs reliably, you must either manually place the font-lock-multiline property on the text before Font Lock mode looks at it, or use font-lock-fontify-region-function.

23.6.9.2 Region to Fontify after a Buffer Change

When a buffer is changed, the region that Font Lock refontifies is by default the smallest sequence of whole lines that spans the change. While this works well most of the time, sometimes it doesn’t—for example, when a change alters the syntactic meaning of text on an earlier line.

You can enlarge (or even reduce) the region to refontify by setting the following variable:

variable font-lock-extend-after-change-region-function

This buffer-local variable is either nil or a function for Font Lock mode to call to determine the region to scan and fontify.

The function is given three parameters, the standard beg, end, and old-len from after-change-functions (see Change Hooks). It should return either a cons of the beginning and end buffer positions (in that order) of the region to fontify, or nil (which means choose the region in the standard way). This function needs to preserve point, the match-data, and the current restriction. The region it returns may start or end in the middle of a line.

Since this function is called after every buffer change, it should be reasonably fast.