A pattern may be followed by any number of modifiers, which are
mostly single characters, though some of these can be qualified
by further characters. Following Perl usage, these are referred
to below as, for example, "the /i modifier", even though the
delimiter of the pattern need not always be a slash, and no slash
is used when writing modifiers. White space may appear between
the final pattern delimiter and the first modifier, and between
the modifiers themselves. For reference, here is a complete list
of modifiers. They fall into several groups that are described in
detail in the following sections.
/8 set UTF mode
/9 set PCRE_NEVER_UTF (locks out UTF mode)
/? disable UTF validity check
/+ show remainder of subject after match
/= show all captures (not just those that are set)
/A set PCRE_ANCHORED
/B show compiled code
/C set PCRE_AUTO_CALLOUT
/D same as /B plus /I
/E set PCRE_DOLLAR_ENDONLY
/F flip byte order in compiled pattern
/f set PCRE_FIRSTLINE
/G find all matches (shorten string)
/g find all matches (use startoffset)
/I show information about pattern
/i set PCRE_CASELESS
/J set PCRE_DUPNAMES
/K show backtracking control names
/L set locale
/M show compiled memory size
/m set PCRE_MULTILINE
/N set PCRE_NO_AUTO_CAPTURE
/O set PCRE_NO_AUTO_POSSESS
/P use the POSIX wrapper
/Q test external stack check function
/S study the pattern after compilation
/s set PCRE_DOTALL
/T select character tables
/U set PCRE_UNGREEDY
/W set PCRE_UCP
/X set PCRE_EXTRA
/x set PCRE_EXTENDED
/Y set PCRE_NO_START_OPTIMIZE
/Z don't show lengths in /B output
/<any> set PCRE_NEWLINE_ANY
/<anycrlf> set PCRE_NEWLINE_ANYCRLF
/<cr> set PCRE_NEWLINE_CR
/<crlf> set PCRE_NEWLINE_CRLF
/<lf> set PCRE_NEWLINE_LF
/<bsr_anycrlf> set PCRE_BSR_ANYCRLF
/<bsr_unicode> set PCRE_BSR_UNICODE
/<JS> set PCRE_JAVASCRIPT_COMPAT
Perl-compatible modifiers
The /i, /m, /s, and /x modifiers set the PCRE_CASELESS,
PCRE_MULTILINE, PCRE_DOTALL, or PCRE_EXTENDED options,
respectively, when pcre[16|32]_compile() is called. These four
modifier letters have the same effect as they do in Perl. For
example:
/caseless/i
Modifiers for other PCRE options
The following table shows additional modifiers for setting PCRE
compile-time options that do not correspond to anything in Perl:
/8 PCRE_UTF8 ) when using the 8-bit
/? PCRE_NO_UTF8_CHECK ) library
/8 PCRE_UTF16 ) when using the 16-bit
/? PCRE_NO_UTF16_CHECK ) library
/8 PCRE_UTF32 ) when using the 32-bit
/? PCRE_NO_UTF32_CHECK ) library
/9 PCRE_NEVER_UTF
/A PCRE_ANCHORED
/C PCRE_AUTO_CALLOUT
/E PCRE_DOLLAR_ENDONLY
/f PCRE_FIRSTLINE
/J PCRE_DUPNAMES
/N PCRE_NO_AUTO_CAPTURE
/O PCRE_NO_AUTO_POSSESS
/U PCRE_UNGREEDY
/W PCRE_UCP
/X PCRE_EXTRA
/Y PCRE_NO_START_OPTIMIZE
/<any> PCRE_NEWLINE_ANY
/<anycrlf> PCRE_NEWLINE_ANYCRLF
/<cr> PCRE_NEWLINE_CR
/<crlf> PCRE_NEWLINE_CRLF
/<lf> PCRE_NEWLINE_LF
/<bsr_anycrlf> PCRE_BSR_ANYCRLF
/<bsr_unicode> PCRE_BSR_UNICODE
/<JS> PCRE_JAVASCRIPT_COMPAT
The modifiers that are enclosed in angle brackets are literal
strings as shown, including the angle brackets, but the letters
within can be in either case. This example sets multiline
matching with CRLF as the line ending sequence:
/^abc/m<CRLF>
As well as turning on the PCRE_UTF8/16/32 option, the /8 modifier
causes all non-printing characters in output strings to be
printed using the \x{hh...} notation. Otherwise, those less than
0x100 are output in hex without the curly brackets.
Full details of the PCRE options are given in the pcreapi
documentation.
Finding all matches in a string
Searching for all possible matches within each subject string can
be requested by the /g or /G modifier. After finding a match,
PCRE is called again to search the remainder of the subject
string. The difference between /g and /G is that the former uses
the startoffset argument to pcre[16|32]_exec() to start searching
at a new point within the entire string (which is in effect what
Perl does), whereas the latter passes over a shortened substring.
This makes a difference to the matching process if the pattern
begins with a lookbehind assertion (including \b or \B).
If any call to pcre[16|32]_exec() in a /g or /G sequence matches
an empty string, the next call is done with the
PCRE_NOTEMPTY_ATSTART and PCRE_ANCHORED flags set in order to
search for another, non-empty, match at the same point. If this
second match fails, the start offset is advanced, and the normal
match is retried. This imitates the way Perl handles such cases
when using the /g modifier or the split() function. Normally, the
start offset is advanced by one character, but if the newline
convention recognizes CRLF as a newline, and the current
character is CR followed by LF, an advance of two is used.
Other modifiers
There are yet more modifiers for controlling the way pcretest
operates.
The /+ modifier requests that as well as outputting the substring
that matched the entire pattern, pcretest should in addition
output the remainder of the subject string. This is useful for
tests where the subject contains multiple copies of the same
substring. If the + modifier appears twice, the same action is
taken for captured substrings. In each case the remainder is
output on the following line with a plus character following the
capture number. Note that this modifier must not immediately
follow the /S modifier because /S+ and /S++ have other meanings.
The /= modifier requests that the values of all potential
captured parentheses be output after a match. By default, only
those up to the highest one actually used in the match are output
(corresponding to the return code from pcre[16|32]_exec()).
Values in the offsets vector corresponding to higher numbers
should be set to -1, and these are output as "<unset>". This
modifier gives a way of checking that this is happening.
The /B modifier is a debugging feature. It requests that pcretest
output a representation of the compiled code after compilation.
Normally this information contains length and offset values;
however, if /Z is also present, this data is replaced by spaces.
This is a special feature for use in the automatic test scripts;
it ensures that the same output is generated for different
internal link sizes.
The /D modifier is a PCRE debugging feature, and is equivalent to
/BI, that is, both the /B and the /I modifiers.
The /F modifier causes pcretest to flip the byte order of the
2-byte and 4-byte fields in the compiled pattern. This facility
is for testing the feature in PCRE that allows it to execute
patterns that were compiled on a host with a different
endianness. This feature is not available when the POSIX
interface to PCRE is being used, that is, when the /P pattern
modifier is specified. See also the section about saving and
reloading compiled patterns below.
The /I modifier requests that pcretest output information about
the compiled pattern (whether it is anchored, has a fixed first
character, and so on). It does this by calling
pcre[16|32]_fullinfo() after compiling a pattern. If the pattern
is studied, the results of that are also output. In this output,
the word "char" means a non-UTF character, that is, the value of
a single data item (8-bit, 16-bit, or 32-bit, depending on the
library that is being tested).
The /K modifier requests pcretest to show names from backtracking
control verbs that are returned from calls to pcre[16|32]_exec().
It causes pcretest to create a pcre[16|32]_extra block if one has
not already been created by a call to pcre[16|32]_study(), and to
set the PCRE_EXTRA_MARK flag and the mark field within it, every
time that pcre[16|32]_exec() is called. If the variable that the
mark field points to is non-NULL for a match, non-match, or
partial match, pcretest prints the string to which it points. For
a match, this is shown on a line by itself, tagged with "MK:".
For a non-match it is added to the message.
The /L modifier must be followed directly by the name of a
locale, for example,
/pattern/Lfr_FR
For this reason, it must be the last modifier. The given locale
is set, pcre[16|32]_maketables() is called to build a set of
character tables for the locale, and this is then passed to
pcre[16|32]_compile() when compiling the regular expression.
Without an /L (or /T) modifier, NULL is passed as the tables
pointer; that is, /L applies only to the expression on which it
appears.
The /M modifier causes the size in bytes of the memory block used
to hold the compiled pattern to be output. This does not include
the size of the pcre[16|32] block; it is just the actual compiled
data. If the pattern is successfully studied with the
PCRE_STUDY_JIT_COMPILE option, the size of the JIT compiled code
is also output.
The /Q modifier is used to test the use of pcre_stack_guard. It
must be followed by '0' or '1', specifying the return code to be
given from an external function that is passed to PCRE and used
for stack checking during compilation (see the pcreapi
documentation for details).
The /S modifier causes pcre[16|32]_study() to be called after the
expression has been compiled, and the results used when the
expression is matched. There are a number of qualifying
characters that may follow /S. They may appear in any order.
If /S is followed by an exclamation mark, pcre[16|32]_study() is
called with the PCRE_STUDY_EXTRA_NEEDED option, causing it always
to return a pcre_extra block, even when studying discovers no
useful information.
If /S is followed by a second S character, it suppresses
studying, even if it was requested externally by the -s command
line option. This makes it possible to specify that certain
patterns are always studied, and others are never studied,
independently of -s. This feature is used in the test files in a
few cases where the output is different when the pattern is
studied.
If the /S modifier is followed by a + character, the call to
pcre[16|32]_study() is made with all the JIT study options,
requesting just-in-time optimization support if it is available,
for both normal and partial matching. If you want to restrict the
JIT compiling modes, you can follow /S+ with a digit in the range
1 to 7:
1 normal match only
2 soft partial match only
3 normal match and soft partial match
4 hard partial match only
6 soft and hard partial match
7 all three modes (default)
If /S++ is used instead of /S+ (with or without a following
digit), the text "(JIT)" is added to the first output line after
a match or no match when JIT-compiled code was actually used.
Note that there is also an independent /+ modifier; it must not
be given immediately after /S or /S+ because this will be
misinterpreted.
If JIT studying is successful, the compiled JIT code will
automatically be used when pcre[16|32]_exec() is run, except when
incompatible run-time options are specified. For more details,
see the pcrejit documentation. See also the \J escape sequence
below for a way of setting the size of the JIT stack.
Finally, if /S is followed by a minus character, JIT compilation
is suppressed, even if it was requested externally by the -s
command line option. This makes it possible to specify that JIT
is never to be used for certain patterns.
The /T modifier must be followed by a single digit. It causes a
specific set of built-in character tables to be passed to
pcre[16|32]_compile(). It is used in the standard PCRE tests to
check behaviour with different character tables. The digit
specifies the tables as follows:
0 the default ASCII tables, as distributed in
pcre_chartables.c.dist
1 a set of tables defining ISO 8859 characters
In table 1, some characters whose codes are greater than 128 are
identified as letters, digits, spaces, etc.
Using the POSIX wrapper API
The /P modifier causes pcretest to call PCRE via the POSIX
wrapper API rather than its native API. This supports only the
8-bit library. When /P is set, the following modifiers set
options for the regcomp() function:
/i REG_ICASE
/m REG_NEWLINE
/N REG_NOSUB
/s REG_DOTALL )
/U REG_UNGREEDY ) These options are not part of
/W REG_UCP ) the POSIX standard
/8 REG_UTF8 )
The /+ modifier works as described above. All other modifiers are
ignored.
Locking out certain modifiers
PCRE can be compiled with or without support for certain features
such as UTF-8/16/32 or Unicode properties. Accordingly, the
standard tests are split up into a number of different files that
are selected for running depending on which features are
available. When updating the tests, it is all too easy to put a
new test into the wrong file by mistake; for example, to put a
test that requires UTF support into a file that is used when it
is not available. To help detect such mistakes as early as
possible, there is a facility for locking out specific modifiers.
If an input line for pcretest starts with the string "< forbid "
the following sequence of characters is taken as a list of
forbidden modifiers. For example, in the test files that must not
use UTF or Unicode property support, this line appears:
< forbid 8W
This locks out the /8 and /W modifiers. An immediate error is
given if they are subsequently encountered. If the character
string contains < but not >, all the multi-character modifiers
that begin with < are locked out. Otherwise, such modifiers must
be explicitly listed, for example:
< forbid <JS><cr>
There must be a single space between < and "forbid" for this
feature to be recognised. If there is not, the line is
interpreted either as a request to re-load a pre-compiled pattern
(see "SAVING AND RELOADING COMPILED PATTERNS" below) or, if there
is a another < character, as a pattern that uses < as its
delimiter.
