The systemtap script language resembles awk and C. There are two
main outermost constructs: probes and functions. Within these,
statements and expressions use C-like operator syntax and
precedence.
GENERAL SYNTAX
Whitespace is ignored. Three forms of comments are supported:
# ... shell style, to the end of line, except for $# and
@#
// ... C++ style, to the end of line
/* ... C style ... */
Literals are either strings enclosed in double-quotes (passing
through the usual C escape codes with backslashes, and with
adjacent string literals glued together, also as in C), or
integers (in decimal, hexadecimal, or octal, using the same
notation as in C). All strings are limited in length to some
reasonable value (a few hundred bytes). Integers are 64-bit
signed quantities, although the parser also accepts (and wraps
around) values above positive 2**63.
In addition, script arguments given at the end of the command
line may be inserted. Use $1 ... $<NN> for insertion unquoted,
@1 ... @<NN> for insertion as a string literal. The number of
arguments may be accessed through $# (as an unquoted number) or
through @# (as a quoted number). These may be used at any place
a token may begin, including within the preprocessing stage.
Reference to an argument number beyond what was actually given is
an error.
PREPROCESSING
A simple conditional preprocessing stage is run as a part of
parsing. The general form is similar to the cond ? exp1 : exp2
ternary operator:
%( CONDITION %? TRUE-TOKENS %)
%( CONDITION %? TRUE-TOKENS %: FALSE-TOKENS %)
The CONDITION is either an expression whose format is determined
by its first keyword, or a string literals comparison or a
numeric literals comparison. It can be also composed of many
alternatives and conjunctions of CONDITIONs (meant as in previous
sentence) using || and && respectively. However, parentheses are
not supported yet, so remembering that conjunction takes
precedence over alternative is important.
If the first part is the identifier kernel_vr or kernel_v to
refer to the kernel version number, with ("2.6.13-1.322FC3smp")
or without ("2.6.13") the release code suffix, then the second
part is one of the six standard numeric comparison operators <,
<=, ==, !=, >, and >=, and the third part is a string literal
that contains an RPM-style version-release value. The condition
is deemed satisfied if the version of the target kernel (as
optionally overridden by the -r option) compares to the given
version string. The comparison is performed by the glibc
function strverscmp. As a special case, if the operator is for
simple equality (==), or inequality (!=), and the third part
contains any wildcard characters (* or ? or [), then the
expression is treated as a wildcard (mis)match as evaluated by
fnmatch.
If, on the other hand, the first part is the identifier arch to
refer to the processor architecture (as named by the kernel build
system ARCH/SUBARCH), then the second part is one of the two
string comparison operators == or !=, and the third part is a
string literal for matching it. This comparison is a wildcard
(mis)match.
Similarly, if the first part is an identifier like
CONFIG_something to refer to a kernel configuration option, then
the second part is == or !=, and the third part is a string
literal for matching the value (commonly "y" or "m").
Nonexistent or unset kernel configuration options are represented
by the empty string. This comparison is also a wildcard
(mis)match.
If the first part is the identifier systemtap_v, the test refers
to the systemtap compatibility version, which may be overridden
for old scripts with the --compatible flag. The comparison
operator is as is for kernel_v and the right operand is a version
string. See also the DEPRECATION section below.
If the first part is the identifier systemtap_privilege, the test
refers to the privilege level that the systemtap script is
compiled with. Here the second part is == or !=, and the third
part is a string literal, either "stapusr" or "stapsys" or
"stapdev".
If the first part is the identifier guru_mode, the test refers to
if the systemtap script is compiled with guru_mode. Here the
second part is == or !=, and the third part is a number, either 1
or 0.
If the first part is the identifier runtime, the test refers to
the systemtap runtime mode. See ALTERNATE RUNTIMES below for more
information on runtimes. The second part is one of the two
string comparison operators == or !=, and the third part is a
string literal for matching it. This comparison is a wildcard
(mis)match.
Otherwise, the CONDITION is expected to be a comparison between
two string literals or two numeric literals. In this case, the
arguments are the only variables usable.
The TRUE-TOKENS and FALSE-TOKENS are zero or more general parser
tokens (possibly including nested preprocessor conditionals), and
are passed into the input stream if the condition is true or
false. For example, the following code induces a parse error
unless the target kernel version is newer than 2.6.5:
%( kernel_v <= "2.6.5" %? **ERROR** %) # invalid token sequence
The following code might adapt to hypothetical kernel version
drift:
probe kernel.function (
%( kernel_v <= "2.6.12" %? "__mm_do_fault" %:
%( kernel_vr == "2.6.13*smp" %? "do_page_fault" %:
UNSUPPORTED %) %)
) { /* ... */ }
%( arch == "ia64" %?
probe syscall.vliw = kernel.function("vliw_widget") {}
%)
PREPROCESSOR MACROS
The preprocessor also supports a simple macro facility, run as a
separate pass before conditional preprocessing.
Macros are defined using the following construct:
@define NAME %( BODY %)
@define NAME(PARAM_1, PARAM_2, ...) %( BODY %)
Macros, and parameters inside a macro body, are both invoked by
prefixing the macro name with an @ symbol:
@define foo %( x %)
@define add(a,b) %( ((@a)+(@b)) %)
@foo = @add(2,2)
Macro expansion is currently performed in a separate pass before
conditional compilation. Therefore, both TRUE- and FALSE-tokens
in conditional expressions will be macroexpanded regardless of
how the condition is evaluated. This can sometimes lead to
errors:
// The following results in a conflict:
%( CONFIG_UTRACE == "y" %?
@define foo %( process.syscall %)
%:
@define foo %( **ERROR** %)
%)
// The following works properly as expected:
@define foo %(
%( CONFIG_UTRACE == "y" %? process.syscall %: **ERROR** %)
%)
The first example is incorrect because both @defines are
evaluated in a pass prior to the conditional being evaluated.
Normally, a macro definition is local to the file it occurs in.
Thus, defining a macro in a tapset does not make it available to
the user of the tapset. Publically available library macros can
be defined by including .stpm files on the tapset search path.
These files may only contain @define constructs, which become
visible across all tapsets and user scripts. Optionally, within
the .stpm files, a public macro definition can be surrounded by a
preprocessor conditional as described above.
CONSTANTS
Tapsets or guru-mode user scripts can access header file constant
tokens, typically macros, using built-in @const() operator. The
respective header file inclusion is possible either via the
tapset library, or using a top-level guru mode embedded-C
construct. This results in appropriate embedded C pragma
comments setting.
@const("STP_SKIP_BADVARS")
VARIABLES
Identifiers for variables and functions are an alphanumeric
sequence, and may include _ and $ characters. They may not start
with a plain digit, as in C. Each variable is by default local
to the probe or function statement block within which it is
mentioned, and therefore its scope and lifetime is limited to a
particular probe or function invocation.
Scalar variables are implicitly typed as either string or
integer. Associative arrays also have a string or integer value,
and a tuple of strings and/or integers serving as a key. Here
are a few basic expressions.
var1 = 5
var2 = "bar"
array1 [pid()] = "name" # single numeric key
array2 ["foo",4,i++] += 5 # vector of string/num/num keys
if (["hello",5,4] in array2) println ("yes") # membership test
The translator performs type inference on all identifiers,
including array indexes and function parameters. Inconsistent
type-related use of identifiers signals an error.
Variables may be declared global, so that they are shared amongst
all probes and functions and live as long as the entire systemtap
session. There is one namespace for all global variables,
regardless of which script file they are found within.
Concurrent access to global variables is automatically protected
with locks, see the SAFETY AND SECURITY section for more details.
A global declaration may be written at the outermost level
anywhere, not within a block of code. Global variables which are
written but never read will be displayed automatically at session
shutdown. The translator will infer for each its value type, and
if it is used as an array, its key types. Optionally, scalar
globals may be initialized with a string or number literal. The
following declaration marks variables as global.
global var1, var2, var3=4
Global variables can also be set as module options. One can do
this by either using the -G option, or the module must first be
compiled using stap -p4. Global variables can then be set on the
command line when calling staprun on the module generated by stap
-p4. See staprun(8) for more information.
The scope of a global variable may be limited to a tapset or user
script file using private keyword. The global keyword is optional
when defining a private global variable. Following declaration
marks var1 and var2 private globals.
private global var1=2
private var2
Arrays are limited in size by the MAXMAPENTRIES variable -- see
the SAFETY AND SECURITY section for details. Optionally, global
arrays may be declared with a maximum size in brackets,
overriding MAXMAPENTRIES for that array only. Note that this
doesn't indicate the type of keys for the array, just the size.
global tiny_array[10], normal_array, big_array[50000]
Arrays may be configured for wrapping using the '%' suffix. This
causes older elements to be overwritten if more elements are
inserted than the array can hold. This works for both associative
and statistics typed arrays.
global wrapped_array1%[10], wrapped_array2%
Many types of probe points provide context variables, which are
run-time values, safely extracted from the kernel or userspace
program being probed. These are prefixed with the $ character.
The CONTEXT VARIABLES section in stapprobes(3stap) lists what is
available for each type of probe point. These context variables
become normal string or numeric scalars once they are stored in
normal script variables. See the TYPECASTING section below on
how to to turn them back into typed pointers for further
processing as context variables. There is some automation to
help!
STATEMENTS
Statements enable procedural control flow. They may occur within
functions and probe handlers. The total number of statements
executed in response to any single probe event is limited to some
number defined by the MAXACTION macro in the translated C code,
and is in the neighbourhood of 1000.
EXP Execute the string- or integer-valued expression and throw
away the value.
{ STMT1 STMT2 ... }
Execute each statement in sequence in this block. Note
that separators or terminators are generally not necessary
between statements.
; Null statement, do nothing. It is useful as an optional
separator between statements to improve syntax-error
detection and to handle certain grammar ambiguities.
if (EXP) STMT1 [ else STMT2 ]
Compare integer-valued EXP to zero. Execute the first
(non-zero) or second STMT (zero).
while (EXP) STMT
While integer-valued EXP evaluates to non-zero, execute
STMT.
for (EXP1; EXP2; EXP3) STMT
Execute EXP1 as initialization. While EXP2 is non-zero,
execute STMT, then the iteration expression EXP3.
foreach (VAR in ARRAY [ limit EXP ]) STMT
Loop over each element of the named global array,
assigning current key to VAR. The array may not be
modified within the statement. By adding a single + or -
operator after the VAR or the ARRAY identifier, the
iteration will proceed in a sorted order, by ascending or
descending index or value. If the array contains
statistics aggregates, adding the desired @operator
between the ARRAY identifier and the + or - will specify
the sorting aggregate function. See the STATISTICS
section below for the ones available. Default is @count.
Using the optional limit keyword limits the number of loop
iterations to EXP times. EXP is evaluated once at the
beginning of the loop.
foreach ([VAR1, VAR2, ...] in ARRAY [ limit EXP ]) STMT
Same as above, used when the array is indexed with a tuple
of keys. A sorting suffix may be used on at most one VAR
or ARRAY identifier.
foreach ([VAR1, VAR2, ...] in ARRAY [INDEX1, INDEX2, ...] [ limit
EXP ]) STMT
Same as above, where iterations are limited to elements in
the array where the keys match the index values specified.
The symbol * can be used to specify an index and will be
treated as a wildcard.
foreach (VAR0 = VAR in ARRAY [ limit EXP ]) STMT
This variant of foreach saves current value into VAR0 on
each iteration, so it is the same as ARRAY[VAR]. This
also works with a tuple of keys. Sorting suffixes on VAR0
have the same effect as on ARRAY.
foreach (VAR0 = VAR in ARRAY [INDEX1, INDEX2, ...] [ limit EXP ])
STMT
Same as above, where iterations are limited to elements in
the array where the keys match the index values specified.
The symbol * can be used to specify an index and will be
treated as a wildcard.
break, continue
Exit or iterate the innermost nesting loop (while or for
or foreach) statement.
return EXP
Return EXP value from enclosing function. If the
function's value is not taken anywhere, then a return
statement is not needed, and the function will have a
special "unknown" type with no return value.
next Return now from enclosing probe handler. This is
especially useful in probe aliases that apply event
filtering predicates. When used in functions, the
execution will be immediately transferred to the next
overloaded function.
try { STMT1 } catch { STMT2 }
Run the statements in the first block. Upon any run-time
errors, abort STMT1 and start executing STMT2. Any errors
in STMT2 will propagate to outer try/catch blocks, if any.
try { STMT1 } catch(VAR) { STMT2 }
Same as above, plus assign the error message to the string
scalar variable VAR.
delete ARRAY[INDEX1, INDEX2, ...]
Remove from ARRAY the element specified by the index
tuple. If the index tuple contains a * in place of an
index, the * is treated as a wildcard and all elements
with keys that match the index tuple will be removed from
ARRAY. The value will no longer be available, and
subsequent iterations will not report the element. It is
not an error to delete an element that does not exist.
delete ARRAY
Remove all elements from ARRAY.
delete SCALAR
Removes the value of SCALAR. Integers and strings are
cleared to 0 and "" respectively, while statistics are
reset to the initial empty state.
EXPRESSIONS
Systemtap supports a number of operators that have the same
general syntax, semantics, and precedence as in C and awk.
Arithmetic is performed as per typical C rules for signed
integers. Division by zero or overflow is detected and results
in an error.
binary numeric operators
* / % + - >> << & ^ | && ||
binary string operators
. (string concatenation)
numeric assignment operators
= *= /= %= += -= >>= <<= &= ^= |=
string assignment operators
= .=
unary numeric operators
+ - ! ~ ++ --
binary numeric, string comparison or regex matching operators
< > <= >= == != =~ !~
ternary operator
cond ? exp1 : exp2
grouping operator
( exp )
function call
fn ([ arg1, arg2, ... ])
array membership check
exp in array
[exp1, exp2, ... ] in array
[*, *, ... ] in array
REGULAR EXPRESSION MATCHING
The scripting language supports regular expression matching. The
basic syntax is as follows:
exp =~ regex
exp !~ regex
(The first operand must be an expression evaluating to a string;
the second operand must be a string literal containing a
syntactically valid regular expression.)
The regular expression syntax supports most of the features of
POSIX Extended Regular Expressions, except for subexpression
reuse ("\1") functionality.
After a successful match, the contents of the matched string and
subexpressions can be extracted using the matched() and ngroups()
tapset functions as follows:
if ("an example string" =~ "str(ing)") {
matched(0) // -> returns "string", the matched substring
matched(1) // -> returns "ing", the 1st matched subexpression
ngroups() // -> returns 2, the number of matched groups
}
PROBES
The main construct in the scripting language identifies probes.
Probes associate abstract events with a statement block ("probe
handler") that is to be executed when any of those events occur.
The general syntax is as follows:
probe PROBEPOINT [, PROBEPOINT] { [STMT ...] }
probe PROBEPOINT [, PROBEPOINT] if (CONDITION) { [STMT ...] }
Events are specified in a special syntax called "probe points".
There are several varieties of probe points defined by the
translator, and tapset scripts may define further ones using
aliases. Probe points may be wildcarded, grouped, or listed in
preference sequences, or declared optional. More details on
probe point syntax and semantics are listed on the
stapprobes(3stap) manual page.
The probe handler is interpreted relative to the context of each
event. For events associated with kernel code, this context may
include variables defined in the source code at that spot. These
"context variables" are presented to the script as variables
whose names are prefixed with "$". They may be accessed only if
the kernel's compiler preserved them despite optimization. This
is the same constraint that a debugger user faces when working
with optimized code. In addition, the objects must exist in
paged-in memory at the moment of the systemtap probe handler's
execution, because systemtap must not cause (suppresses) any
additional paging. Some probe types have very little context.
See the stapprobes(3stap) man pages to see the kinds of context
variables available at each kind of probe point. As of systemtap
version 4.3, functions called from the handlers of some probe
point types may also refer to context variables. These are
treated as if a clone of that function was inlined into the
calling probe handler and $variables evaluated in its context.
Probes may be decorated with an arming condition, consisting of a
simple boolean expression on read-only global script variables.
While disarmed (inactive, condition evaluates to false), some
probe types reduce or eliminate their run-time overheads. When
an arming condition evaluates to true, probes will be soon re-
armed, and their probe handlers will start getting called as the
events fire. (Some events may be lost during the arming
interval. If this is unacceptable, do not use arming conditions
for those probes.) Example of the syntax:
probe timer.us(TIMER) if (enabled) {
}
New probe points may be defined using "aliases". Probe point
aliases look similar to probe definitions, but instead of
activating a probe at the given point, it just defines a new
probe point name as an alias to an existing one. There are two
types of alias, i.e. the prologue style and the epilogue style
which are identified by "=" and "+=" respectively.
For prologue style alias, the statement block that follows an
alias definition is implicitly added as a prologue to any probe
that refers to the alias. While for the epilogue style alias, the
statement block that follows an alias definition is implicitly
added as an epilogue to any probe that refers to the alias. For
example:
probe syscall.read = kernel.function("sys_read") {
fildes = $fd
if (execname() == "init") next # skip rest of probe
}
defines a new probe point syscall.read, which expands to
kernel.function("sys_read"), with the given statement as a
prologue, which is useful to predefine some variables for the
alias user and/or to skip probe processing entirely based on some
conditions. And
probe syscall.read += kernel.function("sys_read") {
if (tracethis) println ($fd)
}
defines a new probe point with the given statement as an
epilogue, which is useful to take actions based upon variables
set or left over by the the alias user. Please note that in each
case, the statements in the alias handler block are treated
ordinarily, so that variables assigned there constitute mere
initialization, not a macro substitution.
Aliases can also be defined to include both a prologue and an
epilogue.
probe syscall.read = kernel.function("sys_read") {
fildes = $fd
if (execname() == "init") next
},{
if (tracethis) println ($fd)
}
An alias is used just like a built-in probe type.
probe syscall.read {
printf("reading fd=%d\n", fildes)
if (fildes > 10) tracethis = 1
}
Probes with an alias can make use of the @probewrite predicate.
This check is used to detect whether a script variable or target
variable has been written to in the probe handler body.
@probewrite(var)
expands to 1 iff var has been written to in the probe
handler body, otherwise it expands to 0.
In the following example, @probewrite(var) expands to 1 because
var has been written to in the probe handler body and
consequently, the conditional statement will run.
probe foo = begin { var = 0 }, { if (@probewrite(var)) println(var) }
probe foo {
var = 1
}
FUNCTIONS
Systemtap scripts may define subroutines to factor out common
work. Functions take any number of scalar (integer or string)
arguments, and must return a single scalar (integer or string).
An example function declaration looks like this:
function thisfn (arg1, arg2) {
return arg1 + arg2
}
Note the general absence of type declarations, which are instead
inferred by the translator. However, if desired, a function
definition may include explicit type declarations for its return
value and/or its arguments. This is especially helpful for
embedded-C functions. In the following example, the type
inference engine need only infer type type of arg2 (a string).
function thatfn:string (arg1:long, arg2) {
return sprint(arg1) . arg2
}
Functions may call others or themselves recursively, up to a
fixed nesting limit. This limit is defined by the MAXNESTING
macro in the translated C code and is in the neighbourhood of 10.
Functions may be marked private using the private keyword to
limit their scope to the tapset or user script file they are
defined in. An example definition of a private function follows:
private function three:long () { return 3 }
Functions terminating without reaching an explicit return
statement will return an implicit 0 or "", determined by type
inference.
Functions may be overloaded during both runtime and compile time.
Runtime overloading allows the executed function to be selected
while the module is running based on runtime conditions and is
achieved using the "next" statement in script functions and
STAP_NEXT macro for embedded-C functions. For example,
function f() { if (condition) next; print("first function") }
function f() %{ STAP_NEXT; print("second function") %}
function f() { print("third function") }
During a functioncall f(), the execution will transfer to the
third function if condition evaluates to true and print "third
function". Note that the second function is unconditionally
nexted.
Parameter overloading allows the function to be executed to be
selected at compile time based on the number of arguments
provided to the functioncall. For example,
function g() { print("first function") }
function g(x) { print("second function") }
g() -> "first function"
g(1) -> "second function"
Note that runtime overloading does not occur in the above
example, as exactly one function will be resolved for the
functioncall. The use of a next statement inside a function while
no more overloads remain will trigger a runtime exception Runtime
overloading will only occur if the functions have the same arity,
functions with the same name but different number of parameters
are completely unrelated.
Execution order is determined by a priority value which may be
specified. If no explicit priority is specified, user script
functions are given a higher priority than library functions.
User script functions and library functions are assigned a
default priority value of 0 and 1 respectively. Functions with
the same priority are executed in declaration order. For example,
function f():3 { if (condition) next; print("first function") }
function f():1 { if (condition) next; print("second function") }
function f():2 { print("third function") }
Since the second function has highest priority, it is executed
first. The first function is never executed as there no "next"
statements in the third function to transfer execution.
PRINTING
There are a set of function names that are specially treated by
the translator. They format values for printing to the standard
systemtap output stream in a more convenient way (note that data
generated in the kernel module need to get transferred to user-
space in order to get printed).
The sprint* variants return the formatted string instead of
printing it.
print, sprint
Print one or more values of any type, concatenated
directly together.
println, sprintln
Print values like print and sprint, but also append a
newline.
printd, sprintd
Take a string delimiter and two or more values of any
type, and print the values with the delimiter interposed.
The delimiter must be a literal string constant.
printdln, sprintdln
Print values with a delimiter like printd and sprintd, but
also append a newline.
printf, sprintf
Take a formatting string and a number of values of
corresponding types, and print them all. The format must
be a literal string constant.
The printf formatting directives similar to those of C, except
that they are fully type-checked by the translator:
%b Writes a binary blob of the value given, instead of
ASCII text. The width specifier determines the
number of bytes to write; valid specifiers are %b
%1b %2b %4b %8b. Default (%b) is 8 bytes.
%c Character.
%d,%i Signed decimal.
%m Safely reads kernel (without #) or user (with #)
memory at the given address, outputs its content.
The optional precision specifier (not field width)
determines the number of bytes to read - default is
1 byte. %10.4m prints 4 bytes of the memory in a
10-character-wide field. Note, on some
architectures user memory can still be read without
#.
%M Same as %m, but outputs in hexadecimal. The
minimal size of output is double the optional
precision specifier - default is 1 byte (2 hex
chars). %10.4M prints 4 bytes of the memory as 8
hexadecimal characters in a 10-character-wide
field. %.*M hex-dumps a given number of bytes
from a given buffer.
%o Unsigned octal.
%p Unsigned pointer address.
%s String.
%u Unsigned decimal.
%x Unsigned hex value, in all lower-case.
%X Unsigned hex value, in all upper-case.
%% Writes a %.
The # flag selects the alternate forms. For octal, this prefixes
a 0. For hex, this prefixes 0x or 0X, depending on case. For
characters, this escapes non-printing values with either C-like
escapes or raw octal. In the case of %#m/%#M, this safely
accesses user space memory rather than kernel space memory.
Examples:
a = "alice", b = "bob", p = 0x1234abcd, i = 123, j = -1, id[a] = 1234, id[b] = 4567
print("hello")
Prints: hello
println(b)
Prints: bob\n
println(a . " is " . sprint(16))
Prints: alice is 16
foreach (name in id) printdln("|", strlen(name), name, id[name])
Prints: 5|alice|1234\n3|bob|4567
printf("%c is %s; %x or %X or %p; %d or %u\n",97,a,p,p,p,j,j)
Prints: a is alice; 1234abcd or 1234ABCD or 0x1234abcd; -1 or 18446744073709551615\n
printf("2 bytes of kernel buffer at address %p: %2m", p, p)
Prints: 2 byte of kernel buffer at address 0x1234abcd: <binary data>
printf("%4b", p)
Prints (these values as binary data): 0x1234abcd
printf("%#o %#x %#X\n", 1, 2, 3)
Prints: 01 0x2 0X3
printf("%#c %#c %#c\n", 0, 9, 42)
Prints: \000 \t *
STATISTICS
It is often desirable to collect statistics in a way that avoids
the penalties of repeatedly exclusive locking the global
variables those numbers are being put into. Systemtap provides a
solution using a special operator to accumulate values, and
several pseudo-functions to extract the statistical aggregates.
The aggregation operator is <<<, and resembles an assignment, or
a C++ output-streaming operation. The left operand specifies a
scalar or array-index lvalue, which must be declared global. The
right operand is a numeric expression. The meaning is intuitive:
add the given number to the pile of numbers to compute statistics
of. (The specific list of statistics to gather is given
separately, by the extraction functions.)
foo <<< 1
stats[pid()] <<< memsize
The extraction functions are also special. For each appearance
of a distinct extraction function operating on a given
identifier, the translator arranges to compute a set of
statistics that satisfy it. The statistics system is thereby
"on-demand". Each execution of an extraction function causes the
aggregation to be computed for that moment across all processors.
Here is the set of extractor functions. The first argument of
each is the same style of lvalue used on the left hand side of
the accumulate operation. The @count(v), @sum(v), @min(v),
@max(v), @avg(v), @variance(v[, b]) extractor functions compute
the number/total/minimum/maximum/average/variance of all
accumulated values. The resulting values are all simple
integers. Arrays containing aggregates may be sorted and
iterated. See the foreach construct above.
Variance uses Welford's online algorithm. The calculations are
based on integer arithmetic, and so may suffer from low precision
and overflow. To improve this, @variance(v[, b]) accepts an
optional parameter b, the bit-shift, ranging from 0 (default) to
62, for internal scaling. Only one value of bit-shift may be
used with given global variable. A larger bitshift value
increases precision, but increases the likelihood of overflow.
$ stap -e \
> 'global x probe oneshot { for(i=1;i<=5;i++) x<<<i println(@variance(x)) }'
12
$ stap -e \
> 'global x probe oneshot { for(i=1;i<=5;i++) x<<<i println(@variance(x,1)) }'
2
$ python3 -c 'import statistics; print(statistics.variance([1, 2, 3, 4, 5]))'
2.5
$
Overflow (from internal multiplication of large numbers) may
occur and may cause a negative variance result. Consider
normalizing your input data. Adding or subtracting a fixed value
from all variance inputs preserves the original variance.
Dividing the variance inputs by a fixed value shrinks the
original variance by that value squared.
Histograms are also available, but are more complicated because
they have a vector rather than scalar value.
@hist_linear(v,start,stop,interval) represents a linear histogram
from "start" to "stop" (inclusive) by increments of "interval".
The interval must be positive. Similarly, @hist_log(v) represents
a base-2 logarithmic histogram. Printing a histogram with the
print family of functions renders a histogram object as a tabular
"ASCII art" bar chart.
probe timer.profile {
x[1] <<< pid()
x[2] <<< uid()
y <<< tid()
}
global x // an array containing aggregates
global y // a scalar
probe end {
foreach ([i] in x @count+) {
printf ("x[%d]: avg %d = sum %d / count %d\n",
i, @avg(x[i]), @sum(x[i]), @count(x[i]))
println (@hist_log(x[i]))
}
println ("y:")
println (@hist_log(y))
}
The counts of each histogram bucket may be individually accessed
via the [index] operator. Each bucket is addressed from 1
through N (for each natural bucket). In addition bucket #0
counts all the samples beneath the start value, and bucket #N+1
counts all the samples above the stop value. Histogram buckets
(including the two out-of-range buckets) may also be iterated
with foreach.
global x
probe oneshot {
x <<< -100
x <<< 1
x <<< 2
x <<< 3
x <<< 100
foreach (bucket in @hist_linear(x,1,3,1))
// expecting 1 out-of-range-low bucket
// 3 payload buckets
// 1 out-of-range-high bucket
printf("bucket %d count %d\n",
bucket, @hist_linear(x,1,3,1)[bucket])
}
TYPECASTING
Once a pointer (see the CONTEXT VARIABLES section of
stapprobes(3stap)) has been saved into a script integer variable,
the translator attempts to keep the type information necessary to
access members from that pointer.
The translator attempts to track DWARF typing associated with
script variables assigned from addresses of context $variables,
@cast or @var operators. Depending on the complexity of the
script code, this association may pass to related variables, so
that -> and [] operators may be used on them, just as on the
original context variable. For example:
foo = $param->foo; printf("x:%d y:%d\n", foo->x, foo->y)
printf("my value is %d\n", ($type == 42 ? $foo : $bar)->value)
printf("my parent pid is %d\n", task_parent(task_current())->tgid)
However, if this association heuristic doesn't work for a script,
using the @cast() operator tells the translator how to interpret
the number as a typed pointer.
@cast(p, "type_name"[, "module"])->member
This will interpret p as a pointer to a struct/union named
type_name and dereference the member value. Further ->subfield
expressions may be appended to dereference more levels. Note that
for direct dereferencing of a pointer
{kernel,user}_{char,int,...}($p) should be used. (Refer to
stapfuncs(5) for more details.) NOTE: the same dereferencing
operator -> is used to refer to both direct containment or
pointer indirection. Systemtap automatically determines which.
The optional module tells the translator where to look for
information about that type. Multiple modules may be specified
as a list with : separators. If the module is not specified, it
will default either to the probe module for dwarf probes, or to
"kernel" for functions and all other probes types.
The translator can create its own module with type information
from a header surrounded by angle brackets, in case normal
debuginfo is not available. For kernel headers, prefix it with
"kernel" to use the appropriate build system. All other headers
are built with default GCC parameters into a user module.
Multiple headers may be specified in sequence to resolve a
codependency.
@cast(tv, "timeval", "<sys/time.h>")->tv_sec
@cast(task, "task_struct", "kernel<linux/sched.h>")->tgid
@cast(task, "task_struct",
"kernel<linux/sched.h><linux/fs_struct.h>")->fs->umask
Values acquired by @cast may be pretty-printed by the $ and $$
suffix operators, the same way as described in the CONTEXT
VARIABLES section of the stapprobes(3stap) manual page.
When in guru mode, the translator will also allow scripts to
assign new values to members of typecasted pointers.
Typecasting is also useful in the case of void* members whose
type may be determinable at runtime.
probe foo {
if ($var->type == 1) {
value = @cast($var->data, "type1")->bar
} else {
value = @cast($var->data, "type2")->baz
}
print(value)
}
EMBEDDED C
When in guru mode, the translator accepts embedded C code in the
top level of the script. Such code is enclosed between %{ and %}
markers, and is transcribed verbatim, without analysis, in some
sequence, into the top level of the generated C code. At the
outermost level, this may be useful to add #include instructions,
and any auxiliary definitions for use by other embedded code.
Another place where embedded code is permitted is as a function
body. In this case, the script language body is replaced
entirely by a piece of C code enclosed again between %{ and %}
markers. This C code may do anything reasonable and safe. There
are a number of undocumented but complex safety constraints on
atomicity, concurrency, resource consumption, and run time
limits, so this is an advanced technique.
The memory locations set aside for input and output values are
made available to it using macros STAP_ARG_* and STAP_RETVALUE.
Errors may be signalled with STAP_ERROR. Output may be written
with STAP_PRINTF. The function may return early with STAP_RETURN.
Here are some examples:
function integer_ops (val) %{
STAP_PRINTF("%d\n", STAP_ARG_val);
STAP_RETVALUE = STAP_ARG_val + 1;
if (STAP_RETVALUE == 4)
STAP_ERROR("wrong guess: %d", (int) STAP_RETVALUE);
if (STAP_RETVALUE == 3)
STAP_RETURN(0);
STAP_RETVALUE ++;
%}
function string_ops (val) %{
strlcpy (STAP_RETVALUE, STAP_ARG_val, MAXSTRINGLEN);
strlcat (STAP_RETVALUE, "one", MAXSTRINGLEN);
if (strcmp (STAP_RETVALUE, "three-two-one"))
STAP_RETURN("parameter should be three-two-");
%}
function no_ops () %{
STAP_RETURN(); /* function inferred with no return value */
%}
The function argument and return value types have to be inferred
by the translator from the call sites in order for this to work.
The user should examine C code generated for ordinary script-
language functions in order to write compatible embedded-C ones.
The last place where embedded code is permitted is as an
expression rvalue. In this case, the C code enclosed between %{
and %} markers is interpreted as an ordinary expression value.
It is assumed to be a normal 64-bit signed number, unless the
marker /* string */ is included, in which case it's treated as a
string.
function add_one (val) {
return val + %{ 1 %}
}
function add_string_two (val) {
return val . %{ /* string */ "two" %}
}
@define SOME_STAP_MACRO %( %{ SOME_C_MACRO %} %)
probe begin {
printf("SOME_C_MACRO has value: %d\n", @SOME_STAP_MACRO);
}
The embedded-C code may contain markers to assert optimization
and safety properties.
/* pure */
means that the C code has no side effects and may be
elided entirely if its value is not used by script code.
/* stable */
means that the C code always has the same value (in any
given probe handler invocation), so repeated calls may be
automatically replaced by memoized values. Such functions
must take no parameters, and also be pure.
/* unprivileged */
means that the C code is so safe that even unprivileged
users are permitted to use it.
/* myproc-unprivileged */
means that the C code is so safe that even unprivileged
users are permitted to use it, provided that the target of
the current probe is within the user's own process.
/* guru */
means that the C code is so unsafe that a systemtap user
must specify -g (guru mode) to use this. (Tapsets are
permitted and presumed to call them safely.)
/* unmangled */
in an embedded-C function, means that the legacy (pre-1.8)
argument access syntax should be made available inside the
function. Hence, in addition to STAP_ARG_foo and
STAP_RETVALUE one can use THIS->foo and THIS->__retvalue
respectively inside the function. This is useful for
quickly migrating code written for SystemTap version 1.7
and earlier.
/* unmodified-fnargs */
in an embedded-C function, means that the function
arguments are not modified inside the function body.
/* string */
in embedded-C expressions only, means that the expression
has const char * type and should be treated as a string
value, instead of the default long numeric.
Script level global variables may be accessed in embedded-C
functions and blocks. To read or write the global variable var ,
the /* pragma:read:var */ or /* pragma:write:var */ marker must
be first placed in the embedded-C function or block. This
provides the macros STAP_GLOBAL_GET_* and STAP_GLOBAL_SET_*
macros to allow reading and writing, respectively. For example:
global var
global var2[100]
function increment() %{
/* pragma:read:var */ /* pragma:write:var */
/* pragma:read:var2 */ /* pragma:write:var2 */
STAP_GLOBAL_SET_var(STAP_GLOBAL_GET_var()+1); //var++
STAP_GLOBAL_SET_var2(1, 1, STAP_GLOBAL_GET_var2(1, 1)+1); //var2[1,1]++
%}
Variables may be read and set in both embedded-C functions and
expressions. Strings returned from embedded-C code are decayed
to pointers. Variables must also be assigned at script level to
allow for type inference. Map assignment does not return the
value written, so chaining does not work.
BUILT-INS
A set of builtin probe point aliases are provided by the scripts
installed in the directory specified in the stappaths(7) manual
page. The functions are described in the stapprobes(3stap)
manual page.
DEREFERENCING
Integers can be dereferenced from pointers saved as a script
integer variables using the @kderef() or @uderef() operators.
@kderef() is used for kernel space addresses and @uderef() is
used for user space addresses.
@kderef(SIZE, addr)
@uderef(SIZE, addr)
This will interpret addr as a kernel/user address and read SIZE
bytes starting at that address. SIZE should be either 1, 2, 4 or
8 bytes.
REGISTERS
The value stored within a register can be accessed using the
@kregister() or @uregister() operators. @kregister() is used for
kernel space registers and @uregister() is used for user space
registers. The register of interest is specified using its DWARF
number.
@kregister(0)
@uregister(5)
