расширенный мониторинг системы и процессов (Advanced System and Process Monitor)
OUTPUT DESCRIPTION - PROCESS LEVEL
Following the system level information, the processes are shown
from which the resource utilization has changed during the last
interval. These processes might have used cpu time or issued
disk or network requests. However a process is also shown if
part of it has been paged out due to lack of memory (while the
process itself was in sleep state).
Per process the following fields may be shown (in alphabetical
order), depending on the current output mode as described in the
section INTERACTIVE COMMANDS and depending on the current width
of your window:
AVGRSZ
The average size of one read-action on disk.
AVGWSZ
The average size of one write-action on disk.
CID
Container ID (Docker) of 12 hexadecimal digits,
referring to the container in which the process/thread
is running. If a process has been started and finished
during the last interval, a `?' is shown because the
container ID is not part of the standard process
accounting record.
CMD
The name of the process. This name can be surrounded by
"less/greater than" signs (`<name>') which means that
the process has finished during the last interval.
Behind the abbreviation `CMD' in the header line, the
current page number and the total number of pages of the
process/thread list are shown.
COMMAND-LINE
The full command line of the process (including
arguments). If the length of the command line exceeds
the length of the screen line, the arrow keys -> and <-
can be used for horizontal scroll.
Behind the verb `COMMAND-LINE' in the header line, the
current page number and the total number of pages of the
process/thread list are shown.
CPU
The occupation percentage of this process related to the
available capacity for this resource on system level.
CPUNR
The identification of the CPU the (main) thread is
running on or has recently been running on.
CTID
Container ID (OpenVZ). If a process has been started
and finished during the last interval, a `?' is shown
because the container ID is not part of the standard
process accounting record.
DSK
The occupation percentage of this process related to the
total load that is produced by all processes (i.e. total
disk accesses by all processes during the last
interval).
This information is shown when per process "storage
accounting" is active in the kernel.
EGID
Effective group-id under which this process executes.
ENDATE
Date that the process has been finished. If the process
is still running, this field shows `active'.
ENTIME
Time that the process has been finished. If the process
is still running, this field shows `active'.
ENVID
Virtual environment identified (OpenVZ only).
EUID
Effective user-id under which this process executes.
EXC
The exit code of a terminated process (second position
of column `ST' is E) or the fatal signal number (second
position of column `ST' is S or C).
FSGID
Filesystem group-id under which this process executes.
FSUID
Filesystem user-id under which this process executes.
GPU
When the pmdanvidia
daemon does not run with root
privileges, the GPU percentage reflects the GPU memory
occupation percentage (memory of all GPUs is 100%).
When the pmdanvidia
daemon runs with root privileges,
the GPU percentage reflects the GPU busy percentage.
GPUBUSY
Busy percentage on all GPUs (one GPU is 100%).
When the pmdanvidia
daemon does not run with root
privileges, this value is not available.
GPUNUMS
Comma-separated list of GPUs used by the process during
the interval. When the comma-separated list exceeds the
width of the column, a hexadecimal value is shown.
LOCKSZ
The virtual amount of memory being locked (i.e. non-
swappable) by this process (or user).
MAJFLT
The number of page faults issued by this process that
have been solved by creating/loading the requested
memory page.
MEM
The occupation percentage of this process related to the
available capacity for this resource on system level.
MEMAVG
Average memory occupation during the interval on all
used GPUs.
MEMBUSY
Busy percentage of memory on all GPUs (one GPU is 100%),
i.e. the time needed for read and write accesses on
memory.
When the pmdanvidia
daemon does not run with root
privileges, this value is not available.
MEMNOW
Memory occupation at the moment of the sample on all
used GPUs.
MINFLT
The number of page faults issued by this process that
have been solved by reclaiming the requested memory page
from the free list of pages.
NET
The occupation percentage of this process related to the
total load that is produced by all processes (i.e.
consumed network bandwidth of all processes during the
last interval).
This information will only be shown when the pmdabcc(1)
module `netproc' has been installed.
NICE
The more or less static priority that can be given to a
process on a scale from -20 (high priority) to +19 (low
priority).
NPROCS
The number of active and terminated processes
accumulated for this user or program.
PID
Process-id.
POLI
The policies 'norm' (normal, which is SCHED_OTHER),
'btch' (batch) and 'idle' refer to timesharing
processes. The policies 'fifo' (SCHED_FIFO) and 'rr'
(round robin, which is SCHED_RR) refer to realtime
processes.
PPID
Parent process-id.
PRI
The process' priority ranges from 0 (highest priority)
to 139 (lowest priority). Priority 0 to 99 are used for
realtime processes (fixed priority independent of their
behavior) and priority 100 to 139 for timesharing
processes (variable priority depending on their recent
CPU consumption and the nice value).
PSIZE
The proportional memory size of this process (or user).
Every process shares resident memory with other
processes. E.g. when a particular program is started
several times, the code pages (text) are only loaded
once in memory and shared by all incarnations. Also the
code of shared libraries is shared by all processes
using that shared library, as well as shared memory and
memory-mapped files. For the PSIZE calculation of a
process, the resident memory of a process that is shared
with other processes is divided by the number of
sharers. This means, that every process is accounted
for a proportional part of that memory. Accumulating
the PSIZE values of all processes in the system gives a
reliable impression of the total resident memory
consumed by all processes.
Since gathering of all values that are needed to
calculate the PSIZE is a very time-consuming task, the
'R' key (or '-R' flag) should be active. Gathering
these values also requires superuser privileges
(otherwise '?K' is shown in the output).
RDDSK
When the kernel maintains standard io statistics (>=
2.6.20):
The read data transfer issued physically on disk (so
reading from the disk cache is not accounted for).
Unfortunately, the kernel aggregates the data tranfer of
a process to the data transfer of its parent process
when terminating, so you might see transfers for
(parent) processes like cron, bash or init, that are not
really issued by them.
RDELAY
Runqueue delay, i.e. time spent waiting on a runqueue.
RGID
The real group-id under which the process executes.
RGROW
The amount of resident memory that the process has grown
during the last interval. A resident growth can be
caused by touching memory pages which were not
physically created/loaded before (load-on-demand). Note
that a resident growth can also be negative e.g. when
part of the process is paged out due to lack of memory
or when the process frees dynamically allocated memory.
For a process which started during the last interval,
the resident growth reflects the total resident size of
the process at that moment.
RSIZE
The total resident memory usage consumed by this process
(or user). Notice that the RSIZE of a process includes
all resident memory used by that process, even if
certain memory parts are shared with other processes
(see also the explanation of PSIZE).
RTPR
Realtime priority according the POSIX standard. Value
can be 0 for a timesharing process (policy 'norm',
'btch' or 'idle') or ranges from 1 (lowest) till 99
(highest) for a realtime process (policy 'rr' or
'fifo').
RUID
The real user-id under which the process executes.
S
The current state of the (main) thread: `R' for running
(currently processing or in the runqueue), `S' for
sleeping interruptible (wait for an event to occur), `D'
for sleeping non-interruptible, `Z' for zombie (waiting
to be synchronized with its parent process), `T' for
stopped (suspended or traced), `W' for swapping, and `E'
(exit) for processes which have finished during the last
interval.
SGID
The saved group-id of the process.
ST
The status of a process.
The first position indicates if the process has been
started during the last interval (the value N means 'new
process').
The second position indicates if the process has been
finished during the last interval.
The value E means 'exit' on the process' own initiative;
the exit code is displayed in the column `EXC'.
The value S means that the process has been terminated
unvoluntarily by a signal; the signal number is
displayed in the in the column `EXC'.
The value C means that the process has been terminated
unvoluntarily by a signal, producing a core dump in its
current directory; the signal number is displayed in the
column `EXC'.
STDATE
The start date of the process.
STTIME
The start time of the process.
SUID
The saved user-id of the process.
SWAPSZ
The swap space consumed by this process (or user).
SYSCPU
CPU time consumption of this process in system mode
(kernel mode), usually due to system call handling.
TCPRASZ
The average size of a received TCP buffer in bytes.
This information will only be shown when the BCC PMDA is
active and the `netproc' module is enabled.
TCPRCV
The number of tcp_recvmsg()/tcp_cleanup_rbuf() calls
from this process. This information will only be shown
when the BCC PMDA is active and the `netproc' module is
enabled.
TCPSASZ
The average size of a TCP buffer requested to be
transmitted in bytes. This information will only be
shown when the BCC PMDA is active and the `netproc'
module is enabled.
TCPSND
The number of tcp_sendmsg() calls from this process.
This information will only be shown when the BCC PMDA is
active and the `netproc' module is enabled.
THR
Total number of threads within this process. All
related threads are contained in a thread group,
represented by pcp-atop
as one line or as a separate
line when the 'y' key (or -y flag) is active.
TID
Thread-id. All threads within a process run with the
same PID but with a different TID. This value is shown
for individual threads in multi-threaded processes (when
using the key 'y').
TRUN
Number of threads within this process that are in the
state 'running' (R).
TSLPI
Number of threads within this process that are in the
state 'interruptible sleeping' (S).
TSLPU
Number of threads within this process that are in the
state 'uninterruptible sleeping' (D).
UDPRASZ
The average size of a received UDP buffer in bytes.
This information will only be shown when the BCC PMDA is
active and the `netproc' module is enabled.
UDPRCV
The number of udp_recvmsg()/skb_consume_udp() calls from
this process. This information will only be shown when
the BCC PMDA is active and the `netproc' module is
enabled.
UDPSASZ
The average size of a UDP buffer requested to be
transmitted in bytes. This information will only be
shown when the BCC PMDA is active and the `netproc'
module is enabled.
UDPSND
The number of udp_sendmsg() calls from this process.
This information will only be shown when the BCC PMDA is
active and the `netproc' module is enabled.
USRCPU
CPU time consumption of this process in user mode, due
to processing the own program text.
VDATA
The virtual memory size of the private data used by this
process (including heap and shared library data).
VGROW
The amount of virtual memory that the process has grown
during the last interval. A virtual growth can be
caused by e.g. issueing a malloc() or attaching a shared
memory segment. Note that a virtual growth can also be
negative by e.g. issueing a free() or detaching a shared
memory segment. For a process which started during the
last interval, the virtual growth reflects the total
virtual size of the process at that moment.
VPID
Virtual process-id (within an OpenVZ container). If a
process has been started and finished during the last
interval, a `?' is shown because the virtual process-id
is not part of the standard process accounting record.
VSIZE
The total virtual memory usage consumed by this process
(or user).
VSLIBS
The virtual memory size of the (shared) text of all
shared libraries used by this process.
VSTACK
The virtual memory size of the (private) stack used by
this process
VSTEXT
The virtual memory size of the (shared) text of the
executable program.
WCHAN
Wait channel of thread in sleep state, i.e. the name of
the kernel function in which the thread has been put
asleep.
Since determining the name string of the kernel function
is a relatively time-consuming task, the 'W' key (or
'-W' flag) should be active.
WRDSK
When the kernel maintains standard io statistics (>=
2.6.20):
The write data transfer issued physically on disk (so
writing to the disk cache is not accounted for). This
counter is maintained for the application process that
writes its data to the cache (assuming that this data is
physically transferred to disk later on). Notice that
disk I/O needed for swapping is not taken into account.
Unfortunately, the kernel aggregates the data tranfer of
a process to the data transfer of its parent process
when terminating, so you might see transfers for
(parent) processes like cron, bash or init, that are not
really issued by them.
WCANCL
When the kernel maintains standard io statistics (>=
2.6.20):
The write data transfer previously accounted for this
process or another process that has been cancelled.
Suppose that a process writes new data to a file and
that data is removed again before the cache buffers have
been flushed to disk. Then the original process shows
the written data as WRDSK, while the process that
removes/truncates the file shows the unflushed removed
data as WCANCL.