trace Daemon
Purpose
Records selected system events.
Syntax
trace [ -a [ -g ] ] [ -f | -l ] [ -b | -B] [ -c] [ -C [ CPUList | all ]] [ -d ] [ -e string-cmd ] [ -h ] [ -j EventList ] [ -k EventgroupList ] [ -J EventgroupList ] [ -K EventgroupList ] [ -m Message ] [ -M ] [ -N ] [ -n ] [ -o Name ] [ -o- ] [ -p ] [ -r reglist ] [ -s ] [ -A ProcessIDList ] [ -t ThreadIDList ] [ -x program-specification | -X program-specification ] [ -I ] [ -P trace-propagation ][ -L Size ] [ -T Size ] [ -W ] [ -@ WparList ]
Description
The trace daemon configures a trace session and starts the collection of system events. The data collected by the trace function is recorded in the trace log. A report from the trace log can be generated with the trcrpt command.
When invoked with the -a, -x, or -X flags, the trace daemon is run asynchronously (for example, as a background task). Otherwise, it is run interactively and prompts you for subcommands.
To put the WPARconfigured ID (CID) in the trace hooks, use the -W flag.
To trace specific WPAR, use the -@ flag with a list of WPAR names that you want to trace.
You can use the System Management Interface Tool (SMIT) to run the trace daemon. To use SMIT, enter:
smit trace
The following are modes of trace data collection:
Item | Description |
---|---|
Alternate (the default) | All trace events are captured in the trace log file. |
Circular ( -l) | The trace events wrap within the in-memory buffers and are not captured in the trace log file until the trace data collection is stopped. |
Single ( -f) | The collection of trace events stops when the in-memory trace buffer fills up and the contents of the buffer are captured in the trace log file. |
Buffer Allocation | Trace buffers are allocated from either the kernel heap, or are put into separate segments. By default, buffers are allocated from the kernel heap unless the buffer size requested is too large for buffers to fit in the kernel heap, in which case they are allocated in separate segments. Allocating buffers from separate segments hinders trace performance somewhat. However, buffers in separate segments will not take up paging space, just pinned memory. The type of buffer allocation can be specified with the optional -b or -B flags. |
You can elect to trace only selected processes or threads.
You can also trace a single program. You can specify whether the trace
is to be propagated or extended to newly created processes or threads.
You can optionally include interrupt events in such traces. This is
only valid for trace channel 0
.
- Unless the trace is started before the process that is being traced, the process startup events are not captured. If the trace is started before the process that is being traced, some events from processes other than the process being traced will be captured as well.
- When trace uses memory from the kernel heap which is the case for the -B option (32-bit kernel only), this memory remains part of kernel memory until the next reboot of the system. Thus, care should be taken when using large buffers.
Flags
Item | Description |
---|---|
-@ WparList | Traces the workload partitions that you specify in the WparList parameter. Multiple WPAR names can either be separated by commas or enclosed in quotation marks and separated by spaces. To include the current Global system in the trace, specify Global. You can only specify the -@ flag in the Global system in a workload partition environment. |
-a | Runs the trace daemon asynchronously (i.e. as a background task). Once trace has been started this way, you can use the trcon, trcoff, and trcstop commands to respectively start tracing, stop tracing, or exit the trace session. These commands are implemented as links to trace. |
-A ProcessIDList | Traces only the processes and, optionally, their children specified with the
ProcessIDList. A process ID is a decimal number. Multiple process IDs can
either be separated by commas or enclosed in quotation marks and separated by spaces. The -A
flag is only valid for trace channel 0 ; the -A and -g flags are
incompatible. All threads existing for the specified processes when tracing is started are traced. By default, if after the trace starts, the processes being traced create additional threads or processes, these are not traced unless the -P flag is specified. |
-b | Allocate buffers from the kernel heap. If the requested buffer space can not be obtained from
the kernel heap, the command fails. Restriction: The -b flag is
only valid with the 32–bit kernel.
|
-B | Allocate buffers in separate segments. Restriction: The
-B flag is only valid with the 32–bit kernel.
|
-c | Saves the trace log file, adding .old to its name. |
-C [ CPUList | all ] | Traces using one set of buffers per processor in the CPUList. The processors can be
separated by commas, or enclosed in double quotation marks and separated by commas or blanks. To
trace all processors, specify all. Since this flag uses one set of buffers per processor, and
produces one file per processor, it can consume large amounts of memory and file space, and should
be used with care. The files produced are named trcfile, trcfile-0, trcfile-1,
etc., where 0, 1, etc. are the processor numbers. If -T or -L are
specified, the sizes apply to each set of buffers and each file. On a uniprocessor system, you may
specify -C all, but -C with a list of processor numbers is ignored. Attention: The -C flag can only be used by the root user.
|
-d | Disables the automatic start of trace data collection. Delays starting of trace data collection. Normally, the collection of trace data starts automatically when you issue the trace daemon. Use the trcon command to start the collection of trace data. |
-e string-cmd | Configures Component Trace by running ctctrl with string-cmd as an argument before the trace is started. In other words, it runs ctctrl string-cmd. Passing multiple -e options is allowed and is equivalent to successively running the ctctrl command with each string-cmd of arguments. This option can be used to configure the system trace mode (by setting the system trace mode to On, changing the level of trace, and so on) for some components just before starting to trace the system. |
-f | Runs trace in a single mode. Causes the collection of trace data to stop as soon as the in-memory buffer is filled up. The trace data is then written to the trace log. Use the trcon command to restart trace data collection and capture another full buffer of data. If you issue the trcoff subcommand before the buffer is full, trace data collection is stopped and the current contents of the buffer are written to the trace log. |
-g | Starts a trace session on a generic trace channel (channels 1 through 7). This flag works only when trace is run asynchronously (-a). The return code of the command is the channel number; the channel number must subsequently be used in the generic trace subroutine calls. To stop the generic trace session, use the command trcstop -<channel_number>. |
-h | Omits the header record from the trace log. Normally, the tracedaemon writes a header record with the date and time (from the date command) at the beginning of the trace log; the system name, version and release, the node identification, and the machine identification (from the uname -a command); and a user-defined message. At the beginning of the trace log, the information from the header record is included in the output of the trcrpt command. |
-I | Trace interrupt events. When specified with -A or -t, the -I flag includes interrupt events along with the events for the processes or threads specified. When -I is specified, but neither -A nor -t is specified, only interrupt level events are traced. The -I flag is only valid for trace channel 0; the -I and -g flags are incompatible. |
-j EventList | Specifies the user-defined events to collect trace data. The list items specified in the
EventList parameter can either be separated by commas or enclosed in quotation marks and
separated by commas or spaces. In AIX® 6.1
and earlier releases, specifying a two-digit hook ID in the form hh specifies hh00,
hh10,...,hhF0. Specifying a three-digit hook ID in the form hhh specifies hhh0.
Specifying a four-digit hook ID in the form hhhh specifies hhhh. If any of these events is missing, the information reported by the trcrpt command will be incomplete. Consequently, when using the -j flag, include all these events in the EventList. If starting the trace with SMIT, or the -J flag, these events are in the tidhk group. |
-J EventgroupList | Specifies the event groups to be included. The list items specified in the EventgroupList parameter can either be separated by commas or enclosed in quotation marks and separated by commas or spaces. The -J and -K flags work like -j and -k, except with event groups instead of individual hook IDs. You can specify each flag -j, -J, -k, and -K within the command. |
-k EventgroupList | Specifies the user-defined events to exclude trace data. The list items specified in the
EventgroupList parameter can either be separated by commas or enclosed in quotation marks and
separated by commas or spaces. In AIX 6.1
and earlier releases, specifying a two-digit hook ID in the form hh specifies hh00,
hh10,...,hhF0. Specifying a three-digit hook ID in the form hhh specifies hhh0.
Specifying a four-digit hook ID in the form hhhh specifies hhhh. Tip: The following events are used to determine the pid, the
cpuid, and the exec path name in the trcrpt report:
If any of these events is missing, the information reported by the trcrpt command will be incomplete. When using the -k flag, do not include these events in the EventgroupList parameter. If starting the trace with SMIT, or the -J flag, these events are in the tidhk group. |
-K EventgroupList | Specifies the event groups to be excluded. The list items specified in the EventgroupList parameter can either be separated by commas or enclosed in quotation marks and separated by commas or spaces. The -J and -K flags work like -j and -k, except with event groups instead of individual hook IDs. You can specify each flag -j, -J, -k, and -K within the command. |
-l | Runs trace in a circular mode. The trace daemon writes the trace data to the trace log when the collection of trace data is stopped. Only the last buffer of trace data is captured. When you stop trace data collection using the trcoff command, restart it using the trconcommand. |
-L Size | Overrides the default trace log file size of 1 MB with the value stated. Specifying a file
size of zero sets the trace log file size to the default size.
Note: In the circular and the alternate modes, the trace log file size must be at least twice the size of the trace buffer. In the single mode, the trace log file must be at least the size of the buffer. See the -T flag for information on controlling the trace buffer size. |
-m Message | Specifies text to be included in the message field of the trace log header record. |
-M | Dumps the address map of running processes into the trace. The -M flag must be specified if the trace file is to be processed by the tprof command. |
-n | Adds information to the trace log header: lock information, hardware information, and, for each loader entry, the symbol name, address, and type. |
-N | Dump the address map of specified processes into the trace. The -N option is used in conjunction with -M option. |
-o Name | Overrides the /var/adm/ras/trcfile default trace log file and writes trace data to a user-defined file. |
-o - | Overrides the default trace log name and writes trace data to standard output. The -c flag is ignored when using this flag. An error is produced if -o - and -C are specified. |
-p | Includes the cpuid of the current processor with each hook. This flag is only valid for
64-bit kernel traces.
Note: The trcrpt command can report the cpuid whether or not this option is specified. |
-P propagation | The propagation is specified with the letters p for propagation across
process creation, t for propagation across thread creation, and n
for no propagation. Propagation across process creation implies propagation across thread creation.
For example, if -A is specified to trace a process, all threads for that process that exist
at the time the trace was started are traced. The -Pt flags causes all threads subsequently
created by that process to be traced as well. If -Pp is specified, all processes and threads
subsequently created by that process are traced. If -t all was specified to trace all
threads, -P is ignored. The -P flag is only valid for trace channel
0 ; the -P and -g flags are incompatible. |
-r reglist | Optional, and only valid for a trace run on a 64-bit
kernel. reglist items are separated by commas, or enclosed in quotation
marks and separated by blanks. Up to 8 registers may be specified. Valid
reglist values are:
Restriction: Not all registers are valid for all
processors.
|
-s | Stops tracing when the trace log fills. The trace daemon normally wraps the trace log when it fills up and continues to collect trace data. During asynchronous operation, this flag causes the trace daemon to stop trace data collection. (During interactive operation, the quit subcommand must be used to stop trace.) |
-t ThreadIDList | Traces only the threads specified with the ThreadIDList parameter. A thread ID is a
decimal number. Multiple thread IDs can either be separated by commas or enclosed in quotation marks
and separated by spaces. Also, the thread list can be The -t flag is only valid for trace channel |
-T Size |
Overrides the default trace buffer size of 128 KB with the value stated. You must be root to request more than 1 MB of buffer space. The maximum possible size is 268435184 bytes, unless the -f flag is used, in which case it is 536870368 bytes. The smallest possible size is 8192 bytes, unless the -f flag is used, in which case it is 16392 bytes. Sizes between 8192 and 16392 will be accepted when using the -f flag; however, the actual size used will be 16392 bytes. Note: In the circular and the alternate modes, the trace buffer size must be one-half or less the size of the trace log file. In the single mode, the trace log file must be at least the size of the buffer. See the -L flag for information on controlling the trace log file size. Also note that trace buffers use pinned memory, which means they are not pageable. Therefore, the larger the trace buffers, the less physical memory is available to applications. The -f flag actually uses two buffers, which behave as a single buffer (except that a buffer wraparound trace hook will be recorded when the first buffer is filled). |
-W | Use the -W flag to include the workload partitionconfigured ID (CID) for the current
process with each hook. This flag is only valid in the Global system in a workload partition environment. Tip: The trcrpt command can report the workload partitionCID whether or not this option is
specified.
|
-x program-specification | Traces the specified program. The program-specification specifies a program and
parameters as they would be when running the program from the shell, except that the program
specification must be in quotes if more than just the program's name is given. The trace is stopped
automatically when the program exits, and returns the program's return code. By default, any
processes and threads created by the program are also traced; as if -Pp was specified. To
change this behavior, use -Pn to specify no trace propagation, or -Pt to propagate
trace only to threads created by the program's original process. Tip: The -x flag implies asynchronous tracing, as if the -a flag had also been
specified.
|
-X program-specification | The -X flag works like the -x flag, except that the trace is not automatically stopped when the program exits. This is useful for tracing programs which fork processes, and then terminate, and you want these new processes traced as well. |
Subcommands
When run interactively, trace recognizes the following subcommands:
Item | Description |
---|---|
trcon | Starts the collection of trace data. |
trcoff | Stops the collection of trace data. |
q or quit [-serial | -dd ] | Stops the collection of trace data and exits trace. If the -s option is specified then this serializes any pending I/O operations. If the -d option is specified, any pending I/O operation is discarded. |
! Command | Runs the shell command specified by the Command parameter. |
? | Displays the summary of trace subcommands. |
Signals
The INTERRUPT signal acts as a toggle to start and stop the collection of trace data. Interruptions are set to SIG_IGN for the traced process.
Security
Item | Description |
---|---|
aix.ras.trace.tracech0 | Required to run the trace command on channel 0. |
Item | Description |
---|---|
aix.ras.trace.trace | Required to perform all trace operations. |
Examples
- To use trace interactively, enter trace, (the > prompt
is displayed), then specify the subcommands you
want. For example, to trace system events during the run of the anycmd command,
enter:
trace > !anycmd > q
- To avoid delays when the command finishes, start trace asynchronously
( -a), using only
one command line, enter:
trace -a; anycmd; trcstop
- To trace the system itself for a period of 10 seconds, enter:
trace -a; sleep 10; trcstop
- To output trace data to a specific trace log file (instead of
the /var/adm/ras/trcfile default trace log file), :
trace -a -o /tmp/my_trace_log; anycmd; trcstop
- To capture the execution of a cp command, excluding specific
events from the collection process:
trace -a -k "20e,20f" -x "cp /bin/track /tmp/junk"
In the example above, the -k option suppresses the collection of events from the lockl and unlockl functions (20e and 20f events).
Also notice that the -x flag was used, so only hooks associated with the cp command process will be traced, and no interrupt activity will be traced.
- To trace hook 234 and the hooks that will allow you to see the
process names, use:
trace -a -j 234 -J tidhk
- To have trace use one set of buffers per processor, specify:
trace -aC all
The files produced are /var/adm/ras/trcfile, /var/adm/ras/trcfile-0, /var/adm/ras/trcfile-1, etc. up to /var/adm/ras/trcfile-(n-1), where n is the number of procssors in the system.Tip: trace -aCall -o mylog produces the files mylog, mylog-0, mylog-1, ... - To trace a program that starts a daemon process, and to continue
tracing the daemon after the original program has finished, use
The trace must be stopped with trcstop.trace -X "mydaemon"
- To trace mydaemon, which is currently running, use:
Where mydaemon-process-id is the process for mydaemon as returned by the ps command. The -Pp flag tells trace to also trace any processes and threads created by mydaemon while the trace is running.trace -A mydaemon-process-id -Pp
- To capture the PURR, and PMC1 and PMC2, type:
trace -ar "PURR PMC1 PMC2"
- To trace hooks 1A00,1A10,...,1AF0, DCA0 and 1AB1, enter:
trace -aj 1A,DCA,1AB1
Files
Item | Description |
---|---|
/usr/include/sys/trcmacros.h | Defines trchook and utrchook macros. |
/var/adm/ras/trcfile | Contains the default trace log file. |