ProbeVue: Extended Users Guide Specification (Draft)

Contents

1.   Introduction
1.1   Terminology
1.2   Vue Programming Language
1.3   Probevue command

2.   Vue: An overview
2.1   Structure of a Vue script
2.2   Probe Point Specification
2.3   Action Block
2.4   Predicates

3.   Variables
3.1   Data Types in Vue
3.2   Variable Classes
3.3   Value and Type Assignment
3.3.1   External Variables
3.3.2   Script Variables

4.   Elements of C
4.1   Differences from C language

5.   Other Vue Features
5.1   Vue Functions
5.2   Predicates
5.3   Symbolic Constants
5.4   Header Files
5.5   Supported Shell Elements
5.6   Trace capture facilities
5.6.1   Tentative Tracing

6.   Vue Functions
6.1   printf
6.2   trace
6.3   stktrace
6.4   get_probe
6.5   get_function
6.6   get_location_point
6.7   list
6.8   append
6.9   sum
6.10   avg
6.11   min
6.12   max
6.13   count
6.14   start_tentative, end_tentative
6.15   commit_tentative, discard_tentative
6.16   get_userstring
6.17   timestamp
6.18   diff_time
6.19   exit
6.20   atoi
6.21   strstr

7.   Probe Managers
7.1   System Call Probe Manager
7.2   UFT Probe Manager
7.3   Interval Probe Manager

8.   Authorizations and Privileges
8.1 RBAC-enabled mode  
8.1.1   ProbeVue Privileges
8.2 Legacy or RBAC-disabled mode

9.   ProbeVue parameters

10.   Miscellaneous
10.1   Running in a WPAR

11.   Sample Programs

Appendix A.  System Calls supported by the syscall probe manager

Appendix B.  Shell helper programs for ProbeVue

Scripts Index

1. Introduction

ProbeVue is a dynamic tracing facility. It has the following features:

• Trace hooks do not have to be pre-compiled. ProbeVue works on unmodified kernel or user applications.
• The trace points or probes have no effect (do not exist) until they are dynamically enabled.
• Actions (specified by the instrumentation code) to be executed at a probe point or the probe actions are provided dynamically at the time the probe is enabled.
• Trace data captured as part of the probe actions are available for viewing immediately and can be displayed as terminal output or saved to a file for later viewing.

ProbeVue can be used for performance analysis as well as for debugging problems. It is designed to be safe to run on production systems and provides protection against errors in the instrumentation code.

The next sub-section of this chapter defines some of the terminology used in this document. The subsequent sub-sections of this chapter introduce Vue, the programming language used by ProbeVue and the probevue command that is used to start a tracing session.

1.1 Terminology

1.2 Vue Programming Language

The Vue programming language is used to provide your tracing specifications to ProbeVue. In this document, The Vue programming language is often abbreviated to the "Vue language" or just to "Vue". A Vue script or Vue program is a program written in Vue. You can use a Vue script to:

1. Identify the probe points where a probe is to be dynamically enabled.
2. Identify the conditions, if any, which must be satisfied for the actions to be executed when a probe fires.
3. Identify the actions to be executed including what trace data to capture.
4. Associate the same set of actions for multiple probe points.

In short, a Vue script tells ProbeVue where to trace, when to trace and what to trace.

It is recommended that Vue scripts have a file suffix of ".e" to distinguish them from other file types, although this is not a requirement.

1.3 Probevue command

The probevue command is used to start a dynamic tracing session or a probevue session. The probevue command takes a Vue script as input reading from a file or from the command-line and activates a probevue session. Any trace data that is captured by the probevue session can be printed to the terminal or saved to a user-specified file as per options passed in the command line.

The probevue session stays active until a Ctrl-C in typed on the terminal or an exit action is executed from within the Vue script.

Each invocation of the probevue command activates a separate dynamic tracing session. Multiple tracing sessions may be active at one time, but each session presents only the trace data that is captured in that session.

Running the probevue command is considered a privileged operation and privileges are required for non-root users who wish to initiate a dynamic tracing session.

2. Vue: An overview

Vue is both a programming and a script language. It is NOT simply an extension of C or a simple mix of C and "awk". It has been specifically designed as a dedicated dynamic tracing language. Vue supports a subset of C and scripting syntax that is most beneficial for dynamic tracing purposes.

The following list provides a summary of Vue features:

• Subset of C (almost all C operators supported)
• C-89 data types and Vue-specific data types (String, List data types, etc)
• Data with different scopes: Thread-local data, global data, OS kernel variables etc.
• Read access to parameters to functions and return value.
• Special built-in variables that return useful information like __pid that returns traced process's process ID. __tid that returns its thread ID, __pname that returns its executable name, etc.
• Auxiliary functions like stktrace() that prints a stack trace, timestamp() that records a timestamp, get_userstring() that reads user-space data, etc.
• Conditional execution of enabled probe points using the “"when <predicate>"” syntax.
• Shell syntax: Can access shell environment variables and positional parameters.

Each of these features is described in further detail in subsequent chapters. The rest of this chapter describes the basic layout and elements of a Vue script.

2.1 Structure of a Vue script

A Vue script consists of one or more clauses. The clauses in a Vue script can be specified in any order. The following is a typical layout of a Vue script.

@@BEGIN {        <statement 1> ;      ...     <statement n> ;  }

<-------   optional BEGIN clause




<probe point specification>,....,<probe point specification>   when (<predicate>)   <---- optional predicate {        <statement 1> ;      ...     <statement n> ;    <-----  action block }

<-------   one or more probe clauses




@@END {        <statement 1> ;      ...     <statement n> ;  }

<-------   optional END clause

Each clause of a Vue script consists of the following three elements:

1. Probe point specification
   The probe point specification identifies the probe points to be dynamically enabled.
   
   
2. Action Block
   The action block is used to identify the set of probe actions to be performed when the probe fires.
   
   
3. An optional predicate
   The predicate, if present, identifies a condition that is to be checked at the time the probe is triggered. The predicate must evaluate to TRUE for the probe actions of the clause to be executed.

In the following example clause, the superscripts identify the elements of a Vue clause.

	/* 
	 * File: count.e
	 *
	 * Count number of times the read or write system call is entered
	 * by process with Id 400 
	 */

	@@syscall:*:read:entry, @@syscall:*:write:entry [1]
		when (__pid == 400)[3]

	{
		int count;
		count++;[2]

	}

The elements of a Vue clause are covered in more detail following sub-sections.

2.2 Probe Point Specification

	@@<probetype>:<one or more probetype-specific fields separated by colons>:<location>

	- @@uft:34568:*:foo:entry
	  Probe at entry into any function called foo() (in the main executable
	  or any of the loaded modules including libraries) in process
	  with ID = 34568

	- @@syscall:*:read:exit
	  Probe at exit of read system call 

	- @@interval:*:clock:500
	  Probe to fire every 500 milliseconds (wall clock time).

2.3 Action Block

2.4 Predicates

Predicates should be used when execution of clauses at probe points must be performed conditionally. The predicate section is identified by the presence of the "when" keyword immediately after the probe specification section. The predicate itself consists of regular C-style conditional expressions with the enclosing parentheses.

A predicate has the following format:

	when ( <condition> )

For example, when ( __pid == 1678 ).

Section 5.2 has more details about predicates.

3. Variables

The Vue language supports all the traditional C data types, namely those recognized by the C-89 specifications. In addition, Vue includes some extensions to allow simple, but powerful dynamic tracing programs to be written easily.

Vue supports variables with three different scope rules, variables that are local to one action block only, variables that have global scope and variables that have thread-local scope. In addition, Vue allows access to variables with external scope like global variables in the kernel or user data in application being probed.

The following subsection describes the special non-C variable types supported by Vue. The next two subsections describe the variable classes of Vue - each variable class has distinct rules on scope initialization and modification.

In general, variables need to be declared before their first use in the script, although Vue also supports a very limited form of implicit type recognition. Variable declaration statements inside an action block must appear before any of the executable statements. In some cases, Vue allows you to declare variables outside any of the action blocks, but in this case, all such declarations must appear before the first action block.

3.1 Data types in Vue

3.2 Variable Classes

1. Automatic Class Variables

   An automatic is a clause-specific variable and is similar to an automatic or stack variable in C. It has scope only within the action block portion of the clause where it is defined or used and is recreated for each invocation of the action block. Automatic variables are always undefined at the start of an action block and must be initialized before they can be used in an expression or statement.

   An automatic variable is identified by using the "auto:" prefix, for example auto:lticks indicates an automatic variable. You can also declare automatic variables using the __auto declaration statement in which case the auto: prefix can be omitted.

   Automatic class variables may not be used in the predicate section of the e-clause.

   	__auto int i;      /* Explicit declaration */
   	auto:j = 0;        /* Implicit declaration */
   

2. Thread-local Class Variables

   A thread-local variable is instantiated per traced thread the first time it executes an action block that uses the variable. Once created the thread-local variable exists as long as the Vue script is active and the traced thread does not exit. The value of the thread-local variable is thread-specific and retained across executions of any of the clauses of the same program. In other words, variables of this class are visible everywhere within the Vue script. However, each thread that executes the Vue script obtains its own copy of these variables and the variables in each such copy are accessible and modifiable only by the thread that instantiated them.

   A thread-local variable is distinguished by using the thread: prefix. For example, thread:count indicates a thread-local variable. You can also declare thread-local variables using the __thread declaration statement in which case the thread: prefix can be omitted with one exception, which is indicated below.

   A thread-local variable may be used in the predicate section of a Vue clause even before it is instantiated. Predicates with un-instantiated thread-local variables are always evaluated to a value of FALSE. When used in the predicate section, the thread: prefix must currently always be included to identify it as a thread-local variable.

   	__thread int i;      /* Explicit declaration */
   	thread:j = 0;        /* Implicit declaration */
   

   Note:

   Although thread-locals may be declared inside the @@BEGIN and @@END probes, any other references to them in these special probes produce undefined behavior.

3. Global Class Variables

   Variables of global class have global scope and are visible everywhere within a Vue script. A global variable can be used in one or more clauses of a Vue script. They can also be declared at the beginning textually before the first clause for clarity. Global variables are initialized to zero or NULL as appropriate.

   All variables in a Vue script are by default assigned global class, unless an explicit non-global class specifier is prefixed to the declaration. Global variables may also be explicitly declared by using the __global class specifier when declaring a variable. List variables are, by definition, always created as variables of global class.

   Reads and updates of global variables are not serialized unless they are of a list type. There are no guarantees on data races when probes are executed simultaneously. Global variables, which are not of type list, are useful for collecting profiling and other statistics, but may not be a good choice when accuracy is important.

   Global variables may be used in the predicate section of a Vue clause .

   The following are examples for initializing and using global variables:

   Figure 2

   int wcount; /* Global variable declared before first clause */ @@BEGIN { int f_count; /* Global variable declared inside @@BEGIN */ __global int z_count; /* Global variable declared with __global prefix */ f_count = 12; } @@syscall:*:read:entry when (z_count == 0) { int m_count; /* Global variable declared inside a probe */ m_count += f_count; printf("m_count = %d\n", m_count); if (wcount == 1) exit(); } @@syscall:*:write:entry { m_count++; } @@syscall:*:write:exit { wcount = 1; }

   
   
   

4. Kernel Global Class Variables

   ProbeVue allows you to access kernel global variables inside the action block of any Vue clause, even for probe points in user space like the uft probe points. Before using or referring to the kernel variable in the Vue script, it should be explicitly declared using the __kernel declaration statement. Only variables exported by the kernel, that is, only those are present in the export list of /unix are accessible.

   You can access integral type kernel variables and kernel variables that are structures or unions (and even pointers, although AIX does not export any useful pointer types). Further, you can also refer to the member names of kernel structures and unions or unions in a Vue script. It is preferable to access pinned kernel variables. If the page containing the kernel variable is not in memory (has been paged out), ProbeVue will return a value of zero for that variable.

   There is an example of how kernel variables may be declared and used in a Vue script in the sample programs section (number 5).

   Kernel variables cannot appear in the predicate section of a clause. Kernel variables are always treated as read-only variables in a trace script. Any attempt to write to a kernel variable will either cause a syntax error or fail later with a script abort message.

5. Entry Class Variables

   Clauses associated with a probe point that is at the entry location point of a kernel or user routine can access the arguments passed to the routine being probed.

   Probes at entry location points are supported by the system call and the user function tracing probe managers. For example, the read system call takes three arguments: a file descriptor ID, a pointer to a user buffer and a value for the size of the data to be read. The values of these three arguments can be accessed if the probe specification is @@syscall:*:read:entry, which specifies a probe at the read system call entry point.

   Parameters to routines are specified indicated using the special built-in entry class variable names __arg1, __arg2, __arg3, ... up to the number of arguments passed to the function. For instance, in the clause associated with the read system call entry point as described above, __arg1 would refer to the value of file descriptor Id parameter, __arg2 would refer to the value of the buffer pointer parameter and __arg3 to the size of the data to be read.

   Use of entry class variables in a Vue clause is legal only if the probe specification associated with the clause identifies a unique probe point. Thus, entry class variables cannot be used in a Vue clause that has multiple probe points specified in the probe specification. Furthermore, the C-style declaration of the function being probed, specifically the data type of the parameters being passed to the function, must be provided in the Vue script.

   Exit class variables can be used in the predicate section of a clause.

   	int read(int fd, char *buf, unsigned long size);
   
   	@@syscall:*:read:entry 
   	{
   		printf("Number of bytes to read = %d\n", __arg3);
   	}
   

   Note that in the above, the definition of the read system call function specified in the script does not exactly match what is given in /usr/include/unistd.h, but it should work just as well.

   The following is an illegal script and will cause the ProbeVue compiler to fail with a syntax error since the probe specification includes two probe points:

   	int read(int fd, char *buf, unsigned long size);
   	int write(int fd, char *buf, unsigned long size);
   
   	@@syscall:*:read:entry, @@syscall:*:write:entry 
   	{
   		/* Cannot use __arg3 in here, as this clause has multiple probe 
   		 * points associated with it. This script will fail with a
   		 * syntax error in the compilation phase of the probevue command.
   		 */
   		printf("Number of bytes to read/write = %d\n", __arg3);
   	}
   

   The script can be modified as follows for it to work:

   	int read(int fd, char *buf, unsigned long size);
   	int write(int fd, char *buf, unsigned long size);
   
   	@@syscall:*:read:entry
   	{
   		printf("Number of bytes to read = %d\n", __arg3);
   	}
   	@@syscall:*:write:entry 
   	{
   		printf("Number of bytes to write = %d\n", __arg3);
   	}
   

6. Exit Class Variables

   Clauses associated with a probe point that is at the exit location points of a routine can access the return state of the routine being probed.

   There is only exit class variable that is defined by the Vue language. This is the return value from a routine, which can be accessed by using the special built-in variable name __rv.

   Probes at exit location points are currently supported by the system call probe manager. For instance, the read system call returns the actual number of bytes read or an error return code of -1. This returned value can be accessed at the @@syscall:*:read:exit probe point, which identifies all exit points from the read system call.

   Similar to entry class variables, the use of exit class variables in a Vue clause is legal only if the probe specification associated with the clause identifies a unique probe point. Thus, __rv cannot be used in a Vue clause that has multiple probe points specified in the probe specification. Furthermore, the C-style declaration of the function being probed, specifically the data type of the return value, must be explicitly provided in the Vue script. In fact, it is an error to specify a function declaration without providing its return type.

   Exit class variables can be used in the predicate section of a clause.

   	/* Bad example. Will fail with syntax error since the
   	 * return type of the read() function is not specified
   	 */
   	read(int fd, char *buf, unsigned long size);
   
   	@@syscall:*:read:entry
   		when (__rv < 0)
   	{
   		/* Entered on read success: return value = # of bytes read */
   		printf("Number of bytes read = %d\n", __rv);
   	}
   

   The script can be modified as follows for it to work:

   	/* Good example.  */
   
   	int read(int fd, char *buf, unsigned long size);
   
   	@@syscall:*:read:entry
   		when (__rv < 0)
   	{
   		/* Entered on read success: return value = # of bytes read */
   		printf("Number of bytes read = %d\n", __rv);
   	}
   

7. Built-in Class Variables

   In addition to the special built-in variables, __argv1 through __argv32 and __rv, Vue also defines a set of general-purpose built-in variables. Built-in class variables are essentially functions, but are treated as variables by ProbeVue. Because of this, built-in variables can be used in the predicate section of a Vue clause.

   The following built-in variables are supported in Vue. The table shows the largest value that can be returned by the built-in variable, which is not necessarily the same as its defined type. Section 3.3.1 discusses the types of built-in variables.

   __tid	Thread ID of traced thread.	32-bit integer
   __pid	Process ID of traced thread.	32-bit integer
   __ppid	Parent process ID of traced thread.	32-bit integer
   __pgid	Process group ID of traced thread.	32-bit integer
   __pname	Process name of traced thread.	String
   __uid, __euid	Real and effective user ID of traced thread.	32-bit integers
   __trcid	Process ID of tracing traced (probevue command)	32-bit integer
   __errno	Current errno value for the traced thread.	32-bit integer
   __kernelmode	Current executable mode: Is either 1 (in kernel mode) or 0 (in user mode).	32-bit integer
   __r3, ..., __r10	General Purpose Register values (for function parameters or return values).	64-bit integers

   The following is an example script that uses built-in variables:

   	@@syscall:*:read:entry
   	{
   		printf("Thread ID:%d, Process ID:%d, Parent Process ID:%d\n",
   				__tid, __pid, __ppid);
   		printf("Process Group ID: %d\n", __pgid);
   		printf("Process name = %s\n", __pname);
   	
   		printf("Real UID=%d, Effective UID=%d\n", __uid, __euid);
   	
   		printf("probevue command process ID = %d\n", __trcid);
   	
   		printf("Errno = %d\n", __errno);
   		printf("Mode = %s\n", __kernelmode == 1 ? "kernel" : "user");
   	
   		printf("Current values of GPRs: r3=0x%016llx, r4=0x%016llx, r5=0x%016llx\n",
   				__r3, __r4, __r5);
   		printf("                        r6=0x%016llx, r7=0x%016llx, r8=0x%016llx\n",
   				__r6, __r7, __r8);
   		printf("                        r9=0x%016llx, r10=0x%016llx\n",
   				__r9, __r10);
   	}
   

3.3 Value and Type Assignment

3.3.1 External Variables

3.3.2 Script Variables

• Implicit type determination for integer types

  To be assigned an integer type, the right-hand side of the assignment must be one of:

  1. A constant number.
  2. Another variable of type integer including built-in variables. Assigning to a variable whose type is not known is an error.
  3. A Vue function that returns an integer like diff_time().
  4. Casting the expression on the right-hand side to an integral type, although this might give a warning in some cases.
  5. An expression involving any of the above.

  The variable will take its type in addition to the value based on the expression on the right-hand side. In addition, the class of the variable may be assigned to the variable by prefixing it to the variable. The following script demonstrates some examples of the above:

  Figure 3

  /* * File: implicit2.e * Usage: Demonstrates implicit assignment for integer types */ int read(int fd, char *p, int size); @@BEGIN { count = 404; /* count: int of global class */ zcount = 2 * (count - 4); /* zcount: int of global class */ llcount = 33459182089021LL; /* lcount: long long of global class */ lxcount = 0xF00000000245B20LL; /* xcount: long long of global class */ } @@syscall:$1:read:entry { __auto unsigned long long ts1, ts2; int gsize; ts1 = timestamp(); auto:dcount = llcount - lxcount; /* dcount: long long of auto class */ auto:mypid = __pid; /* mypid: pid_t (64-bit integer) of automatic class */ fd = __arg1; /* fd: int of global class */ /* The following cast will likely cause a compiler warning * but can be ignored here */ global:bufaddr = (long)__arg2; /* bufaddr: long of global class */ gsize = __arg3; thread:size = gsize + 400; /* size: int of thread-local class */ printf("count = %d, zcount = %lld\n", count, zcount); printf("llcount = %lld, lxcount = 0x%016llx, diff = %lld\n", llcount, lxcount, dcount); printf("mypid = %ld, fd = %d, size = %d\n", mypid, fd, size); printf("bufaddr = 0x%08x\n", bufaddr); ts2 = timestamp(); auto:diff = diff_time(ts1, ts2, MICROSECONDS); /* diff: int of automatic class */ printf("Time to execute = %d microseconds\n", diff); exit(); }

  Assuming that you are probing a process that has process ID 250000, the following shows an example of running this script (whose name is implicit2.e):

  # probevue implicit2.e 250000
  WRN-100: Line:29 Column:26 Incompatible cast
  count = 404, zcount = 800
  llcount = 33459182089021, lxcount = 0x0f00000000245b20, diff = -1080830451389212643
  mypid = 250000, fd = 10, size = 4496
  bufaddr = 0x20033c00
  Time to execute = 11 microseconds
  

• Implicit type determination for String type

  To be assigned a String type, the right-hand side of the assignment must be one of:

  1. A string literal, that is, a sequence of characters within double quotes.
  2. Another variable of type integer including built-in variables.
  3. A Vue function that returns a string like get_userstring().
  4. An expression involving any of the above.

  The following example demonstrates implicit String type assignment.

  Figure 4

  /* * File: implicit3.e * Usage: Demonstrates implicit assignment for string types */ int write(int fd, char *p, int size); @@BEGIN { s1 = "Write system call:\n"; } @@syscall:$1:write:entry { String s2[40]; wbuf = get_userstring(__arg2, __arg3); s2 = s1; zbuf = s2; pstring = zbuf + wbuf; printf("%s\n", pstring); } @@syscall:$1:write:exit { ename = __pname; printf("Exec name = %s\n", ename); exit(); }

• Implicit type determination for List type variables

  To be assigned a String type, the right-hand side of the assignment must be the list() function. Further, currently, the list() function is only supported when invoked from the @@BEGIN probe.

  There is no way to explicitly declare a list variable.

4. Elements of C

Vue supports a subset of C. Not all C features are supported and even supported features may work differently. some differences even for features that are supported. The table below describes how C-keywords are handled by the probevue compiler. All C-keywords remain restricted in Vue. Use of any of them as variable names or other symbols is not flagged as a syntax error. However, you are strongly discouraged from doing this, as the behavior of such usage is undefined.

C keywords in Vue

Supported	Allowed in header files or outside clauses only	Unsupported
char	auto	break
double	const	case
else	extern	continue
enum	register	default
float	static	do
if	typedef	for
int	volatile	goto
long		switch
return		while
short
signed
sizeof
struct
union
unsigned
void

The following list describes the set of C-language features that are supported by Vue.

• Statements: All C-statements are supported except for loop and some control flow statements.
• Operators: All C unary, binary and ternary operators will be supported except for the comma operator. Operator precedence and associativity will follow C language rules.
• Data Types: All legal C-89 defined variable types are supported including all statements and keywords (struct, union, enum, typedef, etc ) for declaring types subject to the restrictions above. This includes the types for kernel or application variables and parameters. Note that Vue has its own rules on scope and storage-classes as described earlier.
• Type Conversions: Implicit type conversions as well explicit conversions with type casts are supported.
• Subroutine: The syntax for calling subroutines and passing parameters to functions are supported. There are, however, some restrictions on what functions can be called as described in the differences section below.
• Variable Names: The naming conventions for variables follow C identifier rules. The complete variable specification can include colons if a variable class name is prefixed to the variable name.
• Header files: Header files may be included for explicitly declaring the types of kernel global variables or the prototypes of functions in applications and the kernel. Currently, there are some restrictions on how the header files are included. See the differences section below.
• Punctuators: All C punctuators are supported and their rules are enforced. Thus, statements must be separated by the ';' symbol. All C white space rules are followed.
• Literals: Representation of strings (using double quotes), octal and hexadecimal integers, and special characters like the \n and \t escape sequences are supported.
• Comments: C and C++ style comments are supported. Comments can appear within functions or outside.

4.1 Differences from C language

5. Other Vue Features

5.1 Vue Functions

Unlike programs written in "C" or in FORTRAN or in a native language, scripts written in Vue do not have access to the routines provided by the AIX system libraries or any user libraries. However, Vue supports its own special library of functions useful for dynamic tracing programs. Functions include:

1. Tracing-specific functions
   • get_function:  Returns the name of the function that encloses the current probe
   • timestamp:  Returns the current timestamp.
   • diff_time:  Finds the difference between two time stamps
2. Trace capture functions
   • printf:  Formats and prints values of variables and expressions
   • trace:  Prints data without formatting.
   • stktrace:  Prints and formats the stack trace
3. List functions
   • list:  Instantiate a list variable.
   • append:  Append a new item to list.
   • sum, max, min, avg, count:  Aggregation functions that can be applied on a list variable
4. C-library functions
   • atoi, strstr:  Standard string functions
5. Functions to support tentative tracing
   • start_tentative, end_tentative:  Indicators for start and end of tentative tracing.
   • commit_tentative, discard_tentative:  Commit or discard data in tentative buffer
6. Miscellaneous functions
   • exit:  Terminates the E-program.
   • get_userstring:  Read string from user memory

The Vue string functions can be applied only on variables of string type and not on a pointer variable. Standard string functions like strcpy(), strcat(), etc., are not necessary in Vue, because they are supported through the language syntax itself.

The probevue compiler will validates the data types of the parameters passed to Vue functions. However, it does not validate that the arguments passed to the printf() function match the conversion specifications indicated in the format string. Casting is allowed to circumvent type mismatches, but incompatible casts usually generate a warning.

Functions are not legal in the predicate section of a Vue clause unlike built-in variables.

Chapter 6 documents the complete list of supported functions.

5.2 Predicates

As mentioned earlier, predicates should be used when execution of clauses at probe points must be performed conditionally. The predicate section is identified by the presence of the "when" keyword immediately after the probe specification section. The predicate itself consists of regular C-style conditional expressions with the enclosing parentheses.

There are some restrictions on the expressions allowed inside the predicate section:

• Kernel class variables are not permitted in the predicate.
• Automatic class variables are not permitted in the predicate.
• Floating-point type variables are not permitted in the predicate.
• Vue functions are not allowed inside a predicate.
• Side-effects are not supported inside a predicate and so the "=" assignment operator and its derivatives like "+=", "|=", and so on are not allowed
• The ninth and higher parameters passed to a function (that the entry class variables __arg9, __arg10, etc) are not supported in the predicate.

Conditional execution of specific actions within a clause are possible by using the "if ... else" statement which works like the analogous statement in C. However, if the entire clause is to be executed conditionally, it is preferable to use predicates instead because ProbeVue is designed to optimize execution of predicates. Note that when a probe point can fire for more than one process, then using thread-local variables inside the predicate is an excellent way to reduce overall performance impact of enabling the probe. In future, the predicate section will be further optimized to execute even more efficiently, so putting your conditional checks inside a predicate is preferable to using the "if" statement.

The following script uses thread-local variables inside predicates to efficiently detect when a particular character string is written to a specific file. It also shows an example of using the "if" statement within the action block of a clause with a predicate. Both the file name and character string are passed as parameters to the script using shell positional parameters (see Section 5.5 for details on passing parameters to a Vue script).

/*
 * Filename : chkfilewrite.e
 *
 * Capture when someone writes a particular word to a specific file 
 * takes 2 arguments: filename and word
 *
 * assumes file name is < 128
 *
 * Usage: probevue chkfilewrite.e \"<filename>\" \"<string>\"
 */

int open(char *fname, int m, int p);
int write(int fd, char *s, int size);

@@syscall:*:open:entry
{
	__auto String fname[128];

	fname = get_userstring(__arg1, -1);

	if (fname == $1)
		thread:tracing = 1;
}

@@syscall:*:open:exit
	when (thread:tracing == 1)
{
	thread:fd = __rv;
	thread:tracing = 0;

}

@@syscall:*:write:entry
	when (thread:fd == __arg1)
{
	__auto String buf[128];

	if (__arg3 < 128)
		buf = get_userstring(__arg2, __arg3);
	else
		buf = get_userstring(__arg2, 128);

	if (strstr(buf, $2)) {
		printf("%d wrote word to file.\n", __pid);
		exit();
	}
}

To run this program to check when someone writes the string "Error" to the foo.log file, this would be executed as:

	probevue chkfilewrite.e \"foo.log\" \"Error\"

5.3 Symbolic Constants

Vue supports some pre-defined symbolic constants, which are commonly used while programming in AIX. These constants are treated as keywords in Vue. During compilation, the constants will be replaced by their definitions in AIX system header files.

• NULL

  To set pointer types to a NULL or zero value.

• Error numbers or "errno" names

  These are the standard error names like EPERM, EAGAIN, ESRCH, ENOENT, etc., specified by the POSIX and ANSI standards and defined in the /usr/include/sys/errno.h header file.

  The following script traces when the bind() system call fails with errno set to EADDRINUSE or address already in use.

  Figure 6

  /* * File: bind.e */ /* * Okay to use void for parameters since we are not planning to * access them in this script. */ int bind(void); @@syscall:*:bind:exit when (__rv == -1) { if (__errno == EADDRINUSE) printf("%d failed with EADDRINUSE for bind() call.\n", __pid); }

  
  
  
• Signal names

  These are the standard signal names like SIGSEGV, SIHGUP, SIGILL, SIGABRT, etc., specified by the ANSI standards and defined in the /usr/include/sys/signal.h header file.

  The following script shows how to debug "who" killed a particular process by sending it a specific signal.

  Figure 7

  /* * File: signal.e * * Who sent SIGKILL to my process ? */ /* Process IDs are < 2^32, so using an 'int' here instead of pid_t is * good enough */ int kill(int pid, int signo); @@syscall:*:kill:entry when (__arg1 == $1 && __arg2 == SIGKILL) { /* Trace sender of SIGKILL */ printf("Stack trace of %s: (PID = %d)\n", __pname, __pid); stktrace(PRINT_SYMBOLS|GET_USER_TRACE, -1); exit(); }

  
  
  
• FUNCTION_ENTRY

  Identifies if a probe point is a function entry point. Used with get_location_point().

• FUNCTION_EXIT

  Identifies if a probe point is a function exit point. Used with get_location_point().

5.4 Header files

You can include header files in a Vue script through the -I option of the probevue command. The header file must be written in the syntax of the C language. However, it should not contain any C executable statements. C preprocessor operators and directives are generally ignored, but operators like "#ifdef" and "#if" may cause undefined behavior. Thus, standard AIX header files from the /usr/include directories cannot be directly included. Instead, you should always run the C-preprocessor directly on the set of relevant header files and generate a post-processed header file, which can then be passed to the probevue command. Another option is to hand-code the header file to include the type definitions and function declarations that are used in your Vue script.

The following example shows a hand-coded header file with a typedef definition that is used in a Vue script:

	/* My header file : myheader.i */

	typedef int myid_t;

The following Vue script uses this typedef definition:

	/* Program name: myscript.e */
	@@BEGIN
	{
		myid_t id;
		id = 0;
	}

The probevue command should be run as follows:

	probevue -I myheader.i myscript.e

5.5 Supported Shell Elements

The Vue language syntax includes support for shell variables identified by the "$" prefix like exported shell variables and positional parameters (arguments to the script).

Vue shell variables can appear anywhere in the Vue script. They can be part of the probe specification, be used in predicates or within statements in action blocks. However, unlike in a shell script, they are not expanded if used within double quoted strings.

The arguments passed from the command line to the script are referenced within the script as $1, $2, $3, and so on. Consider the following Vue script:

	/* Program name: myscript.e */
	@@syscall:*:read:entry
		when (__pid == $1)

	{
		count++;
	}

In the following example, the process ID of the process running the myprog program replaces $1 in the preceding script:

	probevue myscript.e 'ps -ef | grep myprog | cut -d ' ' -f 6'

Environment variables exported from the shell can also be referenced in the script using the $ operator. Consider the following Vue script:

	/* Program name: myscript2.e */
	@@syscall:*:read:entry
		when (__pid == $PID)

	{
		count++;
	}

	/* program to be traced has a function called 'foo' */
	@@user:$PID:foo:entry
	{
		printf("Read system call was invoked %d times\n", count);
	}

In the following example, 3243 replaces $PID in the preceding script:

	PID=3423 probevue myscript2.e

If an environment variable need to be recognized as a string inside the ProbeVue script, the value of the environment variable must include the enclosing double quotes that identify it as a string. For example, the following script captures trace output when a specific file is opened in the system.

	/* Program name: stringshell.e */
	int open(char *path, int oflag);
	@@syscall:*:open:entry
	{
		String s[40]; 
		s = get_userstring(__arg1, -1);
		if (s == $FILE_NAME) {
			printf("pid %d (uid %d) opened %s\n",__pid,__uid, s);
			exit();
		}
	}

The script expects that $FILE_NAME is the name of an exported shell environment variable, which includes the double quotes in its value. For instance:

	export FILE_NAME=\"/etc/passwd\"
	probevue stringshell.e

If the value of an existing environment variable that does not have double quotes is required in a script, then a new environment variable will need to be constructed using double quotes around the existing environment variable. For example:

	export FILE_NAME=\"$HOME\"
	probevue stringshell.e

Vue supports two special environment variables which are useful when the process to be probed is started by the probevue command itself using the "-X" flag. The $__CPID environment variable indicates the process ID of the child process created by the probevue command, and the $__CTID environment variable indicates its thread ID. The -X flag is useful to probe short-lived processes especially for debugging purposes.

A Vue script can be executed directly (like a shell script) by setting the first line to

	#!/usr/bin/probevue

The probevue command can also read the Vue script from standard input like the shell does. This can be accomplished by omitting the script file from the command line. You might find this useful to test short scripts.

Vue does not support special shell parameters such as "$$" and "$@" which are created internally by the shell.

5.6 Trace Capture Facilities

5.6.1 Tentative Tracing

The following script is an example of using tentative tracing functions to capture trace data only when necessary.

/* 
 * File: tentative.e
 *
 *	Print details when write system call takes longer than a
 *	specified number of microseconds
 *
 *	Usage: probevue tentative.e 
 */
int write(int fd, char *buf, int size);

@@BEGIN
{
	unsigned long long ts1, ts2;
}

@@syscall:$1:write:entry 
{
	__auto String buf[256];

	if (__arg3 < 256)
		buf = get_userstring(buf, __arg3);
	else
		buf = get_userstring(buf, __arg3);

	start_tentative("write");

	/* print out all the data associated with the write */
	stktrace(PRINT_SYMBOLS|GET_USER_TRACE, -1);

	printf("fd = %d, size = %d\n", __arg1, __arg3);

	/* Prints 256 bytes of buf, even though size may be < 256 */
	trace(buf);

	end_tentative("write");

	/* Get timestamp for when we entered write: do this at the end of
	 * the probe to reduce probe effect
	 */
	ts1 = timestamp();
}
	
/* If we started probing in the middle of write, ts1 will be zero,
 * ignore that case with a predicate
 */
@@syscall:$1:write:exit
	when (ts1 != 0)
{
	int micros;

	/* Get timestamp for when we exited write: do this at the beginning of
	 * the probe to reduce probe effect
	 */
	ts2 = timestamp();

	micros = diff_time(ts1, ts2, MICROSECONDS);

	start_tentative("write");
	printf("Return value from write = %d\n", __rv);
	end_tentative("write");

	if (micros > $2) {
		/* Can mix normal trace with tentative also  */
		printf("Time to write = %d, limit =%d micro seconds\n",
			micros, $2);
		commit_tentative("write");
		exit();
	}
	else
		discard_tentative("write");
}

6. Vue Functions

The Vue language supports the following list of functions:

• printf - Formats and copies data to the trace buffer
• trace - Copies raw data to the trace buffer.
• stktrace - Generates and prints a runtime stack trace.
• get_probe - Returns the current probe point specification.
• get_function - Returns the name of the function that is being traced.
• get_location_point - Returns the current probe location point.
• list - Creates and returns a new empty list.
• append - Appends a value to a list.
• sum - Returns the sum of all the elements in a list.
• avg - Returns the average of all the elements in a list.
• min - Returns the minimum of all the elements in a list.
• max - Returns the maximum of all the elements in a list.
• count - Returns the count of the number of elements in a list.
• start_tentative, end_tentative - Indicates the start and the end of a tentative tracing section.
• commit_tentative, discard_tentative - Commits or discards the data in a tentative trace buffer.
• get_userstring - Copies data from user memory.
• timestamp - Returns the current timestamp.
• diff_time - Returns the difference between two timestamps.
• exit - Terminates the Vue script.
• atoi - Returns the integer value of a string.
• strstr - Return a string inside another string.

6.1  printf

Purpose

Formats and copies data to the trace buffer.

Syntax

void printf( String format, [ data, ... ] );

Description

The printf() function converts, formats and copies the data parameter values to the trace buffer under the control of the format parameter. As indicated by the syntax, a varying list of arguments can be passed as data parameters to the printf() function. Vue supports all the conversion specifiers supported by the printf routine provided by the C library except for the "%p" conversion specifier.

There is no type checking for the arguments passed to the printf function except of the initial format string. Further, there is no checking if the number of arguments match the number of conversion specifiers in the format string. Strange output can be displayed if the type and number of arguments do not match the type and number of the conversion specifiers in the format string.

The following script demonstrates some examples of using the printf() function.

	String s[128];
	int i;
	float f;
	f = 2.3;

	s = "Test: %d, float = %f\n";
	i = 44;

	printf(s, i, f);

	s = "Note:";
	printf("%s Value of i (left justified) = %-12d and right-justified = %12d\n",
				s, i, i);

	printf("With a long format string that may span multiple lines, you " +
			"can use the '+' operator to concatenate the strings " +
			"in multiple lines into a single string: 0x%08x\n", i); 

Parameters

format	A string that contains plain characters that are directly copied to the trace buffer without any change and one or more conversion specifiers that provide indication on how to format the data parameters. Please refer to the man pages for the AIX printf() subroutine in the "Technical Reference: Base Operating System and Extensions, Volume 1" manual for more information on conversion specifiers.
data	Specifies zero or more arguments that map to the conversion specifiers in the format parameter.

6.2  trace

Purpose

Copies raw data to the trace buffer.

Syntax

void trace( data );

Description

The trace() function takes a single parameter which must be a variable. Expressions are not allowed.

The trace() function copies the value of the passed-in argument to the trace buffer as raw data with no formatting. The argument can be of any data type and the size of the data copied to the trace buffer is based on its innate size. Thus, four bytes are copied for an integer argument, four or eight bytes for pointers (depending upon whether the execution is in 32-bit or 64-bit mode) and the size of the structure for arguments of type struct. For a variable of type String, the number of bytes copied over is the declared string length (which is not the same as the length of the string contained in the variable).

The trace() function copies the data to the trace buffers with no formatting and the trace reporter displays the data as it is stored internally in hexadecimal bytes.

Parameters

data	Data argument to be copied to trace buffer

6.3  stktrace

Purpose

Generates and prints a runtime stack trace.

Syntax

void stktrace( int flags, int levels );

Description

This routine prints the stack trace at the current probe point. By default, the stack trace is generated in compact form with only call chain addresses for only up to two levels. You can use the flags and the levels parameters to modify the format and contents of the stack trace. Currently, ProbeVue cannot read paged out data, so the stack trace is truncated if a page fault is encountered when accessing the stack.

The stktrace() function does not return any values.

section 5.3 has an example script that uses the stktrace() function.

Parameters

flags	Either set to 0 to specify default behavior or specify one or more of the following flags: PRINT_SYMBOLS Prints symbol names instead of addresses. GET_USER_TRACE By default, the stack trace is stopped at the system call boundary if the probe location is in kernel space. This flag indicates to trace all the way into user space also up to the number of levels specified by the levels parameter.
levels	Indicates the number of levels up to which the stack trace is to be printed. A value of -1 indicates that the stack back chain is to be traversed to the extent possible. Default value of 0 tracks back to 2 levels.

6.4   get_probe

Returns the current probe point specification.

Syntax

String get_probe( );

Description

This routine returns the internal representation of the current probe point specification. When saved internally, the probe point does not include the initial "@@" prefix or the process ID, if any.

The following shows an example of using the get_probe() function:

	#cat get_probe.e 
	@@uft:312678:*:run_prog:entry
	{
		printf("function '%s' probe '%s'\n", get_function(), get_probe());
	}

	#probevue get_probe.e
	function 'run_prog' probe 'uft:*:*:run_prog:entry'

Parameters

The get_probe() function does not take any parameters.

6.5  get_function

Purpose

Returns the name of the function that is being traced.

Syntax

String get_function( );

Description

This routine returns the name of the function which encloses the current probe point and which is currently being traced.

section 6.2 has an example script that uses the get_function() function.

The function returns an empty string when called from the interval probe manager.

Parameters

The get_function() function does not take any parameters.

6.6  get_location_point

Purpose

Returns the current probe location point.

Syntax

int get_location_point( );

Description

This routine returns the current probe location point as an offset from the enclosing function entry point. Specifically, it will return FUNCTION_ENTRY or zero if the probe point is at the entry point of the function and FUNCTION_EXIT or -1 if it is at any exit point; else, it returns the actual address offset.

The following shows an example of using the get_location_point() function:

	@@syscall:$1:read:entry, @@syscall:$1:read:exit
	{
		probev_timestamp_t ts1, ts2;
		int diff;

		if (get_location_point() == FUNCTION_ENTRY) 
			ts1 = timestamp();
		else if (get_location_point() == FUNCTION_EXIT) {
			ts2 = timestamp();
			diff = diff_time(ts1, ts2, MICROSECONDS);
			printf("Time for read system call = %d\n", diff);
		}
	}

This function is not supported when called from the interval probe manager.

Parameters

The get_location_point() function does not take any parameters.

6.7  list

Purpose

Creates and returns a new empty list.

Syntax

List list( );

Description

This routine is the constructor function for the list data type. It returns a new empty list and auto-declares the target to be a list data type. There is no explicit way to declare a variable to be a list data type.

Currently, the list() function can only be invoked from within the @@BEGIN probe.

A list variable can be used to collect values that are of integral type. Any value stored in the list is automatically promoted to a long long (or 64-bit integer) data type. Section 3.1 describes the list data type in more detail.

The following shows an example of using the list() function. It assumes that the sprobevue shell program, which encloses double quotes around each argument, is used to invoke the Vue script.



Figure 12

/* File: list.e * * Collect execution time for read system call statistics * * Usage: sprobevue list.e <-s|-d> * * Pass -s for summary and -d for detailed information */ int read(int fd, void *buf, int n); @@BEGIN { String s[10]; int detail; times = list(); /* declare and create a list */ /* Check for parameters */ s = $1; if (s == "-d") detail = 1; else if (s == "-s") detail = 0; else { printf("Usage: sprobevue list.e <-s|-d>\n"); exit(); } } @@syscall:*:read:entry { /* * Save entry time in a thread-local to ensure that * in the read:exit probe point we can access our thread's value for * entry timestamp. If we use a global, then the variable can be * overlaid by the next thread to enter read and this can give * wrong values when we try to find the difference at read:exit * time since we use this later value instead of the original value. */ __thread unsigned long long t1; t1 = timestamp(); } @@syscall:*:read:exit when (thread:t1 != 0) { __auto t2; __auto long difft; /* Get exit time */ t2 = timestamp(); difft = diff_time(t1, t2, MICROSECONDS); /* Append read time to list */ append(times, difft); /* print detail data if "-d" was passed to script */ if (detail) printf("%s (%ld) read time = %d micros\n", __pname, __pid, difft); } @@interval:*:clock:10000 { /* Print statistics every 10 seconds */ printf("Read calls so far = %d, total time = %d, max time = %d, " + "min = %d, avg = %d\n", count(times), sum(times), max(times), min(times), avg(times)); }

Parameters

The list() function does not take any parameters.

6.8  append

Purpose

Appends a value to a list.

Syntax

void append( List listvar, long long val );

Description

The append() function is the only list concatenation function currently available in Vue. It appends the value specified by the second parameter to the list variable specified by the first parameter. Each call to append() adds a new value to the set of values already saved in the list variable and the size of the list variable grows.

Note that the value added to the list must be a parameter of integral type or this will result in a syntax error. The probevue compiler will accept parameters that have any of the integer data types: char, short, int, long or long long. No casts are needed.

The append function has no return values.

Section 3.1 describes the list data type in more detail and section 6.7 has an example script that uses the append() function.

Parameters

listvar	A variable of type list.
val	Value to be appended.

6.9  sum

Purpose

Returns the sum of all the elements in a list.

Syntax

long long sum( List listvar );

Description

This routine returns the sum of all the elements that have been appended to the list variable specified by the listvar parameter.

Section 3.1 describes the list data type in more detail and section 6.7 has an example script that uses the sum() function.

Parameters

listvar

A variable of type list.

6.10  avg

Purpose

Returns the average of all the elements in a list.

Syntax

long long avg( List listvar );

Description

This routine returns the average of all the elements that have been appended to the list variable specified by the listvar parameter.

Section 3.1 describes the list data type in more detail and section 6.7 has an example script that uses the avg() function.

Parameters

listvar

A variable of type list.

6.11  min

Purpose

Returns the minimum of all the elements in a list.

Syntax

long long min( List listvar );

Description

This routine returns the minimum of all the elements that have been appended to the list variable specified by the listvar parameter.

Section 3.1 describes the list data type in more detail and section 6.7 has an example script that uses the min() function.

Parameters

listvar

A variable of type list.

6.12  max

Purpose

Returns the maximum of all the elements in a list.

Syntax

long long max( List listvar );

Description

This routine returns the maximum of all the elements that have been appended to the list variable specified by the listvar parameter.

Section 3.1 describes the list data type in more detail and section 6.7 has an example script that uses the max() function.

Parameters

listvar

A variable of type list.

6.13  count

Purpose

Returns the count of the number of elements in a list.

Syntax

long long count( List listvar );

Description

This routine returns the count of the number of elements that have been appended to the list variable specified by the listvar parameter.

Section 3.1 describes the list data type in more detail and section 6.7 has an example script that uses the count() function.

Parameters

listvar

A variable of type list.

6.14  start_tentative, end_tentative

Purpose

Indicates the start and the end of a tentative tracing section.

Syntax

void start_tentative( String bufID );
void end_tentative( String bufID );

Description

These functions indicate the start and end of a tentative tracing section within a Vue clause. Trace data generated by trace output functions enclosed within the tentative tracing section is saved on a tentative basis until a commit_tentative() or a discard_tentative() function is called to commit or discard this data. The end_tentative() function is optional and if it is not specified, the end of the Vue clause is implicitly assumed to indicate the end of the tentative tracing section.

The generated tentative trace data is identified by the bufID parameter, which needs to be a string constant and not a variable. Tentative trace data may be collected under different IDs simultaneously which can then be committed or discarded as a separate block each. Currently, ProbeVue supports up to 16 tentative trace buffers in the same dynamic tracing session, so up to 16 different trace IDs may be used in a Vue script. A single Vue clause may contain more than one tentative tracing section with one or more different IDs.

Section 5.6.1 describes tentative tracing in more detail and includes an example Vue script that uses tentative tracing.

Parameters

bufID

A string constant that indicates the tentative trace buffer ID.

6.15  commit_tentative, discard_tentative

Purpose

Commits or discards the data in a tentative trace buffer

Syntax

void commit_tentative( String bufID );
void discard_tentative( String bufID );

Description

The commit_tentative() function commits the trace data associated with the tentative trace buffer identified by the bufID parameter. This saves the data and make it available to the trace consumer.

The discard_tentative function discards all the data in the tentative trace buffer indicated by the bufID parameter. This frees up the space in the trace buffers occupied by the tentative trace data.

When tentative trace data is being saved along with regular trace data, erstwhile tentative, but later committed trace data and regular trace data will be made available to the trace consumer in timestamp order. Thus, it is a good idea to commit or discard tentative data at the earliest to free up the trace buffers.

All tentative trace data that has not been committed is discarded when the probevue session ends.

Section 5.6.1 describes tentative tracing in more detail and includes an example Vue script that uses tentative tracing.

Parameters

bufID

A string constant that indicates the tentative trace buffer ID.

6.16  get_userstring

Purpose

Copies data from user memory.

Syntax

String get_userstring( char * addr, int size );

Description

The get_userstring function reads data in user memory into a variable of type String.

Data in user space must be copied in before it can be used in expressions or passed as parameters to a Vue function. The target of the get_userstring() function is generally a variable of type String.

This function is only allowed in user space probes (like the uft probe type) or probes provided by the syscall probe manager. If a page fault occurs while copying the data, the copy operation is terminated and the String variable will contain only the data that was successfully copied. If an exception occurs while executing this function as for instance, when a bad user address is passed to the function, the probevue session is aborted with an error message.

Section 5.2 has an example script that uses the get_userstring() function.

Note:

You may modify the data type of the target of the copy operation sing casts, although this will generate a warning message from the compiler. Thus, the get_userstring() function may be used to copy not only strings, but also data laid out as structures and in other data formats from user space into ProbeVue space. The following script is an example of how to do this kind of data manipulation:

Figure 13

/* File: string2int.e * * Reads an integer passed in a pointer using get_userstring() * */ int get_file_sizep(int *fd); @@BEGIN { String s[8]; int i; } @@uft:$1:*:get_file_sizep:entry { i = *(int *)(get_userstring(__arg1, 4)); printf("fd = %d\n", i); }

Note that the target of the copy operation should be a String variable whose declared length is large enough to accept the copied in data or the probevue session may be aborted. The get_userstring() will accept any value for the size of the data to be copied in, but the maximum size that can be copied is limited by the memory limits of the probevue session.

Parameters

addr	Address of the user space data.
size	The size of the user data to be copied. A value of -1 indicates that the user data be treated as a "C" string and the copy to continue until a null byte (the '\0' character) is read. Be careful when specifying -1 as the value of the Size parameter.

6.17  timestamp

Purpose

Returns the current timestamp.

Syntax

probev_timestamp_t timestamp( );

Description

This routine returns the current timestamp in the probev_timestamp_t abstract data type. Although, abstract, the value has the following properties:

• It returns the same value when called at exactly the same time from multiple Vue scripts running on different CPUs.
• It is guaranteed to return a higher value if called a second time as long as the system has not been rebooted in between.

There is no relationship between the values returned by the timestamp() function on two different systems. Although, the compiler will let you treat the returned value as a 64-bit integer and even print its value using the printf function, this may introduce compatibility problems in future. The only legal use of the timestamp value is as subsequent input to the diff_time() function.

Section 6.6 or Section 6.7 have example scripts that use the timestamp() function.

Note:

The lbolt kernel variable, whose value indicates the number of ticks since boot-up, or the time kernel variable whose value indicates the number of seconds since epoch (Jan 1, 1970) can both be used instead of this function if a lower-resolution timestamp is acceptable.

	typedef long long time_t;
	__kernel time_t lbolt;         	/* number of ticks since last boot     */
	__kernel time_t time;          	/* memory mapped time in secs since epoch  */

Parameters

The timestamp() function does not take any parameters.

6.18  diff_time

Purpose

Returns the difference between two timestamps.

Syntax

long long diff_time( probev_timestamp_t ts1, probev_timestamp_t ts2, int format );

Description

This diff_time() returns the time difference between two timestamps that were recorded using the timestamp() function. This function can return time difference in microseconds or milliseconds as specified by the format parameter.

Section 6.6 or Section 6.7 have example scripts that use the diff_time() function.

Parameters

ts1	Indicates the earlier timestamp.
ts2	Indicates the later timestamp.
format	Set to one of the following: MILLISECONDS Returns time difference to closest millisecond. MICROSECONDS Returns time difference to closest microsecond

6.19  exit

Purpose

Terminates the Vue script.

Syntax

void exit();

Description

This exit()function terminates the Vue script. This disables all the probes enabled in the dynamic tracing session, discards any tentative trace data, executes the actions indicated in @@END probe and flushes all the captured trace data to the trace consumer. After the trace consumer prints any output traced in the @@END probe, the tracing session is terminated and the probevue process exits.

The same effect can be obtained by typing a Ctrl-C in the terminal that where the probevue command was executed if it is running as the foreground task. Alternatively, you can directly send a SIGINT to the probevue process using the kill command or the kill() system call.

Section 6.7 has an example script that uses the exit() function. Section 6.20 has an example script for how to exit at the same time as the process that you are probing terminates.

Note that this function does not take any arguments unlike the exit() subroutine provided by the C library.

Parameters

The exit() function does not take any parameters.

6.20  atoi

Purpose

Returns the integer value of a string.

Syntax

int atoi( String str );

Description

The atoi() function returns the integer whose value is represented by the string specified by the str parameter. It reads the string up to the first character that is not a numerical digit (0-9) and translates the scanned characters into the equivalent integer. Leading white-space characters are ignored, and an optional sign indicator may precede the digits.

The atoi() function is useful in converting strings back to integers when running the sprobevue shell script that wraps double-quotes around all arguments. The following is an example script that captures a process forking faster than expected.

Figure 14

/* File: ffork.e * * Usage: sprobevue ffork.e processname delta * * Traces whenever a specified process is forking faster than * the "delta" value passed. Specify a process name and the time * in milliseconds as parameters. */ /* Ignore other parameters to execve */ int execve(char *path); @@BEGIN { int done; int pid; pname = $1; /* name of process we are monitoring */ /* * Since sprobevue is used, need to extract the integer value * from the string (double quotes around the delta). */ delta = atoi($2); /* minimum delta in millisecs between forks */ printf("pname = %s, delta = %d\n", pname, delta); } @@syscall:*:execve:entry when (done == 0) { __auto String exec[128]; __thread int myproc; /* Find process being 'exec'ed */ exec = get_userstring(__arg1, 128); /* Got it. Set a thread-local and reset 'done' so that we * avoid entering this probe from now on. */ if (exec == pname) { pid = __pid; myproc = 1; done = 1; printf("Process name = %s, pid = %d\n", __pname, pid); } } @@syscall:*:fork:entry when (thread:myproc == 1) { /* old_ts is initialized to zero */ unsigned long long old_ts, new_ts; unsigned long long interval; /* Get current time stamp */ new_ts = timestamp(); /* Find time from last fork */ if (old_ts != 0) { interval = diff_time(old_ts, new_ts, MILLISECONDS); /* if below the specified limit, trace that */ if (interval < delta) printf("%s (%ld) forking too fast (%d milliseconds)\n", pname, __pid, interval); } /* Save current fork timestamp */ old_ts = new_ts; } @@syscall:*:exit:entry when (__pid == pid) { /* Catch process exit and terminate ourselves */ printf("Process '%s' exited.\n", pname); exit(); }

Parameters

str	String to be converted.

6.21  strstr

Purpose

Return a string inside another string.

Syntax

String strstr( String s1, String str2 );

Description

The strstr function finds the first occurrence of the string specified by the s2 in the string specified by the s1 parameter and returns a new string that contains the characters in s1 starting from this location. Neither s1 nor s2 are modified by this operation. If the sequence of characters specified by the s2 parameter does not appear even once in the s1 parameter, then an empty string will be returned.

Section 5.2 has an example script that uses the strstr() function.

Note that this function's behavior is not the same as the strstr() subroutine in the C library.

Parameters

s1	String within which to search.
s2	String to be searched for

7. Probe Managers

The probe manager is not part of the basic ProbeVue framework, but is, nevertheless, an essential component of dynamic tracing. Probe Managers are the providers of the probe points that can be instrumented by ProbeVue.

Probe managers generally support a set of probe points that belong to some common domain and share some common feature or attribute that distinguishes them from other probe points. Probe points are useful at points where control flow changes significantly, at points of state change or other similar points that of significant interest. Probe managers are careful to select probe points only in locations that are safe to instrument.

Probe managers may choose to define their own distinct rules for the probe specifications within the common style that must be followed for all probe specifications as discussed in Section 2.2.

ProbeVue currently supports the following three probe managers:

1. System Call probe manager
2. User function probe manager
3. Interval probe manager

These are discussed in more detail below.

7.1 System Call Probe Manager

The syscall probe manager supports probes at the entry and exit of well-defined and documented base AIX system calls. These are the system calls that have the same interface at the libc.a" (or C library) entry point and in the kernel entry point. Either the system call is a pass-through (the C library, simply imports the symbol from the kernel and the exports it with no code in the library) or there is trivial code for the interface inside the library.

The syscall probe manager accepts a 4-tuple probe specification in one of the following formats:

syscall:*:<system_call_name>:entry
syscall:*:<system_call_name>:exit

Additionally, the syscall probe manager also accepts a 4-tuple probe specification in one of the following formats:

syscall:<process_ID>:<system_call_name>:exit
syscall:<process_ID>:<system_call_name>:exit

The system call names accepted by the syscall probe manager are the names of the libc.a interfaces and not the internal system call names. For instance, the read() subroutine is exported by libc.a, but the actual system call name or kernel entry point is kread. The syscall probe manager will internally translate a libc interface to its kernel entry point and enable the probe at entry into the kread kernel routine. Because of this, if multiple C library interfaces invoke the kread routine, the probe point will fire for those interfaces also. Generally, this is not a problem because for most of the system calls supported by the syscall probe manager, there is a 1-to-1 mapping between the libc interface and the kernel routine.

For each syscall probe, there is an equivalent probe point in the library code provided by the uft probe manager. The uft probe manager does support all library interfaces including those not supported by the syscall probe manager. However, the syscall probe manager has two advantages:

1. It can be used to probe every process in the system by specifying asterisk as the second field.
2. It is more efficient than the uft probe manager because it does not need the switch from user mode to kernel mode and back to run the probe actions.

Appendix A provides the full list of system calls currently supported by the syscall probe manager.

7.2 UFT Probe Manager

The uft or the user function tracing probe manager supports probing user space functions that are visible in the XCOFF symbol table of a process. However, currently, the uft probe manager restricts support only to probe points that are at entry points of functions whose source is a C language text file even the symbol table may contain symbols whose sources are from a language other than C.

The uft probe manager currently accepts a 5-tuple probe specification only in the following format:

	
	uft:<processID>:*:<function_name>:entry

Note that the uft probe manager requires the process ID for the process to be traced and the complete function name of the function at whose entry point the probe is to be placed. Further, the uft probe manager currently requires that the third field be set to '*" to indicate that the function name is to be searched in any of the modules loaded into the process address space including the main executable and shared modules. This implies that if a program contains more than one C function with this name (for example, functions with static class that are contained in different object modules), then probes will be applied to the entry point of every one of these functions.

ProbeVue supports enabling probes in more than one process at the same time. However, you will need privileges even for probing processes that belong to you. This is described in more detail in Section 8.1.

As indicated above, the uft probe manager supports probes in shared modules like shared library modules. The following shows an example script that traces mutex activity by enabling probes in the thread library's mutex lock and unlock routines.

/* pthreadlocks.e */
/* Trace pthread mutex activity for a given multi-threaded process */
/* The following defines are from /usr/include/sys/types.h */

typedef long long pid_t;
typedef long long thread_t;

typedef struct {
	int	__pt_mutexattr_status;	
	int	__pt_mutexattr_pshared;	
	int	__pt_mutexattr_type;
} pthread_mutexattr_t;

typedef struct __thrq_elt thrq_elt_t;

struct __thrq_elt {
	thrq_elt_t	*__thrq_next;
	thrq_elt_t	*__thrq_prev;
};

typedef volatile unsigned char _simplelock_t;

typedef struct __lwp_mutex {
	char		__wanted;
	_simplelock_t	__lock;
} lwp_mutex_t;

typedef struct {
	lwp_mutex_t		__m_lmutex;
	lwp_mutex_t		__m_sync_lock;
	int			__m_type;
	thrq_elt_t		__m_sleepq;
	int			__filler[2];
} mutex_t;

typedef struct {
	mutex_t			__pt_mutex_mutex;
	pid_t			__pt_mutex_pid;
	thread_t		__pt_mutex_owner;
	int			__pt_mutex_depth;
	pthread_mutexattr_t	__pt_mutex_attr;
} pthread_mutex_t;

int pthread_mutex_lock(pthread_mutex_t *mutex);
int pthread_mutex_unlock(pthread_mutex_t *mutex);

@@uft:$__CPID:*:pthread_mutex_lock:entry
{
	printf("thread %d: mutex 0x%08x locked\n", __tid, __arg1);
}

@@uft:$__CPID:*:pthread_mutex_unlock:entry
{
	printf("thread %d: mutex 0x%08x unlocked\n", __tid, __arg1);
}

7.3 Interval Probe Manager

The interval probe manager supports The interval probe manager provides probe points that fire at a user-defined time-interval. The probe points are not located in kernel or application code, but instead are based on wall clock time interval based probe events.

The interval probe manager accepts a 4-tuple probe specification in the following format:

        
        @@interval:*:clock:<# milliseconds>

The second field is * indicating that the probe can be fired in any process. Currently, the interval probe manager does not filter probe events by process IDs. Currently, the only value supported by the interval probe manager for the third field is the clock keyword that identifies the probe specification as being for a wall clock probe. The fourth or last field, that is, the <# milliseconds> field identifies the number of milliseconds between firings of the probe. Currently, the interval probe manager requires that the value for this field be exactly divisible by 100 and consist only of digits 0-9. Thus, probe events that are apart by 100ms, 200ms, 300ms, and so on, are allowed.

Note that the interval probe manager does not guarantee that a probe will be fired exactly the number of milliseconds apart as indicated by value of the fourth field. Higher-priority interrupts and code that runs after disabling all interrupts may cause the probe to fire later than the specification.

The interval probe manager requires only basic dynamic tracing privileges. See Section 8.1 for details. The interval probe manager enforces the following limits on the number of probes it supports to prevent malicious users from running the kernel out of memory by creating a huge numbers of interval probes.

The interval probe manager currently does not support the following functions. If used inside an interval manager probe point, these functions will generate an empty string or zero as output.

• stktrace
• get_function
• get_probe
• get_location_point

The interval probe manager is useful for summarizing statistics collected over an interval of time.

8. Authorizations and Privileges

Dynamic tracing is only allowed for users with privileges or for the superuser. This is unlike the static tracing facilities in AIX, which enforce relatively limited privilege checking. There is a reason for requiring privileges to run probevue. A Vue script can potentially produce more severe impacts on system performance than a static tracing facility like AIX System Trace. This is because probe points for System Trace are pre-defined and restricted. ProbeVue can potentially support many more probe points and the probe locations can potentially be defined almost anywhere. Further, ProbeVue trace actions at a probe point can take much longer to execute than the System Trace actions at a probe point since those are limited to explicit data capture.

Further ProbeVue allows users to trace processes and read kernel global variables, both of which need to be controlled to prevent security exposures. Last but not least, a probevue session can consume a lot of pinned memory and restricting usage of ProbeVue to users with privilege reduces the risk of denial of service attacks. ProbeVue also allows administrators to control the memory usage of probevue sessions through the smitty interface.

Privileges for dynamic tracing are obtained differently depending upon whether RBAC (Role-based Access Control) is enabled or not. Please refer to the AIX man pages for more information about enabling and disabling RBAC.

8.1 RBAC-enabled mode

Privileges in an RBAC system are obtained through authorizations. An authorization is a text string associated with security-related functions or commands. Authorizations provide the mechanism to grant rights to users to perform privileged actions. Only a user with sufficient authorizations is allowed to execute the probevue command and start a dynamic tracing session.

• aix.ras.probevue.trace.user.self

  This authorization allow users to trace their applications in user space. The real user ID of the process to be traced must be equal to the real user ID of the user invoking the probevue command. This authorization allows you to enable probe points provided by the uft probe manager for your processes.

• aix.ras.probevue.trace.user

  This authorization allow users to trace any application in user space including applications started by the superuser. Be careful when handing out this authorization. This authorization allows you to execute the probevue command and enable probe points provided by the uft probe manager for any process on the system.

• aix.ras.probevue.trace.syscall.self

  This authorization allow users to trace system calls made by their applications. The real user ID of the process making the system call must be equal to the real user ID of the user invoking the probevue command. This authorization allows you to enable probe points provided by the syscall probe manager for your processes. The second field of the probe specification must indicate the process ID for a process started by you.

• aix.ras.probevue.trace.syscall

  This authorization allow users to trace system calls made by any application on the system including applications started by the superuser. Be careful when handing out this authorization. This authorization allows you execute the probevue command and enable probe points provided by the syscall probe manager for any process. The second field of the probe specification may either be set to a process ID to probe a specific process or to '*' to probe all processes.

• aix.ras.probevue.trace

  This authorization allow users to trace the entire system and includes all the authorizations defined above. It also allows users to access and read kernel variables when running the probevue command. Be careful when handing out this authorization.

• aix.ras.probevue.manage

  This authorization allow users to administer ProbeVue. This includes changing the values of the different ProbeVue parameters, starting/stopping ProbeVue and viewing details of dynamic tracing sessions of all users when running the probevctrl command. Without this authorization, you may use the probevctrl command to view session data for dynamic tracing sessions started by you or view the current values for ProbeVue parameters.

• aix.ras.probevue

  This authorization grants all dynamic tracing privileges and is equivalent to all the above authorizations combined.

The superuser (or root) has all these authorizations assigned by default. Other users will need to have authorizations assigned to them by first creating a "role" with a set of authorizations and assigning the role to the user. The user will also need to switch roles to a role that has the required authorizations defined for dynamic tracing before invoking the probevue command. Please refer to the RBAC man pages for more information on roles and authorizations. Here is an example of how to provide user "joe" authorization to enable user space and system call probes for processes started by joe.

	mkrole authorizations="aix.ras.probevue.trace.user.self,aix.ras.probevue.trace.syscall.self" apptrace
	chuser roles=apptrace joe
	setkst -t role		# Copy roles to kernel (Or wait until system reboots)

User "joe" may be set up to always have all roles acquired by default when logging in or can switch to the role as needed using the following command:

	swrole apptrace

Note that the interval probe manager does not have a specific authorization associated with it. You can enable interval probe points if you have any of the aix.ras.probevue.trace* authorizations.

8.1.1 ProbeVue Privileges

The privileges that are available for ProbeVue are listed in the following table. A description of each privilege and the authorizations that map to that privilege is provided. Privileges form a hierarchy where the parent privilege contains all of the rights that are associated with the privileges of its children or it can include additional privileges also.

8.2 Legacy or RBAC-disabled mode

In legacy mode, there are no authorizations. Regular users cannot acquire privileges to run the probevue command to start a dynamic tracing session or run the probevctrl command to administer ProbeVue. Only the superuser will have privileges for both these functions. It is recommended that you do not disable RBAC when using ProbeVue unless and you prefer to restrict this facility to root users only.

9. ProbeVue parameters

AIX provides a set of parameters that allow you to tune ProbeVue. of the ProbeVue framework. The parameters allow you to specify both global limits on resource usage by the ProbeVue framework and to specify resource usage for individual users. Note that probe managers are not contained within the ProbeVue framework and hence these limits do not apply them.

All ProbeVue parameters can be modified through smitty interface (use the "smitty probevue" fast path) or directly through the probevctrl command. ProbeVue can be stopped if there are no active dynamic tracing sessions and it can be restarted after stopping it without requiring a reboot. However, currently, ProbeVue may fail to stop if any sessions that used thread-local variables had been previously active.

The following table summarizes the parameters defined for dynamic tracing sessions. In the description below, a privileged user refers to the superuser or a user with aix.ras.probevue.trace authorization and a non-privileged user is one who does not have this authorization.

The initial values in the table above refer to the values when AIX 6.1 is initially installed. You can change any of these values through smitty or the probevctrl command. Both the current values and the values for next boot can be updated.

Please refer to the AIX man pages for more information on the probevue and probevctrl commands.

10. Miscellaneous

10.1 Running in a WPAR

Workload partitions or WPARs are virtualized operating system environments within a single instance of the AIX operating system. The WPAR environment is somewhat different from the standard AIX operating system environment. Please refer to the AIX man pages for more information about WPARs including how to start and stop them and how to make them mobile.

Dynamic tracing is supported in the WPAR environment. By default, when creating a WPAR, only the PV_PROBEVUE_TRC_USER_SELF and the PV_PROBEVUE_TRC_USER privileges are assigned to the WPAR and the superuser (root) on a WPAR system will be grated these privileges. An admin user from the global partition can change the value of the the default WPAR privilege set or can explicitly assign additional privileges when creating the WPAR.

Privileges on WPAR have generally the same meanings as on a global partition. Be careful when assigning PV_PROBEVUE_TRC_KERNEL or the PV_PROBEVUE_TRC_MANAGE to a WPAR. Any user with PV_PROBEVUE_TRC_KERNEL privilege can access global kernel variables while a user with PV_PROBEVUE_TRC_MANAGE can change the values of ProbeVue parameters or shutdown ProbeVue. These changes affect all users even those in other partitions.

When you execute the probevue command in a WPAR, processes running in other WPARs or in the global partition are not visible to it. Because of this, you can only probe processes in your same WPAR and the probevue command will fail if the probe specification contains a process ID that is outside its partition. The PV_PROBEVUE_TRC_USER and PV_PROBEVUE_TRC_SYSCALL privileges in a WPAR only allow you to probe user space functions or system calls of processes that are in your WPAR. When probing system calls, the second field of the syscall probe specification must be currently set to a valid WPAR-visible process ID. Assigning the value "*" to the second field is not supported.

When a probevue session is initiated in a mobile WPAR, that will temporarily switch the WPAR to a non-checkpointable state. After the probevue session terminates, the WPAR is checkpointable again.

11. Sample Programs

A few sample Vue scripts follow:

1. The following canonical "Hello World" program prints "Hello World" into the trace buffer and exits.
   
   

   Figure 16

   #!/usr/bin/probevue /* Hello World in probevue */ /* Program name: hello.e */ @@BEGIN { printf("Hello World\n"); exit(); }

   
   
   
2. The following "Hello World" program prints "Hello World" when Ctrl-C is typed on the keyboard.
   
   

   Figure 17

   #!/usr/bin/probevue /* Hello World 2 in probevue */ /* Program name: hello2.e */ @@END { printf("Hello World\n"); }

   
   
   
3. The following program shows how to use thread-local variables. This Vue script counts the number of bytes written to a particular file. It assumes that the processes are single-threaded or those threads that open files are the same ones that write to them. It also assumes that all write operations are successful.
   
   

   Figure 18

   #!/usr/bin/probevue /* Program name: countbytes.e */ int open( char * Path, int OFlag, int mode ); int read( int fd, char * buf, int sz); @@syscall:*:open:entry when (__arg1 == "/tmp/foo" ) { thread:trace = 1; } @@syscall:*:open:exit when (thread:trace) { thread:fd = __rv; } @@syscall:*:write:entry when (thread:trace && __arg1 == thread:fd) { bytes += __arg3; /* number of bytes is 3rd arg */ } @@END { printf("Bytes written = %d\n", bytes); }

   
   
   
4. The following tentative tracing program shows how to trace the arguments passed to the read system call only if it returns zero bytes when reading the foo.data file.
   
   

   Figure 19

   #!/usr/bin/probevue /* File: ttrace.e */ /* Example of tentative tracing */ /* Capture parameters to read system call only if read fails */ int open ( char* Path, int OFlag , int mode ); int read ( int fd, char * buf, int sz); @@syscall:*:open:entry { filename = get_userstring(__arg1, -1); if (filename == "foo.data") { thread:open = 1; start_tentative("read"); printf("File foo.data opened\n"); } } @@syscall:*:open:exit when (thread:open == 1) { thread:fd = __rv; start_tentative("read"); printf("fd = %d\n", thread:fd); thread:open = 0; } @@syscall:*:read:entry when (__arg1 == thread:fd) { start_tentative("read"); printf("Read fd = %d, input buffer = 0x%08x, bytes = %d,", __arg1, __arg2, __arg3); end_tentative("read"); thread:read = 1; } @@syscall:*:read:exit when (thread:read == 1) { if (__rv < 0) { /* The printf below even though non-tentative is only * executed in error cases and merges with the * previously printed tentative data */ printf(" errno = %d\n", __errno); commit_tentative("read"); } else discard_tentative("read"); thread:read = 0; }

   A possible output if the read failed because a bad address (say 0x1000) was passed as input buffer pointer could look like:

   #probevue ttrace.e
   File foo.data opened
   fd = 4
   Read fd = 4, input buffer = 0x00001000, bytes = 256, errno = 14
   

   
   
   
5. The following Vue script prints the values of some kernel variables and exits immediately. Note the exit() function in the @@BEGIN probe.
   
   

   Figure 20

   /* File: kernel.e */ /* Example of accessing kernel variables */ /* System configuration structure from /usr/include/sys/systemcfg.h */ struct system_configuration { int architecture; /* processor architecture */ int implementation; /* processor implementation */ int version; /* processor version */ int width; /* width (32 || 64) */ int ncpus; /* 1 = UP, n = n-way MP */ int cache_attrib; /* L1 cache attributes (bit flags) */ /* bit 0/1 meaning */ /* -------------------------------------*/ /* 31 no cache / cache present */ /* 30 separate I and D / combined */ int icache_size; /* size of L1 instruction cache */ int dcache_size; /* size of L1 data cache */ int icache_asc; /* L1 instruction cache associativity */ int dcache_asc; /* L1 data cache associativity */ int icache_block; /* L1 instruction cache block size */ int dcache_block; /* L1 data cache block size */ int icache_line; /* L1 instruction cache line size */ int dcache_line; /* L1 data cache line size */ int L2_cache_size; /* size of L2 cache, 0 = No L2 cache */ int L2_cache_asc; /* L2 cache associativity */ int tlb_attrib; /* TLB attributes (bit flags) */ /* bit 0/1 meaning */ /* -------------------------------------*/ /* 31 no TLB / TLB present */ /* 30 separate I and D / combined */ int itlb_size; /* entries in instruction TLB */ int dtlb_size; /* entries in data TLB */ int itlb_asc; /* instruction tlb associativity */ int dtlb_asc; /* data tlb associativity */ int resv_size; /* size of reservation */ int priv_lck_cnt; /* spin lock count in supevisor mode */ int prob_lck_cnt; /* spin lock count in problem state */ int rtc_type; /* RTC type */ int virt_alias; /* 1 if hardware aliasing is supported */ int cach_cong; /* number of page bits for cache synonym */ int model_arch; /* used by system for model determination */ int model_impl; /* used by system for model determination */ int Xint; /* used by system for time base conversion */ int Xfrac; /* used by system for time base conversion */ int kernel; /* kernel attributes */ /* bit 0/1 meaning */ /* -----------------------------------------*/ /* 31 32-bit kernel / 64-bit kernel */ /* 30 non-LPAR / LPAR */ /* 29 old 64bit ABI / 64bit Large ABI */ /* 28 non-NUMA / NUMA */ /* 27 UP / MP */ /* 26 no DR CPU add / DR CPU add support */ /* 25 no DR CPU rm / DR CPU rm support */ /* 24 no DR MEM add / DR MEM add support */ /* 23 no DR MEM rm / DR MEM rm support */ /* 22 kernel keys disabled / enabled */ /* 21 no recovery / recovery enabled */ /* 20 non-MLS / MLS enabled */ long long physmem; /* bytes of OS available memory */ int slb_attr; /* SLB attributes */ /* bit 0/1 meaning */ /* -----------------------------------------*/ /* 31 Software Managed */ int slb_size; /* size of slb (0 = no slb) */ int original_ncpus; /* original number of CPUs */ int max_ncpus; /* max cpus supported by this AIX image */ long long maxrealaddr; /* max supported real memory address +1 */ long long original_entitled_capacity; /* configured entitled processor capacity */ /* at boot required by cross-partition LPAR */ /* tools. */ long long entitled_capacity; /* entitled processor capacity */ long long dispatch_wheel; /* Dispatch wheel time period (TB units) */ int capacity_increment; /* delta by which capacity can change */ int variable_capacity_weight; /* priority weight for idle capacity*/ /* distribution */ int splpar_status; /* State of SPLPAR enablement */ /* 0x1 => 1=SPLPAR capable; 0=not */ /* 0x2 => SPLPAR enabled 0=dedicated; */ /* 1=shared */ int smt_status; /* State of SMT enablement */ /* 0x1 = SMT Capable 0=no/1=yes */ /* 0x2 = SMT Enabled 0=no/1=yes */ /* 0x4 = SMT threads bound true 0=no/1=yes */ int smt_threads; /* Number of SMT Threads per Physical CPU */ int vmx_version; /* RPA defined VMX version, 0=none/disabled */ long long sys_lmbsize; /* Size of an LMB on this system. */ int num_xcpus; /* Number of exclusive cpus on line */ signed char errchecklevel;/* Kernel error checking level */ char pad[3]; /* pad to word boundary */ int dfp_version; /* RPA defined DFP version, 0=none/disabled */ /* if MSbit is set, DFP is emulated */ }; __kernel struct system_configuration _system_configuration; @@BEGIN { String s[40]; int j; __kernel int max_sdl; /* Atomic RAD system decomposition level */ __kernel long lbolt; /* Ticks since boot */ printf("No. of online CPUs\t\t= %d\n", _system_configuration.ncpus); /* Print SMT status */ printf("SMT status\t\t\t="); if (_system_configuration.smt_status == 0) printf(" None"); else { if (_system_configuration.smt_status & 0x01) printf(" Capable"); if (_system_configuration.smt_status & 0x02) printf(" Enabled"); if (_system_configuration.smt_status & 0x04) printf(" BoundThreads"); } printf("\n"); /* Print error checking level */ if (_system_configuration.errchecklevel == 1) s = "Minimal"; else if (_system_configuration.errchecklevel == 3) s = "Normal"; else if (_system_configuration.errchecklevel == 7) s = "Detail"; else if (_system_configuration.errchecklevel == 9) s = "Maximal"; printf("Error checking level\t\t= %s\n",s); printf("Atomic RAD system detail level\t= %d\n", max_sdl); /* Long in the kernel is 64-bit, so we use %lld below */ printf("Number of ticks since boot\t= %lld\n", lbolt); exit(); }

   A possible output when running on a Power 5 dedicated partition with default kernel attributes is:

   
   # probevue kernel.e
   No. of online CPUs              = 4
   SMT status                      = Capable Enabled BoundThreads
   Error checking level            = Normal
   Atomic RAD system detail level  = 2
   Number of ticks since boot      = 34855934
   

Appendix A:
System Calls supported by the syscall probe manager

The following table lists the system calls currently supported by the syscall probe manager along with the actual entry name in the kernel. Note that the kernel entry name is provided here only for documentation purposes. The kernel entry names may change between releases or even after a service update.

Appendix B:
Shell helper programs for ProbeVue

The following shell scripts are useful when running ProbeVue.

1. Shell script that wraps all arguments in double quotes:

   #!/usr/bin/ksh
   #
   # sprobevue:
   #
   #	Simple helper function for probevue
   #	Wraps arguments to probevue in double quotes
   #
   #	Usage: sprobevue <probevue flags> <script> <args>
   #		Doesn't support the -c and -A flags of probevue
   #
   
   usage()
   {
   	echo "Usage: sprobevue <probevue flags> <script> <args>" >&2
   	echo "	Doesn't support the -c and -A flags of probevue" >&2
   	exit 1
   }
   
   CMD=probevue
   # Generate command to execute
   
   while getopts 'c:A:I:s:o:t:X:' zargs
   do
   	case $zargs in
   		I|s|o|t|X) CMD="$CMD -$zargs $OPTARG"      	;;
   		?) usage
   	esac
   done
   
   shift $(($OPTIND -1))
   
   if [ -n "$1" ]
   then
   	CMD="$CMD $1"
   	shift
   fi
   
   for i
   do
   	CMD="$CMD \"$i\""
   done
   
   # Execute command 
   $CMD
   

   
   
   
2. Shell script that prints process ID given process name

   #/usr/bin/ksh
   #
   # prtree:
   #
   #	Simple helper function for probevue
   #	Prints all process IDs with given process name
   #
   # 	Need options to print only one process
   #	  to print process belong to a certain UID
   #	Also, maybe check that "grep $1" starts 8th field. Avoid
   #	some corner cases like "foo "grep foo" where foo takes
   #	grep and foo as arguments to itself.
   #
   #	Usage: prgrep <process_name>
   #	
   
   [ -z "$1" ] && {
   	echo "Usage: prgrep <process_name>" >&2
   	exit 1
   }
   
   ps -ef | grep $1 | grep -v "grep $1" | awk '{print $2}'
   
   

   ---

   ---

   Trademarks

   The following are trademarks of International Business Machines Corporation in the United States, other countries, or both:

   • AIX
   • IBM

   Other company, product, or service names may be trademarks or service marks of others.