I need to write a program to monitor the cpu and memory usage of some
critical application processes in a system. I read about perfstat()
APIs, however they only provide system-wide info instead of per process
data. Are there APIs I should use to get the resource usage data for a
particular process?
Thanks in advance for your help.
-stevie

"steviwong@gmail.com" <steviwong@gmail.com> writes:

> I need to write a program to monitor the cpu and memory usage of some
> critical application processes in a system. I read about perfstat()
> APIs, however they only provide system-wide info instead of per process
> data. Are there APIs I should use to get the resource usage data for a
> particular process?
> Thanks in advance for your help.
> -stevie
>

The AIX Performance Toolbox and Performance AIDE provide programs and
APIs to collect the following per process statistics.

(Here's a snippet from the output of xmpeek -l from one of my AIX 5.3
machines, who's name happens to be 'studio'.)

/studio/Proc/356482~X/ Process X (356482 %cpu 6.0, PgSp: 8.3mb, uid: root
/studio/Proc/356482~X/nice Process nice value
/studio/Proc/356482~X/repage Process repage count
/studio/Proc/356482~X/majflt Process page faults involving IO
/studio/Proc/356482~X/minflt Process page faults not involving IO
/studio/Proc/356482~X/cpums CPU time in milliseconds in interval
/studio/Proc/356482~X/cpuacc CPU time in milliseconds in life
of process
/studio/Proc/356482~X/cpupct CPU time in percent in interval
/studio/Proc/356482~X/usercpu Process CPU use in user mode
(percent)
/studio/Proc/356482~X/kerncpu Process CPU use in kernel mode
(percent)
/studio/Proc/356482~X/workmem Physical memory used by process
private data (4K pages)
/studio/Proc/356482~X/codemem Physical memory used by process
code (4K pages)
/studio/Proc/356482~X/pagsp Page space used by process private
data (4K pages)
/studio/Proc/356482~X/nsignals Signals received by process
/studio/Proc/356482~X/nvcsw Voluntary context switches by
process
/studio/Proc/356482~X/tsize Code size (bytes)
/studio/Proc/356482~X/maxrss Maximum code+data resident set
size (4K pages)

The sample programs shipped with AIDE and Toolbox will give you a good
start towards collecting these yourself, either across the network
using the RSi interface, or on the local machine using the Spmi
interface.

If you want to roll your own from scratch, I beleive the getprocs64()
API is probably the place to start.

I don't know what mechanism AIDE / Toolbox uses to get the underlying
data for the per process statistics.

Hope this helps.

--
#include <disclaimer.std> /* I don't speak for IBM ... */
/* Heck, I don't even speak for myself */
/* Don't believe me ? Ask my wife :-) */
Richard D. Latham lathamr@us.ibm.com

I am looking at the /usr/samples/perfagent/server directory for a
sample program that i may use as a base. Here is what i see:
--------------------------------
Make.mib SpmiDds.c SpmiSupl.c filter.cf
lfiltd.c xmservd.res
Make.samples SpmiHotlog.c SpmiSupl1.c iphosts.c lfiltd.h
xmtrend.cf
Resptime.cf SpmiLogger.c armpeek jtopas.cf
lfiltd.lex xmtrend_wlm.cf
SpmiArmd.cf SpmiPeek.c cfg_jtopas lchmon.c
lfilter.cf
--------------------------------
I don't see a README or something like that, and all those *.c files
are quite confusing to read... not even sure i am at the right starting
point (the code subdirectory). Could you give additional info as to
which program (and where) might be a good sample for collecting cpu and
memory usage data?
Thank you very much.

"steviwong@gmail.com" <steviwong@gmail.com> writes:

> I am looking at the /usr/samples/perfagent/server directory for a
> sample program that i may use as a base. Here is what i see:
> --------------------------------
> Make.mib SpmiDds.c SpmiSupl.c filter.cf
> lfiltd.c xmservd.res
> Make.samples SpmiHotlog.c SpmiSupl1.c iphosts.c lfiltd.h
> xmtrend.cf
> Resptime.cf SpmiLogger.c armpeek jtopas.cf
> lfiltd.lex xmtrend_wlm.cf
> SpmiArmd.cf SpmiPeek.c cfg_jtopas lchmon.c
> lfilter.cf
> --------------------------------
> I don't see a README or something like that, and all those *.c files
> are quite confusing to read... not even sure i am at the right starting
> point (the code subdirectory). Could you give additional info as to
> which program (and where) might be a good sample for collecting cpu and
> memory usage data?
> Thank you very much.
>

OK.

There are seperate sample directories , one for the local machine
interface, aka Spmi, and the other for the remote interface, aka the
RSi interface.

I suggest starting with the local interface samples in
perfagent/server.

( BTW, none of this stuff is going to work if you didn't set up
xmservd in inetd.conf. That's the daemon that does all the heavy
lifting underneath these APIs )

-rw-r--r-- 1 lathamr system 9027 Feb 07 16:58 SpmiDds.c
-rw-r--r-- 1 lathamr system 23746 May 04 17:22 SpmiHotlog.c
-rw-r--r-- 1 lathamr system 25266 Feb 07 16:58 SpmiLogger.c
-rw-r--r-- 1 lathamr system 11757 May 11 10:28 SpmiPeek.c
-rw-r--r-- 1 lathamr system 14192 Feb 07 16:58 SpmiSupl.c
-rw-r--r-- 1 lathamr system 4625 Feb 07 16:58 SpmiSupl1.c
-rw-r--r-- 1 lathamr system 6478 Feb 07 16:58 iphosts.c
-rw-r--r-- 1 lathamr system 24396 Feb 07 16:58 lchmon.c
-rw-r--r-- 1 lathamr system 48204 May 01 14:35 lex.lfiltd.c
-rw-r--r-- 1 lathamr system 43758 Feb 07 16:58 lfiltd.c

SpmiPeek.c will show you how to recurse thru the tree of available
contexts / statistics.

SpmiDds.c is a (IMHO) feeble explaination of Dynamic Data
Suppliers. :-)

lchmon.c is (almost) the source to topas . Note that the per process
CPU statistics here are handled "specially". When you retreive a
context , there are certain values returned without requiring you to
have added the context to a StatSet. It seems "lucky somehw" that
enough info comes back in the context that you can implement something
like "top" easily :-)

You will probably want to have a look next at SpmiLogger.c. This shows
you how to construct Statsets, and subscribe to a feed of the values
associated with the StatSet on a defined interval.

Next have a look at SpmiHotlog.c . If you just want the "heavy
hitters" from a set of statistics, say 'the top 20 CPU consuming
processing in the interval", HotSets are for you.

What you won't find is a sample program that re-generates the StatSet
for the process context every interval. What that means , in practice,
is that transient processes aren't reported by normal StatSet
processing.

Transients are reported in a /Proc HotSet, but there's a max of 40
values returned in a HotSet.

Traversing the /Proc context does return the current list of
processes, but there's only limited performance information returned
in the context struct.

So, If you absolutely require the full list of processes, AND you require
the full set of statistics on all those processes ... you'll have to
piece together a program to do all that from the combination of
chmon.c (or SpmiPeek.c) and SpmiLogger.c.

I haven't created any code to do that myself, and I don't know whether
the API is really intended to be creating and deleting StatSets every
few seconds. I suspect that the underlying code wasn't designed to be
scalable when abused in that fashion.

Now, color me surprised that this sort of stuff wasn't placed into the
/Proc filesystem ... seems like this would have been the natural place
to expose these values.

--
#include <disclaimer.std> /* I don't speak for IBM ... */
/* Heck, I don't even speak for myself */
/* Don't believe me ? Ask my wife :-) */
Richard D. Latham lathamr@us.ibm.com

This is great information... thank you very very much!!
- stv