Arne S <apsodal@NOSPAM.online.no> wrote in message news:<40FD2C54.60400@NOSPAM.online.no>...
> Steve Nottingham wrote:
>
> > Arne S <apsodal@NOSPAM.online.no> wrote in message news:<40fccb2a$1@news.broadpark.no>...
> >
> >>kthr memory page faults cpu
> >>----- ----------- ------------------------ ------------ -----------
> >> r b avm fre re pi po fr sr cy in sy cs us sy id wa
> >>18 0 8483035 1571613 0 0 0 0 0 0 6499 99823 13804 68 22 4 7
> >>17 1 8482913 1571734 0 0 0 0 0 0 6778 86403 14977 78 13 6 4
> >> 8 2 8483831 1570812 0 0 0 0 0 0 6969 79401 16112 67 15 10 8
> >> 8 1 8489595 1565047 0 0 0 0 0 0 6191 209083 17598 70 18 6 6
> >>11 1 8482764 1571877 0 0 0 0 0 0 6049 271784 15108 68 20 5 7
> >>10 1 8482126 1572515 0 0 0 0 0 0 6051 73168 14993 57 11 18 13
> >>12 1 8481532 1573107 0 0 0 0 0 0 6081 87441 15317 60 15 13 12
> >>10 1 8481446 1573193 0 0 0 0 0 0 6344 75612 14889 71 14 7 7
> >>16 1 8483620 1571017 0 0 0 0 0 0 7486 65699 16522 74 13 6 7
> >>
> >>I believe this machine has serious problems with CPU and not with memory (columns "pi" and
> >>"po" are always 0).
> >>
> >>But my question is: Why is the column "us" under CPU so high instead of the column "sy"?
> >>
> >>Anybody have other comments to the output?
> >>
> >
> >
> > us is for user processes, sys is for system processes, Oracle and HA
> > will come under us. System looks fine to me, no 0's under idle You
> > have a bit of I/O wait, but if you have very fast processors then that
> > is normal, as disk speeds ahve increased as fast as processor speeds.
> >
> > I wouldn't be concerned about this system, yet, when you start getting
> > 0 in the id column for a long time, then you have a CPU bottleneck.
> > Are users complaining of slow response ?
> Thanks for response!
>
> I thought if sum of "us" and "sy" was above 80%, then you had a CPU bottleneck. True or not?
> I have done some measures for a week or so, and the "id" is quite low between 07:00 am and
> 04:00 pm (from 5% to 10%). And yes, the users complain about slow performance....
> The oracle procs are stealing the most of the CPU power, and many of the oracle procs have
> lots of CPU time (Oracle db procs (smon, pmon,arch... not included).
>
> The column "r" to the left from vmstat output often shows a value above 30, is that
> normal? I have even seen the value 98 in this column, I think that is too high to get good
> performance?
>
> Both of the nodes have 6 CPU's.
>
> Arne S


The "r" column is way too high (kernel threads in the run queue). I
would expect
poor performance when this starts to exceed 2 X the number of cpu's. I
bet when
this number is 10 or less your performance is fine. Also, you have a
very large amount of "freelist" (fre). AIX would normally keep the
free list fairly low.
(maybe Oracle is using raw area's)

Post some additional info, like:
Release of Oracle (32 or 64 bit)
Operating system release, and maint level.
output from vmtune (/usr/samples/kernel/vmtune) (vmo -L command for
Aix 5.2)
we need to see settings for
minfree
maxfree
minperm
maxperm
are your filesystems for Oracle raw? jfs? jfs2?
asyncronous io server settings if using jfs(2)

Should be pretty easy to trouble shoot.