we recently upgraded a clients old 1-way 333mhz dhcp server from AIX
4.3 to 5.2... now the machine is suffering SERIOUS performance
problems.... it has basically slowed down to about 1/5 of its previous
perf. capacity. other than the os upgrade, another disk was added, and
a datavg was seperated out from the (previously solitary) rootvg. no
other apps were added, and there is no additional workload. i have run
perf.tools and nmon, but cannot seem to find anything extraordinarily
obvious going on. does ANYONE know what could be up with this piece?

also... i know IBM says it meets HW req's, but do you think its just
too weak of a setup to run 5.2??? i mean, this little POS only has one
333mhz proc and 256mbram!!! sales dept. wants this guy to just upgrade
to a new pSeries!

christiancolborn wrote:
> we recently upgraded a clients old 1-way 333mhz dhcp server from AIX
> 4.3 to 5.2... now the machine is suffering SERIOUS performance
> problems.... it has basically slowed down to about 1/5 of its previous
> perf. capacity.

You will have to isolate the slowness to a degree.

What is slow? DHCP authentication or network performance? Have you
checked your network card duplexity for all cards after the upgrade?

What is a transaction in your space and what makes its end to end
performance good/bad? If the disks are not busy, then tick it off and
check something else. What is your memory consumption? (svmon -G)

0xbeef

christiancolborn wrote:
> we recently upgraded a clients old 1-way 333mhz dhcp server from AIX
> 4.3 to 5.2... now the machine is suffering SERIOUS performance
> problems.... it has basically slowed down to about 1/5 of its previous
> perf. capacity. other than the os upgrade, another disk was added, and
> a datavg was seperated out from the (previously solitary) rootvg. no
> other apps were added, and there is no additional workload. i have run
> perf.tools and nmon, but cannot seem to find anything extraordinarily
> obvious going on. does ANYONE know what could be up with this piece?

The upgrade overlaid a lot of your prior config files and added some
default "features" that you might not want.

If you're not having IO or CPU issues, you might want to start looking
at your config files.... and check to see if you're running full duplex
etc.

The OS upgrade, per se, shouldn't do this.

>
> also... i know IBM says it meets HW req's, but do you think its just
> too weak of a setup to run 5.2??? i mean, this little POS only has one
> 333mhz proc and 256mbram!!! sales dept. wants this guy to just upgrade
> to a new pSeries!

You might be paying more in yearly maint than a new system would
cost

How much memory do you have installed on your system? AIX 5L
needs a lot more that AIX 4 to run efficiently.

Regards,
Jerry M


christiancolborn <christiancolborn@yahoo.com> wrote:
> we recently upgraded a clients old 1-way 333mhz dhcp server from AIX
> 4.3 to 5.2... now the machine is suffering SERIOUS performance
> problems.... it has basically slowed down to about 1/5 of its previous
> perf. capacity. other than the os upgrade, another disk was added, and
> a datavg was seperated out from the (previously solitary) rootvg. no
> other apps were added, and there is no additional workload. i have run
> perf.tools and nmon, but cannot seem to find anything extraordinarily
> obvious going on. does ANYONE know what could be up with this piece?

> also... i know IBM says it meets HW req's, but do you think its just
> too weak of a setup to run 5.2??? i mean, this little POS only has one
> 333mhz proc and 256mbram!!! sales dept. wants this guy to just upgrade
> to a new pSeries!

Jerald H. Mathews wrote:
> How much memory do you have installed on your system? AIX 5L
> needs a lot more that AIX 4 to run efficiently.
>
> Regards,
> Jerry M
>
>
> christiancolborn <christiancolborn@yahoo.com> wrote:
>
>>we recently upgraded a clients old 1-way 333mhz dhcp server from AIX
>>4.3 to 5.2... now the machine is suffering SERIOUS performance
>>problems.... it has basically slowed down to about 1/5 of its previous
>>perf. capacity. other than the os upgrade, another disk was added, and
>> a datavg was seperated out from the (previously solitary) rootvg. no
>>other apps were added, and there is no additional workload. i have run
>>perf.tools and nmon, but cannot seem to find anything extraordinarily
>>obvious going on. does ANYONE know what could be up with this piece?
>
>
>>also... i know IBM says it meets HW req's, but do you think its just
>>too weak of a setup to run 5.2??? i mean, this little POS only has one
>>333mhz proc and 256mbram!!! sales dept. wants this guy to just upgrade
>>to a new pSeries!
>
>
>
FWIW: I've been running several OLD RS6K's (166Mhz 128MB, 200Mhz 128MB,
375Mhz 256MB) on 5.1L ML04 and they all run just perfectly FINE.

My guess is you have some DNS issue that's stalling everything. Be sure
you turn off /disable any network service you are NOT using. I see no
reason why your performance should be poor.

Good luck.

christiancolborn schrieb:

> we recently upgraded a clients old 1-way 333mhz dhcp server from AIX
> 4.3 to 5.2... now the machine is suffering SERIOUS performance
> problems.... it has basically slowed down to about 1/5 of its previous
> perf. capacity. [...]
> also... i know IBM says it meets HW req's, but do you think its just
> too weak of a setup to run 5.2???

Yes. It definitely is too weak to run 5.2 with decent performance.

Andreas Schulze wrote:
>>we recently upgraded a clients old 1-way 333mhz dhcp server from AIX
>>4.3 to 5.2... now the machine is suffering SERIOUS performance
>>problems.... it has basically slowed down to about 1/5 of its previous
>>perf. capacity. [...]
>
> Yes. It definitely is too weak to run 5.2 with decent performance.

I disagree. While 5.2 certainly appreciates more memory, mostly for all
the services added since 4.3 there should not be a big impact on
performance.

But before we can give more advice we need to know more about what went
wrong with your performance.

- Is this all the machine is doing serving DHCP ?
- What filesystems are now on datavg (I'm confused, DHCP needs almost no
disk space) ?
- How is the performance measured ?
- What does topas indicates ?
Is there paging (pgsin/out) ?
Is the cpu load high ?
Is the network load high ?
Is there a disk with high load ?

Markus

can you post some perf stats please ... i.e. post the following while
the system is under load

vmstat -v
vmstat 1 20

i have seen cases where maxperm/minperm were tuned at 4.3.3 but the
settings got lost after the migration, this could be your issue...

Rgds
Mark Taylor

here are the results of vmstat -v and vmstat 1 20:

# vmstat 1 20
System Configuration: lcpu=1 mem=256MB
kthr memory page faults cpu
----- ----------- ------------------------ ------------ -----------
r b avm fre re pi po fr sr cy in sy cs us sy id wa
2 3 52014 390 0 2 2 45 134 0 607 5700 932 5 5 77 12
0 0 52018 384 0 1 0 0 0 0 194 2078 71 0 0 98 2
0 0 52018 384 0 0 0 0 0 0 191 2063 67 0 1 99 0
0 0 52018 384 0 0 0 0 0 0 198 2436 76 2 1 94 3
0 0 52018 384 0 0 0 0 0 0 193 2064 68 0 0 99 0
0 0 52018 383 0 0 0 0 0 0 195 2159 75 1 1 96 2
0 0 52018 383 0 0 0 0 0 0 194 2064 62 0 1 99 0
0 0 52398 123 0 0 42 120 799 0 226 2851 99 2 3 95 0
0 0 52419 126 0 0 8 24 141 0 221 2879 121 2 4 93 1
0 0 52018 718 0 0 64 192 1567 0 224 3827 98 3 8 89 0
0 0 52391 346 0 0 1 0 0 0 196 2851 75 0 7 93 0
2 0 52463 266 0 2 0 0 0 0 232 2773 154 4 1 92 3
0 1 53062 126 0 0 82 464 3780 0 288 23574 325 14 53 14 19
2 0 52519 668 0 0 0 0 0 0 226 13942 225 9 38 45 8
0 0 52519 668 0 0 0 0 0 0 192 2172 73 0 1 99 0
0 0 52924 271 0 0 1 8 73 0 212 8963 141 6 17 74 3
4 0 52521 674 0 0 0 0 0 0 233 14434 249 15 35 44 7
1 0 52521 674 0 0 0 0 0 0 197 2226 94 1 1 98 0
0 0 52521 674 0 0 0 0 0 0 196 2436 74 0 2 96 2
3 0 52596 599 0 0 0 0 0 0 206 5324 104 5 6 88 1
# vmstat -v
65536 memory pages
59212 lruable pages
191 free pages
1 memory pools
10049 pinned pages
80.1 maxpin percentage
20.0 minperm percentage
80.0 maxperm percentage
61.2 numperm percentage
36268 file pages
0.0 compressed percentage
0 compressed pages
0.0 numclient percentage
80.0 maxclient percentage
0 client pages
0 remote pageouts scheduled
0 pending disk I/Os blocked with no pbuf
64573 paging space I/Os blocked with no psbuf
412862 filesystem I/Os blocked with no fsbuf
0 client filesystem I/Os blocked with no fsbuf
0 external pager filesystem I/Os blocked with no
fsbuf

the columns a re a bit screwey ... but it looks like you are paging
out.. maxperm is set to default ..

these are also of concern .. so you probably want to up your fsbufs

64573 paging space I/Os blocked with no psbuf
412862 filesystem I/Os blocked with no fsbuf

avm is 52596 *4096 == 215433216 == 205MB

Your numperm is 61.2% so you are going to be paging out .. you need to
stop it paging by reducing the level of persisitent pages that you
cache in realmem .. you do this by lowering maxperm .. i would go with
the following as your avm is about ~ 80% of your realmem ..

maxperm%=10
minperm%=5

i.e.

vmo -p -o maxperm%=10 -o minperm%=5 -o maxclient%=10%

And see how you get on ..

check out man ioo and man vmo for tuning tips.

HTH
Mark Taylor