SEO

I have a machine that is showing a high %sys in sar.

Used cpuhog/iohog/memhog along with u386mon + top and nothing shows
the offending process(es) - ps listings are normal. No sign of swapping.
Resorted to brute force and killed process after process and sar
still showing high %sys.

Anyone any ideas?

Running OSR 5.0.7 + Update Pack 2
Single Pentium P4 + 512Mb Ram + Adaptec Raid
HyperThreading Enabled.


BackupEDGE for OpenServer 5 (ver 02.00.01)
GNU Development Tools (ver 5.0.7g)
GNU Ghostscript, a PostScript and PDF interpreter (ver 6.51)
SCO OpenServer Enterprise System (ver 5.0.7Hw)
SCO OpenServer Linker and Application Development Libraries (ver 5.2
SCO Symmetrical Multiprocessing (ver 1.1.1Hw)
Squid Caching Proxy Server v2.5.1 (ver 5.0.7f)
Large Filesystem Performance Supplement (ver 1.1.0c)
OSS662A - OpenServer 5.0.7 MP1 Supplement (ver 1.0.0)
SCO OpenServer Release 5.0.7 Maintenance Pack 1 (ver 1.0.0Bs)
SCO OpenServer Release 5.0.7 Update Pack 2 (ver 1.0.0Ca)


SCO_SV shepton 3.2v5.0.7 Pent4 04/07/2004

15:41:25 %usr %sys %wio %idle (-u)
16:25:00 0 21 0 79
16:30:00 0 22 0 78
16:40:00 0 20 0 79
16:45:00 0 21 0 79
16:50:00 0 22 0 78
16:55:00 0 22 0 78
17:00:01 0 21 0 79
17:05:00 0 21 0 79
17:15:00 0 21 0 78


Tom

--
================================================== ======================
Tom Melvin tom@tkrh.demon.co.uk http://www.tkrh.demon.co.uk
Veterinary Solutions Ltd
================================================== ======================

Tom Melvin wrote:
> Anyone any ideas?
>
> Running OSR 5.0.7 + Update Pack 2
> Single Pentium P4 + 512Mb Ram + Adaptec Raid
> HyperThreading Enabled.

Adaptec 2100S I20 perhaps? We have a client on 5.0.6 + 2100S that %sys
is ~10% but it is on a 1.2Ghz P3. The machine is otherwise fast and
responsive and handles anything the client can throw at it so I never
persued it further :-/
--
Darryl Krasman
Ideal Computer Group Inc.

Tom Melvin wrote:

> I have a machine that is showing a high %sys in sar.
>
> Used cpuhog/iohog/memhog along with u386mon + top and nothing shows
> the offending process(es) - ps listings are normal. No sign of swapping.
> Resorted to brute force and killed process after process and sar
> still showing high %sys.
>
> Anyone any ideas?
>
> Running OSR 5.0.7 + Update Pack 2
> Single Pentium P4 + 512Mb Ram + Adaptec Raid
> HyperThreading Enabled.

Link scodb into the kernel ('Y' in /etc/conf/sdevice.d/scodb; relink;
reboot). While the problem is happening, do some ad hoc kernel
profiling by repeatedly breaking into scodb. At a text console
multiscreen, hit Ctrl-Alt-D. At the resulting scodb prompt, "b" (a
short alias for "stack"), then "q". Repeat many times. Get a feel for
what the kernel is doing when you expect it to be idle.

>Bela<

Bela Lubkin made comment on Thu Apr 8 08:10:14 2004 :
> Tom Melvin wrote:
>
> > I have a machine that is showing a high %sys in sar.
> >
> > Used cpuhog/iohog/memhog along with u386mon + top and nothing shows
> > the offending process(es) - ps listings are normal. No sign of swapping.
> > Resorted to brute force and killed process after process and sar
> > still showing high %sys.
> >
> > Anyone any ideas?
> >
> > Running OSR 5.0.7 + Update Pack 2
> > Single Pentium P4 + 512Mb Ram + Adaptec Raid
> > HyperThreading Enabled.
>
> Link scodb into the kernel ('Y' in /etc/conf/sdevice.d/scodb; relink;
> reboot). While the problem is happening, do some ad hoc kernel
> profiling by repeatedly breaking into scodb. At a text console
> multiscreen, hit Ctrl-Alt-D. At the resulting scodb prompt, "b" (a
> short alias for "stack"), then "q". Repeat many times. Get a feel for
> what the kernel is doing when you expect it to be idle.

Is there another way to 'access' scodb - I am remote so will either be
using a telnet session or dial-up modem - while the modem is a character
based tty line CTRL X or ALT-CTRL-D does not want to operate.

Tom


--
================================================== ======================
Tom Melvin tom@tkrh.demon.co.uk http://www.tkrh.demon.co.uk
Veterinary Solutions Ltd
================================================== ======================

Tom Melvin wrote:

> Bela Lubkin made comment on Thu Apr 8 08:10:14 2004 :
> > Tom Melvin wrote:
> >
> > > I have a machine that is showing a high %sys in sar.
> > >
> > > Used cpuhog/iohog/memhog along with u386mon + top and nothing shows
> > > the offending process(es) - ps listings are normal. No sign of swapping.
> > > Resorted to brute force and killed process after process and sar
> > > still showing high %sys.
> > >
> > > Anyone any ideas?
> > >
> > > Running OSR 5.0.7 + Update Pack 2
> > > Single Pentium P4 + 512Mb Ram + Adaptec Raid
> > > HyperThreading Enabled.
> >
> > Link scodb into the kernel ('Y' in /etc/conf/sdevice.d/scodb; relink;
> > reboot). While the problem is happening, do some ad hoc kernel
> > profiling by repeatedly breaking into scodb. At a text console
> > multiscreen, hit Ctrl-Alt-D. At the resulting scodb prompt, "b" (a
> > short alias for "stack"), then "q". Repeat many times. Get a feel for
> > what the kernel is doing when you expect it to be idle.
>
> Is there another way to 'access' scodb - I am remote so will either be
> using a telnet session or dial-up modem - while the modem is a character
> based tty line CTRL X or ALT-CTRL-D does not want to operate.

There is a way; whether it can be used in this case, you will have to
determine.

The way is: have someone local to the machine hook its COM1 serial port
up to a serial port on a neighboring machine. Let's call the machines
"victim" and "buddy". If buddy is running Unix, what you're going to do
is connect to it -- doesn't matter if this is via ssh, telnet, dialup,
whatever -- and `cu` out the port that's connected to victim. If buddy
is running something else then you'll have to figure this part out.

At some point you have to reconfigure victim to use a serial console.
If you are able to get a serial login (coming in through buddy), you can
do this all by yourself, remotely. Otherwise you'll need at least a
little on-the-spot assistance.

To change victim to a serial console for the next and subsequent
reboots, edit /etc/default/boot, add a line that reads:

SYSTTY=1

To change it for a single reboot, use a local assistant. Reboot the
machine. Have them type at the boot prompt,

Boot
: systty=1

You should then see a boot prompt on your serial connection.

If scodb is already linked into the kernel, you can change the console
"on the fly", with the help of a local assistant. Make sure they're
looking at a text multiscreen (have them hit Ctrl-Alt-F1). Then have
them hit Ctrl-Alt-D and enter:

scodb> dbtty(1)

A scodb prompt should appear on your serial connection. 'q' to get the
system back in gear. Login, monitor for the %sys-too-high condition;
once you see it, start doing ad hoc profiling.

In all cases, configure the comm program that will be talking to the
serial console for 9600-8-N-1.

================================================== ===========================

Actually there may be an even easier way. Rereading what you wrote,
apparently you're dialing in directly to a modem on victim. In order
for that to work as a serial console, the modem needs to be on COM1
(tty1A). They've probably got it on some sort of intelligent multiport
board. Move it onto COM1 and try again.

Well, ^X still won't work, because at that point the console is the
video board, not COM1. So you still need to either (1) reboot with
"SYSTTY=1" in /etc/default/boot, (2) reboot and enter "systty=1" at the
boot prompt; or (3) break into scodb on the video console, enter
"dbtty(1)" to move the console to COM1.

I prefer the buddy system. In particular, I like to set up two OSR5
systems side by side with COM1 hooked to COM1. Then either of them can
have its console temporarily changed to COM1 (using "SYSTTY=1" in
/etc/default/boot). ssh into buddy and `cu -ltty1a direct` to get
access to victim's console. Even better: install GNU screen on buddy.
Run it. Within screen, `cu -ltty1a direct` to victim's console. Now
you have a persistent connection to the console, won't be dropped if
your serial/telnet/ssh connection in to buddy goes down. It'll only be
dropped if buddy itself goes down.

>Bela<