SEO

#1 (**permalink**) 01-04-2008, 09:20 PM

Last night we had a problem on our p660-6H1 where one of the disks in
our mirrored pair of internal scsi disks went to 100% busy and but
iostat did not show any reads or writes. As a result the server was
unusable until we rebooted it which did in fact fix the problem. This
particular server is 4.3.3 and has 6 cpus and 8 gbs of ram running
oracle 8i.

Anyone else seen anything similar to this before on any scsi based
system?

TIA,

Matt

#2 (**permalink**) 01-04-2008, 09:21 PM

On Thu, 15 Jan 2004 10:44:52 -0800, Matt Raines wrote:

> Last night we had a problem on our p660-6H1 where one of the disks in
> our mirrored pair of internal scsi disks went to 100% busy and but
> iostat did not show any reads or writes. As a result the server was
> unusable until we rebooted it which did in fact fix the problem. This
> particular server is 4.3.3 and has 6 cpus and 8 gbs of ram running
> oracle 8i.
>
> Anyone else seen anything similar to this before on any scsi based
> system?

An operation (such as ls) on a directory with hundreds of thousands of
files in it will make the disk go to 100% with no visible I/O. But IME it
doesn't make the machine unusable. Maybe it does if it's in a system FS.

Look for filesystems with abnormally high inodes used.

Regards, Ian

#3 (**permalink**) 01-04-2008, 09:21 PM

Matt Raines wrote:

> Last night we had a problem on our p660-6H1 where one of the disks in
> our mirrored pair of internal scsi disks went to 100% busy and but
> iostat did not show any reads or writes. As a result the server was
> unusable until we rebooted it which did in fact fix the problem. This
> particular server is 4.3.3 and has 6 cpus and 8 gbs of ram running
> oracle 8i.
>
> Anyone else seen anything similar to this before on any scsi based
> system?
>
> TIA,
>
> Matt

Did the errpt show anything?

I remember a while back that when one of my disks in a mirrored pair,
internal to the server, went awry, the CPU usage increased dramatically.
Not sure of the disk percentage though.
You state that the server was unusable, but you could still run commands
with no problems, correct?

TCP

#4 (**permalink**) 01-04-2008, 09:21 PM

Ian Northeast <ian@house-from-hell.demon.co.uk> wrote in message news:<pan.2004.01.16.00.13.22.628684@house-from-hell.demon.co.uk>...
> On Thu, 15 Jan 2004 10:44:52 -0800, Matt Raines wrote:
>
> > Last night we had a problem on our p660-6H1 where one of the disks in
> > our mirrored pair of internal scsi disks went to 100% busy and but
> > iostat did not show any reads or writes. As a result the server was
> > unusable until we rebooted it which did in fact fix the problem. This
> > particular server is 4.3.3 and has 6 cpus and 8 gbs of ram running
> > oracle 8i.
> >
> > Anyone else seen anything similar to this before on any scsi based
> > system?
>
> An operation (such as ls) on a directory with hundreds of thousands of
> files in it will make the disk go to 100% with no visible I/O. But IME it
> doesn't make the machine unusable. Maybe it does if it's in a system FS.
>
> Look for filesystems with abnormally high inodes used.
>
> Regards, Ian

I agree and have seen in the past on other systems, however the
filesystem with the largest number of inodes used is 9% which is ~
52,000 so I'm thinking that's not it.

Thanks.

#5 (**permalink**) 01-04-2008, 09:21 PM

************* <aixman@attbi.com> wrote in message news:<4007D184.435B65A8@attbi.com>...
> Matt Raines wrote:
>
> > Last night we had a problem on our p660-6H1 where one of the disks in
> > our mirrored pair of internal scsi disks went to 100% busy and but
> > iostat did not show any reads or writes. As a result the server was
> > unusable until we rebooted it which did in fact fix the problem. This
> > particular server is 4.3.3 and has 6 cpus and 8 gbs of ram running
> > oracle 8i.
> >
> > Anyone else seen anything similar to this before on any scsi based
> > system?
> >
> > TIA,
> >
> > Matt
>
> Did the errpt show anything?
>
> I remember a while back that when one of my disks in a mirrored pair,
> internal to the server, went awry, the CPU usage increased dramatically.
> Not sure of the disk percentage though.
> You state that the server was unusable, but you could still run commands
> with no problems, correct?
>
> TCP

That seemed to the be strangest part, nothing showed up the errpt.

You could run commands but it took a very long time to do so
especially if you had to write anything to disk. For instance, we were
editing a 3k file and it took 10 minutes to save it. The system kept
trying to sync the mirror because it could still see the disk and
thought everything was ok and I suppose it was except that performance
was very degraded.

We did contact IBM and created a PMR with them and they were as
stumped as we were. They suggested we update a bunch of microcodes but
did not suspect that to be a cause.

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

SEO

One of mirrored pair goes to 100% busy