Steve,

> Sun box with 4-disc array (4GB RAM. 4 167GB 10K SCSI RAID10
> LSI MegaRAID 128MB). This is after 8 runs.
>
> dbserver-dual-opteron-centos,08/08/06,Tuesday,20,us,12,2,5
> dbserver-dual-opteron-centos,08/08/06,Tuesday,20,sy,59,50,53
> dbserver-dual-opteron-centos,08/08/06,Tuesday,20,wa,1,0,0
> dbserver-dual-opteron-centos,08/08/06,Tuesday,20,id,45,26,38
>
> Average TPS is 75
>
> HP box with 8GB RAM. six disc array RAID10 on SmartArray 642
> with 192MB RAM. After 8 runs, I see:
>
> intown-vetstar-amd64,08/09/06,Tuesday,23,us,31,0,3
> intown-vetstar-amd64,08/09/06,Tuesday,23,sy,16,0,1
> intown-vetstar-amd64,08/09/06,Tuesday,23,wa,99,6,50
> intown-vetstar-amd64,08/09/06,Tuesday,23,id,78,0,42
>
> Average TPS is 31.

Note that the I/O wait (wa) on the HP box high, low and average are all
*much* higher than on the Sun box. The average I/O wait was 50% of one
CPU, which is huge. By comparison there was virtually no I/O wait on
the Sun machine.

This is indicating that your HP machine is indeed I/O bound and
furthermore is tying up a PG process waiting for the disk to return.

- Luke


---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?

http://archives.postgresql.org

Luke,

I thought so. In my test, I tried to be fair/equal since my Sun box has two
4-disc arrays each on their own channel. So, I just used one of them which
should be a little slower than the 6-disc with 192MB cache.

Incidently, the two internal SCSI drives, which are on the 6i adapter,
generated a TPS of 18.

I thought this server would impressive from notes I've read in the group.
This is why I thought I might be doing something wrong. I stumped which way
to take this. There is no obvious fault but something isn't right.

Steve

On 8/8/06, Luke Lonergan <LLonergan@greenplum.com> wrote:
>
> Steve,
>
> > Sun box with 4-disc array (4GB RAM. 4 167GB 10K SCSI RAID10
> > LSI MegaRAID 128MB). This is after 8 runs.
> >
> > dbserver-dual-opteron-centos,08/08/06,Tuesday,20,us,12,2,5
> > dbserver-dual-opteron-centos,08/08/06,Tuesday,20,sy,59,50,53
> > dbserver-dual-opteron-centos,08/08/06,Tuesday,20,wa,1,0,0
> > dbserver-dual-opteron-centos,08/08/06,Tuesday,20,id,45,26,38
> >
> > Average TPS is 75
> >
> > HP box with 8GB RAM. six disc array RAID10 on SmartArray 642
> > with 192MB RAM. After 8 runs, I see:
> >
> > intown-vetstar-amd64,08/09/06,Tuesday,23,us,31,0,3
> > intown-vetstar-amd64,08/09/06,Tuesday,23,sy,16,0,1
> > intown-vetstar-amd64,08/09/06,Tuesday,23,wa,99,6,50
> > intown-vetstar-amd64,08/09/06,Tuesday,23,id,78,0,42
> >
> > Average TPS is 31.
>
> Note that the I/O wait (wa) on the HP box high, low and average are all
> *much* higher than on the Sun box. The average I/O wait was 50% of one
> CPU, which is huge. By comparison there was virtually no I/O wait on
> the Sun machine.
>
> This is indicating that your HP machine is indeed I/O bound and
> furthermore is tying up a PG process waiting for the disk to return.
>
> - Luke
>
>

Luke,

I check dmesg one more time and I found this regarding the cciss driver:

Filesystem "cciss/c1d0p1": Disabling barriers, not supported by the
underlying device.

Don't know if it means anything, but thought I'd mention it.

Steve

On 8/8/06, Steve Poe <steve.poe@gmail.com> wrote:
>
> Luke,
>
> I thought so. In my test, I tried to be fair/equal since my Sun box has
> two 4-disc arrays each on their own channel. So, I just used one of them
> which should be a little slower than the 6-disc with 192MB cache.
>
> Incidently, the two internal SCSI drives, which are on the 6i adapter,
> generated a TPS of 18.
>
> I thought this server would impressive from notes I've read in the group.
> This is why I thought I might be doing something wrong. I stumped which way
> to take this. There is no obvious fault but something isn't right.
>
> Steve
>
>
> On 8/8/06, Luke Lonergan <LLonergan@greenplum.com> wrote:
> >
> > Steve,
> >
> > > Sun box with 4-disc array (4GB RAM. 4 167GB 10K SCSI RAID10
> > > LSI MegaRAID 128MB). This is after 8 runs.
> > >
> > > dbserver-dual-opteron-centos,08/08/06,Tuesday,20,us,12,2,5
> > > dbserver-dual-opteron-centos,08/08/06,Tuesday,20,sy,59,50,53
> > > dbserver-dual-opteron-centos,08/08/06,Tuesday,20,wa,1,0,0
> > > dbserver-dual-opteron-centos,08/08/06,Tuesday,20,id,45,26,38
> > >
> > > Average TPS is 75
> > >
> > > HP box with 8GB RAM. six disc array RAID10 on SmartArray 642
> > > with 192MB RAM. After 8 runs, I see:
> > >
> > > intown-vetstar-amd64,08/09/06,Tuesday,23,us,31,0,3
> > > intown-vetstar-amd64,08/09/06,Tuesday,23,sy,16,0,1
> > > intown-vetstar-amd64,08/09/06,Tuesday,23,wa,99,6,50
> > > intown-vetstar-amd64,08/09/06,Tuesday,23,id,78,0,42
> > >
> > > Average TPS is 31.
> >
> > Note that the I/O wait (wa) on the HP box high, low and average are all
> > *much* higher than on the Sun box. The average I/O wait was 50% of one
> > CPU, which is huge. By comparison there was virtually no I/O wait on
> > the Sun machine.
> >
> > This is indicating that your HP machine is indeed I/O bound and
> > furthermore is tying up a PG process waiting for the disk to return.
> >
> > - Luke
> >
> >
>

On Tue, Aug 08, 2006 at 10:45:07PM -0700, Steve Poe wrote:
> Luke,
>
> I thought so. In my test, I tried to be fair/equal since my Sun box has two
> 4-disc arrays each on their own channel. So, I just used one of them which
> should be a little slower than the 6-disc with 192MB cache.
>
> Incidently, the two internal SCSI drives, which are on the 6i adapter,
> generated a TPS of 18.

You should try putting pg_xlog on the 6 drive array with the data. My
(limited) experience with such a config is that on a good controller
with writeback caching enabled it won't hurt you, and if the internal
drives aren't caching writes it'll probably help you a lot.

> I thought this server would impressive from notes I've read in the group.
> This is why I thought I might be doing something wrong. I stumped which way
> to take this. There is no obvious fault but something isn't right.
>
> Steve
>
> On 8/8/06, Luke Lonergan <LLonergan@greenplum.com> wrote:
> >
> >Steve,
> >
> >> Sun box with 4-disc array (4GB RAM. 4 167GB 10K SCSI RAID10
> >> LSI MegaRAID 128MB). This is after 8 runs.
> >>
> >> dbserver-dual-opteron-centos,08/08/06,Tuesday,20,us,12,2,5
> >> dbserver-dual-opteron-centos,08/08/06,Tuesday,20,sy,59,50,53
> >> dbserver-dual-opteron-centos,08/08/06,Tuesday,20,wa,1,0,0
> >> dbserver-dual-opteron-centos,08/08/06,Tuesday,20,id,45,26,38
> >>
> >> Average TPS is 75
> >>
> >> HP box with 8GB RAM. six disc array RAID10 on SmartArray 642
> >> with 192MB RAM. After 8 runs, I see:
> >>
> >> intown-vetstar-amd64,08/09/06,Tuesday,23,us,31,0,3
> >> intown-vetstar-amd64,08/09/06,Tuesday,23,sy,16,0,1
> >> intown-vetstar-amd64,08/09/06,Tuesday,23,wa,99,6,50
> >> intown-vetstar-amd64,08/09/06,Tuesday,23,id,78,0,42
> >>
> >> Average TPS is 31.
> >
> >Note that the I/O wait (wa) on the HP box high, low and average are all
> >*much* higher than on the Sun box. The average I/O wait was 50% of one
> >CPU, which is huge. By comparison there was virtually no I/O wait on
> >the Sun machine.
> >
> >This is indicating that your HP machine is indeed I/O bound and
> >furthermore is tying up a PG process waiting for the disk to return.
> >
> >- Luke
> >
> >

--
Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com
Pervasive Software http://pervasive.com work: 512-231-6117
vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461

---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

Jim,

I'll give it a try. However, I did not see anywhere in the BIOS
configuration of the 642 RAID adapter to enable writeback. It may have been
mislabled cache accelerator where you can give a percentage to read/write.
That aspect did not change the performance like the LSI MegaRAID adapter
does.

Steve

On 8/9/06, Jim C. Nasby <jnasby@pervasive.com> wrote:
>
> On Tue, Aug 08, 2006 at 10:45:07PM -0700, Steve Poe wrote:
> > Luke,
> >
> > I thought so. In my test, I tried to be fair/equal since my Sun box has
> two
> > 4-disc arrays each on their own channel. So, I just used one of them
> which
> > should be a little slower than the 6-disc with 192MB cache.
> >
> > Incidently, the two internal SCSI drives, which are on the 6i adapter,
> > generated a TPS of 18.
>
> You should try putting pg_xlog on the 6 drive array with the data. My
> (limited) experience with such a config is that on a good controller
> with writeback caching enabled it won't hurt you, and if the internal
> drives aren't caching writes it'll probably help you a lot.
>
> > I thought this server would impressive from notes I've read in the
> group.
> > This is why I thought I might be doing something wrong. I stumped which
> way
> > to take this. There is no obvious fault but something isn't right.
> >
> > Steve
> >
> > On 8/8/06, Luke Lonergan <LLonergan@greenplum.com> wrote:
> > >
> > >Steve,
> > >
> > >> Sun box with 4-disc array (4GB RAM. 4 167GB 10K SCSI RAID10
> > >> LSI MegaRAID 128MB). This is after 8 runs.
> > >>
> > >> dbserver-dual-opteron-centos,08/08/06,Tuesday,20,us,12,2,5
> > >> dbserver-dual-opteron-centos,08/08/06,Tuesday,20,sy,59,50,53
> > >> dbserver-dual-opteron-centos,08/08/06,Tuesday,20,wa,1,0,0
> > >> dbserver-dual-opteron-centos,08/08/06,Tuesday,20,id,45,26,38
> > >>
> > >> Average TPS is 75
> > >>
> > >> HP box with 8GB RAM. six disc array RAID10 on SmartArray 642
> > >> with 192MB RAM. After 8 runs, I see:
> > >>
> > >> intown-vetstar-amd64,08/09/06,Tuesday,23,us,31,0,3
> > >> intown-vetstar-amd64,08/09/06,Tuesday,23,sy,16,0,1
> > >> intown-vetstar-amd64,08/09/06,Tuesday,23,wa,99,6,50
> > >> intown-vetstar-amd64,08/09/06,Tuesday,23,id,78,0,42
> > >>
> > >> Average TPS is 31.
> > >
> > >Note that the I/O wait (wa) on the HP box high, low and average are all
> > >*much* higher than on the Sun box. The average I/O wait was 50% of one
> > >CPU, which is huge. By comparison there was virtually no I/O wait on
> > >the Sun machine.
> > >
> > >This is indicating that your HP machine is indeed I/O bound and
> > >furthermore is tying up a PG process waiting for the disk to return.
> > >
> > >- Luke
> > >
> > >
>
> --
> Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com
> Pervasive Software http://pervasive.com work: 512-231-6117
> vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461
>

On Wed, 2006-08-09 at 16:11, Steve Poe wrote:
> Jim,
>
> I'll give it a try. However, I did not see anywhere in the BIOS
> configuration of the 642 RAID adapter to enable writeback. It may have
> been mislabled cache accelerator where you can give a percentage to
> read/write. That aspect did not change the performance like the LSI
> MegaRAID adapter does.

Nope, that's not the same thing.

Does your raid controller have batter backed cache, or plain or regular
cache? write back is unsafe without battery backup.

The default is write through (i.e. the card waits for the data to get
written out before acking an fsync). In write back, the card's driver
writes the data to the bb cache, then returns on an fsync while the
cache gets written out at leisure. In the event of a loss of power, the
cache is flushed on restart.

---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend

I believe it does, I'll need to check.Thanks for the correction.

Steve

On 8/9/06, Scott Marlowe <smarlowe@g2switchworks.com> wrote:
>
> On Wed, 2006-08-09 at 16:11, Steve Poe wrote:
> > Jim,
> >
> > I'll give it a try. However, I did not see anywhere in the BIOS
> > configuration of the 642 RAID adapter to enable writeback. It may have
> > been mislabled cache accelerator where you can give a percentage to
> > read/write. That aspect did not change the performance like the LSI
> > MegaRAID adapter does.
>
> Nope, that's not the same thing.
>
> Does your raid controller have batter backed cache, or plain or regular
> cache? write back is unsafe without battery backup.
>
> The default is write through (i.e. the card waits for the data to get
> written out before acking an fsync). In write back, the card's driver
> writes the data to the bb cache, then returns on an fsync while the
> cache gets written out at leisure. In the event of a loss of power, the
> cache is flushed on restart.
>

Scott,

Do you know how to activate the writeback on the RAID controller from HP?

Steve

On 8/9/06, Scott Marlowe <smarlowe@g2switchworks.com> wrote:
>
> On Wed, 2006-08-09 at 16:11, Steve Poe wrote:
> > Jim,
> >
> > I'll give it a try. However, I did not see anywhere in the BIOS
> > configuration of the 642 RAID adapter to enable writeback. It may have
> > been mislabled cache accelerator where you can give a percentage to
> > read/write. That aspect did not change the performance like the LSI
> > MegaRAID adapter does.
>
> Nope, that's not the same thing.
>
> Does your raid controller have batter backed cache, or plain or regular
> cache? write back is unsafe without battery backup.
>
> The default is write through (i.e. the card waits for the data to get
> written out before acking an fsync). In write back, the card's driver
> writes the data to the bb cache, then returns on an fsync while the
> cache gets written out at leisure. In the event of a loss of power, the
> cache is flushed on restart.
>

Jim,

I tried as you suggested and my performance dropped by 50%. I went from
a 32 TPS to 16. Oh well.

Steve

On Wed, 2006-08-09 at 16:05 -0500, Jim C. Nasby wrote:
> On Tue, Aug 08, 2006 at 10:45:07PM -0700, Steve Poe wrote:
> > Luke,
> >
> > I thought so. In my test, I tried to be fair/equal since my Sun box has two
> > 4-disc arrays each on their own channel. So, I just used one of them which
> > should be a little slower than the 6-disc with 192MB cache.
> >
> > Incidently, the two internal SCSI drives, which are on the 6i adapter,
> > generated a TPS of 18.
>
> You should try putting pg_xlog on the 6 drive array with the data. My
> (limited) experience with such a config is that on a good controller
> with writeback caching enabled it won't hurt you, and if the internal
> drives aren't caching writes it'll probably help you a lot.
>
> > I thought this server would impressive from notes I've read in the group.
> > This is why I thought I might be doing something wrong. I stumped which way
> > to take this. There is no obvious fault but something isn't right.
> >
> > Steve
> >
> > On 8/8/06, Luke Lonergan <LLonergan@greenplum.com> wrote:
> > >
> > >Steve,
> > >
> > >> Sun box with 4-disc array (4GB RAM. 4 167GB 10K SCSI RAID10
> > >> LSI MegaRAID 128MB). This is after 8 runs.
> > >>
> > >> dbserver-dual-opteron-centos,08/08/06,Tuesday,20,us,12,2,5
> > >> dbserver-dual-opteron-centos,08/08/06,Tuesday,20,sy,59,50,53
> > >> dbserver-dual-opteron-centos,08/08/06,Tuesday,20,wa,1,0,0
> > >> dbserver-dual-opteron-centos,08/08/06,Tuesday,20,id,45,26,38
> > >>
> > >> Average TPS is 75
> > >>
> > >> HP box with 8GB RAM. six disc array RAID10 on SmartArray 642
> > >> with 192MB RAM. After 8 runs, I see:
> > >>
> > >> intown-vetstar-amd64,08/09/06,Tuesday,23,us,31,0,3
> > >> intown-vetstar-amd64,08/09/06,Tuesday,23,sy,16,0,1
> > >> intown-vetstar-amd64,08/09/06,Tuesday,23,wa,99,6,50
> > >> intown-vetstar-amd64,08/09/06,Tuesday,23,id,78,0,42
> > >>
> > >> Average TPS is 31.
> > >
> > >Note that the I/O wait (wa) on the HP box high, low and average are all
> > >*much* higher than on the Sun box. The average I/O wait was 50% of one
> > >CPU, which is huge. By comparison there was virtually no I/O wait on
> > >the Sun machine.
> > >
> > >This is indicating that your HP machine is indeed I/O bound and
> > >furthermore is tying up a PG process waiting for the disk to return.
> > >
> > >- Luke
> > >
> > >
>


---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?

http://archives.postgresql.org

On Wed, Aug 09, 2006 at 08:29:13PM -0700, Steve Poe wrote:
>I tried as you suggested and my performance dropped by 50%. I went from
>a 32 TPS to 16. Oh well.

If you put data & xlog on the same array, put them on seperate
partitions, probably formatted differently (ext2 on xlog).

Mike Stone

---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?

http://archives.postgresql.org