Sun Donated a Sun Fire T2000 to the PostgreSQL community

I am thrill to inform you all that Sun has just donated a fully loaded
T2000 system to the PostgreSQL community, and it's being setup by Corey
Shields at OSL (osuosl.org) and should be online probably early next
week. The system has

* 8 cores, 4 hw threads/core @ 1.2 GHz. Solaris sees the system as
having 32 virtual CPUs, and each can be enabled or disabled individually
* 32 GB of DDR2 SDRAM memory
* 2 @ 73GB internal SAS drives (10000 RPM)
* 4 Gigabit ethernet ports

For complete spec, visit
http://www.sun.com/servers/coolthrea...ifications.jsp

I think this system is well suited for PG scalability testing, among
others. We did an informal test using an internal OLTP benchmark and
noticed that PG can scale to around 8 CPUs. Would be really cool if all
32 virtual CPUs can be utilized!!!

Anyways, if you need to access the system for testing purposes, please
contact Josh Berkus.

Regards,

Robert Lor
Sun Microsystems, Inc.
01-510-574-7189




---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?

http://archives.postgresql.org

Folks,

> I am thrill to inform you all that Sun has just donated a fully loaded
> T2000 system to the PostgreSQL community, and it's being setup by Corey
> Shields at OSL (osuosl.org) and should be online probably early next
> week. The system has

So this system will be hosted by Open Source Lab in Oregon. It's going to
be "donated" to Software In the Public Interest, who will own for the
PostgreSQL fund.

We'll want to figure out a scheduling system to schedule performance and
compatibility testing on this machine; I'm not sure exactly how that will
work. Suggestions welcome. As a warning, Gavin Sherry and I have a bunch
of pending tests already to run.

First thing as soon as I have a login, of course, is to set up a Buildfarm
instance.

--
--Josh

Josh Berkus
PostgreSQL @ Sun
San Francisco

---------------------------(end of broadcast)---------------------------
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match

On 16-6-2006 17:18, Robert Lor wrote:
>
> I think this system is well suited for PG scalability testing, among
> others. We did an informal test using an internal OLTP benchmark and
> noticed that PG can scale to around 8 CPUs. Would be really cool if all
> 32 virtual CPUs can be utilized!!!

I can already confirm very good scalability (with our workload) on
postgresql on that machine. We've been testing a 32thread/16G-version
and it shows near-linear scaling when enabling 1, 2, 4, 6 and 8 cores
(with all four threads enabled).

The threads are a bit less scalable, but still pretty good. Enabling 1,
2 or 4 threads for each core yields resp 60 and 130% extra performance.

Best regards,

Arjen

---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

Arjen,

> I can already confirm very good scalability (with our workload) on
> postgresql on that machine. We've been testing a 32thread/16G-version
> and it shows near-linear scaling when enabling 1, 2, 4, 6 and 8 cores
> (with all four threads enabled).

Keen. We're trying to keep the linear scaling going up to 32 cores of
course (which doesn't happen, presently). Would you be interested in
helping us troubleshoot some of the performance issues?

--
--Josh

Josh Berkus
PostgreSQL @ Sun
San Francisco

---------------------------(end of broadcast)---------------------------
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match

> I am thrill to inform you all that Sun has just donated a fully loaded
> T2000 system to the PostgreSQL community, and it's being setup by Corey
> Shields at OSL (osuosl.org) and should be online probably early next
> week. The system has
>
> * 8 cores, 4 hw threads/core @ 1.2 GHz. Solaris sees the system as
> having 32 virtual CPUs, and each can be enabled or disabled individually
> * 32 GB of DDR2 SDRAM memory
> * 2 @ 73GB internal SAS drives (10000 RPM)
> * 4 Gigabit ethernet ports
>
> For complete spec, visit
> http://www.sun.com/servers/coolthrea...ifications.jsp
>
> I think this system is well suited for PG scalability testing, among
> others. We did an informal test using an internal OLTP benchmark and
> noticed that PG can scale to around 8 CPUs. Would be really cool if all
> 32 virtual CPUs can be utilized!!!

Interesting. We (some Japanese companies including SRA OSS,
Inc. Japan) did some PG scalability testing using a Unisys's big 16
(physical) CPU machine and found PG scales up to 8 CPUs. However
beyond 8 CPU PG does not scale anymore. The result can be viewed at
"OSS iPedia" web site (http://ossipedia.ipa.go.jp). Our conclusion was
PG has a serious lock contention problem in the environment by
analyzing the oprofile result.

You can take a look at the detailed report at:
http://ossipedia.ipa.go.jp/capacity/EV0604210111/
(unfortunately only Japanese contents is available at the
moment. Please use some automatic translation services)

Evalution environment was:
PostgreSQL 8.1.2
OSDL DBT-1 2.1
Miracle Linux 4.0
Unisys ES700 Xeon 2.8GHz CPU x 16 Mem 16GB(HT off)
--
Tatsuo Ishii
SRA OSS, Inc. Japan

---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings

Tatsuo Ishii <ishii@sraoss.co.jp> writes:
> Interesting. We (some Japanese companies including SRA OSS,
> Inc. Japan) did some PG scalability testing using a Unisys's big 16
> (physical) CPU machine and found PG scales up to 8 CPUs. However
> beyond 8 CPU PG does not scale anymore. The result can be viewed at
> "OSS iPedia" web site (http://ossipedia.ipa.go.jp). Our conclusion was
> PG has a serious lock contention problem in the environment by
> analyzing the oprofile result.

18% in s_lock is definitely bad :-(. Were you able to determine which
LWLock(s) are accounting for the contention?

The test case seems to be spending a remarkable amount of time in LIKE
comparisons, too. That probably is not a representative condition.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster

> Tatsuo Ishii <ishii@sraoss.co.jp> writes:
> > Interesting. We (some Japanese companies including SRA OSS,
> > Inc. Japan) did some PG scalability testing using a Unisys's big 16
> > (physical) CPU machine and found PG scales up to 8 CPUs. However
> > beyond 8 CPU PG does not scale anymore. The result can be viewed at
> > "OSS iPedia" web site (http://ossipedia.ipa.go.jp). Our conclusion was
> > PG has a serious lock contention problem in the environment by
> > analyzing the oprofile result.
>
> 18% in s_lock is definitely bad :-(. Were you able to determine which
> LWLock(s) are accounting for the contention?

Yes. We were interested in that too. Some people did addtional tests
to determin that. I don't have the report handy now. I will report
back next week.

> The test case seems to be spending a remarkable amount of time in LIKE
> comparisons, too. That probably is not a representative condition.

I know. I think point is 18% in s_lock only appears with 12 CPUs or more.
--
Tatsuo Ishii
SRA OSS, Inc. Japan

---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?

http://archives.postgresql.org

Arjen van der Meijden wrote:

>
> I can already confirm very good scalability (with our workload) on
> postgresql on that machine. We've been testing a 32thread/16G-version
> and it shows near-linear scaling when enabling 1, 2, 4, 6 and 8 cores
> (with all four threads enabled).
>
> The threads are a bit less scalable, but still pretty good. Enabling
> 1, 2 or 4 threads for each core yields resp 60 and 130% extra
> performance.

Wow, what type of workload is it? And did you do much tuning to get
near-linear scalability to 32 threads?

Regards,
-Robert

---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

Tom,

> 18% in s_lock is definitely bad :-(. Â*Were you able to determine which
> LWLock(s) are accounting for the contention?

Gavin Sherry and Tom Daly (Sun) are currently working on identifying the
problem lock using DLWLOCK_STATS. Any luck, Gavin?

--
Josh Berkus
PostgreSQL @ Sun
San Francisco

---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?

http://archives.postgresql.org

On Jun 16, 2006, at 12:01 PM, Josh Berkus wrote:

> Folks,
>
>> I am thrill to inform you all that Sun has just donated a fully
>> loaded
>> T2000 system to the PostgreSQL community, and it's being setup by
>> Corey
>> Shields at OSL (osuosl.org) and should be online probably early next
>> week. The system has
>
> So this system will be hosted by Open Source Lab in Oregon. It's
> going to
> be "donated" to Software In the Public Interest, who will own for the
> PostgreSQL fund.
>
> We'll want to figure out a scheduling system to schedule
> performance and
> compatibility testing on this machine; I'm not sure exactly how
> that will
> work. Suggestions welcome. As a warning, Gavin Sherry and I have
> a bunch
> of pending tests already to run.
>
> First thing as soon as I have a login, of course, is to set up a
> Buildfarm
> instance.
>
> --
> --Josh
>
> Josh Berkus
> PostgreSQL @ Sun
> San Francisco
>
> ---------------------------(end of
> broadcast)---------------------------
> TIP 1: if posting/reading through Usenet, please send an appropriate
> subscribe-nomail command to majordomo@postgresql.org so that
> your
> message can get through to the mailing list cleanly
>

--
Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com
Pervasive Software http://pervasive.com work: 512-231-6117
vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461



---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly