Unix Technical Forum

SEO

vBulletin Search Engine Optimization


Go Back   Unix Technical Forum > Database Server Software > PostgreSQL > Pgsql Performance

Register FAQ Members List Calendar Search Today's Posts Mark Forums Read
  #1 (permalink)  
Old 04-19-2008, 09:18 AM
Alex Deucher
 
Posts: n/a
Default strange performance regression between 7.4 and 8.1

Hello,

I have noticed a strange performance regression and I'm at a loss as
to what's happening. We have a fairly large database (~16 GB). The
original postgres 7.4 was running on a sun v880 with 4 CPUs and 8 GB
of ram running Solaris on local scsi discs. The new server is a sun
Opteron box with 4 cores, 8 GB of ram running postgres 8.1.4 on Linux
(AMD64) on a 4 Gbps FC SAN volume. When we created the new database
it was created from scratch rather than copying over the old one,
however the table structure is almost identical (UTF8 on the new one
vs. C on the old). The problem is queries are ~10x slower on the new
hardware. I read several places that the SAN might be to blame, but
testing with bonnie and dd indicates that the SAN is actually almost
twice as fast as the scsi discs in the old sun server. I've tried
adjusting just about every option in the postgres config file, but
performance remains the same. Any ideas?

Thanks,

Alex

---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #2 (permalink)  
Old 04-19-2008, 09:18 AM
Joshua D. Drake
 
Posts: n/a
Default Re: strange performance regression between 7.4 and 8.1

Alex Deucher wrote:
> Hello,
>
> I have noticed a strange performance regression and I'm at a loss as
> to what's happening. We have a fairly large database (~16 GB). The
> original postgres 7.4 was running on a sun v880 with 4 CPUs and 8 GB
> of ram running Solaris on local scsi discs. The new server is a sun
> Opteron box with 4 cores, 8 GB of ram running postgres 8.1.4 on Linux
> (AMD64) on a 4 Gbps FC SAN volume. When we created the new database
> it was created from scratch rather than copying over the old one,
> however the table structure is almost identical (UTF8 on the new one
> vs. C on the old). The problem is queries are ~10x slower on the new
> hardware. I read several places that the SAN might be to blame, but
> testing with bonnie and dd indicates that the SAN is actually almost
> twice as fast as the scsi discs in the old sun server. I've tried
> adjusting just about every option in the postgres config file, but
> performance remains the same. Any ideas?


Vacuum? Analayze? default_statistics_target? How many shared_buffers?
effective_cache_size? work_mem?

Sincerely,

Joshua D. Drake


>
> Thanks,
>
> Alex
>
> ---------------------------(end of broadcast)---------------------------
> TIP 3: Have you checked our extensive FAQ?
>
> http://www.postgresql.org/docs/faq
>



--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL Replication: http://www.commandprompt.com/products/


---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?

http://archives.postgresql.org

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #3 (permalink)  
Old 04-19-2008, 09:18 AM
Jeff Frost
 
Posts: n/a
Default Re: strange performance regression between 7.4 and 8.1

On Thu, 1 Mar 2007, Joshua D. Drake wrote:

> Alex Deucher wrote:
>> Hello,
>>
>> I have noticed a strange performance regression and I'm at a loss as
>> to what's happening. We have a fairly large database (~16 GB). The
>> original postgres 7.4 was running on a sun v880 with 4 CPUs and 8 GB
>> of ram running Solaris on local scsi discs. The new server is a sun
>> Opteron box with 4 cores, 8 GB of ram running postgres 8.1.4 on Linux
>> (AMD64) on a 4 Gbps FC SAN volume. When we created the new database
>> it was created from scratch rather than copying over the old one,
>> however the table structure is almost identical (UTF8 on the new one
>> vs. C on the old). The problem is queries are ~10x slower on the new
>> hardware. I read several places that the SAN might be to blame, but
>> testing with bonnie and dd indicates that the SAN is actually almost
>> twice as fast as the scsi discs in the old sun server. I've tried
>> adjusting just about every option in the postgres config file, but
>> performance remains the same. Any ideas?

>
> Vacuum? Analayze? default_statistics_target? How many shared_buffers?
> effective_cache_size? work_mem?


Also, an explain analyze from both the 7.4 and 8.1 systems with one of the
10x slower queries would probably be handy.

What do you mean by "created from scratch rather than copying over the old
one"? How did you put the data in? Did you run analyze after loading it?
Is autovacuum enabled and if so, what are the thresholds?

--
Jeff Frost, Owner <jeff@frostconsultingllc.com>
Frost Consulting, LLC http://www.frostconsultingllc.com/
Phone: 650-780-7908 FAX: 650-649-1954

---------------------------(end of broadcast)---------------------------
TIP 7: You can help support the PostgreSQL project by donating at

http://www.postgresql.org/about/donate

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #4 (permalink)  
Old 04-19-2008, 09:18 AM
Alex Deucher
 
Posts: n/a
Default Re: strange performance regression between 7.4 and 8.1

On 3/1/07, Joshua D. Drake <jd@commandprompt.com> wrote:
> Alex Deucher wrote:
> > Hello,
> >
> > I have noticed a strange performance regression and I'm at a loss as
> > to what's happening. We have a fairly large database (~16 GB). The
> > original postgres 7.4 was running on a sun v880 with 4 CPUs and 8 GB
> > of ram running Solaris on local scsi discs. The new server is a sun
> > Opteron box with 4 cores, 8 GB of ram running postgres 8.1.4 on Linux
> > (AMD64) on a 4 Gbps FC SAN volume. When we created the new database
> > it was created from scratch rather than copying over the old one,
> > however the table structure is almost identical (UTF8 on the new one
> > vs. C on the old). The problem is queries are ~10x slower on the new
> > hardware. I read several places that the SAN might be to blame, but
> > testing with bonnie and dd indicates that the SAN is actually almost
> > twice as fast as the scsi discs in the old sun server. I've tried
> > adjusting just about every option in the postgres config file, but
> > performance remains the same. Any ideas?

>
> Vacuum? Analayze? default_statistics_target? How many shared_buffers?
> effective_cache_size? work_mem?
>


I'm running the autovacuum process on the 8.1 server. vacuuming on
the old server was done manually.

default_statistics_target and effective_cache_size are set to the the
defaults on both.

postgres 7.4 server:
# - Memory -
shared_buffers = 82000 # 1000 min 16, at least
max_connections*2, 8KB each
sort_mem = 8000 # 1024 min 64, size in KB
vacuum_mem = 32000 # 8192 min 1024, size in KB
# - Free Space Map -
#max_fsm_pages = 20000 # min max_fsm_relations*16, 6 bytes each
#max_fsm_relations = 1000 # min 100, ~50 bytes each
# - Kernel Resource Usage -
#max_files_per_process = 1000 # min 25

postgres 8.1 server:
# - Memory -
shared_buffers = 100000 # min 16 or max_connections*2, 8KB each
temp_buffers = 2000 #1000 # min 100, 8KB each
max_prepared_transactions = 100 #5 # can be 0 or more
# note: increasing max_prepared_transactions costs ~600 bytes of shared memory
# per transaction slot, plus lock space (see max_locks_per_transaction).
work_mem = 10000 #1024 # min 64, size in KB
maintenance_work_mem = 524288 #16384 # min 1024, size in KB
#max_stack_depth = 2048 # min 100, size in KB

I've also tried using the same settings from the old server on the new
one; same performance issues.

Thanks,

Alex

> Sincerely,
>
> Joshua D. Drake
>
>
> >
> > Thanks,
> >
> > Alex
> >
> > ---------------------------(end of broadcast)---------------------------
> > TIP 3: Have you checked our extensive FAQ?
> >
> > http://www.postgresql.org/docs/faq
> >

>
>
> --
>
> === The PostgreSQL Company: Command Prompt, Inc. ===
> Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
> Providing the most comprehensive PostgreSQL solutions since 1997
> http://www.commandprompt.com/
>
> Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
> PostgreSQL Replication: http://www.commandprompt.com/products/
>
>


---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #5 (permalink)  
Old 04-19-2008, 09:18 AM
Alex Deucher
 
Posts: n/a
Default Re: strange performance regression between 7.4 and 8.1

On 3/1/07, Jeff Frost <jeff@frostconsultingllc.com> wrote:
> On Thu, 1 Mar 2007, Joshua D. Drake wrote:
>
> > Alex Deucher wrote:
> >> Hello,
> >>
> >> I have noticed a strange performance regression and I'm at a loss as
> >> to what's happening. We have a fairly large database (~16 GB). The
> >> original postgres 7.4 was running on a sun v880 with 4 CPUs and 8 GB
> >> of ram running Solaris on local scsi discs. The new server is a sun
> >> Opteron box with 4 cores, 8 GB of ram running postgres 8.1.4 on Linux
> >> (AMD64) on a 4 Gbps FC SAN volume. When we created the new database
> >> it was created from scratch rather than copying over the old one,
> >> however the table structure is almost identical (UTF8 on the new one
> >> vs. C on the old). The problem is queries are ~10x slower on the new
> >> hardware. I read several places that the SAN might be to blame, but
> >> testing with bonnie and dd indicates that the SAN is actually almost
> >> twice as fast as the scsi discs in the old sun server. I've tried
> >> adjusting just about every option in the postgres config file, but
> >> performance remains the same. Any ideas?

> >
> > Vacuum? Analayze? default_statistics_target? How many shared_buffers?
> > effective_cache_size? work_mem?

>
> Also, an explain analyze from both the 7.4 and 8.1 systems with one of the
> 10x slower queries would probably be handy.
>


I'll run some and get back to you.

> What do you mean by "created from scratch rather than copying over the old
> one"? How did you put the data in? Did you run analyze after loading it?
> Is autovacuum enabled and if so, what are the thresholds?


Both the databases were originally created from xml files. We just
re-created the new one from the xml rather than copying the old
database over. I didn't manually run analyze on it, but we are
running the autovacuum process:

autovacuum = on #off # enable autovacuum subprocess?
autovacuum_naptime = 360 #60 # time between autovacuum runs, in secs
autovacuum_vacuum_threshold = 10000 #1000 # min # of tuple updates before
# vacuum
autovacuum_analyze_threshold = 5000 #500 # min # of tuple updates before

Thanks,

Alex

>
> --
> Jeff Frost, Owner <jeff@frostconsultingllc.com>
> Frost Consulting, LLC http://www.frostconsultingllc.com/
> Phone: 650-780-7908 FAX: 650-649-1954
>


---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #6 (permalink)  
Old 04-19-2008, 09:18 AM
Jeff Frost
 
Posts: n/a
Default Re: strange performance regression between 7.4 and 8.1

On Thu, 1 Mar 2007, Alex Deucher wrote:

> On 3/1/07, Jeff Frost <jeff@frostconsultingllc.com> wrote:
>> On Thu, 1 Mar 2007, Joshua D. Drake wrote:
>>
>> > Alex Deucher wrote:
>> >> Hello,
>> >>
>> >> I have noticed a strange performance regression and I'm at a loss as
>> >> to what's happening. We have a fairly large database (~16 GB). The
>> >> original postgres 7.4 was running on a sun v880 with 4 CPUs and 8 GB
>> >> of ram running Solaris on local scsi discs. The new server is a sun
>> >> Opteron box with 4 cores, 8 GB of ram running postgres 8.1.4 on Linux
>> >> (AMD64) on a 4 Gbps FC SAN volume. When we created the new database
>> >> it was created from scratch rather than copying over the old one,
>> >> however the table structure is almost identical (UTF8 on the new one
>> >> vs. C on the old). The problem is queries are ~10x slower on the new
>> >> hardware. I read several places that the SAN might be to blame, but
>> >> testing with bonnie and dd indicates that the SAN is actually almost
>> >> twice as fast as the scsi discs in the old sun server. I've tried
>> >> adjusting just about every option in the postgres config file, but
>> >> performance remains the same. Any ideas?
>> >
>> > Vacuum? Analayze? default_statistics_target? How many shared_buffers?
>> > effective_cache_size? work_mem?

>>
>> Also, an explain analyze from both the 7.4 and 8.1 systems with one of the
>> 10x slower queries would probably be handy.
>>

>
> I'll run some and get back to you.
>
>> What do you mean by "created from scratch rather than copying over the old
>> one"? How did you put the data in? Did you run analyze after loading it?
>> Is autovacuum enabled and if so, what are the thresholds?

>
> Both the databases were originally created from xml files. We just
> re-created the new one from the xml rather than copying the old
> database over. I didn't manually run analyze on it, but we are
> running the autovacuum process:


You should probably manually run analyze and see if that resolves your
problem.

>
> autovacuum = on #off # enable autovacuum subprocess?
> autovacuum_naptime = 360 #60 # time between autovacuum runs, in
> secs
> autovacuum_vacuum_threshold = 10000 #1000 # min # of tuple updates
> before
> # vacuum
> autovacuum_analyze_threshold = 5000 #500 # min # of tuple updates
> before


Most people make autovacuum more aggressive and not less aggressive. In fact,
the new defaults in 8.2 are:

#autovacuum_vacuum_threshold = 500 # min # of tuple updates before
# vacuum
#autovacuum_analyze_threshold = 250 # min # of tuple updates before
# analyze
#autovacuum_vacuum_scale_factor = 0.2 # fraction of rel size before
# vacuum
#autovacuum_analyze_scale_factor = 0.1 # fraction of rel size before

I'd recommend trying those, otherwise you might not vacuum enough.

It'll be interesting to see the explain analyze output after you've run
analyze by hand.

--
Jeff Frost, Owner <jeff@frostconsultingllc.com>
Frost Consulting, LLC http://www.frostconsultingllc.com/
Phone: 650-780-7908 FAX: 650-649-1954

---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #7 (permalink)  
Old 04-19-2008, 09:18 AM
Jeff Frost
 
Posts: n/a
Default Re: strange performance regression between 7.4 and 8.1

On Thu, 1 Mar 2007, Alex Deucher wrote:

>> Vacuum? Analayze? default_statistics_target? How many shared_buffers?
>> effective_cache_size? work_mem?
>>

>
> I'm running the autovacuum process on the 8.1 server. vacuuming on
> the old server was done manually.
>
> default_statistics_target and effective_cache_size are set to the the
> defaults on both.
>
> postgres 7.4 server:
> # - Memory -
> shared_buffers = 82000 # 1000 min 16, at least
> max_connections*2, 8KB each
> sort_mem = 8000 # 1024 min 64, size in KB
> vacuum_mem = 32000 # 8192 min 1024, size in KB
> # - Free Space Map -
> #max_fsm_pages = 20000 # min max_fsm_relations*16, 6 bytes each
> #max_fsm_relations = 1000 # min 100, ~50 bytes each
> # - Kernel Resource Usage -
> #max_files_per_process = 1000 # min 25
>
> postgres 8.1 server:
> # - Memory -
> shared_buffers = 100000 # min 16 or max_connections*2, 8KB
> each
> temp_buffers = 2000 #1000 # min 100, 8KB each
> max_prepared_transactions = 100 #5 # can be 0 or more
> # note: increasing max_prepared_transactions costs ~600 bytes of shared
> memory
> # per transaction slot, plus lock space (see max_locks_per_transaction).
> work_mem = 10000 #1024 # min 64, size in KB
> maintenance_work_mem = 524288 #16384 # min 1024, size in KB
> #max_stack_depth = 2048 # min 100, size in KB
>
> I've also tried using the same settings from the old server on the new
> one; same performance issues.
>


If this is a linux system, could you give us the output of the 'free' command?
Postgresql might be choosing a bad plan because your effective_cache_size is
way off (it's the default now right?). Also, what was the block read/write
speed of the SAN from your bonnie tests? Probably want to tune
random_page_cost as well if it's also at the default.


--
Jeff Frost, Owner <jeff@frostconsultingllc.com>
Frost Consulting, LLC http://www.frostconsultingllc.com/
Phone: 650-780-7908 FAX: 650-649-1954

---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #8 (permalink)  
Old 04-19-2008, 09:18 AM
Alex Deucher
 
Posts: n/a
Default Re: strange performance regression between 7.4 and 8.1

On 3/1/07, Jeff Frost <jeff@frostconsultingllc.com> wrote:
> On Thu, 1 Mar 2007, Alex Deucher wrote:
>
> >> Vacuum? Analayze? default_statistics_target? How many shared_buffers?
> >> effective_cache_size? work_mem?
> >>

> >
> > I'm running the autovacuum process on the 8.1 server. vacuuming on
> > the old server was done manually.
> >
> > default_statistics_target and effective_cache_size are set to the the
> > defaults on both.
> >
> > postgres 7.4 server:
> > # - Memory -
> > shared_buffers = 82000 # 1000 min 16, at least
> > max_connections*2, 8KB each
> > sort_mem = 8000 # 1024 min 64, size in KB
> > vacuum_mem = 32000 # 8192 min 1024, size in KB
> > # - Free Space Map -
> > #max_fsm_pages = 20000 # min max_fsm_relations*16, 6 bytes each
> > #max_fsm_relations = 1000 # min 100, ~50 bytes each
> > # - Kernel Resource Usage -
> > #max_files_per_process = 1000 # min 25
> >
> > postgres 8.1 server:
> > # - Memory -
> > shared_buffers = 100000 # min 16 or max_connections*2, 8KB
> > each
> > temp_buffers = 2000 #1000 # min 100, 8KB each
> > max_prepared_transactions = 100 #5 # can be 0 or more
> > # note: increasing max_prepared_transactions costs ~600 bytes of shared
> > memory
> > # per transaction slot, plus lock space (see max_locks_per_transaction).
> > work_mem = 10000 #1024 # min 64, size in KB
> > maintenance_work_mem = 524288 #16384 # min 1024, size in KB
> > #max_stack_depth = 2048 # min 100, size in KB
> >
> > I've also tried using the same settings from the old server on the new
> > one; same performance issues.
> >

>
> If this is a linux system, could you give us the output of the 'free' command?


total used free shared buffers cached
Mem: 8059852 8042868 16984 0 228 7888648
-/+ buffers/cache: 153992 7905860
Swap: 15631224 2164 15629060


> Postgresql might be choosing a bad plan because your effective_cache_size is
> way off (it's the default now right?). Also, what was the block read/write


yes it's set to the default.

> speed of the SAN from your bonnie tests? Probably want to tune
> random_page_cost as well if it's also at the default.
>


------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
luna12-san 16000M 58896 91 62931 9 35870 5 54869 82 145504 13 397.7 0

effective_cache_size is the default.

Alex

---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #9 (permalink)  
Old 04-19-2008, 09:18 AM
Jeff Frost
 
Posts: n/a
Default Re: strange performance regression between 7.4 and 8.1

On Thu, 1 Mar 2007, Alex Deucher wrote:

> On 3/1/07, Jeff Frost <jeff@frostconsultingllc.com> wrote:
>> On Thu, 1 Mar 2007, Alex Deucher wrote:
>>
>> >> Vacuum? Analayze? default_statistics_target? How many shared_buffers?
>> >> effective_cache_size? work_mem?
>> >>
>> >
>> > I'm running the autovacuum process on the 8.1 server. vacuuming on
>> > the old server was done manually.
>> >
>> > default_statistics_target and effective_cache_size are set to the the
>> > defaults on both.
>> >
>> > postgres 7.4 server:
>> > # - Memory -
>> > shared_buffers = 82000 # 1000 min 16, at least
>> > max_connections*2, 8KB each
>> > sort_mem = 8000 # 1024 min 64, size in KB
>> > vacuum_mem = 32000 # 8192 min 1024, size in KB
>> > # - Free Space Map -
>> > #max_fsm_pages = 20000 # min max_fsm_relations*16, 6 bytes each
>> > #max_fsm_relations = 1000 # min 100, ~50 bytes each
>> > # - Kernel Resource Usage -
>> > #max_files_per_process = 1000 # min 25
>> >
>> > postgres 8.1 server:
>> > # - Memory -
>> > shared_buffers = 100000 # min 16 or max_connections*2,

>> 8KB
>> > each
>> > temp_buffers = 2000 #1000 # min 100, 8KB each
>> > max_prepared_transactions = 100 #5 # can be 0 or more
>> > # note: increasing max_prepared_transactions costs ~600 bytes of shared
>> > memory
>> > # per transaction slot, plus lock space (see max_locks_per_transaction).
>> > work_mem = 10000 #1024 # min 64, size in KB
>> > maintenance_work_mem = 524288 #16384 # min 1024, size in KB
>> > #max_stack_depth = 2048 # min 100, size in KB
>> >
>> > I've also tried using the same settings from the old server on the new
>> > one; same performance issues.
>> >

>>
>> If this is a linux system, could you give us the output of the 'free'
>> command?

>
> total used free shared buffers cached
> Mem: 8059852 8042868 16984 0 228 7888648
> -/+ buffers/cache: 153992 7905860
> Swap: 15631224 2164 15629060


So, I would set effective_cache_size = 988232 (7905860/8).

>
>> Postgresql might be choosing a bad plan because your effective_cache_size
>> is
>> way off (it's the default now right?). Also, what was the block read/write

>
> yes it's set to the default.
>
>> speed of the SAN from your bonnie tests? Probably want to tune
>> random_page_cost as well if it's also at the default.
>>

>
> ------Sequential Output------ --Sequential Input-
> --Random-
> -Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
> --Seeks--
> Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec
> %CP
> luna12-san 16000M 58896 91 62931 9 35870 5 54869 82 145504 13 397.7
> 0
>


So, you're getting 62MB/s writes and 145MB/s reads. Just FYI, that write
speed is about the same as my single SATA drive write speed on my workstation,
so not that great. The read speed is decent, though and with that sort of
read performance, you might want to lower random_page_cost to something like
2.5 or 2 so the planner will tend to prefer index scans.

--
Jeff Frost, Owner <jeff@frostconsultingllc.com>
Frost Consulting, LLC http://www.frostconsultingllc.com/
Phone: 650-780-7908 FAX: 650-649-1954

---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #10 (permalink)  
Old 04-19-2008, 09:18 AM
Alex Deucher
 
Posts: n/a
Default Re: strange performance regression between 7.4 and 8.1

On 3/1/07, Jeff Frost <jeff@frostconsultingllc.com> wrote:
> On Thu, 1 Mar 2007, Alex Deucher wrote:
>
> > On 3/1/07, Jeff Frost <jeff@frostconsultingllc.com> wrote:
> >> On Thu, 1 Mar 2007, Alex Deucher wrote:
> >>
> >> >> Vacuum? Analayze? default_statistics_target? How many shared_buffers?
> >> >> effective_cache_size? work_mem?
> >> >>
> >> >
> >> > I'm running the autovacuum process on the 8.1 server. vacuuming on
> >> > the old server was done manually.
> >> >
> >> > default_statistics_target and effective_cache_size are set to the the
> >> > defaults on both.
> >> >
> >> > postgres 7.4 server:
> >> > # - Memory -
> >> > shared_buffers = 82000 # 1000 min 16, at least
> >> > max_connections*2, 8KB each
> >> > sort_mem = 8000 # 1024 min 64, size in KB
> >> > vacuum_mem = 32000 # 8192 min 1024, size in KB
> >> > # - Free Space Map -
> >> > #max_fsm_pages = 20000 # min max_fsm_relations*16, 6 bytes each
> >> > #max_fsm_relations = 1000 # min 100, ~50 bytes each
> >> > # - Kernel Resource Usage -
> >> > #max_files_per_process = 1000 # min 25
> >> >
> >> > postgres 8.1 server:
> >> > # - Memory -
> >> > shared_buffers = 100000 # min 16 or max_connections*2,
> >> 8KB
> >> > each
> >> > temp_buffers = 2000 #1000 # min 100, 8KB each
> >> > max_prepared_transactions = 100 #5 # can be 0 or more
> >> > # note: increasing max_prepared_transactions costs ~600 bytes of shared
> >> > memory
> >> > # per transaction slot, plus lock space (see max_locks_per_transaction).
> >> > work_mem = 10000 #1024 # min 64, size in KB
> >> > maintenance_work_mem = 524288 #16384 # min 1024, size in KB
> >> > #max_stack_depth = 2048 # min 100, size in KB
> >> >
> >> > I've also tried using the same settings from the old server on the new
> >> > one; same performance issues.
> >> >
> >>
> >> If this is a linux system, could you give us the output of the 'free'
> >> command?

> >
> > total used free shared buffers cached
> > Mem: 8059852 8042868 16984 0 228 7888648
> > -/+ buffers/cache: 153992 7905860
> > Swap: 15631224 2164 15629060

>
> So, I would set effective_cache_size = 988232 (7905860/8).
>
> >
> >> Postgresql might be choosing a bad plan because your effective_cache_size
> >> is
> >> way off (it's the default now right?). Also, what was the block read/write

> >
> > yes it's set to the default.
> >
> >> speed of the SAN from your bonnie tests? Probably want to tune
> >> random_page_cost as well if it's also at the default.
> >>

> >
> > ------Sequential Output------ --Sequential Input-
> > --Random-
> > -Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
> > --Seeks--
> > Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec
> > %CP
> > luna12-san 16000M 58896 91 62931 9 35870 5 54869 82 145504 13 397.7
> > 0
> >

>
> So, you're getting 62MB/s writes and 145MB/s reads. Just FYI, that write
> speed is about the same as my single SATA drive write speed on my workstation,
> so not that great. The read speed is decent, though and with that sort of
> read performance, you might want to lower random_page_cost to something like
> 2.5 or 2 so the planner will tend to prefer index scans.
>


Right, but the old box was getting ~45MBps on both reads and writes,
so it's an improvement for me Thanks for the advice, I'll let you
know how it goes.

Alex

---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
Reply


Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On
Forum Jump


All times are GMT. The time now is 05:02 AM.


Powered by vBulletin® Version 3.6.5
Copyright ©2000 - 2008, Jelsoft Enterprises Ltd.
SEO by vBSEO 3.2.0
UnixAdminTalk.com

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259