Unix Technical Forum

Re: [PATCHES] Trivial patch to double vacuum speed on tables with no indexes

This is a discussion on Re: [PATCHES] Trivial patch to double vacuum speed on tables with no indexes within the pgsql Hackers forums, part of the PostgreSQL category; --> Gregory Stark wrote: > > Bruce Momjian <bruce@momjian.us> writes: > > > Tom Lane wrote: > >> Bruce Momjian ...


Go Back   Unix Technical Forum > Database Server Software > PostgreSQL > pgsql Hackers

FAQ Members List Calendar Search Today's Posts Mark Forums Read
  #1 (permalink)  
Old 04-12-2008, 05:24 AM
Alvaro Herrera
 
Posts: n/a
Default Re: [PATCHES] Trivial patch to double vacuum speed on tables with no indexes

Gregory Stark wrote:
>
> Bruce Momjian <bruce@momjian.us> writes:
>
> > Tom Lane wrote:
> >> Bruce Momjian <bruce@momjian.us> writes:
> >> > Patch applied. Thanks.
> >>
> >> Wait a minute. This patch changes the behavior so that
> >> LockBufferForCleanup is applied to *every* heap page, not only the ones
> >> where there are removable tuples. It's not hard to imagine scenarios
> >> where that results in severe system-wide performance degradation.
> >> Has there been any real-world testing of this idea?

> >
> > I see the no-index case now:
> >
> > + if (nindexes)
> > + LockBuffer(buf, BUFFER_LOCK_SHARE);
> > + else
> > + LockBufferForCleanup(buf);
> >
> > Let's see what Greg says, or revert.

>
> Hm, that's a good point. I could return it to the original method where it
> released the share lock and did he LockBufferForCleanup only if necessary. I
> thought it was awkward to acquire a lock then release it to acquire a
> different lock on the same buffer but it's true that it doesn't always have to
> acquire the second lock.


This rush to apply patches just because no one seems to be capable of
keeping up with them not being reviewed, is starting to get a bit
worrisome.

--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?

http://archives.postgresql.org

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #2 (permalink)  
Old 04-12-2008, 05:25 AM
Bruce Momjian
 
Posts: n/a
Default Re: [PATCHES] Trivial patch to double vacuum speed

Alvaro Herrera wrote:
> > > I see the no-index case now:
> > >
> > > + if (nindexes)
> > > + LockBuffer(buf, BUFFER_LOCK_SHARE);
> > > + else
> > > + LockBufferForCleanup(buf);
> > >
> > > Let's see what Greg says, or revert.

> >
> > Hm, that's a good point. I could return it to the original method where it
> > released the share lock and did he LockBufferForCleanup only if necessary. I
> > thought it was awkward to acquire a lock then release it to acquire a
> > different lock on the same buffer but it's true that it doesn't always have to
> > acquire the second lock.

>
> This rush to apply patches just because no one seems to be capable of
> keeping up with them not being reviewed, is starting to get a bit
> worrisome.


When things are placed in the patches queue, I need to get feedback if
there is a problem with them. I am not sure what other process we can
follow, unless we just keep patches there indefinitely, or just ignore
them and never place them in the queue.

--
Bruce Momjian bruce@momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #3 (permalink)  
Old 04-12-2008, 05:25 AM
Tom Lane
 
Posts: n/a
Default Re: [PATCHES] Trivial patch to double vacuum speed

Bruce Momjian <bruce@momjian.us> writes:
> Alvaro Herrera wrote:
>> This rush to apply patches just because no one seems to be capable of
>> keeping up with them not being reviewed, is starting to get a bit
>> worrisome.


> When things are placed in the patches queue, I need to get feedback if
> there is a problem with them. I am not sure what other process we can
> follow, unless we just keep patches there indefinitely, or just ignore
> them and never place them in the queue.


The problem with the process you're using is that it defaults to
applying patches --- and in fact, lately it seems like it takes a
threat of mayhem to prevent you from applying a patch.

Now, apply-unless-objected-to was the right default back in 1997, when
the code was in bad enough shape that it was hard to make it worse ;-).
I do not believe it's the right default anymore though. The system is a
lot larger and more complicated than it was when it left Berkeley, and
our quality standards are an order of magnitude higher too. We need to
default to *not* applying patches until they've passed some amount of
review.

I don't want to be too hard-nosed about this, since the last thing we
need is another level of bureaucracy added to our processes, especially
for simple trivial stuff. But when there's been some discussion or
objection to a patch, ISTM that just because the patch submitter has
put up a second version should not mean that it's okay to apply. At
that point some actual review is needed.

I'm also having a bit of a problem with the silence-means-assent rule.
Most of the time I'm OK with it, but right now I simply don't have the
time to look at everything that comes in as soon as it comes in;
especially not second versions of patches.

I don't have a concrete proposal to make, but I do think that the
current patch-queue process is not suited to the project as it stands
today. Maybe if this issue-tracking stuff gets off the ground, we
could let developers place ACK or NAK flags on patches they've looked
at, and have some rule about ACK-vs-NAK requirements for something to go
in.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #4 (permalink)  
Old 04-12-2008, 05:25 AM
Joshua D. Drake
 
Posts: n/a
Default Re: [PATCHES] Trivial patch to double vacuum speed


> I don't have a concrete proposal to make, but I do think that the
> current patch-queue process is not suited to the project as it stands
> today. Maybe if this issue-tracking stuff gets off the ground, we
> could let developers place ACK or NAK flags on patches they've looked
> at, and have some rule about ACK-vs-NAK requirements for something to go
> in.


How about *requiring* test cases that prove the patch?

Sincerely,

Joshua D. Drake


>
> regards, tom lane
>
> ---------------------------(end of broadcast)---------------------------
> TIP 2: Don't 'kill -9' the postmaster
>



--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/



---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?

http://archives.postgresql.org

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #5 (permalink)  
Old 04-12-2008, 05:25 AM
Bruce Momjian
 
Posts: n/a
Default Re: [PATCHES] Trivial patch to double vacuum speed

Tom Lane wrote:
> Bruce Momjian <bruce@momjian.us> writes:
> > Alvaro Herrera wrote:
> >> This rush to apply patches just because no one seems to be capable of
> >> keeping up with them not being reviewed, is starting to get a bit
> >> worrisome.

>
> > When things are placed in the patches queue, I need to get feedback if
> > there is a problem with them. I am not sure what other process we can
> > follow, unless we just keep patches there indefinitely, or just ignore
> > them and never place them in the queue.

>
> The problem with the process you're using is that it defaults to
> applying patches --- and in fact, lately it seems like it takes a
> threat of mayhem to prevent you from applying a patch.


I am just trying to keep everyone happy, the developers, user community,
and patch submitters.

> Now, apply-unless-objected-to was the right default back in 1997, when
> the code was in bad enough shape that it was hard to make it worse ;-).
> I do not believe it's the right default anymore though. The system is a
> lot larger and more complicated than it was when it left Berkeley, and
> our quality standards are an order of magnitude higher too. We need to
> default to *not* applying patches until they've passed some amount of
> review.
>
> I don't want to be too hard-nosed about this, since the last thing we
> need is another level of bureaucracy added to our processes, especially
> for simple trivial stuff. But when there's been some discussion or
> objection to a patch, ISTM that just because the patch submitter has
> put up a second version should not mean that it's okay to apply. At
> that point some actual review is needed.
>
> I'm also having a bit of a problem with the silence-means-assent rule.
> Most of the time I'm OK with it, but right now I simply don't have the
> time to look at everything that comes in as soon as it comes in;
> especially not second versions of patches.
>
> I don't have a concrete proposal to make, but I do think that the
> current patch-queue process is not suited to the project as it stands
> today. Maybe if this issue-tracking stuff gets off the ground, we
> could let developers place ACK or NAK flags on patches they've looked
> at, and have some rule about ACK-vs-NAK requirements for something to go
> in.


Yes, I realize this is a very hard time. I know you were pushing for
end-of-week beta, and though I don't think we can hit that date, I am
trying to move things along. I agree it is that second version of the
patch that often doesn't get the thorough review. Should I increase the
amount of time something is in the queue, or ask for someone to state it
is OK to apply, and just keep asking until I get a "yes" from someone?
I can do that pretty easily.

--
Bruce Momjian bruce@momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #6 (permalink)  
Old 04-12-2008, 05:25 AM
Bruce Momjian
 
Posts: n/a
Default Re: [PATCHES] Trivial patch to double vacuum speed

Joshua D. Drake wrote:
>
> > I don't have a concrete proposal to make, but I do think that the
> > current patch-queue process is not suited to the project as it stands
> > today. Maybe if this issue-tracking stuff gets off the ground, we
> > could let developers place ACK or NAK flags on patches they've looked
> > at, and have some rule about ACK-vs-NAK requirements for something to go
> > in.

>
> How about *requiring* test cases that prove the patch?


That doesn't hit most of the failures, which can be portability,
performance, or missing features, or bad style.

--
Bruce Momjian bruce@momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?

http://archives.postgresql.org

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #7 (permalink)  
Old 04-12-2008, 05:25 AM
Tom Lane
 
Posts: n/a
Default Re: [PATCHES] Trivial patch to double vacuum speed

"Joshua D. Drake" <jd@commandprompt.com> writes:
> How about *requiring* test cases that prove the patch?


There are lots and lots of things that can't really be tested with
pg_regress-based tests, and in any case a test does not prove the
absence of bugs; particularly not a test devised by the code author,
since almost by definition it won't test cases he didn't think of.

Certainly test cases have their place, but I have a lot more faith in
code-reading by knowledgeable developers as a way to spot problems
that got past the original author.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?

http://archives.postgresql.org

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #8 (permalink)  
Old 04-12-2008, 05:25 AM
Gavin Sherry
 
Posts: n/a
Default Re: [PATCHES] Trivial patch to double vacuum speed

On Mon, 4 Sep 2006, Joshua D. Drake wrote:

>
> > I don't have a concrete proposal to make, but I do think that the
> > current patch-queue process is not suited to the project as it stands
> > today. Maybe if this issue-tracking stuff gets off the ground, we
> > could let developers place ACK or NAK flags on patches they've looked
> > at, and have some rule about ACK-vs-NAK requirements for something to go
> > in.

>
> How about *requiring* test cases that prove the patch?


People including regression tests is not a replacement for code review.
For a non-trivial patch, an SQL test will only exercise a few code paths.
Moreover, it wont say anything about code quality, maintainability or
general correctness or completeness. It will still have to be reviewed.

Thanks

Gavin

---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #9 (permalink)  
Old 04-12-2008, 05:25 AM
Joshua D. Drake
 
Posts: n/a
Default Re: [PATCHES] Trivial patch to double vacuum speed

Gavin Sherry wrote:
> On Mon, 4 Sep 2006, Joshua D. Drake wrote:
>
>>> I don't have a concrete proposal to make, but I do think that the
>>> current patch-queue process is not suited to the project as it stands
>>> today. Maybe if this issue-tracking stuff gets off the ground, we
>>> could let developers place ACK or NAK flags on patches they've looked
>>> at, and have some rule about ACK-vs-NAK requirements for something to go
>>> in.

>> How about *requiring* test cases that prove the patch?

>
> People including regression tests is not a replacement for code review.


Uhmmm, of course not? . A test case does however help show that the
person thought through what they were doing Even if they were cranked
in the process.

Sincerely,

Joshua D. Drake


> For a non-trivial patch, an SQL test will only exercise a few code paths

..
> Moreover, it wont say anything about code quality, maintainability or
> general correctness or completeness. It will still have to be reviewed.
>
> Thanks
>
> Gavin
>
> ---------------------------(end of broadcast)---------------------------
> TIP 2: Don't 'kill -9' the postmaster
>



--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/



---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #10 (permalink)  
Old 04-12-2008, 05:25 AM
Hannu Krosing
 
Posts: n/a
Default Re: [PATCHES] Trivial patch to double vacuum speed

Ühel kenal päeval, E, 2006-09-04 kell 16:51, kirjutas Joshua D. Drake:
> > I don't have a concrete proposal to make, but I do think that the
> > current patch-queue process is not suited to the project as it stands
> > today. Maybe if this issue-tracking stuff gets off the ground, we
> > could let developers place ACK or NAK flags on patches they've looked
> > at, and have some rule about ACK-vs-NAK requirements for something to go
> > in.

>
> How about *requiring* test cases that prove the patch?


Test cases are good for checking if some latter patch does not break the
functionality provided by current patch.

They are almost useless for proving that current patch is correct.

> Sincerely,
>
> Joshua D. Drake
>
>
> >
> > regards, tom lane
> >
> > ---------------------------(end of broadcast)---------------------------
> > TIP 2: Don't 'kill -9' the postmaster
> >

>
>

--
----------------
Hannu Krosing
Database Architect
Skype Technologies OÜ
Akadeemia tee 21 F, Tallinn, 12618, Estonia

Skype me: callto:hkrosing
Get Skype for free: http://www.skype.com



---------------------------(end of broadcast)---------------------------
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
Reply


Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On
Forum Jump


All times are GMT. The time now is 08:42 PM.


Powered by vBulletin® Version 3.6.5
Copyright ©2000 - 2008, Jelsoft Enterprises Ltd.
SEO by vBSEO 3.2.0
www.UnixAdminTalk.com