Unix Technical Forum

autovacuum multiworkers, patch 5

This is a discussion on autovacuum multiworkers, patch 5 within the Pgsql Patches forums, part of the PostgreSQL category; --> Alvaro Herrera <alvherre@commandprompt.com> wrote: > I manually merged your patch on top of my own. This is the result. ...


Go Back   Unix Technical Forum > Database Server Software > PostgreSQL > Pgsql Patches

Register FAQ Members List Calendar Search Today's Posts Mark Forums Read
  #11 (permalink)  
Old 04-18-2008, 09:56 AM
ITAGAKI Takahiro
 
Posts: n/a
Default Re: autovacuum multiworkers, patch 5


Alvaro Herrera <alvherre@commandprompt.com> wrote:

> I manually merged your patch on top of my own. This is the result.
> Please have a look at whether the new code is correct and behaves sanely
> (I haven't tested it).


The patch seems to be broken -- the latter half is lost.


Regards,
---
ITAGAKI Takahiro
NTT Open Source Software Center



---------------------------(end of broadcast)---------------------------
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #12 (permalink)  
Old 04-18-2008, 09:56 AM
ITAGAKI Takahiro
 
Posts: n/a
Default Re: autovacuum multiworkers, patch 5


Alvaro Herrera <alvherre@commandprompt.com> wrote:

> > > I manually merged your patch on top of my own. This is the result.
> > > Please have a look at whether the new code is correct and behaves sanely
> > > (I haven't tested it).

>
> Huh, you are right, it is broken, even in my outgoing mailbox -- I don't
> know what happened, as the file I have on disk is complete. Here is
> attached again.


I tested your patch on Linux and Windows. It works well on Linux,
where we use fork(), but falls into segfault on Windows, where we
use exec(). Maybe you forgot to initialize the shared memory stuff.
(I haven't find out where to be fixed, sorry.)

Multiworker and balancing seem to work well after they successfully start up.

Regards,
---
ITAGAKI Takahiro
NTT Open Source Software Center



---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #13 (permalink)  
Old 04-18-2008, 09:56 AM
Alvaro Herrera
 
Posts: n/a
Default Re: autovacuum multiworkers, patch 5

ITAGAKI Takahiro wrote:
>
> Alvaro Herrera <alvherre@commandprompt.com> wrote:
>
> > > > I manually merged your patch on top of my own. This is the result.
> > > > Please have a look at whether the new code is correct and behaves sanely
> > > > (I haven't tested it).

> >
> > Huh, you are right, it is broken, even in my outgoing mailbox -- I don't
> > know what happened, as the file I have on disk is complete. Here is
> > attached again.

>
> I tested your patch on Linux and Windows. It works well on Linux,
> where we use fork(), but falls into segfault on Windows, where we
> use exec(). Maybe you forgot to initialize the shared memory stuff.
> (I haven't find out where to be fixed, sorry.)


Ok, thanks, this confirms that I have to try the EXEC_BACKEND code path.

> Multiworker and balancing seem to work well after they successfully start up.


Great.

--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #14 (permalink)  
Old 04-18-2008, 09:56 AM
Alvaro Herrera
 
Posts: n/a
Default Re: autovacuum multiworkers, patch 5

Alvaro Herrera wrote:
> ITAGAKI Takahiro wrote:
> >
> > I tested your patch on Linux and Windows. It works well on Linux,
> > where we use fork(), but falls into segfault on Windows, where we
> > use exec(). Maybe you forgot to initialize the shared memory stuff.
> > (I haven't find out where to be fixed, sorry.)

>
> Ok, thanks, this confirms that I have to try the EXEC_BACKEND code path.


Oh, uh, the problem is that CreateSharedMemoryAndSemaphores wants to
have access to the PGPROC already, but to obtain the PGPROC we need
access to autovac shared memory (per AutoVacuumGetFreeProc). So this
wasn't too bright a choice :-(

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #15 (permalink)  
Old 04-18-2008, 09:56 AM
Alvaro Herrera
 
Posts: n/a
Default Autovacuum PGPROCs in ProcGlobal? (was Re: autovacuum multiworkers)

Alvaro Herrera wrote:

> Oh, uh, the problem is that CreateSharedMemoryAndSemaphores wants to
> have access to the PGPROC already, but to obtain the PGPROC we need
> access to autovac shared memory (per AutoVacuumGetFreeProc). So this
> wasn't too bright a choice :-(


It seems like I'll have to decouple autovacuum PGPROC's from
autovacuum's own shared memory. The most sensible way to do this seems
to be to store them in ProcGlobal, along with the regular backend's
PGPROCs. Is everyone OK with this plan?

Note that this will mean that those PGPROCs will be protected by the
same spinlock that protects the other PGPROCs. I can't think of any
reason why this would be a problem, but if you think otherwise please
speak up.

--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #16 (permalink)  
Old 04-18-2008, 09:56 AM
Tom Lane
 
Posts: n/a
Default Re: Autovacuum PGPROCs in ProcGlobal? (was Re: autovacuum multiworkers)

Alvaro Herrera <alvherre@commandprompt.com> writes:
> It seems like I'll have to decouple autovacuum PGPROC's from
> autovacuum's own shared memory. The most sensible way to do this seems
> to be to store them in ProcGlobal, along with the regular backend's
> PGPROCs. Is everyone OK with this plan?


> Note that this will mean that those PGPROCs will be protected by the
> same spinlock that protects the other PGPROCs. I can't think of any
> reason why this would be a problem, but if you think otherwise please
> speak up.


I thought the separate pool of PGPROCs was a bit weird. If you're going
back to a common pool, I'm all for it.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
Reply


Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On
Forum Jump


All times are GMT. The time now is 08:51 AM.


Powered by vBulletin® Version 3.6.5
Copyright ©2000 - 2008, Jelsoft Enterprises Ltd.
SEO by vBSEO 3.2.0
www.UnixAdminTalk.com