Unix Technical Forum

Autovacuum vs statement_timeout

This is a discussion on Autovacuum vs statement_timeout within the pgsql Hackers forums, part of the PostgreSQL category; --> -----BEGIN PGP SIGNED MESSAGE----- Hash: RIPEMD160 Bruce asks: >>> Particularly consider using psql to restore a pg_dump >>> dump ...


Go Back   Unix Technical Forum > Database Server Software > PostgreSQL > pgsql Hackers

Register FAQ Members List Calendar Search Today's Posts Mark Forums Read
  #31 (permalink)  
Old 04-15-2008, 10:46 PM
Greg Sabino Mullane
 
Posts: n/a
Default Re: Autovacuum vs statement_timeout


-----BEGIN PGP SIGNED MESSAGE-----
Hash: RIPEMD160


Bruce asks:
>>> Particularly consider using psql to restore a pg_dump
>>> dump --- are we going to add "SET statement_timeout=0"
>>> to the pg_dump file?

>
>> I hope not. That should be the user's choice.


> Would anyone want to limit the load time for pg_dump? I
> can hardly see why.


Not for pg_dump, but for psql, as you stated above. I don't
have a problem adding it to pg_dump or pg_restore. They are
single, atomic actions out of the control of the user. Restoring
a pg_dump'ed file through psql, on the other hand, should not
assume that the user might not want to keep or set their own
timeout, perhaps because they want to limit the load on the
server, or because of vacuuming concerns. Recall that pg_dump
is not just used to restore entire systems: we can dump schemas,
tables, and in the near future may even have the ability to
dump different classes (schema, data, constraints).

Hard-coding a forced option to the top of a potentially ginormous
and hard-to-edit file that really has nothing to do with the data
itself seems the wrong way to do things. It's not as if we've been
inundated on the lists with tales of people getting caught on
custom statement_timeouts when importing dumps.

- --
Greg Sabino Mullane greg@turnstep.com
PGP Key: 0x14964AC8 200803111959
http://biglumber.com/x/web?pk=2529DF...9B906714964AC8
-----BEGIN PGP SIGNATURE-----

iEYEAREDAAYFAkfXHqMACgkQvJuQZxSWSsgifQCgthvDCTiKhw/3A4S1na1mvlOB
+MQAn2baL34c8k3FV+f2CUAn7GwDewrN
=x24Q
-----END PGP SIGNATURE-----



--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #32 (permalink)  
Old 04-15-2008, 10:47 PM
Peter Eisentraut
 
Posts: n/a
Default Re: Autovacuum vs statement_timeout

Tom Lane wrote:
> To me, the killer reason for statement_timeout = 0 during pg_dump
> is that without it, routine cron-driven dumps could fail, and the
> user might not notice until he really really needed that dump.


This concrete case if of course valid, but if you take a step back, there are
about half a dozen ways to configure the server to make pg_dump fail. A
misconfigured pg_hba.conf would also do it. And if the server isn't running,
should we start it? If we take a base backup, and archiving isn't enabled,
should we enable it?

Perhaps we shouldn't paper over configuration problems in inconsistent and ad
hoc ways, and instead give useful configuration advice and encourage users to
monitor their background jobs, which they will have to do anyway, even if we
solve a misconfigured statement_timeout for them.

# DON'T SET THIS PARAMETER IN THE CONFIGURATION FILE OR YOUR BACKUPS WILL FAIL

could be a start.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #33 (permalink)  
Old 04-15-2008, 10:47 PM
Tom Lane
 
Posts: n/a
Default Re: Autovacuum vs statement_timeout

Peter Eisentraut <peter_e@gmx.net> writes:
> Tom Lane wrote:
>> To me, the killer reason for statement_timeout = 0 during pg_dump
>> is that without it, routine cron-driven dumps could fail, and the
>> user might not notice until he really really needed that dump.


> This concrete case if of course valid, but if you take a step back,
> there are about half a dozen ways to configure the server to make
> pg_dump fail.


Sure, but I think in a lot of situations there is a use-case for
a nonzero default statement_timeout, so it seems to me worthwhile
to protect pg_dump from that rather than let DBAs find out the
hard way. Also, statement_timeout is particularly nasty in that
you may not see any failure when you first set up and test your
dump strategy. It'll get you after your tables have grown, or
on a day when there's particularly heavy activity concurrent
with the dump.

> # DON'T SET THIS PARAMETER IN THE CONFIGURATION FILE OR YOUR BACKUPS WILL FAIL


If there are any parameters for which that is actually appropriate,
wouldn't it be *more* appropriate to flat-out prevent them from
being set from the config file? I believe we have a GUC variable
flag already for things that shouldn't be in the file, but it's
not enforced.

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
Reply


Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On
Forum Jump


All times are GMT. The time now is 05:04 PM.


Powered by vBulletin® Version 3.6.5
Copyright ©2000 - 2008, Jelsoft Enterprises Ltd.
SEO by vBSEO 3.2.0
www.UnixAdminTalk.com