Unix Technical Forum

Re: [PATCHES] Fix for large file support

This is a discussion on Re: [PATCHES] Fix for large file support within the pgsql Hackers forums, part of the PostgreSQL category; --> [ redirecting to -hackers for wider comment ] Zdenek Kotala <Zdenek.Kotala@Sun.COM> writes: > Tom Lane wrote: > LET_OS_MANAGE_FILESIZE is ...


Go Back   Unix Technical Forum > Database Server Software > PostgreSQL > pgsql Hackers

FAQ Members List Calendar Search Today's Posts Mark Forums Read
  #1 (permalink)  
Old 04-12-2008, 08:12 AM
Tom Lane
 
Posts: n/a
Default Re: [PATCHES] Fix for large file support

[ redirecting to -hackers for wider comment ]

Zdenek Kotala <Zdenek.Kotala@Sun.COM> writes:
> Tom Lane wrote:
> LET_OS_MANAGE_FILESIZE is good way. I think one problem of this option I
> fixed. It is size of offset. I went thru the code and did not see any
> other problem there. However, how you mentioned it need more testing. I
> going to take server with large disk array and I will test it.


> I would like to add --enable-largefile switch to configure file to
> enable access to wide group of users. What you think about it?


Yeah, I was going to suggest the same thing --- but not with that switch
name. We already use enable/disable-largefile to control whether 64-bit
file access is built at all (this mostly affects pg_dump at the moment).

I think the clearest way might be to flip the sense of the variable.
I never found "LET_OS_MANAGE_FILESIZE" to be a good name anyway. I'd
suggest "USE_SEGMENTED_FILES", which defaults to "on", and you can
turn it off via --disable-segmented-files if configure confirms your
OS has largefile support (thus you could not specify both this and
--disable-largefile).

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #2 (permalink)  
Old 04-12-2008, 08:12 AM
Zdenek Kotala
 
Posts: n/a
Default Re: [PATCHES] Fix for large file support

Tom Lane wrote:
> [ redirecting to -hackers for wider comment ]
>
> Zdenek Kotala <Zdenek.Kotala@Sun.COM> writes:
>> Tom Lane wrote:
>> LET_OS_MANAGE_FILESIZE is good way. I think one problem of this option I
>> fixed. It is size of offset. I went thru the code and did not see any
>> other problem there. However, how you mentioned it need more testing. I
>> going to take server with large disk array and I will test it.

>
>> I would like to add --enable-largefile switch to configure file to
>> enable access to wide group of users. What you think about it?

>
> Yeah, I was going to suggest the same thing --- but not with that switch
> name. We already use enable/disable-largefile to control whether 64-bit
> file access is built at all (this mostly affects pg_dump at the moment).


hmm It looks that ./configure largefile detection does not work on
Solaris.

> I think the clearest way might be to flip the sense of the variable.
> I never found "LET_OS_MANAGE_FILESIZE" to be a good name anyway. I'd
> suggest "USE_SEGMENTED_FILES", which defaults to "on", and you can
> turn it off via --disable-segmented-files if configure confirms your
> OS has largefile support (thus you could not specify both this and
> --disable-largefile).


It sounds good. There is one think for clarification (for the present).
How to handle buffile? It does not currently support non segmented
files. I suggest to use same switch to enable/disable segments there.

Zdenek

---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?

http://archives.postgresql.org

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #3 (permalink)  
Old 04-12-2008, 08:13 AM
Tom Lane
 
Posts: n/a
Default Re: [PATCHES] Fix for large file support

Zdenek Kotala <Zdenek.Kotala@Sun.COM> writes:
> It sounds good. There is one think for clarification (for the present).
> How to handle buffile? It does not currently support non segmented
> files. I suggest to use same switch to enable/disable segments there.


Do you think it really matters? terabyte-sized temp files seem a bit
unlikely, and anyway I don't think the performance argument applies;
md.c's tendency to open all the files at once is irrelevant.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #4 (permalink)  
Old 04-12-2008, 08:16 AM
Jim Nasby
 
Posts: n/a
Default Re: [PATCHES] Fix for large file support

If we expose LET_OS_MANAGE_FILESIZE, should we add a flag to the
control file so that you can't start a backend that has that defined
against a cluster that was initialized without it?

On Apr 6, 2007, at 2:45 PM, Tom Lane wrote:

> [ redirecting to -hackers for wider comment ]
>
> Zdenek Kotala <Zdenek.Kotala@Sun.COM> writes:
>> Tom Lane wrote:
>> LET_OS_MANAGE_FILESIZE is good way. I think one problem of this
>> option I
>> fixed. It is size of offset. I went thru the code and did not see any
>> other problem there. However, how you mentioned it need more
>> testing. I
>> going to take server with large disk array and I will test it.

>
>> I would like to add --enable-largefile switch to configure file to
>> enable access to wide group of users. What you think about it?

>
> Yeah, I was going to suggest the same thing --- but not with that
> switch
> name. We already use enable/disable-largefile to control whether
> 64-bit
> file access is built at all (this mostly affects pg_dump at the
> moment).
>
> I think the clearest way might be to flip the sense of the variable.
> I never found "LET_OS_MANAGE_FILESIZE" to be a good name anyway. I'd
> suggest "USE_SEGMENTED_FILES", which defaults to "on", and you can
> turn it off via --disable-segmented-files if configure confirms your
> OS has largefile support (thus you could not specify both this and
> --disable-largefile).
>
> regards, tom lane
>
> ---------------------------(end of
> broadcast)---------------------------
> TIP 5: don't forget to increase your free space map settings
>


--
Jim Nasby jim@nasby.net
EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)



---------------------------(end of broadcast)---------------------------
TIP 7: You can help support the PostgreSQL project by donating at

http://www.postgresql.org/about/donate

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #5 (permalink)  
Old 04-12-2008, 08:16 AM
Tom Lane
 
Posts: n/a
Default Re: [PATCHES] Fix for large file support

Jim Nasby <decibel@decibel.org> writes:
> If we expose LET_OS_MANAGE_FILESIZE, should we add a flag to the
> control file so that you can't start a backend that has that defined
> against a cluster that was initialized without it?


I imagine we'd flag that as relsegsize = 0 or some such.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #6 (permalink)  
Old 04-12-2008, 08:18 AM
Zdenek Kotala
 
Posts: n/a
Default Re: [PATCHES] Fix for large file support

Tom Lane wrote:
> Jim Nasby <decibel@decibel.org> writes:
>> If we expose LET_OS_MANAGE_FILESIZE, should we add a flag to the
>> control file so that you can't start a backend that has that defined
>> against a cluster that was initialized without it?

>
> I imagine we'd flag that as relsegsize = 0 or some such.


Yes I have it in my patch. I put relsegsize = 0 in the control file when
non-segmentation mode is enabled.

Zdenek

---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #7 (permalink)  
Old 04-12-2008, 08:26 AM
Bruce Momjian
 
Posts: n/a
Default Re: [PATCHES] Fix for large file support


This has been saved for the 8.4 release:

http://momjian.postgresql.org/cgi-bin/pgpatches_hold

---------------------------------------------------------------------------

Tom Lane wrote:
> [ redirecting to -hackers for wider comment ]
>
> Zdenek Kotala <Zdenek.Kotala@Sun.COM> writes:
> > Tom Lane wrote:
> > LET_OS_MANAGE_FILESIZE is good way. I think one problem of this option I
> > fixed. It is size of offset. I went thru the code and did not see any
> > other problem there. However, how you mentioned it need more testing. I
> > going to take server with large disk array and I will test it.

>
> > I would like to add --enable-largefile switch to configure file to
> > enable access to wide group of users. What you think about it?

>
> Yeah, I was going to suggest the same thing --- but not with that switch
> name. We already use enable/disable-largefile to control whether 64-bit
> file access is built at all (this mostly affects pg_dump at the moment).
>
> I think the clearest way might be to flip the sense of the variable.
> I never found "LET_OS_MANAGE_FILESIZE" to be a good name anyway. I'd
> suggest "USE_SEGMENTED_FILES", which defaults to "on", and you can
> turn it off via --disable-segmented-files if configure confirms your
> OS has largefile support (thus you could not specify both this and
> --disable-largefile).
>
> regards, tom lane
>
> ---------------------------(end of broadcast)---------------------------
> TIP 5: don't forget to increase your free space map settings


--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
Reply


Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On
Forum Jump


All times are GMT. The time now is 03:12 PM.


Powered by vBulletin® Version 3.6.5
Copyright ©2000 - 2008, Jelsoft Enterprises Ltd.
SEO by vBSEO 3.2.0
www.UnixAdminTalk.com