Unix Technical Forum

Re: [HACKERS] Arrays of Complex Types

This is a discussion on Re: [HACKERS] Arrays of Complex Types within the Pgsql Patches forums, part of the PostgreSQL category; --> On Fri, Mar 30, 2007 at 05:08:42PM -0400, Tom Lane wrote: > David Fetter <david@fetter.org> writes: > > After ...


Go Back   Unix Technical Forum > Database Server Software > PostgreSQL > Pgsql Patches

FAQ Members List Calendar Search Today's Posts Mark Forums Read
  #1 (permalink)  
Old 04-18-2008, 10:50 AM
David Fetter
 
Posts: n/a
Default Re: [HACKERS] Arrays of Complex Types

On Fri, Mar 30, 2007 at 05:08:42PM -0400, Tom Lane wrote:
> David Fetter <david@fetter.org> writes:
> > After several rounds of patches, it appears that it might be easier to
> > create a new typtype entry, which I'll tentatively call 'a' because it
> > seems a little fragile and a lot inelegant and hard to maintain to
> > have typtype='c' and typrelid=InvalidOid mean, "this is an array of
> > complex types."

>
> Uh, wouldn't it be typtype = 'c' and typelem != 0 ?


Right. The attached patch passes the current regression tests and at
least to a "smoke test" level does what it's supposed to do. I'd
really like to help refactor the whole array system to use 'a', tho.

Cheers,
D
--
David Fetter <david@fetter.org> http://fetter.org/
phone: +1 415 235 3778 AIM: dfetter666
Skype: davidfetter

Remember to vote!
Consider donating to PostgreSQL: http://www.postgresql.org/about/donate


---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #2 (permalink)  
Old 04-18-2008, 10:51 AM
Bruce Momjian
 
Posts: n/a
Default Re: [HACKERS] Arrays of Complex Types


Your patch has been added to the PostgreSQL unapplied patches list at:

http://momjian.postgresql.org/cgi-bin/pgpatches

It will be applied as soon as one of the PostgreSQL committers reviews
and approves it.

---------------------------------------------------------------------------


David Fetter wrote:
> On Fri, Mar 30, 2007 at 05:08:42PM -0400, Tom Lane wrote:
> > David Fetter <david@fetter.org> writes:
> > > After several rounds of patches, it appears that it might be easier to
> > > create a new typtype entry, which I'll tentatively call 'a' because it
> > > seems a little fragile and a lot inelegant and hard to maintain to
> > > have typtype='c' and typrelid=InvalidOid mean, "this is an array of
> > > complex types."

> >
> > Uh, wouldn't it be typtype = 'c' and typelem != 0 ?

>
> Right. The attached patch passes the current regression tests and at
> least to a "smoke test" level does what it's supposed to do. I'd
> really like to help refactor the whole array system to use 'a', tho.
>
> Cheers,
> D
> --
> David Fetter <david@fetter.org> http://fetter.org/
> phone: +1 415 235 3778 AIM: dfetter666
> Skype: davidfetter
>
> Remember to vote!
> Consider donating to PostgreSQL: http://www.postgresql.org/about/donate


[ Attachment, skipping... ]

>
> ---------------------------(end of broadcast)---------------------------
> TIP 9: In versions below 8.0, the planner will ignore your desire to
> choose an index scan if your joining column's datatypes do not
> match


--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?

http://archives.postgresql.org

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #3 (permalink)  
Old 04-18-2008, 10:51 AM
Alvaro Herrera
 
Posts: n/a
Default Re: [HACKERS] Arrays of Complex Types

Bruce Momjian wrote:
>
> Your patch has been added to the PostgreSQL unapplied patches list at:
>
> http://momjian.postgresql.org/cgi-bin/pgpatches
>
> It will be applied as soon as one of the PostgreSQL committers reviews
> and approves it.
>


So, hum, what happened to the idea of creating the array types only on
demand?

>
>
> David Fetter wrote:
> > On Fri, Mar 30, 2007 at 05:08:42PM -0400, Tom Lane wrote:
> > > David Fetter <david@fetter.org> writes:
> > > > After several rounds of patches, it appears that it might be easier to
> > > > create a new typtype entry, which I'll tentatively call 'a' because it
> > > > seems a little fragile and a lot inelegant and hard to maintain to
> > > > have typtype='c' and typrelid=InvalidOid mean, "this is an array of
> > > > complex types."
> > >
> > > Uh, wouldn't it be typtype = 'c' and typelem != 0 ?

> >
> > Right. The attached patch passes the current regression tests and at
> > least to a "smoke test" level does what it's supposed to do. I'd
> > really like to help refactor the whole array system to use 'a', tho.
> >
> > Cheers,



--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #4 (permalink)  
Old 04-18-2008, 10:51 AM
David Fetter
 
Posts: n/a
Default Re: [HACKERS] Arrays of Complex Types

On Mon, Apr 02, 2007 at 10:01:44PM -0400, Alvaro Herrera wrote:
> Bruce Momjian wrote:
> >
> > Your patch has been added to the PostgreSQL unapplied patches list
> > at:
> >
> > http://momjian.postgresql.org/cgi-bin/pgpatches
> >
> > It will be applied as soon as one of the PostgreSQL committers
> > reviews and approves it.
> >

>
> So, hum, what happened to the idea of creating the array types only
> on demand?


Scotched, as far as I could tell, partly due to nobody's having
actually done work toward such a thing, and partly because the closest
thing I've heard to an objection is pretty nebulous.

It's a lot simpler to have them always, and it fits in with the larger
picture of making arrays fully composable with other operations like
DOMAIN, ENUM and TYPE.

Cheers,
D
--
David Fetter <david@fetter.org> http://fetter.org/
phone: +1 415 235 3778 AIM: dfetter666
Skype: davidfetter

Remember to vote!
Consider donating to PostgreSQL: http://www.postgresql.org/about/donate

---------------------------(end of broadcast)---------------------------
TIP 7: You can help support the PostgreSQL project by donating at

http://www.postgresql.org/about/donate

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #5 (permalink)  
Old 04-18-2008, 10:51 AM
Tom Lane
 
Posts: n/a
Default Re: [HACKERS] Arrays of Complex Types

David Fetter <david@fetter.org> writes:
> On Mon, Apr 02, 2007 at 10:01:44PM -0400, Alvaro Herrera wrote:
>> So, hum, what happened to the idea of creating the array types only
>> on demand?


> Scotched, as far as I could tell,


More like "you submitted a patch that entirely ignores multiple people's
opinion on what is needed".

Bruce may have put this into the patch queue, but do not labor under
the delusion that that means it'll get applied as-is. The queue is
currently operating as a list of open issues.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #6 (permalink)  
Old 04-18-2008, 10:51 AM
David Fetter
 
Posts: n/a
Default Re: [HACKERS] Arrays of Complex Types

On Tue, Apr 03, 2007 at 02:30:07AM -0400, Tom Lane wrote:
> David Fetter <david@fetter.org> writes:
> > On Mon, Apr 02, 2007 at 10:01:44PM -0400, Alvaro Herrera wrote:
> >> So, hum, what happened to the idea of creating the array types
> >> only on demand?

>
> > Scotched, as far as I could tell,

>
> More like "you submitted a patch that entirely ignores multiple
> people's opinion on what is needed".
>
> Bruce may have put this into the patch queue, but do not labor under
> the delusion that that means it'll get applied as-is.


I assure you I'm not. Two glaring things it's missing are regression
tests and documentation. I should have those in this week.

Cheers,
D
--
David Fetter <david@fetter.org> http://fetter.org/
phone: +1 415 235 3778 AIM: dfetter666
Skype: davidfetter

Remember to vote!
Consider donating to PostgreSQL: http://www.postgresql.org/about/donate

---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #7 (permalink)  
Old 04-18-2008, 10:51 AM
Bruce Momjian
 
Posts: n/a
Default Re: [HACKERS] Arrays of Complex Types

Tom Lane wrote:
> David Fetter <david@fetter.org> writes:
> > On Mon, Apr 02, 2007 at 10:01:44PM -0400, Alvaro Herrera wrote:
> >> So, hum, what happened to the idea of creating the array types only
> >> on demand?

>
> > Scotched, as far as I could tell,

>
> More like "you submitted a patch that entirely ignores multiple people's
> opinion on what is needed".
>
> Bruce may have put this into the patch queue, but do not labor under
> the delusion that that means it'll get applied as-is. The queue is
> currently operating as a list of open issues.


Correct.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #8 (permalink)  
Old 04-18-2008, 10:54 AM
Andrew Dunstan
 
Posts: n/a
Default Re: [HACKERS] Arrays of Complex Types

Tom Lane wrote:
> David Fetter <david@fetter.org> writes:
>
>> On Mon, Apr 02, 2007 at 10:01:44PM -0400, Alvaro Herrera wrote:
>>
>>> So, hum, what happened to the idea of creating the array types only
>>> on demand?
>>>

>
>
>> Scotched, as far as I could tell,
>>

>
> More like "you submitted a patch that entirely ignores multiple people's
> opinion on what is needed".
>
> Bruce may have put this into the patch queue, but do not labor under
> the delusion that that means it'll get applied as-is. The queue is
> currently operating as a list of open issues.
>
>


One of the things that's been bothering me about this proposal is that
it leaves untouched and indeed greatly expands the scope of the typename
mangling we do. (i.e. adding an entry to pg_type with _ prepended). Up
to now we've only used this gadget in a way that might matter a lot on
user defined non-composite types, I think, and now we have expanded that
to include enums, which are really a special case of user defined
non-composites which don't require an extra C module. That's a
comparatively small window, but this proposal will extend it to all
composites, which is quite a large expansion in scope. And since _ is a
perfectly legal initial char for an identifier, if type _foo exists then
any attempt to create a table or view or composite called foo will fail.

Is it possible to fix this, or am I trying to shut the stable door after
the horse has well and truly bolted? If it can be fixed, I'd like to see
it fixed before we fix the problem David is trying to address here.

It's been suggested to me that this is an insignificant corner case. But
I have often seen coding standards that actually require certain classes
of identifier to being with _, so it's very far from a merely
theoretical point.

I'm slightly inclined to agree with David that the danger of catalog
bloat isn't that great, and might not justify the extra work that some
sort of explicit array creation would involve (e.g. changes in grammar,
pg_dump), as long as we are agreed that we don't want array types ever
to have their own user definable names or settable namespace.

cheers

andrew


---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #9 (permalink)  
Old 04-18-2008, 10:54 AM
Tom Lane
 
Posts: n/a
Default Re: [HACKERS] Arrays of Complex Types

Andrew Dunstan <andrew@dunslane.net> writes:
> One of the things that's been bothering me about this proposal is that
> it leaves untouched and indeed greatly expands the scope of the typename
> mangling we do. (i.e. adding an entry to pg_type with _ prepended).


Yeah, that's been bothering me too. One of the problems with the patch
as-is is that it extends the 62-instead-of-63-char limit to table names
as well as type names.

I've been thinking of proposing that we add a column to pg_type that
points from a type to its array type (if any), ie the reverse link
from typelem. If we had that then the parser could follow that to
determine which type is foo[], instead of relying on the _foo naming
convention. I don't suggest that we stop using the naming convention,
but it would no longer be a hard-and-fast rule, just a convention.
In particular we could rejigger things around the edges to reduce
the name conflict problem. For instance the rule for forming array type
names could be "prepend _, truncate to less than 64 bytes if necessary,
then substitute numbers at the end if needed to get something unique".
This is not all that different from what we do now to get unique
serial sequence names, for example.

This would also open the door to supporting

CREATE TYPE foo AS ARRAY OF bar

without having to have any restrictions about the name of foo.
I'd still much rather do things that way for arrays of composites
than invent a ton of pg_type entries that are mostly going to go
unused.

regards, tom lane

PS: Has anyone looked at what it will take to make the entries
in an array-of-composite be something smaller than full tuples?
It's not going to be anything but a toy unless you can get the
per-entry overhead down to something sane. Perhaps the
MinimalTuple representation would work.

---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #10 (permalink)  
Old 04-18-2008, 10:54 AM
Andrew Dunstan
 
Posts: n/a
Default Re: [HACKERS] Arrays of Complex Types

Tom Lane wrote:
>
> I've been thinking of proposing that we add a column to pg_type that
> points from a type to its array type (if any), ie the reverse link
> from typelem. If we had that then the parser could follow that to
> determine which type is foo[], instead of relying on the _foo naming
> convention.


good.


> I don't suggest that we stop using the naming convention,
> but it would no longer be a hard-and-fast rule, just a convention.
> In particular we could rejigger things around the edges to reduce
> the name conflict problem. For instance the rule for forming array type
> names could be "prepend _, truncate to less than 64 bytes if necessary,
> then substitute numbers at the end if needed to get something unique".
> This is not all that different from what we do now to get unique
> serial sequence names, for example.
>


Sounds OK but I'd add something that might make it even more unlikely to
generate a name clash.

> This would also open the door to supporting
>
> CREATE TYPE foo AS ARRAY OF bar
>
> without having to have any restrictions about the name of foo.
> I'd still much rather do things that way for arrays of composites
> than invent a ton of pg_type entries that are mostly going to go
> unused.
>
>
>


ISTM we should either do it all automatically or all manually. If you
want user defined names for array types then we can forget name mangling
for user defined types and do everything manually.

cheers

andrew



---------------------------(end of broadcast)---------------------------
TIP 7: You can help support the PostgreSQL project by donating at

http://www.postgresql.org/about/donate

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
Reply


Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On
Forum Jump


All times are GMT. The time now is 05:41 PM.


Powered by vBulletin® Version 3.6.5
Copyright ©2000 - 2008, Jelsoft Enterprises Ltd.
SEO by vBSEO 3.2.0
www.UnixAdminTalk.com