New functions to examine the contents of heap pages, as discussed
recently on -hackers. These are fully integrated into backend.

Designed to be extended for other page layouts/contents. (Heikki has
some similar functions for index pages).

Docs included, applies cleanly, tests good.

I'll maintain this with immediate fixes/additions as we go up to 8.3 and
beyond, to assist review process of various patches that alter page
contents.

--
Simon Riggs
EnterpriseDB http://www.enterprisedb.com



---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?

http://archives.postgresql.org

On Thu, 2007-03-08 at 15:44 +0000, Simon Riggs wrote:

> Docs included

Just noticed a typo. File mentioned in func.sgml, line 11049 should be
<filename>src/include/access/htup.h</> and not
<filename>src/include/storage/bufpage.h</>

--
Simon Riggs
EnterpriseDB http://www.enterprisedb.com



---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq

Your patch has been added to the PostgreSQL unapplied patches list at:

http://momjian.postgresql.org/cgi-bin/pgpatches

It will be applied as soon as one of the PostgreSQL committers reviews
and approves it.

---------------------------------------------------------------------------


Simon Riggs wrote:
> New functions to examine the contents of heap pages, as discussed
> recently on -hackers. These are fully integrated into backend.
>
> Designed to be extended for other page layouts/contents. (Heikki has
> some similar functions for index pages).
>
> Docs included, applies cleanly, tests good.
>
> I'll maintain this with immediate fixes/additions as we go up to 8.3 and
> beyond, to assist review process of various patches that alter page
> contents.
>
> --
> Simon Riggs
> EnterpriseDB http://www.enterprisedb.com
>

[ Attachment, skipping... ]

>
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Have you searched our list archives?
>
> http://archives.postgresql.org

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings

Your patch has been added to the PostgreSQL unapplied patches list at:

http://momjian.postgresql.org/cgi-bin/pgpatches

It will be applied as soon as one of the PostgreSQL committers reviews
and approves it.

---------------------------------------------------------------------------


Simon Riggs wrote:
> On Thu, 2007-03-08 at 15:44 +0000, Simon Riggs wrote:
>
> > Docs included
>
> Just noticed a typo. File mentioned in func.sgml, line 11049 should be
> <filename>src/include/access/htup.h</> and not
> <filename>src/include/storage/bufpage.h</>
>
> --
> Simon Riggs
> EnterpriseDB http://www.enterprisedb.com
>
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 3: Have you checked our extensive FAQ?
>
> http://www.postgresql.org/docs/faq

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?

http://archives.postgresql.org

This looks useful, but shouldn't it be part of /contrib/pgstattuple
rather than in the backend?

---------------------------------------------------------------------------

Simon Riggs wrote:
> New functions to examine the contents of heap pages, as discussed
> recently on -hackers. These are fully integrated into backend.
>
> Designed to be extended for other page layouts/contents. (Heikki has
> some similar functions for index pages).
>
> Docs included, applies cleanly, tests good.
>
> I'll maintain this with immediate fixes/additions as we go up to 8.3 and
> beyond, to assist review process of various patches that alter page
> contents.
>
> --
> Simon Riggs
> EnterpriseDB http://www.enterprisedb.com
>

[ Attachment, skipping... ]

>
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Have you searched our list archives?
>
> http://archives.postgresql.org

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

On Sat, 2007-04-07 at 11:53 -0400, Bruce Momjian wrote:

> This looks useful, but shouldn't it be part of /contrib/pgstattuple
> rather than in the backend?

Well, this was written with a view to it being usable for writing test
cases that checked the various tuple states as we went. It was
originally proposed when Tom asked "How will we test HOT?" (concurrent
psql is the other half of the required base functionality to write
sensible test cases).

If we see it as a manual test tool only, contrib is OK. But it has to be
in the backend to be usable in the regression test suite, so thats where
it was designed to go.

--
Simon Riggs
EnterpriseDB http://www.enterprisedb.com



---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend

Simon Riggs wrote:
> On Sat, 2007-04-07 at 11:53 -0400, Bruce Momjian wrote:
>
> > This looks useful, but shouldn't it be part of /contrib/pgstattuple
> > rather than in the backend?
>
> Well, this was written with a view to it being usable for writing test
> cases that checked the various tuple states as we went. It was
> originally proposed when Tom asked "How will we test HOT?" (concurrent
> psql is the other half of the required base functionality to write
> sensible test cases).
>
> If we see it as a manual test tool only, contrib is OK. But it has to be
> in the backend to be usable in the regression test suite, so thats where
> it was designed to go.

Well, contrib can have its own regression tests. We can put the HOT
tests in there too.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster

"Simon Riggs" <simon@2ndquadrant.com> writes:
> If we see it as a manual test tool only, contrib is OK. But it has to be
> in the backend to be usable in the regression test suite, so thats where
> it was designed to go.

The core regression tests have depended on some contrib stuff forever,
so the above argument holds no water.

I'm uncomfortable with putting these things in core because I suspect
it's not hard to crash the backend (or worse) by feeding one of them
artfully corrupted data.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 7: You can help support the PostgreSQL project by donating at

http://www.postgresql.org/about/donate

On Sat, 2007-04-07 at 18:19 -0400, Bruce Momjian wrote:
> Simon Riggs wrote:
> > On Sat, 2007-04-07 at 11:53 -0400, Bruce Momjian wrote:
> >
> > > This looks useful, but shouldn't it be part of /contrib/pgstattuple
> > > rather than in the backend?
> >
> > Well, this was written with a view to it being usable for writing test
> > cases that checked the various tuple states as we went. It was
> > originally proposed when Tom asked "How will we test HOT?" (concurrent
> > psql is the other half of the required base functionality to write
> > sensible test cases).
> >
> > If we see it as a manual test tool only, contrib is OK. But it has to be
> > in the backend to be usable in the regression test suite, so thats where
> > it was designed to go.
>
> Well, contrib can have its own regression tests. We can put the HOT
> tests in there too.

OK.

I'll rework it once I've completed the other items on my list. Feel free
to have a hack at it if I'm a little slow. I only have today left before
I'm away for two weeks - travelling tomorrow.

--
Simon Riggs
EnterpriseDB http://www.enterprisedb.com



---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq

Simon Riggs wrote:
> New functions to examine the contents of heap pages, as discussed
> recently on -hackers. These are fully integrated into backend.
> ...
> I'll maintain this with immediate fixes/additions as we go up to 8.3 and
> beyond, to assist review process of various patches that alter page
> contents.

Excellent. I agree with the other comments that these should all be in
contrib.

The functions need to be very careful to handle invalid data gracefully.
For example, this will return bogus data or crash if the offset in the
line pointer is corrupt, past the end of page for example:

+ tuphdr = (HeapTupleHeader) PageGetItem((Page) inter_call_data->page, id);

It's particularly important if the page-examining functions are not
superuser-only, to avoid vulnerabilities. Even if they are, the
functions are for diagnostic purposes and diagnosis is often done on
something that's corrupt.

Dare I propose deprecating the all the system columns except for ctid in
favor of these functions? I guess there's no immediate reason to remove
them, but these functions feel like a much better design than tightly
integrated system columns.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq