SEO

In addition to/instead of abstracting cmin/cmax to a phantom ID, what
about allowing for two versions of the tuple header, one with cid info
and one without? That would allow for cid info to be stripped out when
pages were written to disk.

The downside to this is that we'd have to be able to deal with pages
in-memory potentially being larger than pages on-disk. Since there's
been discussion of separating on-disk and in-memory page formats, maybe
that doesn't kill the proposal outright.
--
Jim Nasby jim@nasby.net
EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)

---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

Jim C. Nasby wrote:
> In addition to/instead of abstracting cmin/cmax to a phantom ID, what
> about allowing for two versions of the tuple header, one with cid info
> and one without? That would allow for cid info to be stripped out when
> pages were written to disk.
>

How exactly would that help? You can't just strip out cid info when
writing to disk, if you don't want to lose the information.

And it's certainly a lot more complicated than the phantom id thing.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com


---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

On Thu, Sep 28, 2006 at 05:13:11PM +0100, Heikki Linnakangas wrote:
> Jim C. Nasby wrote:
> >In addition to/instead of abstracting cmin/cmax to a phantom ID, what
> >about allowing for two versions of the tuple header, one with cid info
> >and one without? That would allow for cid info to be stripped out when
> >pages were written to disk.
> >
>
> How exactly would that help? You can't just strip out cid info when
> writing to disk, if you don't want to lose the information.

Erm, sorry, brainfart... yeah, we'd need to be able to write the info
out to disk.

The reason I thought of this is because once the transaction commits, we
have no use for the cid info. So we could do something like have
bgwriter look for tuples that belong to committed transactions before it
writes a page, and strip the cid out of them.

The problem with *that* is that (AFAIK) you'll need cid info again once
you go to update or delete that tuple. And that might obviously need to
spill to disk before the transaction commits.

Back to the drawing board...
--
Jim Nasby jim@nasby.net
EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)

---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend

"Jim C. Nasby" <jim@nasby.net> wrote:

> The reason I thought of this is because once the transaction commits, we
> have no use for the cid info. So we could do something like have
> bgwriter look for tuples that belong to committed transactions before it
> writes a page, and strip the cid out of them.

Your concept is just like as the experimental method that I suggested before
in http://archives.postgresql.org/pgsql...8/msg01193.php
We can remove cmin and cmax from commited tuples and xmin from frozen tuples
and we might save some bytes in tuple headers.

However, I think our next goal to shrink the headers is 16 bytes. The headers
become 23 bytes using phantom cids and we are limited by alignments, so we will
have no more advantages unless we delete extra 7 bytes in the headers.
....and it seems to be very difficult.

Regards,
---
ITAGAKI Takahiro
NTT Open Source Software Center



---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend

ITAGAKI Takahiro wrote:
> However, I think our next goal to shrink the headers is 16 bytes. The
> headers
> become 23 bytes using phantom cids and we are limited by alignments,
> so we will
> have no more advantages unless we delete extra 7 bytes in the headers.
> ...and it seems to be very difficult.

Yeah, I thought about that too earlier.

If we get rid of VACUUM FULL, or replace it with something that doesn't
need xvac, and keep cmin and cmax in backend-private storage, we could
get rid of the overlayed t_field4, which is 4 bytes. Then we're down to
19 bytes.

We could get rid of t_hoff, because we should always be able to
calculate the header size. Then we're down to 18 bytes.

There's currently 15 bits in use in the infomask. After we remove the
HEAP_MOVED_* fields that we don't need without VACUUM FULL, that's down
to 13 bits. t_natts only needs 11 bits, because MaxHeapAttributeNumber
is 1600. We could move 5 of the bits in infomask to the high 5 bits of
t_natts, and save one byte.

We're now down to 17 bytes. That's as far as I got.

So it seems we could shave off some bytes, but we still can't get down
to 16. And the changes needed in total would be quite massive.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

On Fri, Sep 29, 2006 at 09:35:31AM +0100, Heikki Linnakangas wrote:
> We could get rid of t_hoff, because we should always be able to
> calculate the header size. Then we're down to 18 bytes.

Without t_hoff, how do you know the size of the null bitmap? You could
probably do it only if you assume the null bitmap, if present, always
covers all the columns...

Have a nice day,
--
Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/
> From each according to his ability. To each according to his ability to litigate.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFFHOz7IB7bNG8LQkwRAuH7AJ4x+jTTRWdrzABWj5uzZX Zvfc9tEwCgg1L7
wrqlnUREVnGhdps/vFzcNvU=
=PTSK
-----END PGP SIGNATURE-----

On Fri, Sep 29, 2006 at 10:59:13AM +0100, Heikki Linnakangas wrote:
> Martijn van Oosterhout wrote:
> >On Fri, Sep 29, 2006 at 09:35:31AM +0100, Heikki Linnakangas wrote:
> >>We could get rid of t_hoff, because we should always be able to
> >>calculate the header size. Then we're down to 18 bytes.
> >
> >Without t_hoff, how do you know the size of the null bitmap? You could
> >probably do it only if you assume the null bitmap, if present, always
> >covers all the columns...
>
> I think we assume that already. heap_form_tuple reserves space for the
> bitmap like this:
>
> if (hasnull)
> len += BITMAPLEN(numberOfAttributes);

Ok, now we do an ALTER TABLE blah ADD COLUMN ..., and we have to expand
the bitmaps for the entire table?

Have a nice day,
--
Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/
> From each according to his ability. To each according to his ability to litigate.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFFHO/bIB7bNG8LQkwRArFGAJ9NsWVcI/xYLIgfmebSqJN6a6m6mACdH4+0
z2bKlmrNIQzvsc9vAkSx30Q=
=ZX3O
-----END PGP SIGNATURE-----

Martijn van Oosterhout wrote:
> On Fri, Sep 29, 2006 at 10:59:13AM +0100, Heikki Linnakangas wrote:
>> Martijn van Oosterhout wrote:
>>> On Fri, Sep 29, 2006 at 09:35:31AM +0100, Heikki Linnakangas wrote:
>>>> We could get rid of t_hoff, because we should always be able to
>>>> calculate the header size. Then we're down to 18 bytes.
>>> Without t_hoff, how do you know the size of the null bitmap? You could
>>> probably do it only if you assume the null bitmap, if present, always
>>> covers all the columns...
>> I think we assume that already. heap_form_tuple reserves space for the
>> bitmap like this:
>>
>> if (hasnull)
>> len += BITMAPLEN(numberOfAttributes);
>
> Ok, now we do an ALTER TABLE blah ADD COLUMN ..., and we have to expand
> the bitmaps for the entire table?
No, you'd still have the the number of attributes (t_natts) in the header.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?

http://archives.postgresql.org

Martijn van Oosterhout wrote:
> On Fri, Sep 29, 2006 at 09:35:31AM +0100, Heikki Linnakangas wrote:
>> We could get rid of t_hoff, because we should always be able to
>> calculate the header size. Then we're down to 18 bytes.
>
> Without t_hoff, how do you know the size of the null bitmap? You could
> probably do it only if you assume the null bitmap, if present, always
> covers all the columns...

I think we assume that already. heap_form_tuple reserves space for the
bitmap like this:

if (hasnull)
len += BITMAPLEN(numberOfAttributes);

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?

http://archives.postgresql.org

On Fri, Sep 29, 2006 at 01:15:06PM +0900, ITAGAKI Takahiro wrote:
>
> "Jim C. Nasby" <jim@nasby.net> wrote:
>
> > The reason I thought of this is because once the transaction commits, we
> > have no use for the cid info. So we could do something like have
> > bgwriter look for tuples that belong to committed transactions before it
> > writes a page, and strip the cid out of them.
>
> Your concept is just like as the experimental method that I suggested before
> in http://archives.postgresql.org/pgsql...8/msg01193.php
> We can remove cmin and cmax from commited tuples and xmin from frozen tuples
> and we might save some bytes in tuple headers.
>
> However, I think our next goal to shrink the headers is 16 bytes. The headers
> become 23 bytes using phantom cids and we are limited by alignments, so we will
> have no more advantages unless we delete extra 7 bytes in the headers.
> ...and it seems to be very difficult.

Dumb question... wouldn't getting down to 20 bytes buy us something?
--
Jim Nasby jim@nasby.net
EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)

---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?

http://archives.postgresql.org