> -----Original Message-----
> From: Martijn van Oosterhout [mailto:kleptog@svana.org]
> Sent: 13 April 2006 07:58
> To: Dave Page
> Cc: pgsql-hackers@postgresql.org; Hiroshi Inoue
> Subject: Re: [HACKERS] Practical impediment to supporting
> multiple SSL libraries
>
> On Wed, Apr 12, 2006 at 05:25:47PM +0100, Dave Page wrote:
> > The driver implements all versions of the wire protocol
> itself, but if
> > libpq is available at runtime (it will dynamically load it on
> > platforms that support it) it can use it for connection setup so
> > features like SSL can be provided easily. I'm still not overly
> > familiar with how it works yet, but I'm sure Hiroshi (CC'd) can
> > provide further details if you need them.
>
> Right, so what you're basically doing is setting up the
> connection via libpq then grabbing the SSL pointer and using
> that to continue communicating. If it's not SSL you use
> PQsocket get the socket and continue from there.

Yup.

> Unorthodox usage, but it should work.

Well, we had a pure custom implementation of the protocol, had a pure
libpq based version and after much discussion decided that the best
version of all was the hybrid as it allowed us to hijack features like
SSL, Kerberos, pgpass et al, yet not be constrained by the limitations
of libpq, or copy query results about so much.

Regards, Dave

---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq

On Thu, Apr 13, 2006 at 08:48:54AM +0100, Dave Page wrote:
> Well, we had a pure custom implementation of the protocol, had a pure
> libpq based version and after much discussion decided that the best
> version of all was the hybrid as it allowed us to hijack features like
> SSL, Kerberos, pgpass et al, yet not be constrained by the limitations
> of libpq, or copy query results about so much.

Right. Would you see value in a more formal libpq "hijack-me" interface
that would support making the initial connection and then handing off
the rest to something else?

I'm wondering because obviously with the current setup, if libpq is
compiled with SSL support, psqlODBC must also be. Are there any points
where you have to fight libpq over control of the socket?

I'm thinking that such an interface would need to provide the
following:

read (sync/async)
write (sync/async)
getfd (for select/poll)
ispending (is there stuff to do)
release (for when you're finished)

Is there anything else you might need?
--
Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/
> Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a
> tool for doing 5% of the work and then sitting around waiting for someone
> else to do the other 95% so you can sue them.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFEPgiUIB7bNG8LQkwRAi89AJ9Nrkp5hCU4bf/VPVZxv+KuepsUawCghmra
p8jOYXXxeIdcdcoNNndezNI=
=bCuS
-----END PGP SIGNATURE-----

* Martijn van Oosterhout (kleptog@svana.org) wrote:
> On Thu, Apr 13, 2006 at 08:48:54AM +0100, Dave Page wrote:
> > Well, we had a pure custom implementation of the protocol, had a pure
> > libpq based version and after much discussion decided that the best
> > version of all was the hybrid as it allowed us to hijack features like
> > SSL, Kerberos, pgpass et al, yet not be constrained by the limitations
> > of libpq, or copy query results about so much.
>
> Right. Would you see value in a more formal libpq "hijack-me" interface
> that would support making the initial connection and then handing off
> the rest to something else?
>
> I'm wondering because obviously with the current setup, if libpq is
> compiled with SSL support, psqlODBC must also be. Are there any points
> where you have to fight libpq over control of the socket?
[...]
> Is there anything else you might need?

Instead of having it hijack the libpq connection and implement the
wireline protocol itself, why don't we work on fixing the problems (such
as the double-copying that libpq requires) in libpq to allow the driver
(and others!) to use it in the 'orthodox' way?

I would have spoken up on the ODBC list if I understood that 'hybrid'
really meant 'just using libpq for connection/authentication'. I really
think it's a bad idea to have the ODBC driver reimplement the wireline
protocol because that protocol does change from time to time and someone
using libpq will hopefully have fewer changes (and thus makes the code
easier to maintain) than someone implementing the wireline protocol
themselves (just causing more busy-work that, at least we saw in the
past with the ODBC driver, may end up taking *forever* for someone to
be able to commit the extra required time to implement).

Thanks,

Stephen

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (GNU/Linux)

iD8DBQFEPit8rzgMPqB3kigRAq+qAJ4vfYImh985AHnJzuCN4W HwJoiqVgCglBlH
q/3jzMYyEVD6Im7dh219yek=
=WBds
-----END PGP SIGNATURE-----

On Thu, Apr 13, 2006 at 06:44:12AM -0400, Stephen Frost wrote:
> Instead of having it hijack the libpq connection and implement the
> wireline protocol itself, why don't we work on fixing the problems (such
> as the double-copying that libpq requires) in libpq to allow the driver
> (and others!) to use it in the 'orthodox' way?

Ok. I'm not sure what this "double copying" you're referring to is, but
I'd certaintly like to know why people are reimplementing the protocol
(psqlODBC is hardly the only one).

Is is that people want to use completely different interaction models?
Like work around the wait-for-whole-resultset-before-returing issue? Or
maybe better notice handling? What is it that's so deficient?

Or maybe it's portability? Like DBI PgPP module?

Have a nice day,
--
Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/
> Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a
> tool for doing 5% of the work and then sitting around waiting for someone
> else to do the other 95% so you can sue them.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFEPi21IB7bNG8LQkwRAhewAJ0ZLhN7F4o24BRnzbOmAC rJQrKUwgCfbGcx
higUqJRVHi/q4H3uQgXf69k=
=jFxA
-----END PGP SIGNATURE-----

Martijn van Oosterhout <kleptog@svana.org> writes:
> Right. Would you see value in a more formal libpq "hijack-me" interface
> that would support making the initial connection and then handing off
> the rest to something else?

I think this would just be busywork... the way ODBC is doing it seems
fine to me. In any case, do we really want to encourage random apps to
bypass the library? For one thing, with an API such as you suggest,
it would really be libpq's problem to figure out what to do with regular
vs passthrough calls. As it stands, it's very obviously not libpq's
problem anymore once you hijack the socket.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend

Stephen Frost wrote:

>* Martijn van Oosterhout (kleptog@svana.org) wrote:
>
>
>>On Thu, Apr 13, 2006 at 08:48:54AM +0100, Dave Page wrote:
>>
>>
>>>Well, we had a pure custom implementation of the protocol, had a pure
>>>libpq based version and after much discussion decided that the best
>>>version of all was the hybrid as it allowed us to hijack features like
>>>SSL, Kerberos, pgpass et al, yet not be constrained by the limitations
>>>of libpq, or copy query results about so much.
>>>
>>>
>>Right. Would you see value in a more formal libpq "hijack-me" interface
>>that would support making the initial connection and then handing off
>>the rest to something else?
>>
>>I'm wondering because obviously with the current setup, if libpq is
>>compiled with SSL support, psqlODBC must also be. Are there any points
>>where you have to fight libpq over control of the socket?
>>
>>
>[...]
>
>
>>Is there anything else you might need?
>>
>>
>
>Instead of having it hijack the libpq connection and implement the
>wireline protocol itself, why don't we work on fixing the problems (such
>as the double-copying that libpq requires) in libpq to allow the driver
>(and others!) to use it in the 'orthodox' way?
>
>I would have spoken up on the ODBC list if I understood that 'hybrid'
>really meant 'just using libpq for connection/authentication'. I really
>think it's a bad idea to have the ODBC driver reimplement the wireline
>protocol because that protocol does change from time to time and someone
>using libpq will hopefully have fewer changes (and thus makes the code
>easier to maintain) than someone implementing the wireline protocol
>themselves (just causing more busy-work that, at least we saw in the
>past with the ODBC driver, may end up taking *forever* for someone to
>be able to commit the extra required time to implement).
>
>

Libpq and the psqlodbc driver have walked on another road for a very
long time.
In 6.3 or before, there wasn't a libpq library under Windows. In 6.4 we
had the
libpq library under Windows but it wasn't able to talk to 6.3 or before
unfortunately....
At last in 7.4 the libpq was able to speak both protocol v3 and protocol
v2 but it is
a pretty hard work at least for me to tranfer all the accummulated works
to libpq based
version. I'm not sure what kind of functionalities required for libpq to
make the tranfer
easy. Of cource double-copying issue is big one of them.

regards,
Hiroshi Inue


---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq