There was some discussion about the issues relating to using other SSL
libraries. In a nutshell, it came down to that we couldn't return
anything other than an OpenSSL pointer from PQgetssl because existing
programs simply wouldn't know what to do with it.

So, I was pondering what we might want from an alternative. What I've
come up with is the following:

PGresult *PQgettlsinfo(PGconn *conn);

What it does instead of returning a single pointer is return a PGresult
that has various info depending on the library involved. For example,
if you connected using a libpq compiled with GnuTLS it would contain
the following:

key | value
---------------------+---------------------------------------
tls_library | GnuTLS
tls_library_version | 1.0.16
tls_sslmode | prefer
tls_active | yes
tls_verify_server | yes
tls_peerdn | C=AU,ST=NSW,L=Sydney,O=Home,CN=Myself
tls_peercn | Myself
tls_protocol | TLS 1.0
tls_cipher | AES 256 CBC
tls_keysize | 256 bits
tls_kx | DHE RSA
tls_mac | SHA
tls_compression | NULL
tls_certtype | X.509
(14 rows)

And when you connected with OpenSSL you would get something like:

key | value
---------------------+----------------------------------------
tls_library | OpenSSL
tls_library_version | OpenSSL 0.9.7e 25 Oct 2004
tls_sslmode | prefer
tls_active | yes
tls_peerdn | /C=AU/ST=NSW/L=Sydney/O=Home/CN=Myself
tls_peercn | Myself
tls_cipher | DHE-RSA-AES256-SHA
tls_protocol | TLSv1/SSLv3
tls_keysize | 256 bits
(9 rows)

Now, other than for the first time giving users access to the
information like peer DN and CN, it also provides some other
information they might want. And it's done in a way that's extensible.

Do people like this idea?

Note, I don't return a pointer to the GnuTLS session anywhere. I think
that's a bad idea all round and we need to provide another way for
programs to acheive the same effect.

The thing is, it could be extended to include almost anything. One
example would be if the user authenticated using kerberos, we could add
a few rows indicating that. I suppose you would call it
PQgetconninfo().

Thoughts?

*** PostgreSQL with GnuTLS

I've got it almost completely working and have tested interoperability.
You can find it here:

http://svana.org/kleptog/temp/gnutls.patch

The patch does the following:

- Adds configure stuff for gnutls so it checks for the libraries when
you specify --with-gnutls. You may need to run autoconf and autoheader
after patching.

- Both fe-secure.c and be-secure.c have been made TLS library agnostic.
They only refer to functions that implement TLS specific stuff which
are implemented in the files:

src/interfaces/libpq/fe-secure-openssl.c
src/interfaces/libpq/fe-secure-gnutls.c
src/backend/libpq/be-secure-openssl.c
src/backend/libpq/be-secure-gnutls.c

The makefile determines which (if either) is linked in.

- Implements the PQgettlsinfo() as described above and alters psql to
use it. Hence psql is now also TLS library agnostic.

Differences between the two implementations are:

- GnuTLS generates the DH key on the fly on server start, which takes a
few seconds. The OpenSSL versions use hardcoded keys which can be
overridden by the user. Not sure which is best here.

- This breaks psqlODBC when it uses libpq because it wants to use OpenSSL
and when libpq is compiled with GnuTLS that obviously won't work.
Recent thread on -hackers found no resolution for this problem.

- Both support authentication of the server and authentication of the
client, though more testing is need to test all the different
combinations of keys and certificates that are allowed.

- Different output for PQgettlsinfo()

That about it. There's no real difference from the users point of view,
it Just Works either way. In the future we may be able to use the PGP
support in GnuTLS. In other words, provide the server with a pgp
keyring and it accepts any user which a matching key in the keyring.

I hope to post of -patches sometime soon, once some of the kinks have
been ironed out.

Have a nice day,
--
Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/
> Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a
> tool for doing 5% of the work and then sitting around waiting for someone
> else to do the other 95% so you can sue them.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFEQqu0IB7bNG8LQkwRAp+KAJ9CQbKGfnoNJtMoEJpy8c r1O5t6xQCcCT29
G1W1pgBdCVeN9bCiZC8GJPs=
=dCs9
-----END PGP SIGNATURE-----

Martijn van Oosterhout <kleptog@svana.org> writes:
> Do people like this idea?

Not really ...

> Note, I don't return a pointer to the GnuTLS session anywhere. I think
> that's a bad idea all round and we need to provide another way for
> programs to acheive the same effect.

No, failing to provide that is the bad idea, because then you're buying
into the notion that libpq will provide a universal API that will
incorporate anything anyone could possibly want to do with the
underlying SSL library. The above is *not* that, prima facie because
it is read-only access. Even if this was a feasible goal, it would absorb
a lot of time on our part that could be better spent elsewhere, plus a
lot of time on the part of app programmers rewriting existing
OpenSSL-aware or GnuTLS-aware code to instead use whatever random API we
tell them they ought to use. They have better things to do with their
time, too.

> I've got it almost completely working and have tested interoperability.
> ...
> - This breaks psqlODBC when it uses libpq because it wants to use OpenSSL
> and when libpq is compiled with GnuTLS that obviously won't work.

That alone is sufficient reason why we're not going down that path.
If we expose a GnuTLS-handle-fetching API then it's up to the ODBC
guys to extend their code to handle that SSL library when they feel
like it. But telling them that we're simply going to break their
code and not provide them a path to fix it is not happening.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings

On Sun, Apr 16, 2006 at 05:29:04PM -0400, Tom Lane wrote:
> No, failing to provide that is the bad idea, because then you're buying
> into the notion that libpq will provide a universal API that will
> incorporate anything anyone could possibly want to do with the
> underlying SSL library. The above is *not* that, prima facie because
> it is read-only access. Even if this was a feasible goal, it would absorb

The intention is not to provide access to everything. If people want to
know more about the certificate, we simply export the certificate to
them and they can do with it what they like, including sending it to
another program. I wasn't expecting that list to grow much because
there's not much to export.

Besides, what's wrong with read-only access? What parameters were you
expecting them to want to change? After a session is setup there are no
parameters to change anymore. All you need is read and write.

> a lot of time on our part that could be better spent elsewhere, plus a
> lot of time on the part of app programmers rewriting existing
> OpenSSL-aware or GnuTLS-aware code to instead use whatever random API we
> tell them they ought to use. They have better things to do with their
> time, too.

The whole point is that app writers should not be aware at all which
library we're using. At the moment psqlODBC requires openssl because we
force them to. They only use three OpenSSL functions, SSL_read,
SSL_write, and SSL_get_error.

If we provided a hook to allow people read/write directly, they
wouldn't need to know about the SSL connection at all. I think that's a
much better way to go than adding a new library specific function for
every little feature we add.

You objected to this on the grounds of a problem with the COPY
functions, except I can't see any problem that's relevent. The problem
with copy was that data didn't have a length. Given the user is sending
their own packets, we always have a length.

> > - This breaks psqlODBC when it uses libpq because it wants to use OpenSSL
> > and when libpq is compiled with GnuTLS that obviously won't work.
>
> That alone is sufficient reason why we're not going down that path.
> If we expose a GnuTLS-handle-fetching API then it's up to the ODBC
> guys to extend their code to handle that SSL library when they feel
> like it. But telling them that we're simply going to break their
> code and not provide them a path to fix it is not happening.

By going down this path you're saying that psql will never be able
display the cipher of an SSL connection if the libpq was compiled with
a different library. If we provide a read/write than psqlODBC can
remove code and it will work with GnuTLS. Isn't that much better?

Have a nice day,
--
Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/
> Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a
> tool for doing 5% of the work and then sitting around waiting for someone
> else to do the other 95% so you can sue them.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFEQ6D+IB7bNG8LQkwRApYpAJ911KjiFr6j0y3v22GVrP jaDTZbmQCfaREF
s6SCWN87rwOWIEpjZtTXSdk=
=ovpt
-----END PGP SIGNATURE-----

Martijn van Oosterhout <kleptog@svana.org> writes:
> On Sun, Apr 16, 2006 at 05:29:04PM -0400, Tom Lane wrote:
>> No, failing to provide that is the bad idea, because then you're buying
>> into the notion that libpq will provide a universal API that will
>> incorporate anything anyone could possibly want to do with the
>> underlying SSL library. ...
> [ snip ]
> Besides, what's wrong with read-only access?

Well, psqlODBC seems a sufficient counterexample. But the problem
with this is that you're asking a bunch of non-SSL-experts to design,
evaluate, and then maintain an API for an SSL library. The real
answer to the above is "I don't know, and I doubt you do either."
This is the sort of problem that we should be avoiding, rather than
going out of our way to get involved in. PQgetssl made it possible
for us to stay out of the way for SSL-using applications, and I think
we should continue to follow that philosophy for other SSL libraries.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster

* Martijn van Oosterhout (kleptog@svana.org) wrote:
> The intention is not to provide access to everything. If people want to
> know more about the certificate, we simply export the certificate to
> them and they can do with it what they like, including sending it to
> another program. I wasn't expecting that list to grow much because
> there's not much to export.
>
> Besides, what's wrong with read-only access? What parameters were you
> expecting them to want to change? After a session is setup there are no
> parameters to change anymore. All you need is read and write.

I have to agree with this... Certificate handling isn't growing new
things in leaps and bounds these days and as long as the certificates
are available to the user then the other things are really just
nicities for people who don't want to decode the certs themselves.

> > a lot of time on our part that could be better spent elsewhere, plus a
> > lot of time on the part of app programmers rewriting existing
> > OpenSSL-aware or GnuTLS-aware code to instead use whatever random API we
> > tell them they ought to use. They have better things to do with their
> > time, too.
>
> The whole point is that app writers should not be aware at all which
> library we're using. At the moment psqlODBC requires openssl because we
> force them to. They only use three OpenSSL functions, SSL_read,
> SSL_write, and SSL_get_error.
>
> If we provided a hook to allow people read/write directly, they
> wouldn't need to know about the SSL connection at all. I think that's a
> much better way to go than adding a new library specific function for
> every little feature we add.

I have to agree with Martijn here too. It's not all that expensive to
provide read/write calls to abstract away the specific library being
used (since psqlODBC, at least, couldn't care less which library is
being used, really) rather than ask for application developers to write
their apps to support multiple SSL libraries to handle the case where
libpq is compiled with one library vs. compiled with another...

> > > - This breaks psqlODBC when it uses libpq because it wants to use OpenSSL
> > > and when libpq is compiled with GnuTLS that obviously won't work.
> >
> > That alone is sufficient reason why we're not going down that path.
> > If we expose a GnuTLS-handle-fetching API then it's up to the ODBC
> > guys to extend their code to handle that SSL library when they feel
> > like it. But telling them that we're simply going to break their
> > code and not provide them a path to fix it is not happening.
>
> By going down this path you're saying that psql will never be able
> display the cipher of an SSL connection if the libpq was compiled with
> a different library. If we provide a read/write than psqlODBC can
> remove code and it will work with GnuTLS. Isn't that much better?

I didn't really understand Tom's point here either... In order to not
break psqlODBC when libpq is compiled with GnuTLS we'd have to code up
support for returning an SSL* from the GnuTLS library that would work
for all the OpenSSL functions which just isn't going to happen... I
agree that we probably shouldn't go out of our way to break psqlODBC
when libpq is compiled with OpenSSL (and so we shouldn't really change
PQgetssl but mark it obsolete and ask that people don't use it and
provide some other way for psqlODBC to do what it wants, or not, perhaps
return NULL when using GnuTLS or '0x1', or not define it when libpq is
compiled with GnuTLS...).

Thanks,

Stephen

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (GNU/Linux)

iD8DBQFEQ68urzgMPqB3kigRAqz2AJ9eMUb8m5P+nruwkQW1cz 3wE8ZcwgCaAi6K
tjKpj7bl0jBrcTuZmi2BLaQ=
=MI5U
-----END PGP SIGNATURE-----

Stephen Frost <sfrost@snowman.net> writes:
> I have to agree with Martijn here too. It's not all that expensive to
> provide read/write calls to abstract away the specific library being
> used (since psqlODBC, at least, couldn't care less which library is
> being used, really)

You're failing to consider async applications. AFAICS, the *minimum*
API would be
read
write
read ready?
write ready?
get socket so I can use it in select()
(very possibly there's some stuff I missed, considering I haven't
consumed any caffeine yet today...). And that's just considering
the data transport aspect of it. I'm still concerned that SSL-using
apps may wish to twiddle the SSL library in ways we don't even know
about.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster

On Mon, Apr 17, 2006 at 11:07:26AM -0400, Stephen Frost wrote:
> I didn't really understand Tom's point here either... In order to not
> break psqlODBC when libpq is compiled with GnuTLS we'd have to code up
> support for returning an SSL* from the GnuTLS library that would work
> for all the OpenSSL functions which just isn't going to happen... I
> agree that we probably shouldn't go out of our way to break psqlODBC
> when libpq is compiled with OpenSSL (and so we shouldn't really change
> PQgetssl but mark it obsolete and ask that people don't use it and
> provide some other way for psqlODBC to do what it wants, or not, perhaps
> return NULL when using GnuTLS or '0x1', or not define it when libpq is
> compiled with GnuTLS...).

Just mark PQgetssl() obsolete. We have to return a NULL or a valid
pointer, otherwise existing programs will just crash. The major reason
for this PQgettlsinfo() function is so people can actually *know* which
library is active. Consider stuff like this:

PQgetssl() not NULL -> using openssl
PQgetgnutls not NULL -> using gnutls
else
not using ssl OR using some unknown library

That's not a way to design an interface. This PQgettlsinfo() would tell
you, one way or the other, if there is a TLS library in use. Give the
user the certificates and the cipher and if the user wants more info,
they can use the SSL library of their choice to get the information
they want.

Note, some of the info there is not stuff you can actually get from
the PQgetssl() function currently anyway. You can't ask the SSL library
easily if we provided a certificate during authentication. libpq knows
this easily.

Have a nice day,
--
Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/
> Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a
> tool for doing 5% of the work and then sitting around waiting for someone
> else to do the other 95% so you can sue them.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFEQ7oPIB7bNG8LQkwRAgAfAJsFEQABZlZj12e8cK+J63 GZXFJ1TACeOVNd
wMOmfBF6/JaF8B5dPfNbNlI=
=lJuP
-----END PGP SIGNATURE-----

* Tom Lane (tgl@sss.pgh.pa.us) wrote:
> Stephen Frost <sfrost@snowman.net> writes:
> > I have to agree with Martijn here too. It's not all that expensive to
> > provide read/write calls to abstract away the specific library being
> > used (since psqlODBC, at least, couldn't care less which library is
> > being used, really)
>
> You're failing to consider async applications. AFAICS, the *minimum*

You're assuming I intended just 'read()' and 'write()', I suppose I
should have been more explicit but I certainly understand there's more
than just read() and write() involved, though not by much. I'm not sure
I see a reason we wouldn't just use the existing PQsocket() for the same
socket back to the user for select()-based systems (don't we do this for
SSL-enabled connections too anyway?)... There's not a whole lot special
when it comes to handling data transport, or much that has changed
lately or is likely to change anytime soon.

> consumed any caffeine yet today...). And that's just considering
> the data transport aspect of it. I'm still concerned that SSL-using
> apps may wish to twiddle the SSL library in ways we don't even know
> about.

SSL is all about the certificates, really. As long as we provide
that back to the user in a library-agnostic way they'll be able to
do whatever they like with it, such as compare it to a CRL or use OCSP
to check the current status. Of course, these are usually things you're
more concerned about on the *server* side, which we don't currently
support anyway. With supporting multiple SSL libraries we'll have to
deal with these issues on the *server* side, should we decide to
implement them someday (which would be nice...) anyway so trying to
play like we don't want to or don't care to know about them doesn't
really work. Just saying we won't do CRLs or OCSP ever would just mean
we don't get run in some environments.

Of course, in either case we can play the "if you want it, write the
code" card but I'd really expect to get complaints from admins who want
to use X.509 but need CRL/OCSP support before getting complaints from
application developers that they need libpq to provide something more
for them...

Thanks,

Stephen

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (GNU/Linux)

iD8DBQFEQ77IrzgMPqB3kigRAv6BAKCBYymtY9ZxjSKWmUF/5aDf41ZUMQCghS1/
GV6c9efGzrSdzJQzz9urGY8=
=etcR
-----END PGP SIGNATURE-----

On Mon, Apr 17, 2006 at 11:25:26AM -0400, Tom Lane wrote:
> You're failing to consider async applications. AFAICS, the *minimum*
> API would be
> read
> write
> read ready?
> write ready?
> get socket so I can use it in select()

Actually, you only need two functions:

read
write

The "get socket" already exists as PQsocket().

Both SSL libraries work perfectly fine if the user sets the connection
to non-blocking via PQsetnonblocking(). They just return -EAGAIN. I was
actually thinking of the two functions as follows:

typedef PostgresPollingStatusType pq_read_func( PGconn *conn, void *buf, int *len);
typedef PostgresPollingStatusType pq_write_func( PGconn *conn, const void *buf, int *len);

The existing PostgresPollingStatusType seems to handle both blocking
and non-blocking states just fine.

> (very possibly there's some stuff I missed, considering I haven't
> consumed any caffeine yet today...). And that's just considering
> the data transport aspect of it. I'm still concerned that SSL-using
> apps may wish to twiddle the SSL library in ways we don't even know
> about.

Well, I checked a few libs like libcurl and libldap. They generally
allow you to configure the files containing the certificates but that's
about it.

Seriously, if people want to do really sophisticated things with the
SSL library, they should setup s_tunnel instead. If we wanted to let
users control everything we'd allow Anonymous DH key transfers. The
README.SSL lists the situations were SSL makes sense and if you fall
outside of that you shouldn't be using SSL.

All I'm asking for is that libpq be made SSL-library *agnostic* so that
users like psqlODBC can just *use* the connection with having to jump
through hoops.

Have a nice day,
--
Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/
> Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a
> tool for doing 5% of the work and then sitting around waiting for someone
> else to do the other 95% so you can sue them.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFEQ77yIB7bNG8LQkwRAn9rAJkBxfPu/CgatXjXNACuAHczse+acwCdE0AO
jbgE4JJ6pq17F7Q8J+VJ5ok=
=4ZPG
-----END PGP SIGNATURE-----

* Martijn van Oosterhout (kleptog@svana.org) wrote:
> Seriously, if people want to do really sophisticated things with the
> SSL library, they should setup s_tunnel instead. If we wanted to let

I certainly agree with all the rest but I'm just not sure I can agree
with you here. While s_tunnel is nice it's not always an option and I
think it *would* be nice to have Postgres support things like CRLs and
OCSP but more from the server-side of things than the client-side.

Thanks,

Stephen

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (GNU/Linux)

iD8DBQFEQ8FIrzgMPqB3kigRAshvAKCbc2x4G5tYzc4eh04ZK4 jGKzaeqACfSjWV
t6Uf3E00eRH5XWjYyR8IUoQ=
=ALcC
-----END PGP SIGNATURE-----