SEO

With the 8.0 server release coming later this week, I've put up a new set
of JDBC drivers. I haven't had time to write up a full set of release
notes, so this email will have to suffice for now...

Naturally there are a huge number of minor improvements and fixes,
but these are the big things in 8.0:

- True V3 protocol support.
- Savepoints.
- No more "idle in transaction" problems.
- Binary data doesn't use fifty times the necessary memory.
- SSL connection customization
- driver configuration via properties file
- 5.0 JDK compiles

Major compatibility issues that come to mind immediately.

The driver now always uses some form of server prepared statement. This
means that all data is now strongly typed instead of mashed together into
a query string for the server to figure out. If you have an application
that uses setString(x, myString) for every type, you are not going to be
happy. The driver needs to know what the real column type is in the
database and it infers this from the setXXX call you make. This presents
some tricky problems when dealing with postgresql datatypes that do not
match up exactly with Java datatypes that are only solved by using a
custom PGobject subclass to pass both the data and the correct type to the
driver. This also extends to null values, no longer is setObject(x, null)
valid because it does not contain type information for the driver to use.

Previously Statement.executeQuery("INSERT ... ; SELECT currval('myseq')");
would return a ResultSet. This is no longer true and I don't believe it
was ever strictly legal. This should be correctly written in separate
queries or:

Statement.execute("INSERT ...; SELECT currval('myseq')");
Statement.getMoreResults();
ResultSet rs = Statement.getResultSet();

Calling PreparedStatement.setBinaryStream now requires a correct length
argument. Previously Integer.MAX_VALUE or similar could be used and it
would read the stream to completion, but with the ability to stream data
directly to the server and not use up outrageous amounts of memory means
that we need to know the correct length from the outset.

Kris Jurka

---------------------------(end of broadcast)---------------------------
TIP 7: don't forget to increase your free space map settings

On Monday 17 January 2005 13:48, Kris Jurka wrote:
> With the 8.0 server release coming later this week, I've put up a new set
> of JDBC drivers. I haven't had time to write up a full set of release
> notes, so this email will have to suffice for now...
>
> Naturally there are a huge number of minor improvements and fixes,
> but these are the big things in 8.0:
>
> - True V3 protocol support.
> - Savepoints.
> - No more "idle in transaction" problems.
> - Binary data doesn't use fifty times the necessary memory.
> - SSL connection customization
> - driver configuration via properties file
> - 5.0 JDK compiles
>
> Major compatibility issues that come to mind immediately.
>
> The driver now always uses some form of server prepared statement. This
> means that all data is now strongly typed instead of mashed together into
> a query string for the server to figure out. If you have an application
> that uses setString(x, myString) for every type, you are not going to be
> happy. The driver needs to know what the real column type is in the
> database and it infers this from the setXXX call you make. This presents
> some tricky problems when dealing with postgresql datatypes that do not
> match up exactly with Java datatypes that are only solved by using a
> custom PGobject subclass to pass both the data and the correct type to the
> driver. This also extends to null values, no longer is setObject(x, null)
> valid because it does not contain type information for the driver to use.
>
> Previously Statement.executeQuery("INSERT ... ; SELECT currval('myseq')");
> would return a ResultSet. This is no longer true and I don't believe it
> was ever strictly legal. This should be correctly written in separate
> queries or:
>
> Statement.execute("INSERT ...; SELECT currval('myseq')");
> Statement.getMoreResults();
> ResultSet rs = Statement.getResultSet();
>
> Calling PreparedStatement.setBinaryStream now requires a correct length
> argument. Previously Integer.MAX_VALUE or similar could be used and it
> would read the stream to completion, but with the ability to stream data
> directly to the server and not use up outrageous amounts of memory means
> that we need to know the correct length from the outset.
>
> Kris Jurka

Hi!
I tried using postgresql-8.0.309.jdbc2.jar (and jdbc3) and got the following
NullPointerException:
Caused by: java.lang.NullPointerException
at org.postgresql.Driver.getDefaultProperties(Driver. java:84)
at org.postgresql.Driver.connect(Driver.java:167)
at org.apache.catalina.realm.JDBCRealm.open(JDBCRealm .java:599)

Any explaination?

--
Andreas Joseph Krogh <andreak@officenet.no>
Senior Software Developer / Manager
gpg public_key: http://dev.officenet.no/~andreak/public_key.asc
------------------------+---------------------------------------------+
OfficeNet AS | Can i wash my clothes with my dvd drive? |
Hoffsveien 17 | Or do i need to replace it with a washing |
PO. Box 425 Skøyen | machine?? |
0213 Oslo | |
NORWAY | |
Phone : +47 22 13 01 00 | |
Direct: +47 22 13 10 03 | |
Mobile: +47 909 56 963 | |
------------------------+---------------------------------------------+

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (GNU/Linux)

iD8DBQBB7OovUopImDh2gfQRAq0qAJ43Y52zDgf964rssVh9hl UIh0ulpACeKyiX
HCmlRjxn1V0KV3BC3RH/Pcg=
=Hlat
-----END PGP SIGNATURE-----

On Tuesday 18 January 2005 12:58, you wrote:
> Andreas Joseph Krogh wrote:
> > NullPointerException:
> > Caused by: java.lang.NullPointerException
> > at org.postgresql.Driver.getDefaultProperties(Driver. java:84)
> > at org.postgresql.Driver.connect(Driver.java:167)
> > at org.apache.catalina.realm.JDBCRealm.open(JDBCRealm .java:599)
>
> Driver.java:84 is:
>
> Enumeration urlEnum =
> getClass().getClassLoader().getResources("org/postgresql/driverconfig.prope
>rties");
>
> I guess that org.postgresql.Driver is being loaded by the bootstrap
> classloader (getClassLoader() returns null). How exactly are you getting
> the driver jar into the classpath?

I saw that line too, but would never have guessed that getClassLoader()
returnes null(don't know much about classloaders). I'm trying to load a
ConnectionPool in Tomcat-5.0.28 and had my pg.jar in
$TOMCAT_HOME/common/endorsed/, when I moved it to $TOMCAT_HOME/common/lib/ it
works ok. It used to work in common/endorsed with previous releases(including
8.0.beta4 which is the last one I tried before 8.0 final).

Anyway, the problem is now gone, thanks!

--
Andreas Joseph Krogh <andreak@officenet.no>
Senior Software Developer / Manager
gpg public_key: http://dev.officenet.no/~andreak/public_key.asc
------------------------+---------------------------------------------+
OfficeNet AS | Can i wash my clothes with my dvd drive? |
Hoffsveien 17 | Or do i need to replace it with a washing |
PO. Box 425 Skøyen | machine?? |
0213 Oslo | |
NORWAY | |
Phone : +47 22 13 01 00 | |
Direct: +47 22 13 10 03 | |
Mobile: +47 909 56 963 | |
------------------------+---------------------------------------------+

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (GNU/Linux)

iD8DBQBB7P30UopImDh2gfQRAiHSAKCoOQJYKI7iL4pOwZluEx cbT4Aq3ACfZvyN
jSqGB4DHinmY9+Szd8lgKs0=
=h+VG
-----END PGP SIGNATURE-----

Andreas Joseph Krogh wrote:

> NullPointerException:
> Caused by: java.lang.NullPointerException
> at org.postgresql.Driver.getDefaultProperties(Driver. java:84)
> at org.postgresql.Driver.connect(Driver.java:167)
> at org.apache.catalina.realm.JDBCRealm.open(JDBCRealm .java:599)

Driver.java:84 is:

Enumeration urlEnum =
getClass().getClassLoader().getResources("org/postgresql/driverconfig.properties");

I guess that org.postgresql.Driver is being loaded by the bootstrap
classloader (getClassLoader() returns null). How exactly are you getting
the driver jar into the classpath?

-O

---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives?

http://archives.postgresql.org

Am Montag, 17. Januar 2005 13:48 schrieb Kris Jurka:
> With the 8.0 server release coming later this week, I've put up a new set
> of JDBC drivers. I haven't had time to write up a full set of release
> notes, so this email will have to suffice for now...

Could you please not name the tarballs "postgresql-8.0.309.src.tar.gz" etc. I
think there is a chance that this will be confused with postgresql proper.
Maybe "postgresql-jdbc" is a better stem.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

Oliver Jowett wrote:
> Andreas Joseph Krogh wrote:
>
>> NullPointerException:
>> Caused by: java.lang.NullPointerException
>> at org.postgresql.Driver.getDefaultProperties(Driver. java:84)
>> at org.postgresql.Driver.connect(Driver.java:167)
>> at org.apache.catalina.realm.JDBCRealm.open(JDBCRealm .java:599)
>
> Driver.java:84 is:
>
> Enumeration urlEnum =
> getClass().getClassLoader().getResources("org/postgresql/driverconfig.properties");
>
> I guess that org.postgresql.Driver is being loaded by the bootstrap
> classloader (getClassLoader() returns null). How exactly are you getting
> the driver jar into the classpath?

I can reproduce the exception by running test code when the driver is
loaded from the bootstrap classpath
('-Xbootclasspath/a:jars/postgresql.jar')

It seems reasonable to fall back to the system classloader if the driver
was loaded from the bootstrap classloader. I will commit a change to do
this shortly.

-O

---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives?

http://archives.postgresql.org

Vadim Nasardinov wrote:

> Another reasonable course of action might be to try the context classloader:
> http://java.sun.com/j2se/1.4.2/docs/...ssLoader%28%29
>
> This would typically have the system classloader in its parent hierarchy,
> but it may also give you access to locations that the system classloader
> cannot see.

I am of the opinion that thread context classloaders are evil and should
be avoided if at all possible -- they make code fragile and sensitive to
caller context (almost by definition!)

Also it'd mean that we would have to load the defaults on every call to
getConnection(), rather than once per Driver instance (i.e. most
probably exactly once). Does it make sense to have the driver defaults
change depending on who is obtaining the connection?

-O

---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

On Tuesday 18 January 2005 17:17, Oliver Jowett wrote:
> I can reproduce the exception by running test code when the driver
> is loaded from the bootstrap classpath
> ('-Xbootclasspath/a:jars/postgresql.jar')
>
> It seems reasonable to fall back to the system classloader if the
> driver was loaded from the bootstrap classloader. I will commit a
> change to do this shortly.

Another reasonable course of action might be to try the context classloader:
http://java.sun.com/j2se/1.4.2/docs/...ssLoader%28%29

This would typically have the system classloader in its parent hierarchy,
but it may also give you access to locations that the system classloader
cannot see.

---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

On Tuesday 18 January 2005 18:07, Oliver Jowett wrote:
> I am of the opinion that thread context classloaders are evil

If that's a moral judgment, I have no comment. If a technical one --
where's the evidence? (My ignorance prevents me from taking sides in
this argument. I'm just curious.)

> and should be avoided if at all possible -- they make code fragile
> and sensitive to caller context (almost by definition!)

It doesn't have to be any more sensitive than the current situation.
Current code is also sensitive to caller context. The Driver.class's
classloader depends on the caller. The thrust of your argument seems
to be that this is a one-time dependency. Once established, it
doesn't changed for the remainder of the class's lifecycle.

Well, the same could be true of a context-classloader-based
implementation.

When you register the Driver instance with the DriverManager in the
static initializer, a (weak) reference to the current context
classloader may be saved in a static field. The field can be used
later on in the getDefaultProperties() method, if
getClass().getClassLoader() turns out to be null. (BTW, couldn't this
be Driver.class.getClassLoader() instead?)

> Does it make sense to have the driver defaults change depending on
> who is obtaining the connection?

The fact that this is nonsensical is irrelevant. Changing defaults on
a per connection basis has never been proposed.

More to the point, can someone explain what
/org/postgresql/driverconfig.properties files are actually used for?
Google doesn't seem to know:
http://www.google.com/search?q=%22dr....properties%22


Without knowing what this feature is actually used for, I'd speculate
groundlessly that the context-classloader-based implementation is more
likely to work in the application server scenario. Within the same
app server instance, multiple *.properties files are likely to exist
for use by different apps. These files aren't likely to be on the
CLASSPATH. Rather, they're likely to be in a per-application config
directory that the system classloader can't see.

---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

Vadim Nasardinov wrote:
> On Tuesday 18 January 2005 18:07, Oliver Jowett wrote:
>
>>I am of the opinion that thread context classloaders are evil
>
>
> If that's a moral judgment, I have no comment. If a technical one --
> where's the evidence? (My ignorance prevents me from taking sides in
> this argument. I'm just curious.)

It's just an opinion. I have been bitten by context classloader
weirdness often enough that I try to avoid it whenever I can.

If you use the CCL, you have added hidden coupling between whoever sets
the CCL and the use of the CCL. Hidden coupling is usually bad -- it
makes the code less obvious and more prone to context-sensitive
breakage. Many libraries depend on particular CCL settings *and do not
document this anywhere*. Really hidden coupling!

It gets worse if you are in a managed environment -- the environment may
be using the CCL for its own purposes, and your code may not even have
permission (security policy) to change the CCL.

To use a CCL-using library reliably in that sort of environment, you end
up setting and resetting the CCL on every call to the library (might
involve writing a privileged wrapper -- ugh!). This defeats the purpose
of having a CCL in the first place. Why not just configure the library
with the right ClassLoader via another mechanism, or explicitly pass a
ClassLoader to those methods that need to do class/resource loading?

Sorry .. it's a pet peeve of mine

Given that there is a non-ClassLoader-aware layer (DriverManager)
between the JDBC client and JDBC driver that we cannot change, if we
needed to pass a ClassLoader then using the CCL is probably the best
way. But I don't see that we *need* to pass a ClassLoader at all; see below.

> When you register the Driver instance with the DriverManager in the
> static initializer, a (weak) reference to the current context
> classloader may be saved in a static field. The field can be used
> later on in the getDefaultProperties() method, if
> getClass().getClassLoader() turns out to be null.

Ok, but I don't understand why you'd want to jump through all these
hoops in the first place. It is even less predictable when the static
initializer will be run (and therefore what the CCL will be set to at
that point). What's the point?

> (BTW, couldn't this
> be Driver.class.getClassLoader() instead?)

Yes. getClass() tends to be a bit friendlier to anything that tweaks the
bytecode, though, since a .class reference compiles down to a call to
Class.forName() which breaks if the class is renamed, etc.

>>Does it make sense to have the driver defaults change depending on
>>who is obtaining the connection?
>
> The fact that this is nonsensical is irrelevant. Changing defaults on
> a per connection basis has never been proposed.

*shrug* It's the logical conclusion of using the CCL to find the
defaults. The context can change -- why do the defaults *not* change if
you change the context? What is special about the context of the first
call to getConnection(), or the first action that happens to cause class
initialization (if you go with magic in the static initializer)?

> More to the point, can someone explain what
> /org/postgresql/driverconfig.properties files are actually used for?

Briefly, configuring driver defaults globally without having to mess
around with every application URL.

See http://archives.postgresql.org/pgsql...0/msg00023.php for
the thread that spawned it.

> Without knowing what this feature is actually used for, I'd speculate
> groundlessly that the context-classloader-based implementation is more
> likely to work in the application server scenario. Within the same
> app server instance, multiple *.properties files are likely to exist
> for use by different apps. These files aren't likely to be on the
> CLASSPATH.

I don't think it makes sense to allow a particular application to
override a bit of "global" data for the driver that is shared between
multiple applications. App servers can reconfigure the driver either
globally or on a per-application basis easily enough anyway.

> Rather, they're likely to be in a per-application config directory that the system classloader can't see.

The system classloader only gets involved when the driver has been
loaded in the *bootstrap* classpath. In all other cases, we use the
classloader that loaded the driver to load the defaults. Still, this
will usually be a classloader that cannot see the per-application resources.

If an application wants a customized set of defaults, it can arrange for
a separate instance of the driver packaged with the appropriate defaults
to be loaded with the application; in that case, there's no problem with
finding the right classloader.

-O

---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org