SEO

> Does anyone know if it will be possible to run the server with ANSI/ISO
> string escaping instead of C-style escapes? The C style escaping is a
> shoot-down for our adoption of postgres, since its non-standard.
>
>
> Not yet, but we have a TODO item:
>
> * Allow backslash handling in quoted strings to be disabled for
> portability
>
> The use of C-style backslashes (.e.g. \n, \r) in quoted strings is not
> SQL-spec compliant, so allow such handling to be disabled.
>
> Uh, what is ANSI/ISO escaping actually? I assume you mean only supporting
> '' for literal quotes rather than \' too.
>
> Yes Sir. Being able to disable the backslash-escaping is the desired
> operation. In circles of business deciding wether to move to opensource
> databases this is the silently used excuse (by the critics) to dismiss
> pg since its arguably so fundamental and somewhat dangerous (data loss).
> Getting pg to behave 'normally' would silence this excuse. Is there
> anything I can do to help move this up the todo list?

Uh, yea, this is going to require quite a bit of discussion in the
group, and I am concerned how it will affect other apps using
PostgreSQL. (The mode isn't going to be useful if it breaks plug-in
extensions and stuff.)

I think COPY is always going to need to use backslashes. There is no
other good way to handle special characters and stuff, but I don't see
people complaining that has to be portable.

I think most of it is done in parser/scan.l if you want to hack in there
and get a test implementation working and send in a patch. We can work
on the infrastructure to turn it on and off.

--
Bruce Momjian | http://candle.pha.pa.us
pgman@candle.pha.pa.us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

Bruce Momjian <pgman@candle.pha.pa.us> writes:
>> Does anyone know if it will be possible to run the server with ANSI/ISO
>> string escaping instead of C-style escapes? The C style escaping is a
>> shoot-down for our adoption of postgres, since its non-standard.

> Uh, yea, this is going to require quite a bit of discussion in the
> group, and I am concerned how it will affect other apps using
> PostgreSQL. (The mode isn't going to be useful if it breaks plug-in
> extensions and stuff.)

The hard part of this isn't turning off backslash quoting; the code
changes to do that would be pretty trivial. The hard part is not
breaking vast quantities of existing client code. After our experience
with autocommit, no one is going to want to solve it with a GUC variable
that can be flipped on and off at random. That would make the
compatibility problems that autocommit caused look like a day at the
beach :-(

I don't actually know a way to solve this that wouldn't impose
impossible amounts of pain on our existing users, and I'm afraid that
I rank that consideration higher than acquiring new users who won't
consider changing their own code.

If you can show me a way to provide this behavior without risk of
breaking existing code, I'm all ears.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

>>Uh, yea, this is going to require quite a bit of discussion in the
>>group, and I am concerned how it will affect other apps using
>>PostgreSQL. (The mode isn't going to be useful if it breaks plug-in
>>extensions and stuff.)
>>
>>
>
>The hard part of this isn't turning off backslash quoting; the code
>changes to do that would be pretty trivial. The hard part is not
>breaking vast quantities of existing client code. After our experience
>with autocommit, no one is going to want to solve it with a GUC variable
>that can be flipped on and off at random. That would make the
>compatibility problems that autocommit caused look like a day at the
>beach :-(
>
>I don't actually know a way to solve this that wouldn't impose
>impossible amounts of pain on our existing users, and I'm afraid that
>I rank that consideration higher than acquiring new users who won't
>consider changing their own code.
>
>If you can show me a way to provide this behavior without risk of
>breaking existing code, I'm all ears.
>
> regards, tom lane
>
>
I feel somewhat confident (very actually) that a config option that
disabled the backslash behavior globally(*) would be acceptable, BUT
leave the current backslash behavior turned on by default so that
current users are not impacted at all. Only a conscientious decision by
the db admin to turn it on could cause problems, but _only_ if he/she
didn't warn all his/her users beforehand of the impending change and its
consequences (rtm).

(*Or if it's possible, provide the no-backslash config on a per-catalog
basis perhaps? -or even per-user/group?, --that would allow individuals
to use the legacy mode until they choose otherwise)

I can say, that I for one would enable the no-backslash config option
out of the box -globally -so that we can start using pg now without any
more upper managerial concerns/excuses about language/interface
compliance..I can also say that (what we already know) the longer we
wait to provide the 'right' option, the *more* legacy apps (and
interfaces) will be built around it and consequently suffer when the
need for change eventually comes (almost wholly caused by interop
concerns). And market gain is being hurt now by this incompatibility
with commercial offerings; that's an unfortunate fact.

Better to nip it in the bud sooner than later, imo.

thoughts,
ken



---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

On Sun, 2005-02-27 at 18:25 -0700, Ken Johanson wrote:
> >>Uh, yea, this is going to require quite a bit of discussion in the
> >>group, and I am concerned how it will affect other apps using
> >>PostgreSQL. (The mode isn't going to be useful if it breaks plug-in
> >>extensions and stuff.)
> >>
> >>
> >
> >The hard part of this isn't turning off backslash quoting; the code
> >changes to do that would be pretty trivial. The hard part is not
> >breaking vast quantities of existing client code. After our experience
> >with autocommit, no one is going to want to solve it with a GUC variable
> >that can be flipped on and off at random. That would make the
> >compatibility problems that autocommit caused look like a day at the
> >beach :-(
> >
> >I don't actually know a way to solve this that wouldn't impose
> >impossible amounts of pain on our existing users, and I'm afraid that
> >I rank that consideration higher than acquiring new users who won't
> >consider changing their own code.
> >
> >If you can show me a way to provide this behavior without risk of
> >breaking existing code, I'm all ears.
> >
> > regards, tom lane
> >
> >
> I feel somewhat confident (very actually) that a config option that
> disabled the backslash behavior globally(*) would be acceptable, BUT
> leave the current backslash behavior turned on by default so that
> current users are not impacted at all. Only a conscientious decision by
> the db admin to turn it on could cause problems, but _only_ if he/she
> didn't warn all his/her users beforehand of the impending change and its
> consequences (rtm).
>

I'm a little worried about PostgreSQL having the same problems as PHP.
In PHP, every time you want to download an application, you never see
"This application works on php 4+". Instead, you see "This application
works on php4+ with the following config options set <long list>".
Sometimes these applications have conflicting requirements. From an
administrator's standpoint, it's a mess.

In PostgreSQL I think it would actually be much worse. Right now many
applications build a PostgreSQL layer, but will they build two? I think
this would cause a divide in the application support (some for config
option A some for config option B) in the already smaller-than-we'd-like
set of software that supports PostgreSQL.

Regards,
Jeff Davis




---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org

>I'm a little worried about PostgreSQL having the same problems as PHP.
>In PHP, every time you want to download an application, you never see
>"This application works on php 4+". Instead, you see "This application
>works on php4+ with the following config options set <long list>".
>Sometimes these applications have conflicting requirements. From an
>administrator's standpoint, it's a mess.
>
>In PostgreSQL I think it would actually be much worse. Right now many
>applications build a PostgreSQL layer, but will they build two? I think
>this would cause a divide in the application support (some for config
>option A some for config option B) in the already smaller-than-we'd-like
>set of software that supports PostgreSQL.
>
>Regards,
> Jeff Davis
>
>
There's certainly two perspectives to this. The one you present is
certainly valid, but consider the bigger picture...

"This application requires the following databases: Oracle versionX, MY
SQL version X, Mysql version 5.2 with the no-backslashes option, UltraDB
version x"

Notice the lack of PG - some apps - most notably commercial ones - will
automatically shoot it down if it cant meet certain language
requirements. The database itself could meet the latest SQL03 (or
whatever we're up to) specs for Object Relational stuff, etc to the tee.
The JDBC driver could meet the JDBC spec to the tee for transaction
support, etc - but this one low level problem is a total show stopper,
because it plainly breaks queries sent through various interfaces in
various drivers.

Besides, the version-deprecation / version requirements you mention
exists in every piece of software I've even seen. Sometime they're okay
with a really old version, sometime only the newest will do. This is the
very argument for getting PG to offer an (use-optional) escape behavior
inline with the rest - to mitigate these version requirements down the road.

Thoughts,
ken



---------------------------(end of broadcast)---------------------------
TIP 8: explain analyze is your friend

On Sun, Feb 27, 2005 at 06:25:18PM -0700, Ken Johanson wrote:
> I feel somewhat confident (very actually) that a config option that
> disabled the backslash behavior globally(*) would be acceptable, BUT
> leave the current backslash behavior turned on by default so that
> current users are not impacted at all. Only a conscientious decision by
> the db admin to turn it on could cause problems, but _only_ if he/she
> didn't warn all his/her users beforehand of the impending change and its
> consequences (rtm).

It's not just a question of warning the users, all interfaces to the
database will instantly break. For example: JDBC, Perl DBI, PHP PEAR
etc. They will continue to send queries with the backslashes embedded.
These interfaces would need to be modified to handle both situations
and detect which situation they're dealing with.

The thing is all these interfaces handle the quoting transparently for
you, so the code is portable already. What you're complaining about is
that you have your own query marshalling and it is not portable.

Incidently, if you disable the backslash quoting, how does one enter
raw binary data including NUL (\0) characters?

The only viable solution I can think of is that it is set at
*connection* time (maybe extra parameters), and unchangable for the
rest of the session. This means that unmodified client interfaces won't
see a difference.

> I can say, that I for one would enable the no-backslash config option
> out of the box -globally -so that we can start using pg now without any
> more upper managerial concerns/excuses about language/interface
> compliance..I can also say that (what we already know) the longer we
> wait to provide the 'right' option, the *more* legacy apps (and
> interfaces) will be built around it and consequently suffer when the
> need for change eventually comes (almost wholly caused by interop
> concerns). And market gain is being hurt now by this incompatibility
> with commercial offerings; that's an unfortunate fact.

Even if PostgreSQL implements this now, you will have to wait for new
versions of any client libraries before it's usable. See the autocommit
disaster for an example why people are not rushing into this...

Have a nice day,
--
Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/
> Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a
> tool for doing 5% of the work and then sitting around waiting for someone
> else to do the other 95% so you can sue them.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQFCIuF9Y5Twig3Ge+YRAkI8AJ97gHc6OWjrKav3kGfCew DtqVTlcQCgmpw2
IIbWZUrG80OTJ8s0ydS2ojI=
=jjsh
-----END PGP SIGNATURE-----

On Mon, Feb 28, 2005 at 10:13:00 -0700,
Ken Johanson <pg-user@kensystem.com> wrote:
>
> Besides, the version-deprecation / version requirements you mention
> exists in every piece of software I've even seen. Sometime they're okay
> with a really old version, sometime only the newest will do. This is the
> very argument for getting PG to offer an (use-optional) escape behavior
> inline with the rest - to mitigate these version requirements down the road.

Shouldn't this data be being passed through some standard code that checks if
escaping is needed? If so, is that the right place to handle whether or not
backslashes need to be escaped in addition to single quotes?

---------------------------(end of broadcast)---------------------------
TIP 9: the planner will ignore your desire to choose an index scan if your
joining column's datatypes do not match

Bruno Wolff III wrote:
> On Mon, Feb 28, 2005 at 10:13:00 -0700,
> Ken Johanson <pg-user@kensystem.com> wrote:
>
>>Besides, the version-deprecation / version requirements you mention
>>exists in every piece of software I've even seen. Sometime they're okay
>>with a really old version, sometime only the newest will do. This is the
>>very argument for getting PG to offer an (use-optional) escape behavior
>>inline with the rest - to mitigate these version requirements down the road.
>
>
> Shouldn't this data be being passed through some standard code that checks if
> escaping is needed? If so, is that the right place to handle whether or not
> backslashes need to be escaped in addition to single quotes?
>
>
>

Ideally yes, but its not a requirement in any driver's spec that I'm
familiar with. In fact the driver specs expect or 'claim' some (possibly
implicit) level of sql language compliance -- so that the same query
sent to a different database yields the same result.

insert into tbl (path) values ('c:\test')

The above query *could* and "should* be sent through an escape
preprocessor (PreparedStatement interface) but it is *not* required.
It's also not fair to say that a user can *expect* the above to not work
with PG even though it does with another DB, imo. The user coming from
another DB *won't* expect it to be broken. (I know from experience :-)



---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org

Martijn van Oosterhout wrote:
> Incidently, if you disable the backslash quoting, how does one enter
> raw binary data including NUL (\0) characters?

The bytea type has its own internal quoting/escaping mechanism (which
overlaps with the lexer's mechanism in some ways), so entering binary
data is not an issue.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

---------------------------(end of broadcast)---------------------------
TIP 7: don't forget to increase your free space map settings

On Mon, 2005-02-28 at 10:13 -0700, Ken Johanson wrote:
> >I'm a little worried about PostgreSQL having the same problems as PHP.
> >In PHP, every time you want to download an application, you never see
> >"This application works on php 4+". Instead, you see "This application
> >works on php4+ with the following config options set <long list>".
> >Sometimes these applications have conflicting requirements. From an
> >administrator's standpoint, it's a mess.
> >
> >In PostgreSQL I think it would actually be much worse. Right now many
> >applications build a PostgreSQL layer, but will they build two? I think
> >this would cause a divide in the application support (some for config
> >option A some for config option B) in the already smaller-than-we'd-like
> >set of software that supports PostgreSQL.
> >
> >Regards,
> > Jeff Davis
> >
> >
> There's certainly two perspectives to this. The one you present is
> certainly valid, but consider the bigger picture...
>
> "This application requires the following databases: Oracle versionX, MY
> SQL version X, Mysql version 5.2 with the no-backslashes option, UltraDB
> version x"
>
> Notice the lack of PG -
[snip]

A valid point: that's certainly the issue we're dealing with here.

I think most people agree that being SQL compliant is good. The question
is: is it worth the pain for existing users?

A configurable option does not make the pain disappear. Admins are
forced to choose one side (either sql compliant or c style) and exclude
the other applications. Any app developer that wants to support pre-8.1
apps will have to have a c-style app available. So even if you nip it in
the bud, it's not really gone yet because app developers want to support
old versions of postgres.

I know if we added the option and deprecated the old style, I would be
forced to choose between using deprecated syntax that may not be
supported for long, or doing a lot of work to convert and retest
applications.

> Besides, the version-deprecation / version requirements you mention
> exists in every piece of software I've even seen. Sometime they're okay
> with a really old version, sometime only the newest will do. This is the
> very argument for getting PG to offer an (use-optional) escape behavior
> inline with the rest - to mitigate these version requirements down the road.


I think you may have misunderstood what I meant. I am not suggesting
that we don't change the database at all between versions, my argument
was showing the difficulties when one version has many different shapes
due to many incompatible options.

Regards,
Jeff Davis


---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org