Unix Technical Forum

psqlodbclibpq 8_1_03 unicode problem

This is a discussion on psqlodbclibpq 8_1_03 unicode problem within the pgsql Interfaces odbc forums, part of the PostgreSQL category; --> Hi I am using the versionde psqlodbclibpq 8_01_003, to be able to use ssl connections, and until the moment ...


Go Back   Unix Technical Forum > Database Server Software > PostgreSQL > pgsql Interfaces odbc

Register FAQ Members List Calendar Search Today's Posts Mark Forums Read
  #1 (permalink)  
Old 04-16-2008, 01:19 AM
Mario A. Soto Cordones
 
Posts: n/a
Default psqlodbclibpq 8_1_03 unicode problem

Hi

I am using the versionde psqlodbclibpq 8_01_003, to be able to use ssl
connections, and until the moment works well, but the problem has it with
caracters like:

character Ñ, show like character ?
character ó, show like character ??

that I can make?

Thank you

Mario Soto



---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #2 (permalink)  
Old 04-16-2008, 01:19 AM
=?iso-8859-1?Q?Sime=F3_Reig?=
 
Posts: n/a
Default Re: psqlodbclibpq 8_1_03 unicode problem

> I am using the versionde psqlodbclibpq 8_01_003, to be able to use ssl
> connections, and until the moment works well, but the problem has it with
> caracters like:
>
> character Ñ, show like character ?
> character ó, show like character ??
>
> that I can make?
>
> Thank you


I've the same problem too with spanish and catalan letters, Seems like it
don't support unicode characters.

Thanks a lot



---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #3 (permalink)  
Old 04-16-2008, 01:19 AM
Josef Springer
 
Posts: n/a
Default Re: psqlodbclibpq 8_1_03 unicode problem

I have the same problem and reported a bug.
It seems, that Unicode is readed unencoded (one byte per character)

Josef Springer

Simeó Reig wrote:

>> I am using the versionde psqlodbclibpq 8_01_003, to be able to use ssl
>> connections, and until the moment works well, but the problem has it
>> with
>> caracters like:
>>
>> character Ñ, show like character ?
>> character ó, show like character ??
>>
>> that I can make?
>>
>> Thank you

>
>
> I've the same problem too with spanish and catalan letters, Seems like
> it don't support unicode characters.
>
> Thanks a lot
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 6: explain analyze is your friend





---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #4 (permalink)  
Old 04-16-2008, 01:19 AM
Joel Fradkin
 
Posts: n/a
Default Re: psqlodbclibpq 8_1_03 unicode problem

Not sure why your getting that.
Is the database Unicode?
When I was in SQLASCII mine read as ?
The older driver 7.4 reas SQLASCII ok and presented French characters ok,
but newer version did not.
When I converted to a Unicode database I could read the French characters ok
with the new driver.

By the way been up Since Monday no odbc errors yet

Joel Fradkin

Wazagua, Inc.
2520 Trailmate Dr
Sarasota, Florida 34243
Tel. 941-753-7111 ext 305

jfradkin@wazagua.com
www.wazagua.com
Powered by Wazagua
Providing you with the latest Web-based technology & advanced tools.
© 2004. WAZAGUA, Inc. All rights reserved. WAZAGUA, Inc
This email message is for the use of the intended recipient(s) and may
contain confidential and privileged information. Any unauthorized review,
use, disclosure or distribution is prohibited. If you are not the intended
recipient, please contact the sender by reply email and delete and destroy
all copies of the original message, including attachments.




-----Original Message-----
From: pgsql-odbc-owner@postgresql.org
[mailtogsql-odbc-owner@postgresql.org] On Behalf Of Josef Springer
Sent: Wednesday, August 10, 2005 7:58 AM
To: pgsql-odbc@postgresql.org
Subject: Re: [ODBC] psqlodbclibpq 8_1_03 unicode problem

I have the same problem and reported a bug.
It seems, that Unicode is readed unencoded (one byte per character)

Josef Springer

Simeó Reig wrote:

>> I am using the versionde psqlodbclibpq 8_01_003, to be able to use ssl
>> connections, and until the moment works well, but the problem has it
>> with
>> caracters like:
>>
>> character Ñ, show like character ?
>> character ó, show like character ??
>>
>> that I can make?
>>
>> Thank you

>
>
> I've the same problem too with spanish and catalan letters, Seems like
> it don't support unicode characters.
>
> Thanks a lot
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 6: explain analyze is your friend





---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings


---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #5 (permalink)  
Old 04-16-2008, 01:19 AM
Marko Ristola
 
Posts: n/a
Default Re: psqlodbclibpq 8_1_03 unicode problem


Characters will go into question marks, if the character conversion is
not defined
properly, or the input for the conversion functions is incorrect.


Here is something about ANSI C standard about the character conversions:


setlocale(LC_ALL,"C");
After calling this function in main() C program, the program
works as follows:

LATIN1 works with itself, because it does not need any conversions and
it is single byte
encoding.
UTF-8 works with itself, because it does not need any locale specific
information to work.

Conversion from UTF-8 into LATIN 1does not work. Non-ascii characters
will be converted
into question marks.
Conversion from LATIN1 into UTF-8 does not work. Non-ascii characters
will be converted
into question marks.

setlocale(LC_ALL,""); or setlocale(LC_CTYPE,"");

ANSI C definition for this is, that locale environment settings (Windows
settings, or Linux environment settings), for example
LC_CTYPE=fi_FI.iso8859-1 is used. This way conversions between LATIN1
and UTF-8 will work, if the environment is ANSI C compilant (it should).

So the LC_CTYPE must match with the multibyte data. Windows settings
will be activated
with the "" setting: an empty string means the environment default.

There is a good book about ANSI C 99: "The C standard"


So, how this relates to psqlodbc?

Case 1:
Typically user needs to do the conversions outside US.
Inside psqlodbc, do setlocale(LC_CTYPE,""), do the LATIN1/UTF-8 -conversion
and do setlocale(LC_CTYPE,"C");

Case 2:
Sometimes user does not want to do any charset conversions
because of performance reasons.
Don't support this one.

Case 3:
User has already done setlocale(LC_CTYPE,"");
To make Case 1 and Case 3 work, we do inside ODBC the following:
strncpy(prev_locale,setlocale(LC_CTYPE,NULL),64);
setlocale(LC_CTYPE,""), do the LATIN1/UTF-8 -conversion
and do setlocale(LC_CTYPE,prev_locale);

Case 4:
A threaded application needs to work also.
Under Windows, setlocale() can be altered to behave as thread local, so
that every
thread has it's own locale setting.
Under Linux, at least iconv() conversion facility works, even though it
is not ANSI C.


Regards, Marko Ristola



Joel Fradkin wrote:

>Not sure why your getting that.
>Is the database Unicode?
>When I was in SQLASCII mine read as ?
>The older driver 7.4 reas SQLASCII ok and presented French characters ok,
>but newer version did not.
>When I converted to a Unicode database I could read the French characters ok
>with the new driver.
>
>By the way been up Since Monday no odbc errors yet
>
>Joel Fradkin
>
>Wazagua, Inc.
>2520 Trailmate Dr
>Sarasota, Florida 34243
>Tel. 941-753-7111 ext 305
>
>jfradkin@wazagua.com
>www.wazagua.com
>Powered by Wazagua
>Providing you with the latest Web-based technology & advanced tools.
>© 2004. WAZAGUA, Inc. All rights reserved. WAZAGUA, Inc
> This email message is for the use of the intended recipient(s) and may
>contain confidential and privileged information. Any unauthorized review,
>use, disclosure or distribution is prohibited. If you are not the intended
>recipient, please contact the sender by reply email and delete and destroy
>all copies of the original message, including attachments.
>
>
>
>
>-----Original Message-----
>From: pgsql-odbc-owner@postgresql.org
>[mailtogsql-odbc-owner@postgresql.org] On Behalf Of Josef Springer
>Sent: Wednesday, August 10, 2005 7:58 AM
>To: pgsql-odbc@postgresql.org
>Subject: Re: [ODBC] psqlodbclibpq 8_1_03 unicode problem
>
>I have the same problem and reported a bug.
>It seems, that Unicode is readed unencoded (one byte per character)
>
>Josef Springer
>
>Simeó Reig wrote:
>
>
>
>>>I am using the versionde psqlodbclibpq 8_01_003, to be able to use ssl
>>>connections, and until the moment works well, but the problem has it
>>>with
>>>caracters like:
>>>
>>>character Ñ, show like character ?
>>>character ó, show like character ??
>>>
>>>that I can make?
>>>
>>>Thank you
>>>
>>>

>>I've the same problem too with spanish and catalan letters, Seems like
>>it don't support unicode characters.
>>
>>Thanks a lot
>>
>>
>>---------------------------(end of broadcast)---------------------------
>>TIP 6: explain analyze is your friend
>>
>>

>
>
>
>
>---------------------------(end of broadcast)---------------------------
>TIP 5: don't forget to increase your free space map settings
>
>
>---------------------------(end of broadcast)---------------------------
>TIP 6: explain analyze is your friend
>
>



---------------------------(end of broadcast)---------------------------
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #6 (permalink)  
Old 04-16-2008, 01:19 AM
Mario A. Soto Cordones
 
Posts: n/a
Default Re: psqlodbclibpq 8_1_03 unicode problem

Hi

I find this mail very well, but I don't see I eat up he can help with the
psqlodbc problem.

thank you



>
> Characters will go into question marks, if the character conversion is
> not defined
> properly, or the input for the conversion functions is incorrect.
>
>
> Here is something about ANSI C standard about the character conversions:
>
>
> setlocale(LC_ALL,"C");
> After calling this function in main() C program, the program
> works as follows:
>
> LATIN1 works with itself, because it does not need any conversions and
> it is single byte
> encoding.
> UTF-8 works with itself, because it does not need any locale specific
> information to work.
>
> Conversion from UTF-8 into LATIN 1does not work. Non-ascii characters
> will be converted
> into question marks.
> Conversion from LATIN1 into UTF-8 does not work. Non-ascii characters
> will be converted
> into question marks.
>
> setlocale(LC_ALL,""); or setlocale(LC_CTYPE,"");
>
> ANSI C definition for this is, that locale environment settings (Windows
> settings, or Linux environment settings), for example
> LC_CTYPE=fi_FI.iso8859-1 is used. This way conversions between LATIN1
> and UTF-8 will work, if the environment is ANSI C compilant (it should).
>
> So the LC_CTYPE must match with the multibyte data. Windows settings
> will be activated
> with the "" setting: an empty string means the environment default.
>
> There is a good book about ANSI C 99: "The C standard"
>
>
> So, how this relates to psqlodbc?
>
> Case 1:
> Typically user needs to do the conversions outside US.
> Inside psqlodbc, do setlocale(LC_CTYPE,""), do the LATIN1/UTF-8
> -conversion and do setlocale(LC_CTYPE,"C");
>
> Case 2:
> Sometimes user does not want to do any charset conversions
> because of performance reasons.
> Don't support this one.
>
> Case 3:
> User has already done setlocale(LC_CTYPE,"");
> To make Case 1 and Case 3 work, we do inside ODBC the following:
> strncpy(prev_locale,setlocale(LC_CTYPE,NULL),64);
> setlocale(LC_CTYPE,""), do the LATIN1/UTF-8 -conversion
> and do setlocale(LC_CTYPE,prev_locale);
>
> Case 4:
> A threaded application needs to work also.
> Under Windows, setlocale() can be altered to behave as thread local, so
> that every
> thread has it's own locale setting.
> Under Linux, at least iconv() conversion facility works, even though it
> is not ANSI C.
>
>
> Regards, Marko Ristola
>
>
>
> Joel Fradkin wrote:
>
>>Not sure why your getting that.
>>Is the database Unicode?
>>When I was in SQLASCII mine read as ?
>>The older driver 7.4 reas SQLASCII ok and presented French characters
>> ok, but newer version did not.
>>When I converted to a Unicode database I could read the French
>> characters ok with the new driver.
>>
>>By the way been up Since Monday no odbc errors yet
>>
>>Joel Fradkin
>>
>>Wazagua, Inc.
>>2520 Trailmate Dr
>>Sarasota, Florida 34243
>>Tel. 941-753-7111 ext 305
>>
>>jfradkin@wazagua.com
>>www.wazagua.com
>>Powered by Wazagua
>>Providing you with the latest Web-based technology & advanced tools. ©
>> 2004. WAZAGUA, Inc. All rights reserved. WAZAGUA, Inc
>> This email message is for the use of the intended recipient(s) and may
>>contain confidential and privileged information. Any unauthorized
>> review, use, disclosure or distribution is prohibited. If you are not
>> the intended recipient, please contact the sender by reply email and
>> delete and destroy all copies of the original message, including
>> attachments.
>>
>>
>>
>>
>>-----Original Message-----
>>From: pgsql-odbc-owner@postgresql.org
>>[mailtogsql-odbc-owner@postgresql.org] On Behalf Of Josef Springer
>> Sent: Wednesday, August 10, 2005 7:58 AM
>>To: pgsql-odbc@postgresql.org
>>Subject: Re: [ODBC] psqlodbclibpq 8_1_03 unicode problem
>>
>>I have the same problem and reported a bug.
>>It seems, that Unicode is readed unencoded (one byte per character)
>>
>>Josef Springer
>>
>>Simeó Reig wrote:
>>
>>
>>
>>>>I am using the versionde psqlodbclibpq 8_01_003, to be able to use
>>>> ssl connections, and until the moment works well, but the problem has
>>>> it with
>>>>caracters like:
>>>>
>>>>character Ñ, show like character ?
>>>>character ó, show like character ??
>>>>
>>>>that I can make?
>>>>
>>>>Thank you
>>>>
>>>>
>>>I've the same problem too with spanish and catalan letters, Seems like
>>> it don't support unicode characters.
>>>
>>>Thanks a lot
>>>
>>>
>>>---------------------------(end of
>>> broadcast)--------------------------- TIP 6: explain analyze is your
>>> friend
>>>
>>>

>>
>>
>>
>>
>>---------------------------(end of
>> broadcast)--------------------------- TIP 5: don't forget to increase
>> your free space map settings
>>
>>
>>---------------------------(end of
>> broadcast)--------------------------- TIP 6: explain analyze is your
>> friend
>>
>>

>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 9: In versions below 8.0, the planner will ignore your desire to
> choose an index scan if your joining column's datatypes do not
> match





---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?

http://archives.postgresql.org

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
Reply


Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On
Forum Jump


All times are GMT. The time now is 08:38 AM.


Powered by vBulletin® Version 3.6.5
Copyright ©2000 - 2008, Jelsoft Enterprises Ltd.
SEO by vBSEO 3.2.0
www.UnixAdminTalk.com