Unix Technical Forum

Re: [HACKERS] [ADMIN] invalid multibyte character for locale

This is a discussion on Re: [HACKERS] [ADMIN] invalid multibyte character for locale within the Pgsql Patches forums, part of the PostgreSQL category; --> Tom Lane wrote: > Bjoern Metzdorf <bm@turtle-entertainment.de> writes: > > I assume I could just remove > > #define ...


Go Back   Unix Technical Forum > Database Server Software > PostgreSQL > Pgsql Patches

Register FAQ Members List Calendar Search Today's Posts Mark Forums Read
  #1 (permalink)  
Old 04-17-2008, 11:09 PM
Bruce Momjian
 
Posts: n/a
Default Re: [HACKERS] [ADMIN] invalid multibyte character for locale

Tom Lane wrote:
> Bjoern Metzdorf <bm@turtle-entertainment.de> writes:
> > I assume I could just remove
> > #define USE_WIDE_UPPER_LOWER
> > from oracle_compat.c to emulate the old behaviour. But a cleaner fix
> > would be to check if we are using UNICODE and locale is C or POSIX and
> > only then skip USE_WIDE_UPPER_LOWER.

>
> Perhaps it would be reasonable to do something like this:
>
> #ifdef USE_WIDE_UPPER_LOWER
> /*
> * use wide char code only when max encoding length > one
> * and we aren't in C locale
> */
> if (pg_database_encoding_max_length() > 1 &&
> !lc_ctype_is_c())
> {
>
> where lc_ctype_is_c() is the obvious clone of the existing
> lc_collate_is_c() routine. We can reasonably assume that mbstowcs
> is going to be unable to offer any useful behavior in C locale.


Tom, is this the fix you were thinking of? Seems like it would be a
good improvement.

--
Bruce Momjian | http://candle.pha.pa.us
pgman@candle.pha.pa.us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Index: src/backend/utils/adt/oracle_compat.c
================================================== =================
RCS file: /cvsroot/pgsql/src/backend/utils/adt/oracle_compat.c,v
retrieving revision 1.57
diff -c -c -r1.57 oracle_compat.c
*** src/backend/utils/adt/oracle_compat.c 31 Dec 2004 22:01:22 -0000 1.57
--- src/backend/utils/adt/oracle_compat.c 15 Mar 2005 05:24:11 -0000
***************
*** 166,173 ****
lower(PG_FUNCTION_ARGS)
{
#ifdef USE_WIDE_UPPER_LOWER
! /* use wide char code only when max encoding length > one */
! if (pg_database_encoding_max_length() > 1)
{
text *string = PG_GETARG_TEXT_P(0);
text *result;
--- 166,173 ----
lower(PG_FUNCTION_ARGS)
{
#ifdef USE_WIDE_UPPER_LOWER
! /* use wide char code only when max encoding length > 1 */
! if (pg_database_encoding_max_length() > 1 && !lc_ctype_is_c())
{
text *string = PG_GETARG_TEXT_P(0);
text *result;
***************
*** 229,235 ****
{
#ifdef USE_WIDE_UPPER_LOWER
/* use wide char code only when max encoding length > one */
! if (pg_database_encoding_max_length() > 1)
{
text *string = PG_GETARG_TEXT_P(0);
text *result;
--- 229,235 ----
{
#ifdef USE_WIDE_UPPER_LOWER
/* use wide char code only when max encoding length > one */
! if (pg_database_encoding_max_length() > 1 && !lc_ctype_is_c())
{
text *string = PG_GETARG_TEXT_P(0);
text *result;
***************
*** 294,300 ****
{
#ifdef USE_WIDE_UPPER_LOWER
/* use wide char code only when max encoding length > one */
! if (pg_database_encoding_max_length() > 1)
{
text *string = PG_GETARG_TEXT_P(0);
text *result;
--- 294,300 ----
{
#ifdef USE_WIDE_UPPER_LOWER
/* use wide char code only when max encoding length > one */
! if (pg_database_encoding_max_length() > 1 && !lc_ctype_is_c())
{
text *string = PG_GETARG_TEXT_P(0);
text *result;
Index: src/backend/utils/adt/pg_locale.c
================================================== =================
RCS file: /cvsroot/pgsql/src/backend/utils/adt/pg_locale.c,v
retrieving revision 1.30
diff -c -c -r1.30 pg_locale.c
*** src/backend/utils/adt/pg_locale.c 1 Jan 2005 05:43:07 -0000 1.30
--- src/backend/utils/adt/pg_locale.c 15 Mar 2005 05:24:11 -0000
***************
*** 197,202 ****
--- 197,229 ----


/*
+ * We'd like to cache whether LC_CTYPE is C (or POSIX), so we can
+ * optimize a few code paths in various places.
+ */
+ bool
+ lc_ctype_is_c(void)
+ {
+ /* Cache result so we only have to compute it once */
+ static int result = -1;
+ char *localeptr;
+
+ if (result >= 0)
+ return (bool) result;
+ localeptr = setlocale(LC_CTYPE, NULL);
+ if (!localeptr)
+ elog(ERROR, "invalid LC_CTYPE setting");
+
+ if (strcmp(localeptr, "C") == 0)
+ result = true;
+ else if (strcmp(localeptr, "POSIX") == 0)
+ result = true;
+ else
+ result = false;
+ return (bool) result;
+ }
+
+
+ /*
* Frees the malloced content of a struct lconv. (But not the struct
* itself.)
*/
Index: src/include/utils/pg_locale.h
================================================== =================
RCS file: /cvsroot/pgsql/src/include/utils/pg_locale.h,v
retrieving revision 1.19
diff -c -c -r1.19 pg_locale.h
*** src/include/utils/pg_locale.h 1 Jan 2005 05:43:09 -0000 1.19
--- src/include/utils/pg_locale.h 15 Mar 2005 05:24:16 -0000
***************
*** 32,37 ****
--- 32,38 ----
bool doit, GucSource source);

extern bool lc_collate_is_c(void);
+ extern bool lc_ctype_is_c(void);

/*
* Return the POSIX lconv struct (contains number/money formatting


---------------------------(end of broadcast)---------------------------
TIP 7: don't forget to increase your free space map settings

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #2 (permalink)  
Old 04-17-2008, 11:09 PM
Tom Lane
 
Posts: n/a
Default Re: [HACKERS] [ADMIN] invalid multibyte character for locale

Bruce Momjian <pgman@candle.pha.pa.us> writes:
> Tom Lane wrote:
>> Perhaps it would be reasonable to do something like this:
>>
>> #ifdef USE_WIDE_UPPER_LOWER
>> /*
>> * use wide char code only when max encoding length > one
>> * and we aren't in C locale
>> */
>> if (pg_database_encoding_max_length() > 1 &&
>> !lc_ctype_is_c())


> Tom, is this the fix you were thinking of? Seems like it would be a
> good improvement.


Please fix the comments not only the code ;-). Other than that,
this is exactly what I meant.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
Reply


Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On
Forum Jump


All times are GMT. The time now is 07:19 AM.


Powered by vBulletin® Version 3.6.5
Copyright ©2000 - 2008, Jelsoft Enterprises Ltd.
SEO by vBSEO 3.2.0
www.UnixAdminTalk.com