Character type value including multibyte characters is not padded
with spaces. It reproduces at 7.3.x, 7.4.x and 8.0.x.

create table t (a char(10));
insert into t values ('XXXXX'); -- X is 2byte character.

I expect that 'XXXXX ' is inserted. But 'XXXXX' is inserted.

select a, octed_length(a) from t;

a | octet_length
-------+--------------
XXXXX | 10

If padded with spaces, octet_length(a) is 15. This problem is caused
that string length is calculated by byte length(VARSIZE) in
exprTypmod().

I attache the patch for this problem.

Regards,

--
Yoshiyuki Asaba
y-asaba@sra.co.jp

*** parse_expr.c.orig 2005-01-13 02:32:36.000000000 +0900
--- parse_expr.c 2005-05-22 17:12:37.000000000 +0900
***************
*** 18,23 ****
--- 18,24 ----
#include "catalog/pg_operator.h"
#include "catalog/pg_proc.h"
#include "commands/dbcommands.h"
+ #include "mb/pg_wchar.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
#include "nodes/params.h"
***************
*** 34,40 ****
#include "utils/lsyscache.h"
#include "utils/syscache.h"

-
bool Transform_null_equals = false;

static Node *transformColumnRef(ParseState *pstate, ColumnRef *cref);
--- 35,40 ----
***************
*** 1491,1497 ****
{
case BPCHAROID:
if (!con->constisnull)
! return VARSIZE(DatumGetPointer(con->constvalue));
break;
default:
break;
--- 1491,1503 ----
{
case BPCHAROID:
if (!con->constisnull)
! {
! int32 len = VARSIZE(DatumGetPointer(con->constvalue)) - VARHDRSZ;
!
! if (pg_database_encoding_max_length() > 1)
! len = pg_mbstrlen_with_len(VARDATA(DatumGetPointer(con->constvalue)), len);
! return len + VARHDRSZ;
! }
break;
default:
break;


---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to majordomo@postgresql.org)

Hackers,

The problem he found is not only existing in Japanese characters but
also in any multibyte encodings including UTF-8. For me the patch
looks good and I will commit it to 7.3, 7.4, 8.0 stables and current
if there's no objection.
--
Tatsuo Ishii

> Character type value including multibyte characters is not padded
> with spaces. It reproduces at 7.3.x, 7.4.x and 8.0.x.
>
> create table t (a char(10));
> insert into t values ('XXXXX'); -- X is 2byte character.
>
> I expect that 'XXXXX ' is inserted. But 'XXXXX' is inserted.
>
> select a, octed_length(a) from t;
>
> a | octet_length
> -------+--------------
> XXXXX | 10
>
> If padded with spaces, octet_length(a) is 15. This problem is caused
> that string length is calculated by byte length(VARSIZE) in
> exprTypmod().
>
> I attache the patch for this problem.
>
> Regards,
>
> --
> Yoshiyuki Asaba
> y-asaba@sra.co.jp

---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to majordomo@postgresql.org)

I see Tatsuo already applied this, which is great. I added a little
comment:

/* if multi-byte, take len and find # characters */

---------------------------------------------------------------------------

Yoshiyuki Asaba wrote:
> Character type value including multibyte characters is not padded
> with spaces. It reproduces at 7.3.x, 7.4.x and 8.0.x.
>
> create table t (a char(10));
> insert into t values ('XXXXX'); -- X is 2byte character.
>
> I expect that 'XXXXX ' is inserted. But 'XXXXX' is inserted.
>
> select a, octed_length(a) from t;
>
> a | octet_length
> -------+--------------
> XXXXX | 10
>
> If padded with spaces, octet_length(a) is 15. This problem is caused
> that string length is calculated by byte length(VARSIZE) in
> exprTypmod().
>
> I attache the patch for this problem.
>
> Regards,
>
> --
> Yoshiyuki Asaba
> y-asaba@sra.co.jp

> *** parse_expr.c.orig 2005-01-13 02:32:36.000000000 +0900
> --- parse_expr.c 2005-05-22 17:12:37.000000000 +0900
> ***************
> *** 18,23 ****
> --- 18,24 ----
> #include "catalog/pg_operator.h"
> #include "catalog/pg_proc.h"
> #include "commands/dbcommands.h"
> + #include "mb/pg_wchar.h"
> #include "miscadmin.h"
> #include "nodes/makefuncs.h"
> #include "nodes/params.h"
> ***************
> *** 34,40 ****
> #include "utils/lsyscache.h"
> #include "utils/syscache.h"
>
> -
> bool Transform_null_equals = false;
>
> static Node *transformColumnRef(ParseState *pstate, ColumnRef *cref);
> --- 35,40 ----
> ***************
> *** 1491,1497 ****
> {
> case BPCHAROID:
> if (!con->constisnull)
> ! return VARSIZE(DatumGetPointer(con->constvalue));
> break;
> default:
> break;
> --- 1491,1503 ----
> {
> case BPCHAROID:
> if (!con->constisnull)
> ! {
> ! int32 len = VARSIZE(DatumGetPointer(con->constvalue)) - VARHDRSZ;
> !
> ! if (pg_database_encoding_max_length() > 1)
> ! len = pg_mbstrlen_with_len(VARDATA(DatumGetPointer(con->constvalue)), len);
> ! return len + VARHDRSZ;
> ! }
> break;
> default:
> break;

>
> ---------------------------(end of broadcast)---------------------------
> TIP 2: you can get off all lists at once with the unregister command
> (send "unregister YourEmailAddressHere" to majordomo@postgresql.org)

--
Bruce Momjian | http://candle.pha.pa.us
pgman@candle.pha.pa.us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org