Hi!

I found a bug in FreeBSD libc which prevented Unicode-enabled wxWidgets
from working properly. I committed a fix to FreeBSD CVS (HEAD) and will
merge the change back to RELENG_5 after two weeks.

pgadmin3 from FreeBSD ports (ports/databases/pgadmin3) now works properly.
pkg-message contains information which FreeBSD versions has fixed libc and
where to get patch for versions with broken libc.

I have a few questions:
- Does unicode-enabled pgadmin3 work correctly in non-Latin-1 single-byte
encoding locales (e.g. ru_RU.KOI8-R)? It does not for me on FreeBSD 5.3.
Does it work on other platforms (e.g. Linux)?
- Have you experienced user input problems (non-Latin-1 characters can not
be entered)?
- if column data contains non-ASCII characters pgadmin3 shows empty
columns in result set for me. Can this happen because database encoding
is set to SQL_ASCII?

/fjoe

---------------------------(end of broadcast)---------------------------
TIP 7: don't forget to increase your free space map settings

Hi Max.

From: "Max Khon" <fjoe@samodelkin.net>

>
> I found a bug in FreeBSD libc which prevented Unicode-enabled wxWidgets
> from working properly. I committed a fix to FreeBSD CVS (HEAD) and will
> merge the change back to RELENG_5 after two weeks.

Oh, I believe that patch-kit is very wonderful. I didn't have time to touch them recently.
However, You met our expectation perfectly.:-) You are a great engineer..!
I was very busy with the preparation for the commemoration seminar of PostgreSQL8.0.
See.
http://itpro.nikkeibp.co.jp/free/ITP...050218/156392/
(Sorry, It is Japanese)
Thanks Dave.:-)

>
> pgadmin3 from FreeBSD ports (ports/databases/pgadmin3) now works properly.
> pkg-message contains information which FreeBSD versions has fixed libc and
> where to get patch for versions with broken libc.

We should find it.

>
> I have a few questions:
> - Does unicode-enabled pgadmin3 work correctly in non-Latin-1 single-byte
> encoding locales (e.g. ru_RU.KOI8-R)? It does not for me on FreeBSD 5.3.
> Does it work on other platforms (e.g. Linux)?

I understand only Ja_JP.UTF-8.:-) Can anyone except for me confirm it?

> - Have you experienced user input problems (non-Latin-1 characters can not
> be entered)?

Hmm, I am not clear though I thinks whether it is an IME-Setting problem.

> - if column data contains non-ASCII characters pgadmin3 shows empty
> columns in result set for me. Can this happen because database encoding
> is set to SQL_ASCII?

Hm, You must take it into consideration as follows when a multi-byte set is used.
example-Japanese)

bsd2% initdb -E EUC_JP --no-locale
bsd2% LANG=ja_JP.UTF-8 pgadmin3

regards,
Hiroshi Saito

---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

Hi!

Attached patch fixes coredump in non-Unicode version of pgadmin3.
This allows to build (non-Unicode) pgadmin3 on FreeBSD 4.x.

/fjoe


---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to majordomo@postgresql.org)

Max Khon wrote:
> Hi!
>
> Attached patch fixes coredump in non-Unicode version of pgadmin3.
> This allows to build (non-Unicode) pgadmin3 on FreeBSD 4.x.

This patch appears highly dubious to me (apart from the fact that I'd
call non-unicode unsupported).

> +#if wxUSE_UNICODE
> + size_t buf_len = nLen;
> +#else
> + size_t buf_len = nLen * sizeof(wchar_t);
> +#endif
> +
m_conversion->MB2WC((wchar_t*)(wxChar*)wxStringBuffer(str, buf_len),
(const char*)buffer, (size_t)(nLen+1));

wxStringBuffers expects the second parameter to contain the number of
characters (not bytes), which will be the same for unicode as and
non-unicode. Using sizeof(wchar_t) in non-unicode is just an arbitrary
number with no special meaning, might be the magic constant 42 as well.
The only mechanism of "fix" I could think of is that wxStringBuffer
really should receive nLen+1 instead of nLen.
Max, apparently you have a test case, please check nLen+1 instead of
buf_len.

Regards,
Andreas

---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

Hi!

On Wed, Feb 23, 2005 at 10:26:05PM +0000, Andreas Pflug wrote:

> >Attached patch fixes coredump in non-Unicode version of pgadmin3.
> >This allows to build (non-Unicode) pgadmin3 on FreeBSD 4.x.
>
> This patch appears highly dubious to me (apart from the fact that I'd
> call non-unicode unsupported).
>
> >+#if wxUSE_UNICODE
> >+ size_t buf_len = nLen;
> >+#else
> >+ size_t buf_len = nLen * sizeof(wchar_t);
> >+#endif
> > +
> m_conversion->MB2WC((wchar_t*)(wxChar*)wxStringBuffer(str, buf_len),
> (const char*)buffer, (size_t)(nLen+1));
>
> wxStringBuffers expects the second parameter to contain the number of
> characters (not bytes), which will be the same for unicode as and
> non-unicode.

No. wxStringBuffer uses wxWCharBuffer in Unicode case and
wxCharBuffer in non-Unicode case.

> Using sizeof(wchar_t) in non-unicode is just an arbitrary
> number with no special meaning, might be the magic constant 42 as well.
> The only mechanism of "fix" I could think of is that wxStringBuffer
> really should receive nLen+1 instead of nLen.
>
> Max, apparently you have a test case, please check nLen+1 instead of
> buf_len.

It doesn't help.

/fjoe

---------------------------(end of broadcast)---------------------------
TIP 9: the planner will ignore your desire to choose an index scan if your
joining column's datatypes do not match

Max Khon wrote:
> Hi!
>
> On Wed, Feb 23, 2005 at 10:26:05PM +0000, Andreas Pflug wrote:
>
>
>>>Attached patch fixes coredump in non-Unicode version of pgadmin3.
>>>This allows to build (non-Unicode) pgadmin3 on FreeBSD 4.x.
>>
>>This patch appears highly dubious to me (apart from the fact that I'd
>>call non-unicode unsupported).
>>
>>
>>>+#if wxUSE_UNICODE
>>>+ size_t buf_len = nLen;
>>>+#else
>>>+ size_t buf_len = nLen * sizeof(wchar_t);
>>>+#endif
>>>+
>>
>>m_conversion->MB2WC((wchar_t*)(wxChar*)wxStringBuffer(str, buf_len),
>>(const char*)buffer, (size_t)(nLen+1));
>>
>>wxStringBuffers expects the second parameter to contain the number of
>>characters (not bytes), which will be the same for unicode as and
>>non-unicode.
>
>
> No. wxStringBuffer uses wxWCharBuffer in Unicode case and
> wxCharBuffer in non-Unicode case.

Which is the same.
1 character = wchar_t = wxChar = wxWChar in Unicode,
1 character = char = wxChar in non-Unicode.

Anyhow, having a deeper look at wx unicode conversions it seems that (in
contrast to earlier versions) now utf-conversions are supported in
non-unicode too, making that offending line more than a no-op.
Unfortunately, the usage of wxStringBuffer as argument to MB2WC is
*plain wrong*, because writing wchar_t to char is pure bullsh*t for
non-unicode. The code assumes wxChar==wchar_t, which is true in unicode
only.

Which condenses down to "utffile.cpp is not non-unicode compatible".
I've added a #error accordingly. Until utffile is thoroughly reworked
for non-unicode, compiling pgadmin in unicode mode is *required*.

Regards,
Andreas

---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org

Hi!

On Thu, Feb 24, 2005 at 10:22:22AM +0000, Andreas Pflug wrote:

> >>>Attached patch fixes coredump in non-Unicode version of pgadmin3.
> >>>This allows to build (non-Unicode) pgadmin3 on FreeBSD 4.x.
> >>
> >>This patch appears highly dubious to me (apart from the fact that I'd
> >>call non-unicode unsupported).
> >>
> >>
> >>>+#if wxUSE_UNICODE
> >>>+ size_t buf_len = nLen;
> >>>+#else
> >>>+ size_t buf_len = nLen * sizeof(wchar_t);
> >>>+#endif
> >>>+
> >>
> >>m_conversion->MB2WC((wchar_t*)(wxChar*)wxStringBuffer(str, buf_len),
> >>(const char*)buffer, (size_t)(nLen+1));
> >>
> >>wxStringBuffers expects the second parameter to contain the number of
> >>characters (not bytes), which will be the same for unicode as and
> >>non-unicode.
> >
> >
> >No. wxStringBuffer uses wxWCharBuffer in Unicode case and
> >wxCharBuffer in non-Unicode case.
>
> Which is the same.
> 1 character = wchar_t = wxChar = wxWChar in Unicode,
> 1 character = char = wxChar in non-Unicode.
>
> Anyhow, having a deeper look at wx unicode conversions it seems that (in
> contrast to earlier versions) now utf-conversions are supported in
> non-unicode too, making that offending line more than a no-op.
> Unfortunately, the usage of wxStringBuffer as argument to MB2WC is
> *plain wrong*, because writing wchar_t to char is pure bullsh*t for
> non-unicode. The code assumes wxChar==wchar_t, which is true in unicode
> only.
>
> Which condenses down to "utffile.cpp is not non-unicode compatible".
> I've added a #error accordingly. Until utffile is thoroughly reworked
> for non-unicode, compiling pgadmin in unicode mode is *required*.

You are correct. The patch I submitted is a workaround and was never supposed
to fix the root of the problem. But it allows to run pgadmin3 1.2.0 on FreeBSD
4.x which lacks proper wchar_t support but is still widely used.

I removed non-Unicode support in FreeBSD port initially because of the
problems you described, but the a few users asked me if it is possible to
add it back.

/fjoe

---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq

Max Khon wrote:

>
>
> You are correct. The patch I submitted is a workaround and was never supposed
> to fix the root of the problem. But it allows to run pgadmin3 1.2.0 on FreeBSD
> 4.x which lacks proper wchar_t support but is still widely used.

That hotfix didn't allow reading a file, so disabling the non-working
file operations would have been better.


> I removed non-Unicode support in FreeBSD port initially because of the
> problems you described, but the a few users asked me if it is possible to
> add it back.


While we removed non-unicode builds for win32 a long time ago, I'm not
opposed to supporting it in general. If you contribute some working code
(which would either mean adding a utf handling portion for non-unicode
in utffile or disable unicode file operations throughout the app),
you're welcome.

Regards,
Andreas


---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org

Hi Max.

Sorry,
I am watching argument now for busyness.

----- Original Message -----
From: "Max Khon" <fjoe@samodelkin.net>

> > Which condenses down to "utffile.cpp is not non-unicode compatible".
> > I've added a #error accordingly. Until utffile is thoroughly reworked
> > for non-unicode, compiling pgadmin in unicode mode is *required*.
>
> You are correct. The patch I submitted is a workaround and was never supposed
> to fix the root of the problem. But it allows to run pgadmin3 1.2.0 on FreeBSD
> 4.x which lacks proper wchar_t support but is still widely used.

However, I think that your approach is right. Because, An environment had a limitation.
You are the result which becomes stable how and it thought whether it could be supplied.
Unicode edition will probably meet with the trouble with 4.x. I used special magic therefore.

>
> I removed non-Unicode support in FreeBSD port initially because of the
> problems you described, but the a few users asked me if it is possible to
> add it back.

There must be many users who use the version very much.
Again, your offer was right.:-)

Thank you.

regards,
Hiroshi Saito



---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org