Unix Technical Forum

how to store and still search special characters in Python and MySql

This is a discussion on how to store and still search special characters in Python and MySql within the MySQL forums, part of the Database Server Software category; --> I have an MySQL database called zingers. The structure is: zid - integer, key, autoincrement keyword - varchar citation ...


Go Back   Unix Technical Forum > Database Server Software > MySQL

FAQ Members List Calendar Search Today's Posts Mark Forums Read
  #1 (permalink)  
Old 02-28-2008, 10:08 AM
Ron
 
Posts: n/a
Default how to store and still search special characters in Python and MySql

I have an MySQL database called zingers. The structure is:

zid - integer, key, autoincrement
keyword - varchar
citation - text
quotation - text

I am having trouble storing text, as typed in latter two fields.
Special characters and punctuation all seem not to be stored and
retrieved correctly.

Special apostrophes and single quotes from Microsoft Word are causing
a
special problem, even though I have ''ed all 's

perhaps the encoding of the database itself should be different? it
is
currenlty latin_swedish_ci

Input and output is through a browser.

I think my problem may be that I need to encode the string before
saving it in the databse. Can anyone point me in the right direction
here?

here's the error message:

UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position
95: ordinal not in range(128)
args = ('ascii', "update zingers set keywords =
'a;Action;b;Religi... \n \n \n ' where zid = 422", 95, 96, 'ordinal
not
in range(128)')
encoding = 'ascii'
end = 96
object = "update zingers set keywords = 'a;Action;b;Religi...
\n
\n \n ' where zid = 422"
reason = 'ordinal not in range(128)'
start = 95

the characters I am trying to add are startquote and endquote copied
and pasted from Microsoft Word.

Can anyone help me on this?

bests,

-rsr-

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #2 (permalink)  
Old 02-28-2008, 10:08 AM
Jerry Stuckle
 
Posts: n/a
Default Re: how to store and still search special characters in Python andMySql

Ron wrote:
> I have an MySQL database called zingers. The structure is:
>
> zid - integer, key, autoincrement
> keyword - varchar
> citation - text
> quotation - text
>
> I am having trouble storing text, as typed in latter two fields.
> Special characters and punctuation all seem not to be stored and
> retrieved correctly.
>
> Special apostrophes and single quotes from Microsoft Word are causing
> a
> special problem, even though I have ''ed all 's
>
> perhaps the encoding of the database itself should be different? it
> is
> currenlty latin_swedish_ci
>
> Input and output is through a browser.
>
> I think my problem may be that I need to encode the string before
> saving it in the databse. Can anyone point me in the right direction
> here?
>
> here's the error message:
>
> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position
> 95: ordinal not in range(128)
> args = ('ascii', "update zingers set keywords =
> 'a;Action;b;Religi... \n \n \n ' where zid = 422", 95, 96, 'ordinal
> not
> in range(128)')
> encoding = 'ascii'
> end = 96
> object = "update zingers set keywords = 'a;Action;b;Religi...
> \n
> \n \n ' where zid = 422"
> reason = 'ordinal not in range(128)'
> start = 95
>
> the characters I am trying to add are startquote and endquote copied
> and pasted from Microsoft Word.
>
> Can anyone help me on this?
>
> bests,
>
> -rsr-
>


Ron,

Microsoft Word uses special characters not in latin_swedish_ci. You
should be able to save the data in the database by using
mysql_real_escape_string for the text (you may have to define the
columns as BINARY). But I don't think you'll be able to display it in a
browser.

Better is to not use Microsoft Word for your input.

--
==================
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
jstucklex@attglobal.net
==================
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #3 (permalink)  
Old 02-28-2008, 10:08 AM
Peter H. Coffin
 
Posts: n/a
Default Re: how to store and still search special characters in Python and MySql

On Mon, 12 Feb 2007 05:51:13 -0500, Jerry Stuckle wrote:
> Ron wrote:
>> the characters I am trying to add are startquote and endquote copied
>> and pasted from Microsoft Word.
>>
>> Can anyone help me on this?
>>
>> bests,
>>
>> -rsr-
>>

>
> Ron,
>
> Microsoft Word uses special characters not in latin_swedish_ci. You
> should be able to save the data in the database by using
> mysql_real_escape_string for the text (you may have to define the
> columns as BINARY). But I don't think you'll be able to display it in a
> browser.
>
> Better is to not use Microsoft Word for your input.


UTF-8 charsets may also help. Those do support the characters in
question.

http://dev.mysql.com/doc/refman/5.0/...t-unicode.html

This is not to say that making a single change to the table will fix the
issue. There are *many* traps in the issue of character sets, but mysql
at least provides options to help support a careful programmer in making
a character-safe system.

--
It's not hard, it's just asking for a visit by the fuckup fairy.
-- Peter da Silva
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
Reply


Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On
Forum Jump


All times are GMT. The time now is 05:17 PM.


Powered by vBulletin® Version 3.6.5
Copyright ©2000 - 2008, Jelsoft Enterprises Ltd.
SEO by vBSEO 3.2.0
www.UnixAdminTalk.com