Storing Unicode strings to SQLite database using PeeWee

Question

Working with peewee to create a simple sqlite database that stores items form a parsed MRSS feed, specifically video URLs and video titles.

I have a little dedupe script that I wrote that checks whether or not the URL and video title that's just been parsed is already in the database. I've noticed that when I pass URL strings to the database, there are no issues, dedupe works fine. When I browser through the database for the URLs I've stored there, using an app like sqlitebrowser, I see the URLs with no Unicode encoding, they looks like regular old strings, no u'.

However, when I pass unicode video titles — like this: (u'Animals doing the strangest things',) the dedupe script wasn't "seeing" the video titles in the database. The solution was to do a simple conversion with the video title object like this: videoName = str(videoName) — however, I have no clue why this works now. When I browse the videoName column in my database using sqlitebrowser, I see the video names, but they're still in Unicode and look the same as they did before: (u'Animals doing the strangest things',)

Any idea what the heck is going on here? Does this have something to do with a weird bug with how PeeWee is handling string serialization?

coleifer · Accepted Answer · 2015-06-30 02:32:32Z

1

Peewee uses unicode for both CharField and TextField, meaning that whatever value you have stored on your model is converted to a Python unicode object (well, unicode in Py2, str in Py3).

The database driver (sqlite3, psycopg2, etc) generally handles converting the unicode to the appropriate encoding.

I'm not sure what the bug is you're describing, but maybe you can share some code to replicate it. I can tell you though, that peewee's unicode handling is tested and works.

answered Jun 30, 2015 at 2:32

coleifer

26.4k7 gold badges67 silver badges81 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

AdjunctProfessorFalcon Over a year ago

Thanks for the response, what do you think would account or PeeWee not allowing me to store the video names stored in videoName until I altered it to str(videoName)?

coleifer Over a year ago

What do you mean by "not allowing"? Is there an error?

AdjunctProfessorFalcon Over a year ago

Only when I altered the unicode string held in videoName to str(videoName) would it show up in the SQLite database.

Collectives™ on Stack Overflow

Storing Unicode strings to SQLite database using PeeWee

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related