Python3 UnicodeDecodeError

Question

I have a Django 1.8 app running on a server with Python 3 and I get a UnicodeDecodeError when logging and printing strings with special characters.

>:python --version python 3.4.3

For example, if I try to run a silly method in the shell:

def print_test():
       print('Test: èè') # any 'special char' like ä ç é û...

I get a stack trace:

>>> print_test()
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/home/sailingadmin/sailing-admin/utest.py", line 2, in print_test
    print('This is a test: \xe8\xe8')
UnicodeEncodeError: 'ascii' codec can't encode characters in position 16-17: ordinal not in range(128)

Same with print(u'Test: èè')

Why does this error happen?

utest.py is encoded in utf-8 (Python 3 default for source files)

>:echo $LC_CTYPE UTF-8

All logging and printing raises a UnicodeEncodeError...

What's important here are the LC_* environment variable used by your server process - not the ones defined for your own user account. You may want to check your process's environment and front-end server config. — bruno desthuilliers
– bruno desthuilliers, Commented Nov 27, 2015 at 10:39
@brunodesthuilliers Do you mean the /proc/<id>/environ? LC_CTYPE=UTF-8 and LANG=en_US.UTF-8, no other encoding variables are set there. — Ire
– Ire, Commented Nov 27, 2015 at 10:50
I mean "the environment variables of the process which stdout will be used by your process"... Sorry I've not had to deal with this kind of problem for quite a few years now (thanks to our sysadmin <g>), so I don't remember all the gory details :-/ — bruno desthuilliers
– bruno desthuilliers, Commented Nov 27, 2015 at 10:55
Take a look here, maybe that will help you set the encoding back to UTF-8: stackoverflow.com/questions/2276200/… — Maciek
– Maciek, Commented Nov 27, 2015 at 11:28
@Maciek this does not apply to python 3, reload is not a builtin. (but it is importable) — Ire
– Ire, Commented Nov 27, 2015 at 11:37

Ire · Accepted Answer · 2015-12-04 16:00:01Z

2

LC_CTYPE format as follows: en_US.UTF-8 (char map after the dot) Then python uses it as default encoding for logging and io.

answered Dec 4, 2015 at 16:00

Ire

1,36110 silver badges17 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Python3 UnicodeDecodeError

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related