3

I have a very simple json I can't parse with simplejson module. Reproduction:

import simplejson as json
json.loads(r'{"translatedatt1":"Vari\351es"}')

Result:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/pymodules/python2.5/simplejson/__init__.py", line 307, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/pymodules/python2.5/simplejson/decoder.py", line 335, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/pymodules/python2.5/simplejson/decoder.py", line 351, in raw_decode
    obj, end = self.scan_once(s, idx)
ValueError: Invalid \escape: line 1 column 23 (char 23)

Anyone has an idea what's wrong and how to parse the json above correctly?

The string that is encoded there is: Variées

P.S. I use python 2.5

Thanks a lot!

Improve this question

2 Answers 2

Reset to default
8

That would be quite correct; Vari\351es contains an invalid escape, the JSON standard does not allow for a \ followed by just numbers.

Whatever produced that code should be fixed. If that is impossible, you'll need to use a regular expression to either remove those escapes, or replace them with valid escapes.

If we interpret the 351 number as an octal number, that would point to the unicode code point U+00E9, the é character (LATIN SMALL LETTER E WITH ACUTE). You can 'repair' your JSON input with:

import re

invalid_escape = re.compile(r'\\[0-7]{1,6}')  # up to 6 digits for codepoints up to FFFF

def replace_with_codepoint(match):
    return unichr(int(match.group(0)[1:], 8))


def repair(brokenjson):
    return invalid_escape.sub(replace_with_codepoint, brokenjson)

Using repair() your example can be loaded:

>>> json.loads(repair(r'{"translatedatt1":"Vari\351es"}'))
{u'translatedatt1': u'Vari\xe9es'}

You may need to adjust the interpretation of the codepoints; I choose octal (because Variées is an actual word), but you need to test this more with other codepoints.

Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

This code was produced for/by Venda platform. Unfortunately, I can't change this behavior. BTW - what would be a valid escaper?
Thanks, but the result is not what's supposed to be: after rapair function we get "Varişes", while it should have been "Variées"
@diemacht: Updated already :-) Since you didn't specify what you expected, I had to guess, then updated my guess.
4

You probably did not intend to use a raw string, but a unicode string?

>>> import simplejson as json
>>> json.loads(u'{"translatedatt1":"Vari\351es"}')
{u'translatedatt1': u'Vari\xe9es'}

If you want to quote the data inside the JSON string you need to use \uNNNN:

>>> json.loads(r'{"translatedatt1":"Vari\u351es"}')
{'translatedatt1': u'Vari\u351es'}

Please note that the resulting dict is slightly different in this case. When parsing a unicode string simplejson uses unicode strings for the keys. Otherwise it uses byte string keys.

If your JSON data does in fact use \351e than it is simply broken and no valid JSON.

Improve this answer

4 Comments

Can I do it if the string is in some variable, for example: s=r'{"translatedatt1":"Vari\351es"}' ? Thanks!!!
Just don't create the string that way. Get rid of the r prefix and use u instead if you want to create a string containing unicode data. If you really want to use quoting inside the JSON data you need to use \u351e.
@bikeshedder: I think the OP means that the server sent that data. The r'' makes it easier to show us the raw data sent. Yes, that's broken JSON data...
U+351E is CJK Ideograph Extension A.. not certain that that is the correct interpretation either. I think you were closer with octal..

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.