Using Python to read json data - Beginner question

Question

I am a beginner with Json and trying to read the Automotive 5 core Json file from here : http://jmcauley.ucsd.edu/data/amazon/, with the following code

Python code:

import json
with open('Automotive_5.json') as f:
data = json.load(f)

I keep getting JSONDecodeError: Extra data

Complete Traceback:

runfile('C:/Users/Paul/Google Drive/erg2.py', wdir='C:/Users/Paul/Google Drive')
Traceback (most recent call last):

  File "<ipython-input-122-72136ec568c5>", line 1, in <module>
    runfile('C:/Users/Paul/Google Drive/erg2.py', wdir='C:/Users/Paul/Google Drive')

  File "C:\Users\Paul\Anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 704, in runfile
    execfile(filename, namespace)

  File "C:\Users\Paul\Anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 108, in execfile
    exec(compile(f.read(), filename, 'exec'), namespace)

  File "C:/Users/Paul/Google Drive/erg2.py", line 10, in <module>
    data = json.load(f)

  File "C:\Users\Paul\Anaconda3\lib\json\__init__.py", line 296, in load
    parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)

  File "C:\Users\Paul\Anaconda3\lib\json\__init__.py", line 348, in loads
    return _default_decoder.decode(s)

  File "C:\Users\Paul\Anaconda3\lib\json\decoder.py", line 340, in decode
    raise JSONDecodeError("Extra data", s, end)

JSONDecodeError: Extra data

on this page I see only json compressed with gzip - reviews_Automotive_5.json.gz. Did you uncompress this file ? — furas
– furas, Commented Sep 10, 2019 at 3:17
yes i did uncompress it... OK, edited the original post with the whole error msg — Heavy Load
– Heavy Load, Commented Sep 10, 2019 at 3:27

Matthew Gaiser · Accepted Answer · 2019-09-10 03:30:26Z

2

I downloaded your file and tested things on my own system. I am not sure why, but you need to load each line separately. Hopefully someone else can provide the why, but this code seems to work for me. Maybe it is just too large? My editor complained about the size.

import json
data = []
with open('Automotive_5.json') as f:
  for line in f:
    data.append(json.loads(line))
    print(json.loads(line))

Read each line in the file as JSON and append it to data rather than trying to load it all at once. Runs without error for me.

answered Sep 10, 2019 at 3:30

Matthew Gaiser

4,8431 gold badge22 silver badges36 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

furas Over a year ago

I also checked file by you was faster :) Now I checked web page and there is even information "Format is one-review-per-line in (loose) json." and there is even code which shows how to convert it to normal JSON - see "Convert to 'strict' json".

Matthew Gaiser Over a year ago

"Loose" JSON might do it. Maybe the thing isn't properly formatted?

furas Over a year ago

file is not correctly formated JSON but it seems they did it intentionally.

Matthew Gaiser Over a year ago

That would do it then.

virender nehra · Accepted Answer · 2019-09-10 03:42:01Z

0

It looks like the problem it with the json file itself i.e. you have to read data line by line, the all data in json file should be in one object but you have separate separate object in json file.

If you want to read data in one go then put all the objects of json file in one object with comma separated.

answered Sep 10, 2019 at 3:42

virender nehra

6228 silver badges16 bronze badges

Collectives™ on Stack Overflow

Using Python to read json data - Beginner question

2 Answers 2

4 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

4 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related