Getting JSON records line by line from a file, in python

Question

I am working with JSON files in python. I have keys in a list, for each key I retrieve a JSON record, which I store in a file. Each JSON record contains that key inside.

In a separate process, I need to read that JSON file and retrieve each record, get key and do something else.

The problem is I cannot read JSON lines from file I stored this way:

dataFile = open(output_file, "w")
for line in keys_list:
   json_line = fetch_json_str(line)
   data = simplejson.loads(json_line)
   dataFile.write(simplejson.dumps(data, sort_keys = True))

It reads all into one line, and the length of returned list is 0, len(json_lines).

  json_lines = [line.strip() for line in open(tmp_load_file)]
  for line in json_lines
      data = simplejson.loads(simplejson.dumps(line))

What do I do wrong? Is there any way to get back JSON lines without changing the way I stored them, because that will call for re-processing all the json files I stored this way.

tdelaney · Accepted Answer · 2015-02-17 17:53:48Z

1

You need to add a line separator when you write the json string

dataFile.write(simplejson.dumps(data, sort_keys = True) + '\n')

Otherwise, you just get all of the json records in a single line and the json parser can't figure it out.

answered Feb 17, 2015 at 17:53

tdelaney

77.9k6 gold badges91 silver badges129 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Getting JSON records line by line from a file, in python

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related