1

I am using python, and I have a large 'outputString' that consists of several outputs, each on a new line, to look something like this:

{size:1, title:"Hello", space:0}
{size:21, title:"World", space:10}
{size:3, title:"Goodbye", space:20}

However, there is so much data that I cannot see it all in the terminal, and would like to write code that automatically writes a json file. I am having trouble getting the json to keep the separated lines. Right now, it is all one large line in the json file. I have attached some code that I have tried. I have also attached the code used to make the string that I want to convert to a json. Thank you so much!

for value in outputList:
    newOutputString = json.dumps(value)
    outputString += (newOutputString + "\n")

with open('data.json', 'w') as outfile:
    for item in outputString.splitlines():
        json.dump(item, outfile)
        json.dump("\n",outfile)
Improve this question
8
  • 1
    Does the indent keyword argument of json.dump help you? Commented Jun 23, 2020 at 17:29
  • 1
    Why is outputString a string? Are you forced to have a string here or did you decided to make it a string? Commented Jun 23, 2020 at 17:32
  • Actually the answer to my question is clearly going to be yes... you would do e.g. json.dump(item, outfile, indent=4). But you probably do not want to be writing multiple JSON dumps to the same file in any case... Commented Jun 23, 2020 at 17:32
  • Just construct one object that contains everything, and dump that to the file as a single JSON dump (with pretty-printing using indent if you want). Otherwise your output file is not valid JSON. Commented Jun 23, 2020 at 17:35
  • @alaniwi Thank you! That gave me an error, 'json.decoder.JSONDecodeError: Extra data: line 2 column 1 (char 104)' Do you know why this could be? Commented Jun 23, 2020 at 21:46

2 Answers 2

Reset to default
1

If the input really is a string, you'll probably have to make sure it's some properly formated as json:

outputString = '''{"size":1, "title":"Hello", "space":0}
{"size":21, "title":"World", "space":10}
{"size":3, "title":"Goodbye", "space":20}'''

You could then use pandas to manipulate your data (so it's not a problem of screen size anymore).

import pandas as pd
import json

pd.DataFrame([json.loads(line) for line in outputString.split('\n')])

Which gives:

   size    title  space
0     1    Hello      0
1    21    World     10
2     3  Goodbye     20

On the other hand, from what I understand outputString is not a string but a list of dictionaries, so you could write a simpler version of this:

outputString = [{'size':1, 'title':"Hello", 'space':0},
{'size':21, 'title':"World", 'space':10},
{'size':3, 'title':"Goodbye", 'space':20}]

pd.DataFrame(outputString)

Which gives the same DataFrame as before. Using this DataFrame will allow you to query your data and it will be much more confortable than a JSON. For example

>>> df = pd.DataFrame(outputString)
>>> df[df['size'] >= 3]
   size    title  space
1    21    World     10
2     3  Goodbye     20

You could also to try ipython (or even jupyter/jupyterlab) as it will probably also make your life easier.

Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

Thank you! But I would like the json to be in the same format as the string, rather than the data frame you mentioned. Each Line is indeed a dictionary, but I have converted it to a json that is added to the output String. So the outputString is a dictionary converted into a json and then a new line for each value
0

You can use below code:

json_data = json.loads(outputString)
with open('data.json', 'w') as outfile:
    json.dump(json_data, outfile, indent= 5)
Improve this answer

2 Comments

Thank you! However, that gives me 'json.decoder.JSONDecodeError: Extra data: line 2 column 1 (char 104)' Do you know why this could be?
Please check there is no variable who has same name as "data" may be that will be the issue.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.