Python parsing JSON from url incomplete

Question

I am trying to get a JSON from a URL I have succeeded in doing that but when I print the result it seems like I get only half of the content.

this is what I wrote:

with urllib.request.urlopen("https://financialmodelingprep.com/api/v3/company/stock/list") as url:
    data = json.loads(url.read().decode())
    print(data)

I won't print what I am getting because it's very long(if you look at the source and ctrl+f look up the sentence you will find that its in the middle of the page) but it starts with ties Municipal Fund', 'price': 9.83,

thanks for the help.

I tried to run your code on Jupyter and it seems I can print the whole json. — mttbrt
– mttbrt, Commented Mar 31, 2020 at 13:38

Gabio · Accepted Answer · 2020-03-31 13:46:25Z

2

I think that your output has been cut by your IDE.

When I run the same request and I write it into a file, you can see the that the data is fully written:

import requests
with requests.get("https://financialmodelingprep.com/api/v3/company/stock/list") as url:
    data = url.content.decode()
    with open("file.txt","w") as f:
        f.write(data)

answered Mar 31, 2020 at 13:46

Gabio

9,5643 gold badges17 silver badges38 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Serge Ballesta · Accepted Answer · 2020-03-31 13:42:40Z

1

When the response is large, the first read may only return what is available at that time. If you want to be sure to have the full data, you should loop reading until you get nothing (an empty byte string here).

Here, it would probably be more robust to pass the response stream to json.load and let it read until the json string is completed:

with urllib.request.urlopen(
        "https://financialmodelingprep.com/api/v3/company/stock/list") as url:
    data = json.load(url)
    print(data)

But I think that if the data was partial, json.loads should have choked on it. So the problem is likely elsewhere...

answered Mar 31, 2020 at 13:42

Serge Ballesta

150k13 gold badges137 silver badges267 bronze badges

1 Comment

Serge Ballesta Over a year ago

@bikoman57: this is the evidence that the problem is not is the load part but in the display one...

Collectives™ on Stack Overflow

Python parsing JSON from url incomplete

2 Answers 2

Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related