How do you pull JSON array from API in Python?

Question

I am having issues pulling a JSON array from http://api.divesites.com. When I manually set the data as a string in Python 3.7, I can see the array.

    import json

def main():
    hardcode = """{"request":{"str":null,"timestamp":1572865590,"loc":{"lat":"50.442","lng":"-4.08279999999999"},"mode":"sites","dist":"9","api":1},"sites":[{"currents":null,"distance":"7.47","hazards":null,"lat":"50.3378","name":"Fort Bovisand","water":null,"marinelife":null,"description":null,"maxdepth":null,"mindepth":null,"predive":null,"id":"24387","equipment":null,"lng":"-4.1285"},{"currents":null,"distance":"7.93","hazards":null,"lat":"50.3352","name":"Plymouth Breakwater","water":null,"marinelife":null,"description":null,"maxdepth":null,"mindepth":null,"predive":null,"id":"24388","equipment":null,"lng":"-4.1485"}],"version":1,"loc":{"lat":"50.442","lng":"-4.08279999999999"},"result":true}
        """
    print("Original Data: ---", get_data(hardcode))
    print("Site Data:----", get_data(hardcode)['sites'])

def get_data(data):
    return json.loads(data)

if __name__=='__main__':
    main()

Output

Original Data: --- {'request': {'str': None, 'timestamp': 1572865590, 'loc': {'lat': '50.442', 'lng': '-4.08279999999999'}, 'mode': 'sites', 'dist': '9', 'api': 1}, 'sites': [{'currents': None, 'distance': '7.47', 'hazards': None, 'lat': '50.3378', 'name': 'Fort Bovisand', 'water': None, 'marinelife': None, 'description': None, 'maxdepth': None, 'mindepth': None, 'predive': None, 'id': '24387', 'equipment': None, 'lng': '-4.1285'}, {'currents': None, 'distance': '7.93', 'hazards': None, 'lat': '50.3352', 'name': 'Plymouth Breakwater', 'water': None, 'marinelife': None, 'description': None, 'maxdepth': None, 'mindepth': None, 'predive': None, 'id': '24388', 'equipment': None, 'lng': '-4.1485'}], 'version': 1, 'loc': {'lat': '50.442', 'lng': '-4.08279999999999'}, 'result': True}
Site Data:---- [{'currents': None, 'distance': '7.47', 'hazards': None, 'lat': '50.3378', 'name': 'Fort Bovisand', 'water': None, 'marinelife': None, 'description': None, 'maxdepth': None, 'mindepth': None, 'predive': None, 'id': '24387', 'equipment': None, 'lng': '-4.1285'}, {'currents': None, 'distance': '7.93', 'hazards': None, 'lat': '50.3352', 'name': 'Plymouth Breakwater', 'water': None, 'marinelife': None, 'description': None, 'maxdepth': None, 'mindepth': None, 'predive': None, 'id': '24388', 'equipment': None, 'lng': '-4.1485'}]

However when I try and perform a GET request (using urllib.request.urlopen()), I only receive the objects.

import urllib.request
import json

def get_jsonparsed_data(url):
    response = urllib.request.urlopen(url)
    data = response.read().decode("utf-8")
    return json.loads(data)

def main():
    url = 'http://api.divesites.com/?mode=sites&lat=-50.350874&lng=175.849890&dist=12'
    print("Original Data: ---", get_jsonparsed_data(url))
    print("Sites: ----", get_jsonparsed_data(url)['sites'])



if __name__=='__main__':
    main()

Output

Original Data: --- {'request': {'str': None, 'timestamp': 1572866211, 'loc': {'lat': '-50.350874', 'lng': '175.849890'}, 'mode': 'sites', 'dist': '12', 'api': 1}, 'sites': [], 'version': 1, 'loc': {'lat': '-50.350874', 'lng': '175.849890'}, 'result': True}
Site Data:---- []

Am I doing something wrong, or do I need an extra step?

see your Original Data - there is 'sites': [], - so you get empty list from server and Site Data shows you this empty list. All work correctly. — furas
– furas, Commented Nov 4, 2019 at 11:51
If I use lat=39.8333&lng=-98.5833 then code gives me many results in sites. The same for lat=50.442&lng=-4.08279999999999. Simply you have to use different values lat, lng to get some sites — furas
– furas, Commented Nov 4, 2019 at 12:00

Jimmy jones · Accepted Answer · 2019-11-04 12:34:26Z

1

Solved it. I spent so long focusing on debugging the code I forgot to check the final API call. Turns out I either didn't have enough distance set, or the wrong starting longitude and latitude. This was different to my hard coded JSON.

The URL should have been: http://api.divesites.com/?mode=sites&lat=50.442&lng=-4.08279999999999&dist=9

Stupid mistake. Thanks for helping though. It got me to look.

answered Nov 4, 2019 at 12:34

Jimmy jones

111 bronze badge

Sign up to request clarification or add additional context in comments.

Comments

Dinac23 · Accepted Answer · 2019-11-04 11:55:36Z

0

First, I'd use the requests library instead and then you can find the solution here

answered Nov 4, 2019 at 11:55

Dinac23

2243 silver badges12 bronze badges

1 Comment

furas Over a year ago

problem has nothing to do with requests or JSON - code works correctly with different values lat, lng

Collectives™ on Stack Overflow

How do you pull JSON array from API in Python?

2 Answers 2

Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related