How to efficiently parse JSON data with multiple keys in Python 2.7?

Question

I'm writing a script that will check the CVS COVID vaccine availability for cities in my state of VA. I have been successful getting the data I'm looking for, but my code is hard coded in some areas. I'm specifically asking for help improving my code in the areas number 1 & 2 below:

The JSON file can be found here: https://www.cvs.com//immunizations/covid-19-vaccine.vaccine-status.VA.json?vaccineinfo

I'm trying to access the data in the responsePayloadData key. The only way I could figure out how to do this is to make it the only key. For that reason, I deleted the other key responseMetaData:

#remove the key that we don't need
del obj['responseMetaData']

I'm also not sure how to dynamically loop through the VA items without hard coding the number of cities I know are there in the data:

for x, y in obj.items():
   for a in range(34):

Here's the full code:

    import requests
    import json
    import time
    from datetime import datetime
    import urllib2
    try: 
        import indigo
    except:
        pass
    
    strAvail = "False"
    strAvailCity = "None"
    
    try:
        # download raw json object from CVS Virginia Website
        url = "https://www.cvs.com//immunizations/covid-19-vaccine.vaccine-status.VA.json?vaccineinfo"
        data = urllib2.urlopen(url).read().decode()
    except urllib2.HTTPError, err:
            return {"error": err.reason, "error_code": err.code} 
    
    # parse json object
    obj = json.loads(data)
    
    # remove the key that we don't need
    del obj['responseMetaData']
    
    # loop through the JSON dictionary and check availability
    # status options: {"Fully Booked", "Available"}
    for x, y in obj.items():
        for a in range(34):
            # print('City: ' + y['data']['VA'][a]['city'])
            # print('Total Available: ' + y['data']['VA'][a]['totalAvailable'])
            # print('Percent Available: ' + y['data']['VA'][a]['pctAvailable'])
            # print('Status: ' + y['data']['VA'][a]['status'])
            # print("------------------------------")
            # If there is availability anywhere in the state, take some action. 
            if y['data']['VA'][a]['status'] == "Available":
                strAvail = True
                strAvailCity = y['data']['VA'][a]['city']
    
    # Log timestamp for this check to the JSON
    now = datetime.now()
    strDateTime = now.strftime("%m/%d/%Y %I:%M %p")

EDIT: Since the JSON is not available outside the US. I've pasted it below:

{"responsePayloadData":{"currentTime":"2021-02-11T14:55:00.470","data":{"VA":[{"totalAvailable":"1","city":"ABINGDON","state":"VA","pctAvailable":"0.19%","status":"Fully Booked"},{"totalAvailable":"0","city":"ALEXANDRIA","state":"VA","pctAvailable":"0.00%","status":"Fully Booked"},{"totalAvailable":"0","city":"ARLINGTON","state":"VA","pctAvailable":"0.00%","status":"Fully Booked"},{"totalAvailable":"0","city":"BEDFORD","state":"VA","pctAvailable":"0.00%","status":"Fully Booked"},{"totalAvailable":"0","city":"BLACKSBURG","state":"VA","pctAvailable":"0.00%","status":"Fully Booked"},{"totalAvailable":"0","city":"CHARLOTTESVILLE","state":"VA","pctAvailable":"0.00%","status":"Fully Booked"},{"totalAvailable":"0","city":"CHATHAM","state":"VA","pctAvailable":"0.00%","status":"Fully Booked"},{"totalAvailable":"0","city":"CHESAPEAKE","state":"VA","pctAvailable":"0.00%","status":"Fully Booked"},{"totalAvailable":"1","city":"DANVILLE","state":"VA","pctAvailable":"0.19%","status":"Fully Booked"},{"totalAvailable":"2","city":"DUBLIN","state":"VA","pctAvailable":"0.39%","status":"Fully Booked"},{"totalAvailable":"0","city":"FAIRFAX","state":"VA","pctAvailable":"0.00%","status":"Fully Booked"},{"totalAvailable":"0","city":"FREDERICKSBURG","state":"VA","pctAvailable":"0.00%","status":"Fully Booked"},{"totalAvailable":"0","city":"GAINESVILLE","state":"VA","pctAvailable":"0.00%","status":"Fully Booked"},{"totalAvailable":"0","city":"HAMPTON","state":"VA","pctAvailable":"0.00%","status":"Fully Booked"},{"totalAvailable":"0","city":"HARRISONBURG","state":"VA","pctAvailable":"0.00%","status":"Fully Booked"},{"totalAvailable":"0","city":"LEESBURG","state":"VA","pctAvailable":"0.00%","status":"Fully Booked"},{"totalAvailable":"0","city":"LYNCHBURG","state":"VA","pctAvailable":"0.00%","status":"Fully Booked"},{"totalAvailable":"0","city":"MARTINSVILLE","state":"VA","pctAvailable":"0.00%","status":"Fully Booked"},{"totalAvailable":"0","city":"MECHANICSVILLE","state":"VA","pctAvailable":"0.00%","status":"Fully Booked"},{"totalAvailable":"0","city":"MIDLOTHIAN","state":"VA","pctAvailable":"0.00%","status":"Fully Booked"},
{"totalAvailable":"0","city":"NEWPORT NEWS","state":"VA","pctAvailable":"0.00%","status":"Fully Booked"},{"totalAvailable":"0","city":"NORFOLK","state":"VA","pctAvailable":"0.00%","status":"Fully Booked"},{"totalAvailable":"0","city":"PETERSBURG","state":"VA","pctAvailable":"0.00%","status":"Fully Booked"},{"totalAvailable":"0","city":"PORTSMOUTH","state":"VA","pctAvailable":"0.00%","status":"Fully Booked"},{"totalAvailable":"0","city":"RICHMOND","state":"VA","pctAvailable":"0.00%","status":"Fully Booked"},{"totalAvailable":"0","city":"ROANOKE","state":"VA","pctAvailable":"0.00%","status":"Fully Booked"},
{"totalAvailable":"0","city":"ROCKY MOUNT","state":"VA","pctAvailable":"0.00%","status":"Fully Booked"},{"totalAvailable":"0","city":"STAFFORD","state":"VA","pctAvailable":"0.00%","status":"Fully Booked"},{"totalAvailable":"0","city":"SUFFOLK","state":"VA","pctAvailable":"0.00%","status":"Fully Booked"},
{"totalAvailable":"0","city":"VIRGINIA BEACH","state":"VA","pctAvailable":"0.00%","status":"Fully Booked"},{"totalAvailable":"0","city":"WARRENTON","state":"VA","pctAvailable":"0.00%","status":"Fully Booked"},{"totalAvailable":"0","city":"WILLIAMSBURG","state":"VA","pctAvailable":"0.00%","status":"Fully Booked"},{"totalAvailable":"0","city":"WINCHESTER","state":"VA","pctAvailable":"0.00%","status":"Fully Booked"},{"totalAvailable":"0","city":"WOODSTOCK","state":"VA","pctAvailable":"0.00%","status":"Fully Booked"}]}},"responseMetaData":{"statusDesc":"Success","conversationId":"Id-beb5f68730b34e6aa3bbc1fd927ea12b","refId":"Id-b4a7256078789eb59b8912b4","operation":"getInventorybyCity","statusCode":"0000"}}

Please note that the link to the JSON file you've provided is not accessible outside the US. Maybe you can provide us with a portion of it by copying and pasting it in your post. — dpant
– dpant, Commented Feb 11, 2021 at 21:23
I have fixed the hard coded for loop using len(y['data']['VA']) — ryanbuckner
– ryanbuckner, Commented Feb 11, 2021 at 22:52
@ryanbuckner: you can iterate over lists without knowing their length. Iteration will stop once all items have been visited. See my answer for example. — mhawke
– mhawke, Commented Feb 11, 2021 at 23:02
@ryanbuckner looks like access is denied to that JSON feed now, even inside the US. did you find a way around that? — Keith
– Keith, Commented Feb 25, 2021 at 17:28

mhawke · Accepted Answer · 2021-02-11 22:59:57Z

1

Regarding problem 1, you can just access the data by key. You don't need to delete the other key:

payload = obj['responsePayloadData']

For the second problem, you can just iterate over the items in the list associated with obj['data']['VA']:

for city in payload['data']['VA']:
    print(city)

{'city': 'ABINGDON',
 'pctAvailable': '0.19%',
 'state': 'VA',
 'status': 'Fully Booked',
 'totalAvailable': '1'}
{'city': 'ALEXANDRIA',
 'pctAvailable': '0.00%',
 'state': 'VA',
 'status': 'Fully Booked',
 'totalAvailable': '0'}
...

answered Feb 11, 2021 at 22:59

mhawke

87.5k10 gold badges122 silver badges142 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

ryanbuckner Over a year ago

Thank you! This is exactly what I need

Collectives™ on Stack Overflow

How to efficiently parse JSON data with multiple keys in Python 2.7?

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related