python string to json conversion

Question

I have a python script below which loads a yaml file. Arguments are passed in as a list so I can loop through them to add values to the dictionary. The code below adds the parameters but the json is not properly formatted as displayed in the current and desired json result. Any help in the right direction is appreciated. Thanks.

YAML File

  policies:
  - name: ec2age
    resource: ec2
    filters:      
      - State.Name: running
      - "tag:ResourceCreatedBy": present
      - and:
        - type: instance-age
          days: 30
          op: ge

Python script:

#!/usr/bin/python
import sys
import argparse
import json
import yaml

parser = argparse.ArgumentParser()
parser.add_argument("-y", "--yaml_file", type=str)
parser.add_argument("-b", "--business_unit", type=str)
parser.add_argument("-p", "--platform", type=str)
parser.add_argument("-c", "--client", type=str)
args = parser.parse_args()

with open(args.yaml_file, 'r') as stream:
    data = yaml.load(stream)

inner_dict = [args.business_unit, args.platform, args.client]

for item in list(inner_dict):
    data['policies'][0]['filters'][0]['tag:BUSINESS_UNIT'] = args.business_unit
    data['policies'][0]['filters'][1]['tag:PLATFORM'] = args.platform

print json.dumps(data)

Current Result:

{
"policies": [
    {
    "resource": "ec2",
    "name": "ec2age",
    "filters": [
        {
        "tag:BUSINESS_UNIT": "TEST_FLITE",
        "State.Name": "running"
        },
        {
        "tag:ResourceCreatedBy": "present"
        },
        {
        "and": [
            {
            "type": "instance-age",
            "days": 30,
            "op": "ge"
            }
        ]
        }
    ]
    }
]
}

Desired Result:

{
"policies": [
    {
    "resource": "ec2",
    "name": "ec2age",
    "filters": [
        {
        "tag:BUSINESS_UNIT": "TEST_FLITE"
        },
        {
        "tag:PLATFORM": "Android"
        },
        {            
        "State.Name": "running"
        },
        {
        "tag:ResourceCreatedBy": "present"
        },
        {
        "and": [
            {
            "type": "instance-age",
            "days": 30,
            "op": "ge"
            }
        ]
        }
    ]
    }
]
}

It can be isssue with YAML, without looking at YAML it seems hard to tell. You can verify tabs in your YAML, have you verified by printing dict/data itself? — Nikhil Rupanawar
– Nikhil Rupanawar, Commented Sep 4, 2017 at 6:31

Nikhil Rupanawar · Accepted Answer · 2017-09-04 11:21:03Z

1

In your code,

inner_dict = [args.business_unit, args.platform, args.client]

for item in list(inner_dict):
    """
      following line appends adds key to 0'th dict which is 
     {
    "tag:BUSINESS_UNIT": "TEST_FLITE",
    "State.Name": "running"
    }
    """
    data['policies'][0]['filters'][0]['tag:BUSINESS_UNIT'] = args.business_unit
    data['policies'][0]['filters'][1]['tag:PLATFORM'] = args.platform

You could do something like this,

#!/usr/bin/python

from __future__ import print_function
import sys
import argparse
import json
import yaml

parser = argparse.ArgumentParser()
parser.add_argument("-y", "--yaml_file", type=str)
parser.add_argument("-b", "--business_unit", type=str)
parser.add_argument("-p", "--platform", type=str)
parser.add_argument("-c", "--client", type=str)
args = parser.parse_args()

with open(args.yaml_file, 'r') as stream:
    data = yaml.load(stream)


inner_list = [{"tag:BUSINESS_UNIT" : args.business_unit},
              {"tag:PLATFORM" : args.platform},
              # {"tag:CLIENT" : args.client} #if needed
             ]

for i, item in enumerate(inner_list):
    data['policies'][0]['filters'].insert(i, item)

print(json.dumps(data))

edited Sep 4, 2017 at 11:21

answered Sep 4, 2017 at 8:57

Nikhil Rupanawar

4,22111 gold badges37 silver badges51 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

cezar Over a year ago

Good idea with insert! However the OP didn't mention anything about {"tag:PLATFORM" : args.client}. But he/she should decide how to call that key, if needed.

cezar · Accepted Answer · 2017-09-04 07:12:53Z

This part of your code:

inner_dict = [args.business_unit, args.platform, args.client]

for item in list(inner_dict):
    data['policies'][0]['filters'][0]['tag:BUSINESS_UNIT'] = args.business_unit
    data['policies'][0]['filters'][1]['tag:PLATFORM'] = args.platform

doesn't really make a lot of sense. First, inner_dict is already a list. There is no reason why would we convert it to list again - list(inner_dict).

Second, in the loop you iterate 3 times, once for every item in the list, and repeat the same statements in every iteration step. You don't use the item in those statements.

From here, without knowing the structure of the YAML file, is hard to provide a solution, but you should look for the above mentioned lines. I'd try without loop, just execute:

data['policies'][0]['filters'].append({'tag:BUSINESS_UNIT': args.business_unit})
data['policies'][0]['filters'].append({'tag:PLATFORM': args.platform})

EDIT: after you added the YAML file content I can give you a further explanation. When you call data['policies'][0]['filters'] you call this part:

[
    {            
        "State.Name": "running"
    },
    {
        "tag:ResourceCreatedBy": "present"
    },
    {
        "and": [
            {
                "type": "instance-age",
                "days": 30,
                "op": "ge"
            }
        ]
    }
]

Consequently data['policies'][0]['filters'][0] will return:

{            
    "State.Name": "running"
}

and data['policies'][0]['filters'][0]['tag:BUSINESS_UNIT'] = args.business_unit will add new key:value pair to that dict:

{            
    "State.Name": "running",
    "tag:BUSINESS_UNIT": "TEST_FLITE"
}

The loop doesn't help, it just does the same in every iteration step.

My proposal will append the needed key:value pairs as dictionaries to the end of the list. If the order really matters, then you have to shift the list. Take a look at the module collections.deque.

Collectives™ on Stack Overflow

python string to json conversion

2 Answers 2

1 Comment

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related