How to run shell commands inside python

Question

I need to create AWS lambda function to execute the python program. I need to incorporate the following shell command in it.

curl https://ip-ranges.amazonaws.com/ip-ranges.json | jq -r '.prefixes[] | select(.region=="ap-southeast-1") | .ip_prefix'

Could someone guide me on this.

Does this help ? stackoverflow.com/questions/89228/… Same here: stackoverflow.com/questions/14457756/… — kakou
– kakou, Commented Jun 14, 2021 at 8:29
This smells like an X/Y problem – you really shouldn't shell out to curl and jq to download and parse JSON. — AKX
– AKX, Commented Jun 14, 2021 at 8:30

AKX · Accepted Answer · 2021-06-14 08:34:38Z

1

To simply shell out to curl and jq to get that data,

import subprocess

data = subprocess.check_output("""curl https://ip-ranges.amazonaws.com/ip-ranges.json | jq -r '.prefixes[] | select(.region=="ap-southeast-1") | .ip_prefix'""", shell=True)

but you really probably shouldn't do that since e.g. there's no guarantee you have curl and jq in the Lambda execution environment (not to mention the overhead).

Instead, if you have the requests library,

import requests

resp = requests.get("https://ip-ranges.amazonaws.com/ip-ranges.json")
resp.raise_for_status()
prefixes = {
    r["ip_prefix"]
    for r in resp.json()["prefixes"]
    if r["region"] == "ap-southeast-1"
}

answered Jun 14, 2021 at 8:34

AKX

171k16 gold badges146 silver badges228 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Vidarbha Yalagadde Over a year ago

Many thanks. Is their a way for me to store all the IPs in a single file

AKX Over a year ago

In general, yes, of course you can write them into a file. However, you can't write files in lambda functions.

Vidarbha Yalagadde Over a year ago

Understood. I need to fetch these output IPs and store them in a file, later dump the file to S3 for further processing.

AKX Over a year ago

So you need to store them in S3? Fine, that you can do; the boto3 library is the go-to S3 client for Python.

Collectives™ on Stack Overflow

How to run shell commands inside python

1 Answer 1

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related