Convert curl command from google-colab to python script

Question

I have the following google colab code:

code :

!wget https://bin.equinox.io/c/4VmDzA7iaHb/ngrok-stable-linux-amd64.zip

!unzip ngrok-stable-linux-amd64.zip

LOG_DIR = './log'
get_ipython().system_raw(
    'tensorboard --logdir {} --host 0.0.0.0 --port 6006 &'
    .format(LOG_DIR)
)

get_ipython().system_raw('./ngrok http 6006 &')    
! curl -s http://localhost:4040/api/tunnels | python3 -c \
    "import sys, json; print(json.load(sys.stdin)['tunnels'][0]['public_url'])"

output: https://6a112ff8.ngrok.io

My question is How do I convert the curl pipe python command (last 3 lines) into a python script ? Currently it is being executed in google colab.

I have tried to get close to solution using this code :

import sys, json
import requests
from IPython import get_ipython


LOG_DIR = './log'

get_ipython().system_raw(
    'tensorboard --logdir {} --host 0.0.0.0 --port 6006 &'
    .format(LOG_DIR)
)

response = requests.get('http://localhost:4040/api/tunnels')
# result=json.load(response)
print(json.load(response)['tunnels'][0]['public_url'])

However I get an error:

AttributeError: 'NoneType' object has no attribute 'system_raw'

I tried your solution: I get ''' import requests response = requests.get('http://%7C') ''' I don't know how this would work when there is a curl command piped with python command — Ajinkya
– Ajinkya, Commented Dec 14, 2019 at 14:53
what value has import sys, json; print(json.load(sys.stdin)['tunnels'][0]['public_url']) ?? — GiovaniSalazar
– GiovaniSalazar, Commented Dec 14, 2019 at 15:00
output: 6a112ff8.ngrok.io "have No dashboards are active for the current data set." — GiovaniSalazar
– GiovaniSalazar, Commented Dec 14, 2019 at 15:02

v25 · Accepted Answer · 2019-12-14 23:11:48Z

1

! curl -s http://localhost:4040/api/tunnels | python3 -c \
    "import sys, json; print(json.load(sys.stdin)['tunnels'][0]['public_url'])"

To do this without curl in pure python, you could use the requests library, and the json method of the response object (r in this example) which returns a dictionary:

import requests
try:
    r = requests.get('http://localhost:4040/api/tunnels')
    d = r.json() 
    public_url = d['tunnels'][0]['public_url']
except Exception as e:
    print ('Failed: ', e)

# Do something with `public_url`

answered Dec 14, 2019 at 23:11

v25

7,6912 gold badges24 silver badges38 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Ajinkya Over a year ago

How do I convert get_ipython().system_raw in python ?

v25 Over a year ago

Those commands are just running another program (tensorboard/ngrok) in the background. Why not start these at a separate terminal, rather than trying to bundle them in a single script. Also can you clarify what service is actually running on localhost:4040? I'm guessing this is some kind of ngrok service port, which gives you the public URL of the service (which by the way will change each time your run this script cause ngrok generates a new public URL on launch). Stick this in a separate process somewhere and you'd have a more persistant URL. Are you trying to run this on a local box?

Ajinkya · Accepted Answer · 2019-12-16 09:29:47Z

Thanks to @v25 comment I am able to run it using python. Here's what I understood.

Warning: On Macos there is some kind of permission issue with ngrok, hence I coudnt get the fist demo.py script running on macos. I got it running on Ubuntu 16

I downloaded these files and placed them in a folder where demo.py and demo2.py are present

wget https://bin.equinox.io/c/4VmDzA7iaHb/ngrok-stable-linux-amd64.zip

unzip ngrok-stable-linux-amd64.zip

I made 2 scripts:

demo.py (nominal case)

import os
os.system('./ngrok http 8000 &')

for tensorboard replace os.system with:

os.system('tensorboard --logdir {} --host 0.0.0.0 --port 8001 &'.format(LOG_DIR))

demo.py will run in one terminal window.

demo2.py

import os
import requests

try:
    r = requests.get('http://localhost:4040/api/tunnels')
    d = r.json() 
    public_url = d['tunnels'][0]['public_url']
    print(public_url)
except Exception as e:
    print ('Failed: ', e)

demo2.py will run in another terminal window.

demo2.py will produce a url which I can use. Please refer to v25's comment to see in details why 2 seperate terminal windows are needed. I am adding this solution for my future reference.

Collectives™ on Stack Overflow

Convert curl command from google-colab to python script

2 Answers 2

2 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related