Parsing JSON data with Python

Question

I receive some data from a GitHub webhook service. It contains info about recent commits in a repository. String I receive looks like this:

payload = {
    "ref":"refs/heads/master","commits":[
        {"added":[],"author":{"username":"myname","email":"[email protected]","name":"John Doe"},"timestamp":"2011-03-03T02:04:32-08:00","removed":[],"url":"https://github.com/repository/commit/3da6de4ae4550aa84ff16b9b52d8c5179e126dc5","message":"Setup some functions","modified":["apps/api/__init__.py","main.py","utils.py"],"id":"3da6de4ae4550aa84ff16b9b52d8c5179e126dc5"}
    ],
    "compare":"https://github.com/repository/compare/a270fe9...eb26a23","after":"eb26a2312e1955ccb2b7cb50d43682be87c30faa","repository":{"pushed_at":"2011/03/03 02:23:14 -0800","has_issues":true,"has_downloads":true,"url":"https://github.com/repository","watchers":1,"fork":false,"has_wiki":true,"private":false,"size":2248,"forks":1,"created_at":"2011/02/23 04:41:51 -0800","owner":{"email":"[email protected]","name":"john"},"name":"john","language":"Python","open_issues":0},"forced":false,"before":"a270fe954359caa323a9880afc3d6301055ea566"
}

What is the right way to remove that variable payload= and get only the data between {} so that it would be a correct JSON that I can parse with simplejson? Now I just use json_data = json_data.replace('payload=', ''). I am not sure it's a right way to do this. How can I do it a better way? Thanks.

UPDATE I posted a message on GitHub support board with the issue description.

It might also be worth bugging Github to return valid JSON - it might be a bug at their end. — Thomas K
– Thomas K, Commented Mar 3, 2011 at 13:19
How do you receive it? Is it in a HTTP response or a HTTP post? — Stephen Paulger
– Stephen Paulger, Commented Mar 3, 2011 at 14:03
GitHub sends a POST to the url I provide there in settings. It's described here: help.github.com/post-receive-hooks. So, It sends a POST with a string in my question. — Sergei Basharov
– Sergei Basharov, Commented Mar 3, 2011 at 14:12
Definitely worth bugging github as @Thomas suggests, since there's no sign of a variable in any of their samples on that page. — Nick Johnson
– Nick Johnson, Commented Mar 4, 2011 at 4:56
@Nick: Well, in fact there is. "This is sent as a POST with a single parameter: ‘payload’". In fact is's being sent as an HTTP form, so the correct way to do it is to handle it as such - I think there's something in the cgi module for it. But that's possibly overkill here. — Thomas K
– Thomas K, Commented Mar 5, 2011 at 20:28

Thomas K · Accepted Answer · 2011-03-03 13:16:57Z

6

I would look for the first { character:

jsondata = jsondata[jsondata.index("{"):]

answered Mar 3, 2011 at 13:16

Thomas K

40.7k7 gold badges88 silver badges89 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Nico · Accepted Answer · 2011-03-03 13:28:15Z

1

There's no single "right" way of doing it, unless you want to implement a complete JS parser. That said, the solution suggested by Thomas is more generalizable, eg. less likely to break if github decides to change the format of the file, if you use it on a different data source, etc.

answered Mar 3, 2011 at 13:28

Nico

2,65320 silver badges25 bronze badges

Collectives™ on Stack Overflow

Parsing JSON data with Python

2 Answers 2

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related