0

We have a syslog-ng server that get Mongodb logs that contain JSON in the log, The sample log is below:

2016-10-17 19:54:12 f:local1.p:info h:10.133.126.79 prog:sharmongo-log m:sharmongo-log 2016-10-17T19:54:04.943+0330 I COMMAND  [conn5573] command CLM.BillingAccount command: count { count: "BillingAccount", query: { $or: [ { billingAccount.customerCode: "C8088810719" } ] } } planSummary: COUNT_SCAN { billingAccount.customerCode: 1.0 } keyUpdates:0 writeConflicts:0 numYields:0 reslen:122 locks:{ Global: { acquireCount: { r: 2 } }, Database: { acquireCount: { r: 1 } }, Collection: { acquireCount: { r: 1 } } } protocol:op_query 216ms

We need to insert the incomming logs to the database, first of all I need to parse the log to CSV, I want to write a python script to change the incomming logs to CSV. as you can see a part of the log is JSON (as described in MongoDB documentation ant this girthub link it is in JSON format). the JSON portion of logs are like this:

command: count {
    count: "BillingAccount",
    query: {
        $or: [{
            billingAccount.customerCode: "C8088810719"
        }]
    }
}
planSummary: COUNT_SCAN {
    billingAccount.customerCode: 1.0
}
keyUpdates: 0 writeConflicts: 0 numYields: 0 reslen: 122 locks: {
    Global: {
        acquireCount: {
            r: 2
        }
    },
    Database: {
        acquireCount: {
            r: 1
        }
    },
    Collection: {
        acquireCount: {
            r: 1
        }
    }
}
protocol: op_query 216 ms

but I see a lot of errors in it that when I want to parse it with a python json library I get an error. I need to change it to a form that I doesn't give me an error anymore, first I quoted every key as they were not. but again I got the following errors:

enter image description here enter image description here

besides these errors' one major problem is that I got more than one JSON in log that it caused me more errors , I need to know If there is a python library to help me to parse this JSON. What do you suggest to solve this problem?

1
  • 2
    You need to wrap everything in { } so that it becomes a json object. Commented Oct 22, 2016 at 8:29

1 Answer 1

1
"command": "count" {

Is not JSON and that could probably be the reason.

"command": "count",
           { ......

This might make sense in terms of JSON.

Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.