How to insert txt data into mongodb using python

Question

I need to insert data from file.txt into new MongoDB database.collection using python.

.txt file contains data like:

a={
    "_id"          : "67a2c0e2e559fb56bf055502",
    "field1"       : "value1",
    "field2" : ["value21","value22"],
    "field3"  : {
                       "_id"  : "37a2c0e2e559fb56bf055502",
                       "subfield1" : "subvalue1"
                     }
};

b={
    "_id"          : "67a2c0e2e559fb56bf055503",
    "field1"       : "value1",
    "field2" : ["value21","value22"],
    "field3"  : {
                       "_id"  : "27a2c0e2e559fb56bf055503",
                       "subfield1" : "subvalue1"
                     }
};

c={....
};

I want to insert all documents if we say a=doc1, b=doc2, C=doc3, d... I tried to split them in a list using l=split(read,';') and add it to Mongo with insert_many but I get this error

TypeError: document must be an instance of dict, bson.son.SON, bson.raw_bson.RawBSONDocument, or a type that inherits from collections.MutableMapping

is there any way to insert the data without creating json file? Thanks

Code

def insert():
    client = pymongo.MongoClient(dbStringConnection)

    db = client[dbName]

    collectionR = db[dbCollection]

    list1 = []
    with open (file, 'r') as f:
        reader= f.read()
        #print (reader)
    f.close()
    reader= reader.replace('\n', '')
    reader= reader.replace('  ','')
    list1 = reader.split(';')
   # print(list1[0])

    list2={}

    for i in range(len(lista)-1):

        s=''
        t=list1[i][0:1]
        s=s+str(list1[i][2:len(list1[i])])
        list2[t]=s
    collectionR.insert_many(list2)

Can you show the exact code that you are running?

C.Nivs
– C.Nivs

2019-05-28 19:38:19 +00:00
Commented May 28, 2019 at 19:38 — C.Nivs
– C.Nivs, Commented May 28, 2019 at 19:38

bagerard · Accepted Answer · 2019-05-28 19:49:57Z

collection.insert_many() expects a list of dict, by simply loading the content of the txt file in a string and splitting on ";", you obtain a list of strings like

'a={ "_id" : "67a2c0e2e559fb56bf055502", "field1" : "value1", "field2" : ["value21","value22"], "field3" : { "_id" : "37a2c0e2e559fb56bf055502", "subfield1" : "subvalue1" } }'

And pymongo/mongodb does not let you insert strings, it expects documents (python dict's)

See below (using insert_one but principle is the same with insert_many):

s = 'a={ "_id" : "67a2c0e2e559fb56bf055502", "field1" : "value1", "field2" : ["value21","value22"], "field3" : { "_id" : "37a2c0e2e559fb56bf055502", "subfield1" : "subvalue1" } }'
c.insert_one(s)   # raise TypeError: document must be an instance of dict, bson.son.SON, bson.raw_bson.RawBSONDocument,...

What you need to achieve is to load the string into a dict:

dic = { "_id" : "67a2c0e2e559fb56bf055502", "field1" : "value1", "field2" : ["value21","value22"], "field3" : { "_id" : "37a2c0e2e559fb56bf055502", "subfield1" : "subvalue1" } }
c.insert_one(dic)

If you manage to turn your strings like 'a={"key1":"value1", "key2":"value2"}' into strings like '{"key1":"value1", "key2:value2"}', then you can use eval to turn the string to a dict with my_dict=eval('{"key1":"value1", "key2":"value2"}')

ailb90 · Accepted Answer · 2019-05-28 20:18:14Z

0

Thanks a lot, I was able to insert the data in MongoDB

def insert():
    client = pymongo.MongoClient(dbStringConnection)

    db = client[dbName]

    collectionR = db[dbCollection]

    list1 = []
    with open (file, 'r') as f:
        reader= f.read()

    f.close()
    reader= reader.replace('\n', '')
    reader= reader.replace('  ','')
    list1 = reader.split(';')
    for i in range(len(list1)-1):
        my_dict={}
        my_dict=eval((list1[i][2:len(list1[i])]))
        collectionR.insert_one(my_dict)

answered May 28, 2019 at 20:18

ailb90

156 bronze badges

Collectives™ on Stack Overflow

How to insert txt data into mongodb using python

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related