How to make a python object json-serialized?

Question

I want to serialize a python object, after saved it into mysql(based on Django ORM) I want to get it and pass this object to a function which need this kind of object as a param.

Following two parts are my main logic code:

1 save param part :



class Param(object):
    def __init__(self, name=None, targeting=None, start_time=None, end_time=None):
        self.name = name
        self.targeting = targeting
        self.start_time = start_time
        self.end_time = end_time
    #...

param = Param()
param.name = "name1"
param.targeting= "targeting1"


task_param = {
            "task_id":task_id,              # string
            "user_name":user_name,          # string
            "param":param,                  # Param object
            "save_param":save_param_dict,   # dictionary
            "access_token":access_token,    # string
            "account_id": account_id,       # string
            "page_id": page_id,             # string
            "task_name":"sync_create_ad"    # string
        }


class SyncTaskList(models.Model):
    task_id = models.CharField(max_length=128, blank=True, null=True)
    ad_name = models.CharField(max_length=128, blank=True, null=True)
    user_name = models.CharField(max_length=128, blank=True, null=True)
    task_status = models.SmallIntegerField(blank=True, null=True)
    task_fail_reason = models.CharField(max_length=255, blank=True, null=True)
    task_name = models.CharField(max_length=128, blank=True, null=True)
    start_time = models.DateTimeField()
    end_time = models.DateTimeField(blank=True, null=True)
    task_param = models.TextField(blank=True, null=True)

    class Meta:
        managed = False
        db_table = 'sync_task_list'



SyncTaskList(
                task_id=task_id,
                ad_name=param.name,
                user_name=user_name,
                task_status=0,
                task_param = task_param,
            ).save()

2 use param part


def add_param(param, access_token):
    pass

task_list = SyncTaskList.objects.filter(task_status=0)
for task in task_list:
    task_param = json.loads(task.task_param)
    add_param(task_param["param"], task_param["access_token"]) # pass param object to function add_param

If I directly use Django ORM to save task_param into mysql, I get error,

json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)

for after ORM operation, I get string who's property name enclosed in single quotes like :


# in mysql it saved as 

 task_param: " task_param: {'task_id': 'e4b8b240cefaf58fa9fa5a591221c90a',
              'user_name': 'jimmy',
              'param': Param(name='name1',
                                    targeting='geo_locations',
                                   ),
              'save_param': {}}"

I am now confused with serializing an python object, then how to load this original object and pass it to a function?

Any commentary is very welcome. great thanks.

update my solution so far

task_param = {
            # ...
            "param":vars(param),            # turn Param object to dictionary 
            # ...
            }

SyncTaskList(
                #...
                task_param = json.dumps(task_param),
                #...
            ).save()

#task_list = SyncTaskList.objects.filter(task_status=0)
#for task in task_list:
    task_param = json.loads(task.task_param)
    add_param(Param(**task_param["param"]), task_param["access_token"])

update based on @AJS's answer

directly pickle dumps and saved it as an binary field, then pickle loadsit also works

Any better solution for this?

If i understand it correctly in MySql you are saving task_param as string,under column task_param, and you are saving it with single quotes. Please correct me if i am wrong — AJS
– AJS, Commented Mar 1, 2019 at 4:10
@KlausD. It will raise error TypeError: Object of type 'Param' is not JSON serializable — jia Jimmy
– jia Jimmy, Commented Mar 1, 2019 at 4:31
@AJS yeah, that's true if i directly save it as my above codes showed , it automaticly turn to single quotes style — jia Jimmy
– jia Jimmy, Commented Mar 1, 2019 at 4:34
The obj has to be a simple Python structure, made out of builtin types like dictionaries, list, strings, numbers... Your param key is not. — Klaus D.
– Klaus D., Commented Mar 1, 2019 at 4:36

booleys1012 · Accepted Answer · 2019-03-01 06:01:42Z

1

Try looking into msgpack https://msgpack.org/index.html

unlike pickle, which is python-specific, msgpack is supported by many languages (so the language you use to write to mysql can be different than the language used to read).

There are also some projects out there that integrate these serializer-libraries into Django model fields:

answered Mar 1, 2019 at 6:01

booleys1012

7313 silver badges10 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

AJS · Accepted Answer · 2019-03-01 06:13:44Z

0

You can use pickle basically you are serializing your python object and save it as bytes in your MySQL db using BinaryField as your model field type in Django, as i don't think JSON serialization would work in your case as you have a python object as a value as well in your dict, when you fetch your data from db simpily unpickle it syntax is similar to json library see below.

import pickle
#to pickle
data = pickle.dumps({'name':'testname'})
# to unpickle just do
pickle.loads(data)

so in your case when you unpickle your object you should get your data in same form as it was before you did pickle.

Hope this helps.

edited Mar 1, 2019 at 6:13

answered Mar 1, 2019 at 4:44

AJS

2,02317 silver badges26 bronze badges

2 Comments

jia Jimmy Over a year ago

thanks for your reply, after pickle.dump I can get a bytes data, but when saved it into mysql by Django ORM it turn to be a string, and I cannot loads it back anymore

AJS Over a year ago

use pickle.dumps it should be string, you missed s

Collectives™ on Stack Overflow

How to make a python object json-serialized?

2 Answers 2

Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related