Detailed Analysis of pickle json for Serialization and Deserialization of Python Objects

  • 2021-12-04 19:09:13
  • OfStack

Catalogue introduction picklejson ending

Introduction

The process of converting the state information of an object into a form that can be stored or transmitted is called serialization

Similarly, converting serialized data into corresponding objects is called deserialization

This article introduces Python Two modules for serializing and de-ordering objects

picklejson pickle

pickle


#  Serialization 
In [19]: num = 66

In [20]: s = 'python'

In [21]: pi = 3.14

In [22]: li = [1, 2, 3]

In [27]: b_num = pickle.dumps(num)

In [28]: b_s = pickle.dumps(s)

In [29]: b_pi = pickle.dumps(pi)

In [30]: b_li = pickle.dumps(li)

In [31]: b_num
Out[31]: b'\x80\x03KB.'

In [32]: b_s
Out[32]: b'\x80\x03X\x06\x00\x00\x00pythonq\x00.'

In [33]: b_pi
Out[33]: b'\x80\x03G@\t\x1e\xb8Q\xeb\x85\x1f.'

In [34]: b_li
Out[34]: b'\x80\x03]q\x00(K\x01K\x02K\x03e.'
    
In [35]: type(b_li)
Out[35]: bytes
    
#  Deserialization     
In [47]: pickle.loads(b_num)
Out[47]: 66

In [48]: pickle.loads(b_s)
Out[48]: 'python'

In [49]: pickle.loads(b_pi)
Out[49]: 3.14

In [50]: li = pickle.loads(b_li)

In [51]: li
Out[51]: [1, 2, 3]

In [52]: type(li)
Out[52]: list    

Custom objects can also be serialized


class User:
    
    def __init__(self, name, sex):
        self.name = name
        self.sex = sex
        
In [38]: user = User('hui', ' Male ')

In [39]: b_user = pickle.dumps(user)

In [40]: b_user
Out[40]: b'\x80\x03c__main__\nUser\nq\x00)\x81q\x01}q\x02(X\x04\x00\x00\x00nameq\x03X\x03\x00\x00\x00huiq\x04X\x03\x00\x00\x00sexq\x05X\x03\x00\x00\x00\xe7\x94\xb7q\x06ub.'

In [41]: type(b_user)
Out[41]: bytes

In [42]: user = pickle.loads(b_user)

In [43]: type(user)
Out[43]: __main__.User

In [44]: user.name
Out[44]: 'hui'

In [45]: user.sex
Out[45]: ' Male '        

Note: pickle Serialized data is of byte (bytes) type

pickle You can also serialize an object and save it to a file, and then de-order it back to the object from the file.


import pickle

class User:
    
    def __init__(self, name, sex):
        self.name = name
        self.sex = sex
        
user = User('ithui', ' Male ')
f = open('user.txt', mode='wb')
pickle.dump(user, f)
f.close()

To de-order from a file back to an object


In [3]: f = open('user.txt', 'rb')
   ...: user = pickle.load(f)
   ...: f.close()
   ...: 
In [4]: user
Out[4]: <__main__.User at 0x16c58ebef08>

In [5]: user.name
Out[5]: 'ithui'

In [6]: user.sex
Out[6]: ' Male '

pickle Module can serialize objects, but it is only applicable to Python Language, so it is inconvenient to exchange data. For example, if you send data to the front end, js You can't turn the data into what you want.

json

If we want to pass objects between different programming languages, we must serialize the objects into standard formats, such as json , because json It is represented as a string, which can be read by all languages, and can be conveniently stored on disk or transmitted through network for data exchange.

json The object represented by the string is js Object of, json And Python The built-in data types correspond as follows:

JSON类型 Python类型
{} dict
[] list
"string" 'str' 或 u'unicode'
3.14 int 或 float
true / false True / False
null None

In [7]: import json

In [8]: info_dict = {
   ...:     'name': 'hui',
   ...:     'age': 22,
   ...:     'is_admin': True,
   ...:     'hobbies': [' Play chess ', ' Write code '],
   ...:     'other': None
   ...: }

In [9]: info_json = json.dumps(info_dict)

In [10]: info_json
Out[10]: '{
    		"name": "hui", 
    		"age": 22, 
             "is_admin": true, 
             "hobbies": ["\\u4e0b\\u8c61\\u68cb", "\\u5199\\u4ee3\\u7801"], 
             "other": null
        }'
   
#  Corresponding deserialization 
In [16]: info_d = json.loads(info_json)

In [17]: info_d
Out[17]:
{'name': 'hui',
 'age': 22,
 'is_admin': True,
 'hobbies': [' Play chess ', ' Write code '],
 'other': None}

In [18]: type(info_d)
Out[18]: dict        

See if custom class objects can json Serialization


In [21]: import json

In [22]: class User:
    ...:
    ...:     def __init__(self, name, sex):
    ...:         self.name = name
    ...:         self.sex = sex
    ...:

In [23]: user = User('ithui', ' Male ')

In [24]: json.dumps(user)
    
TypeError: Object of type User is not JSON serializable    

Make an error, say User Object cannot json Serialization. Is there a way to make custom objects turn into json , there must be.

The general idea is to first turn User objects into objects that can be json Serialized objects, such as dict Wait, and then give the serializable object to json Module.


In [28]: def user2dict(obj):
    ...:     return {'name': obj.name, 'sex': obj.sex}
    ...:
    ...:

In [29]: user = User('ithui', ' Male ')

In [30]: user_dict = user2dict(user)

In [31]: user_dict
Out[31]: {'name': 'ithui', 'sex': ' Male '}

In [32]: user_json = json.dumps(user_dict)

In [33]: user_json
Out[33]: '{"name": "ithui", "sex": "\\u7537"}'

You can also specify 1 converter at serialization time, optional parameter default Is to change any one object into an object that can be sequenced as JSON. We only need to User Specially write a conversion function, and then pass the function in:


In [28]: def user2dict(obj):
    ...:     return {'name': obj.name, 'sex': obj.sex}
    ...:
    ...:
        
In [34]: user_json = json.dumps(user, default=user2dict)

In [35]: user_json
Out[35]: '{"name": "ithui", "sex": "\\u7537"}'

In this way, although you can convert custom class objects into json However, to customize different converters for different classes is repetitive and troublesome, so I think of using each class __dict__ Property, which is a dict Object for storing instance variables. There are a few exceptions, such as defining the __slots__ Adj. class


In [36]: user.__dict__
Out[36]: {'name': 'ithui', 'sex': ' Male '}
    
In [41]: json.dumps(user.__dict__)
Out[41]: '{"name": "ithui", "sex": "\\u7537"}'

Note: If an attribute in an object is nested with another one, it cannot be directly json Serialized object, using the __dict__ Attributes still cannot be serialized normally.

Final language

Write the world with Code to make life more interesting. ️

Qian Shan is always in love. Can you praise it before you go? ️

The code word is not easy, and I hope you heroes will give me more support. ️


Related articles: