python base (20): serialization, json module, pickle module

1. Serialization

What is serialization - The process of converting an original dictionary, list, etc. into a string is called serialization.

1.1 Why serialization

Why convert other data types to strings?
Because only bytes can be transmitted over the network,
Only bytes and STRs can be stored in the file.

For example, one of the data we calculate in python code needs to be used by another program, so how do we give it?
The way we can think of now is to have a file in it, and then another python program reads it out.
But we all know that there is no concept of a dictionary for a file, so we can only convert the data into a dictionary and put it in a file.
You must ask, it's easy to convert a dictionary into a string, that's str(dic). Why do we have to learn the serialization module?
Yes, the serialization process is from DIC to STR (dic).Now you can convert a dictionary named DIC into a string by str(dic),
But how do you convert a string into a dictionary?
Smart you must have thought of eval(), if we pass a string-type dictionary str_dic to eval, we'll get a return dictionary type.
The eval() function is very powerful, but what does Eval do?Eofficial demo interprets the string str as a valid expression to evaluate and return the result of the calculation.
But.Powerful functions come at a cost.Security is its biggest disadvantage.
Imagine if we read a destructive sentence like "Delete File" instead of a data structure from a file, the consequences would be unthinkable.
The use of eval carries this risk.
Therefore, eval is not recommended for deserialization (converting str to a data structure in python)

1.2 Purpose of serialization

1. Persist the custom object in some form of storage;
2. Transfer objects from one place to another.
3. Make the program more maintainable.



2. json module

The Json module provides four functions: dumps, dump, loads, load.

Import the json module:

import json

2.1 loads and dumps

import json
dic = {'k1':'v1','k2':'v2','k3':'v3'}
str_dic = json.dumps(dic)  #Serialization: Converts a dictionary to a string
print(type(str_dic),str_dic)  #<class 'str'> {"k3": "v3", "k1": "v1", "k2": "v2"}
#Be careful, json A Dictionary of converted String Types consists of""Represented

dic2 = json.loads(str_dic)  #Deserialization: Converts a string-formatted dictionary into a dictionary
#Note that the json Of loads Strings in a dictionary of type string handled by a function must be replaced by""Express
print(type(dic2),dic2)  #<class 'dict'> {'k1': 'v1', 'k2': 'v2', 'k3': 'v3'}

list_dic = [1,['a','b','c'],3,{'k1':'v1','k2':'v2'}]
str_dic = json.dumps(list_dic) #Nested data types can also be handled 
print(type(str_dic),str_dic) #<class 'str'> [1, ["a", "b", "c"], 3, {"k1": "v1", "k2": "v2"}]
list_dic2 = json.loads(str_dic)
print(type(list_dic2),list_dic2) #<class 'list'> [1, ['a', 'b', 'c'], 3, {'k1': 'v1', 'k2': 'v2'}]

2.2 load and dump

import json
f = open('json_file','w')
dic = {'k1':'v1','k2':'v2','k3':'v3'}
json.dump(dic,f)  #dump Method receives a file handle and converts the dictionary directly to json String Write to File

f = open('json_file')
dic2 = json.load(f)  #load Method receives a file handle that directly transfers the json String to Data Structure Return

2.3 ensure_ascii keyword parameter

import json
f = open('file','w')
ret = json.dumps({'nationality':'China'})
ret = json.dumps({'nationality':'U.S.A'},ensure_ascii=False)

2.4 Explanation of other parameters

Serialize obj to a JSON formatted str.(String Represented json object) 
Skipkeys: Default value is False,If dict Of keys The data in is not python Basic types(str,unicode,int,long,float,bool,None),Set to False When you do, you will report TypeError Error.Set to True,Will skip this class key 
ensure_ascii:,When it is True When, All Not ASCII Code characters are displayed as\uXXXX Sequence, just in dump Will ensure_ascii Set to False Save in now json The Chinese language can be displayed normally.) 
If check_circular is false, then the circular reference check for container types will be skipped and a circular reference will result in an OverflowError (or worse). 
If allow_nan is false, then it will be a ValueError to serialize out of range float values (nan, inf, -inf) in strict compliance of the JSON specification, instead of using the JavaScript equivalents (NaN, Infinity, -Infinity). 
indent: Should be a non-negative integer, if 0 is the top row display, if empty is the most compact row display, otherwise it will wrap and follow indent Numeric values are displayed in front of blank lines, which are printed json Data is also called pretty-printed json 
separators: The separator, in fact(item_separator, dict_separator)A tuple whose default is(',',':');This means dictionary within keys Use ","Separate, and KEY and value Separate them with':'. 
default(obj) is a function that should return a serializable version of obj or raise TypeError. The default simply raises TypeError. 
sort_keys: Base data on keys The values are sorted. 
To use a custom JSONEncoder subclass (e.g. one that overrides the .default() method to serialize additional types), specify it with the cls kwarg; otherwise JSONEncoder is used.

Formatted output of 2.5 json

import json
data = {'username':['Li Hua','Fructus Erbiae'],'sex':'male','age':16}
json_dic2 = json.dumps(data,sort_keys=True,indent=2,separators=(',',':'),ensure_ascii=False)

3. Pile module

Two modules for serialization:

json, used to convert between strings and python data types.

pickle, used to convert between Python-specific types and python's data types.

The pickle module provides four functions: dumps, dump (serialization, memory), loads (deserialization, reading), load (not only can serialize dictionaries, lists... can serialize any data type in python).

import pickle
dic = {'k1':'v1','k2':'v2','k3':'v3'}
str_dic = pickle.dumps(dic)
print(str_dic)  #A string of binary content

dic2 = pickle.loads(str_dic)
print(dic2)    #Dictionaries

import time
struct_time  = time.localtime(1000000000)
f = open('pickle_file','wb')

f = open('pickle_file','rb')
struct_time2 = pickle.load(f)

Now, wise you have to say, why do you want to learn json since pickle is so powerful?
Here we want to show that json is a data structure that all languages can recognize.
If we serialize a dictionary or a json into a file, then the java or js code can also be used.
But if we use pickle s for serialization, other languages will not be able to understand what this is.
So, if your serialized content is a list or dictionary, we highly recommend that you use the json module
But if for some reason you have to serialize other data types and you'll use python to deserialize this data in the future, you can use pickle.

Tags: Python JSON network ascii

Posted on Fri, 08 Nov 2019 21:41:45 -0500 by helppy