Python Serialization – Table of Content
Serialization in Python
Serialization in python is a process to serialize data in a species that is user-friendly, human-readable, and easily inspected. There are two very common python serialization libraries that serialize data objects in python. They are ‘HDF5’ and ‘Pickle’ which take dictionaries as well as Tensorflow models for storage purposes and transmission.
Become a Python Certified professional by learning this HKR Python Training !
Why Python Serialization?
The serialization process allows the python user to send, receive and save his data alongside maintaining the original structure also. The user finds it very useful to save a certain kind of data in the database so that he can reuse it later whenever it is needed. It can also be used to transmit data on a server network and the user can access it on any system later on.
The process of serialization is also very helpful for projects related to data science. For instance, the process of dataset preprocessing can be very time-consuming, hence preprocessing is done just once that too before saving the data on the disk. It is preferred that the user performs preprocessing each time he uses it. It also eliminates memory limitation problems for big data too which is heavy for loading in the memory as a single piece. So when the data is split into smaller chunks, the user is able to load every single chunk for preprocessing, and he can then save the outputs to the disk, removing all the data chunks from the memory.
Python Serialization: Text Based
The process of textual serialization means serializing the data in some specific format that is easy to understand, human-readable as well as easily inspected. Formats which are text-based are mainly language agnostic and they can be formed with the help of any language related to programming.
JSON is a standard format that is used to exchange data between servers and web clients. JSON is known to serialize the objects in a plain text file format and allow for easy visual identification to the user. JSON stores the objects in the form of key-value pairs, just like a dictionary in Python. JSON is a built-in library in python which makes it a breeze for the user to work with JSON.
It is very easy to perform JSON serialization just like creating a JSON file and dumping the object. This is done with the help of the dump() method. This method has two arguments which are:
- The object user is serializing
- File which will store the serialized object.
Python JSON has two main functions which it works with:
- dump(): This function helps to convert a Python object into JSON format
- Loads(): This function helps to convert the JSON string back into a Python object.
The table below will show the conversion of the python data type into a JSON type:
dict-objectList, tuple- array
str- String
True- true
Int, float- Number
False- false
None- null
Check out our Python Spark sample resumes and take your career to the next level!
Python Training Certification
- Master Your Craft
- Lifetime LMS & Faculty Access
- 24/7 online expert support
- Real-world & Project Based Learning
YAML
YAML is not a Markup Language but it is actually a parent set of JSON made in a way to be more comprehensible to the user. The most important and distinguishing feature of YAML is the capacity to create references for other objects in the same file. Another most important advantage is that it is possible to write comments in python. This feature has proved very useful to work with the configuration files also.
Python Serialization: Binary Formats
It is not possible for binary formats in serialization to be human-readable; however they are faster in general and also require much lesser space than text-based counterparts. Let us see some very popular binary formats below:
Pickle
It is a very popular format for python serialization. It is used to serialize almost all the Python object types. Pickle is considered to be an original serialization format used for Python, hence when a user plans to serialize objects in python that he expects to share and he must use with many other languages used for programming, he has to be mindful of the issues such as cross-compatibility. Similarly, pickle works in the same way for various Python versions. The user cannot unpickle a file present in the XXX version, which he picked in the python ZZZ version. So by doing such unnecessary changes, the execution of malicious code gets tough.
Let us see an example below and understand how pickling is performed in python:
import pickle
class example_class:
x_number = 10
x_string = "Welcome to the tutorial"
x_list = [10, 20, 30]
x_dict = {"Heya": "x", "How": 5, "you": [10, 20, 30]}
x_tuple = (2, 3)
my_object = example_class()
my_pickled_object = pickle.dumps(my_object)
print(f"This would be pickled object:\n{my_pickled_object}\n")
my_object.a_dict = None
my_unpickled_object = pickle.loads(my_pickled_object)
print(
f"The dictionary of unpickled object is:\n{my_unpickled_object.a_dict}\n")
Output
This would be pickled object:
b'\x80\x04\x95!\x00\x00\x00\x00\x00\x00\x00\x8c\x08__main__\x94\x8c\rexample_class\x94\x93\x94)\x81\x94.'
Traceback (most recent call last):
File "", line 19, in
AttributeError: 'example_class' object has no attribute 'a_dict'
>
Enroll in our Python training in Singapore program today and elevate your skills!
Subscribe to our YouTube channel to get new updates..!
Module Interface for Pickling and Unpickling
The data format is always Python-specific for the pickle module. That is why it is always important to write the essentially required code when the user is performing the process of serialization or deserialization. dumps() is the Python function that is used to serialize an object hierarchy whereas loads() is the function that is used to de-serialize the same.
Pickle Protocols
Protocols in pickle act like the convention measures to deconstruct and construct the python objects. There are in total of 5 protocols that a user can use in pickling. Whenever a user uses a higher protocol version, he will need the latest version of Python to obtain the highly compatible as well as readable pickle.
Protocol version 0: This version is readable by humans. It is compatible to use with data and interfaces from the older python versions.
Protocol version 1: It is known to be an old binary format. Just like protocol version 0, it is also compatible with older python versions.
Protocol version 2: It came into effect during the release of python version 2.3. This version is well known for providing new styles in picking.
Protocol version 3: This version was discovered during the release of python version 3.0. It is famous for supporting byte objects however the major drawback with this version is it gets unpicked by python version 2.0
Protocol version 4: This version was discovered during the release of python version 3.4. This is able to support large objects and various different objects can be picked too. It is also famous for supporting data optimization.
If you have any doubts on Python, then get them clarified from python Industry experts on our Python Community
Numpy
It is a very popular python library used by the user to work with large and multidimensional arrays as well as matrices. It stands for numerical python. They are open source and free to use but slow to process. NumPy arrays can be stored in one continuous place in the memory; however this same is not possible for lists. Processes can therefore access as well as manipulate the arrays very efficiently.
Let us see an example below and understand how the Numpy library is used in python:
import numpy as nparr = np.array( [[ 10, 20, 30],
[ 40, 20, 50]] )
print("The type of array is: ", type(arr))
print("The no of dimensions are: ", arr.ndim)
print("The shape of the array is: ", arr.shape)
print("The size of the array is: ", arr.size)
print("Array stores elements of the type: ", arr.dtype)
Output
The type of array is: <class 'numpy.ndarray'>
The no of dimensions are: 2
The shape of the array is: (2, 3)
The size of the array is: 6
Array stores elements of the type: int64
>
Top 50 frequently asked Python interview Question and answers !
Python Training Certification
Weekday / Weekend Batches
Conclusion
Serialization is a process that aims at simplifying the data storage methods for a data scientist. Serialization in Python is one of the most important features that ease the data conversion interface of the data. In this article, we have talked about why we need serialization. The serialization process allows the python user to send, receive and save his data alongside maintaining the original structure also. The user finds it very useful to save a certain kind of data in the database so that he can reuse it later whenever it is needed.
We have also discussed JSON and YAML in python. Then we talked about binary formats of python serialization which are pickle and NumPy. In this sub-topic, we will also have a glance at module instances of pickling and unpickling along with pickle protocols. Now we will be discussing some frequently asked questions by the developers and will give solutions for them.
Related Articles
