Python Generators | Step by Step Guide of Generators in Python


Python Generators – Table of Content

Generators

The main purpose of a generator is to help us in creating our own iterators. It is a special type of function that returns an iterable set.The iterators that we create with the generator are referred to as lazy iterators. The contents of lazy iterators will not be stored in memory.If you want to iterate through large files, data streams, CSV files, etc., generators will be a good choice.Generators are introduced in PEP 255 and they are available since python 2.2 version.

How to create generator functions

Let us create a sample generator. Create a new file in any text editor and copy the below code.

def sample():

a = ["Hello", "Welcome"]

yield a

for i in sample():

print("This is a sample generator")

In this code, the sample() is the generator function name. Yield is used to return items to the caller. Unlike return in normal function, you won’t exit the function here. Once a generator is defined, it is called similar to a normal function. But the execution gets paused when it encounters a yield keyword.

Save the file with script.py as the name. Open command prompt, navigate to the script file location path, and execute the below command.

python script.py

You should be able to see an output that says ‘This is a sample generator’ on the command prompt. Let us look at one more example that returns squared root numbers to the range of numbers defined.

def Squared_numbers(num):

for num in range(num):

yield num**2

for i in Squared_numbers(5):

print(i)

This program calls Squared_numbers generator with 5 as a range. The generator will iterate from 0 and yields the square root of 5 numbers. The output for this program will be as follows.

0

1

4

9

16

Importance of yield statements in generators

Yield controls the flow of a generator function. When we call a generator expression or a generator function, we will get an iterator in return. This is nothing but a generator. 

We have to assign the generator to a variable and then use it. When we call a generator function, it only gets executed until it encounters a yield statement. The yielded value is sent back to the caller. 

  Become a python Certified professional  by learning this HKR Python Training !

Python Training Certification

  • Master Your Craft
  • Lifetime LMS & Faculty Access
  • 24/7 online expert support
  • Real-world & Project Based Learning

Creating a generator object with generator expressions

Generator expressions are similar to list comprehensions. They help us to create a generator object with minimal code. We can create generator objects that do not hold the entire object in memory before iteration. Let us create a list and a generator object and look at the difference between the two.

#Creating a list

numbers_list = [num for num in range(5)]

#Creating a generator object

numbers_generatorObject = (num for num in range(5))

#output

numbers_list

numbers_generatorObject

In the above code, we have created a list and a generator object for numbers. The syntax will be very much similar, but the difference will be the type of parentheses that we use. When you execute the above code, this will be the output.

[0, 1, 2, 3, 4]

at 0x7f776b77dd58>

You can observe here that the numbers_list is a list, so the numbers were printed on the command line. Whereas the numbers_generatorObject has got created as a generator object. You can also see the location at which the generator object is created.

Evaluating generator performance

As I mentioned before, generators optimize memory. Let’s consider the same example that we have taken above and increase numbers up to 150. Let us see how much size the list and generator objects take to hold the same numbers. Here is a small program that we can use to get the size.

import sys

#Creating a list

numbers_list = [num for num in range(150)]

print("The size of the list is", sys.getsizeof(numbers_list))

#Creating a generator object

numbers_generatorObject = (num for num in range(150))

print("The size of the generator is", sys.getsizeof(numbers_generatorObject))

The output for the above program will be as follows.

The size of the list is 1448

The size of the generator is 88

You can see that the list took 1448 bytes, whereas the generator object is only 88 bytes. You can observe a huge difference when you work with a larger dataset.

Acquire NLP certification by enrolling in the HKR NLP Training program in Hyderabad!

HKR Trainings Logo

Subscribe to our YouTube channel to get new updates..!

Advanced generator methods

Generators provide three special methods which were introduced in PEP 342 and is available since the python 2.5 version.

send() – It is a method used to send values to the generator iterators. The value specified in the send() method is used to continue with the next yield. If we do not pass any value to the send() method, it will be equivalent to the next() call. 

throw() – It is a method used to throw exceptions from the generator. We can add a throw() method when we might need to catch an exception. The value or exception specified in the throw() method will be sent to the caller.

close() – It is a method used to stop a generator. This will be really helpful when we want to stop a program when it goes into an infinity loop. 

Realted Article, List to String in Python !

Creating data pipelines with generators

When you have a huge dataset that needs processing, we can’t really do all the processing at a single place. To avoid this, we can create a pipeline. Each method in a pipeline receives an item, applies transformations on it, and returns the transformed item. This way, we can even change the order of transformations.

For example, if we want to process data in a CSV file, we have to read all the lines of data in the file. Identify the column names,split each row into a list of values,and filter out any unwanted data.Create dictionaries for the column names and lists.Apply the transformations that you want on the rows. All the created generators will function as a pipeline.

  Top 50 frequently asked Python interview Question and answers !

Python Training Certification

Weekday / Weekend Batches

Conclusion

As you have learned, generators simplify code. Generator expressions simplify code much further. They might be a little confusing at first. But when you put enough effort and practice them, you will get to understand them completely. Then you will know how easy it is to code in python with the help of generators.

Generators are especially useful when dealing with huge datasets.We can create pipelines and make the developer’s job easier.The calculations on data will be performed on-demand. We can use generators to simulate concurrency.Enjoy coding with python!

Related Articles:

1. Python Partial Functions

2. Python Split Method

3. Running Scripts in Python

4. Python List Length



Source link

Leave a Reply

Subscribe to Our Newsletter

Get our latest articles delivered straight to your inbox. No spam, we promise.

Recent Reviews


Navicat for PostgreSQL – Table of Content

Navicat for PostgreSQL

In this blog we will begin with how to start with Navicat and then will explore into the concepts of Navicat for PostgreSQL with Schemas and Databases, Tables, Views, Materialized views, Functions/Procedures, Types, Foreign Servers, Other Objects and Maintain Objects.

      Get ahead in your career by learning PostgreSQL course through hkrtrainings PostgreSQL online training

1. Getting Started with Navicat

Navicat is a database management program with multiple connections that enables you to connect to MySQL, Oracle, PostgreSQL, SQLite, SQL Server, and/or MariaDB databases, making database administration simple. It also has Amazon RDS and Amazon Redshift management capabilities. Navicat’s features are sophisticated enough to meet the needs of experienced developers while also being simple to learn for those who are new to database servers. Navicat’s well-designed GUI allows you to quickly and effortlessly generate, organize, access, and share data in a secure and simple manner.

Linux, Mac OS X, and Microsoft Windows are the three platforms on which Navicat is available. It could connect users to a local or remote server and provides various utility tools to help with data upkeep, including Data Modeling, Data Transfer, Data/Structure Synchronization, Import/Export, Backup/Restore, Report Builder, and Schedule.

Getting Started with Navicat

1) Navicat Main Toolbar:

Connections, users, tables, backup, scheduling, and other fundamental objects and functionality are all accessible through the Navicat Main Toolbar. Simply right-click the toolbar and disable Use Big Icons or Show Caption to use small icons or hide the caption.

2) Connection:

Connections, databases, and database objects are all navigated through the Connection pane. It uses a tree structure that allows you to rapidly and simply interact with the database and its items through pop-up menus. The Connection pane would be divided into Navicat Cloud and My Connections sections after you log in to the Navicat Cloud feature. Select View -> Show Only Active Objects from the main menu to display only the opened objects. Select View -> Show Connection from the main menu to show or conceal the Connection window.

3) Tab Bar:

You can switch between the Object List and the tabbed windows using the Tab Bar. You may choose whether to always open pop-ups in a new tab or a new window. If you have numerous tabs open, you can quickly switch between them by using CTRL+TAB. Options can also be found here.

4) Toolbar for Object Lists:

Other controls for manipulating the objects are available in the Object List Toolbar.

5) Object List:

The Object List pane shows a list of objects like tables, views, and queries.

6) Object Information:

The Object Information pane shows the server and Navicat objects’ comprehensive information. From the main menu, select View -> Show Object Information to show or conceal the Object Information window.

7) Navicat Cloud Activity:

The project participants and actions are displayed in the Navicat Cloud Activity pane. In the Connection pane, you choose a project, and in the Object List pane, you choose a Navicat Cloud object. Select View -> Show Navicat Cloud Activity from the main menu to show or conceal the Navicat Cloud Activity window.

2. Schemas and Databases in PostgreSQL Navicat

You must first construct and open a connection before you can begin dealing with the server objects. Create a new database and/or schema if the server is empty.

Creating a new database

  • Right-click a connection in the Navigation pane and choose New Database.
  • In the pop-up window, type the database properties.

Editing an existing database

  • Right-click a database in the Navigation pane and choose Edit Database.
  • In the pop-up window, change the database’s properties.

Creating a new schema

  • Right-click a database in the Navigation pane and choose New Schema.
  • In the pop-up window, type the schema properties.

Editing an existing schema

  • Right-click a schema in the Navigation pane and choose Edit Schema.
  • In the pop-up window, edit the schema properties.
  •       Database Management & Administrations, navicat-for-postgresql-description-2

  • Get ahead in your career by learning PostgreSQL course through hkrtrainings PostgreSQL online training in Hyderabad !

PostgreSQL Training

  • Master Your Craft
  • Lifetime LMS & Faculty Access
  • 24/7 online expert support
  • Real-world & Project Based Learning

3. Tables in PostgreSQL Navicat

Tables are the database objects that hold all of a database’s data. A table is made up of rows and columns, with fields at their intersections. To open the table object list, click  in the main window.

You can make tables that are Normal, Foreign, or Partitioned. Choose the table type by clicking the down arrow next to  on the object toolbar.

You can open a table with graphical fields in one of two ways: right-click the table and select:
1) Open Table: BLOB fields (images) are loaded by Navicat when the table is opened.

2) Open Table (Quick): BLOB fields (images) will not be loaded until you click on the cell, resulting in faster performance while opening the graphical table. (By default, it is hidden until you right-click the table while holding down the SHIFT key.)

By right-clicking a table in the Objects tab and selecting Create Open Table Shortcut from the pop-up menu, you may create a table shortcut. This option is provided to give you a quick method to open your table and start adding data without having to open the Navicat main window.

Right-click the table you want to empty and choose Empty Table from the pop-up menu. When you want to erase all existing entries without resetting the auto-increment value, you can use this option. Use Truncate Table to reset the auto-increment value when emptying your table.

Table Designer

The basic Navicat tool for working with tables is Table Designer. You may use it to create, update, and delete table fields, indexes, foreign keys, and more.

You may find a field name in the Fields tab by selecting Edit -> Find or pressing CTRL+F. You can add fields or rearrange the order of the fields when building a new table.

Note: The designer’s tabs and options are determined on the table type and server version.

Table Viewer

Table Viewer displays data as a grid when you open a table. There are two modes to display data: Form View and Grid View.

4. Views in PostgreSQL Navicat

A view enables users to access a collection of tables as if they were one. Views can be used to restrict row access. To open the view object list, 
clickView in the main window.

By right-clicking a view in the Objects tab and selecting Create Open View Shortcut from the pop-up menu, you can create a view shortcut. This option is intended to provide a quick way to open your view without having to open the Navicat main window.

View Designer

The basic Navicat tool for working with views is View Designer. In the Definition tab, you can edit the view definition as a SQL statement (it implements the SELECT statement). You can pick File -> Import SQL to import SQL statements from a SQL file into the editor.

Buttons

1)Preview: Preview the data view.

2)Explain: The view’s Query Plan is displayed.

3)View Builder: Create a visual representation of the view. It enables you to create and change views without having any SQL experience. 

4)Beautify SQL: Beautify SQL options in Editor can be used to format the codes.

Hint: By selecting View -> Result -> Show Below Editor or Show in New Page, you can display the preview results below the editor or in a new tab.

View Viewer

View Viewer displays data as a grid when you open a view. There are two modes to display data: Grid View and Form View are two different types of views.

5. Materialized Views in PostgreSQL Navicat

Materialized Views are schema objects for summarising, computing, replicating, and distributing data. To open the materialised view object list, click 
Materialized View in the main window.

By right-clicking a materialised view in the Objects tab and selecting Create Open Materialized View Shortcut from the pop-up menu, you can create a materialised view shortcut. This option is designed to give you a quick way to open your materialised view without having to open the Navicat main window.

Right-click a materialised view in the Objects tab and choose Refresh Materialized View With -> Data or No Data from the pop-up menu to totally replace its contents.

5.1 Materialized View Designer

Navicat’s Materialized View Designer is the entry-level tool for working with materialized views. In the Definition tab, you can edit the view definition as a SQL statement (it implements the SELECT statement). You can use File -> Import SQL to import SQL statements from a SQL file into the editor.

Buttons:

1)Preview: Preview materialized view data.

2)Explain: Displays the materialized view’s Query Plan.

3)View Builder: Create a visual representation of the materialized view. It enables you to create and change materialized views without having any SQL knowledge. 

4)Beautify SQL: Beautify SQL settings in Editor can be used to format the codes.

Hint: By selecting View -> Result -> Show Below Editor or Show in New Page, you can display the preview results below the editor or in a new tab.

5.2 Materialized View Viewer
Materialized View Viewer presents data as a grid when you access a materialized view. There are two modes to display data: Form View and Grid View. 

                                            Click here to get latest PostgreSQL interview questions and answers for 2021!

Database Management & Administrations, navicat-for-postgresql-description-2, Database Management & Administrations, navicat-for-postgresql-description-3

Subscribe to our YouTube channel to get new updates..!

6. Functions/Procedures in PostgreSQL Navicat

Functions and Procedures are schema objects that are kept on the server and consist of a series of SQL statements. PostgreSQL 11 supports procedures. To access the function object list, go to the main window and click&nbs;Function.

6.1 Function Wizard
On the object toolbar, select  New Function. The Function Wizard appears, allowing you to quickly define a function.

  • Choose the routine type: Function or Procedure.
  • The parameters should be defined. Under the relevant columns, set the Mode, Type Schema, Type, Name, and Default Value.
  • Select the Schema and Return Type from the list when creating a function.

Hint: After unchecking the option to Show wizard next time, go to Options to enable it.

6.2 Function Designer
The basic Navicat tool for working with functions/procedures is Function Designer. In the Definition tab, you could enter a valid SQL statement. It could be a basic statement like INSERT or SELECT, or it could be a compound statement including BEGIN and END. 

6.3 Results
Click Execute on the toolbar to run the procedure/function. The query will be executed if the SQL statement is accurate, and the Result tab will open with the data returned if the statement is expected to produce data. If an error occurs during the execution of the procedure/function, the program is terminated and the relevant error message is displayed. The Input Parameter dialogue will popup if the procedure/function requires input parameters. To transfer the supplied values to the procedure/function without quotation marks, select Raw Mode.

Note: Navicat is capable of returning up to 20 result sets.

6.4 Debug (Only the Non-Essentials Edition is available)
Install the pldbgapi extension before debugging PL/pgSQL procedures/functions. Install pldbgapi Extension can be done by right-clicking anywhere in the function object list.

Note: Only PostgreSQL 9.1 or later has this option. If you’re using PostgreSQL 8.3 to 9.0 on your server, you’ll need to manually enable the debugger plugin.

Then, open a PL/pgSQL function/procedure. By clicking in the grey area beside each statement, you can add/remove breakpoints for debugging.

To use the PostgreSQL Debugger, go to the toolbar and select 

7. Types in PostgreSQL Navicat

Types are used to create new data types that can be used in the current database. To open the type object list, click Others -> Type in the main window.

Base, Composite, Enum, and Range types can all be created. Choose the type by clicking the down arrow next to  New Type on the object toolbar.

7.1 Type Designer
Type Designer is the basic Navicat tool for working with types. It allows you to create or edit a type.

Note: The designer’s tabs and options are determined by the server version and type you select.

8. Foreign Servers in PostgreSQL Navicat

The connection information that a foreign-data wrapper needs to access an external data resource is often encapsulated in a foreign server. To open the foreign server object list, click Others -> Foreign Server in the main window. 

Right-click anywhere in the foreign server object list and select Install postgres_fdw Extension to install the postgres_fdw extension for accessing data stored in external PostgreSQL servers.

8.1 Foreign Server Designer
The basic Navicat tool for working with international servers is Foreign Server Designer. You can use it to make or edit a foreign server.

PostgreSQL Training

Weekday / Weekend Batches

9. Other Objects in PostgreSQL Navicat

Navicat also permits you to handle other PostgreSQL objects: Domain, Conversion, Aggregate, Operator Class, Operator, Index, Tablespace, Trigger, Sequence, Language and Cast. To open the object list, click Others in the main window and choose an object.

10. Maintain Objects in PostgreSQL Navicat

Navicat is a complete solution for PostgreSQL object maintenance.

  • Select objects in the Navigation pane or the Objects tab in the main window.
  • Perform Right-click over the selected objects.
  • Select Maintain from the pop-up menu, and then select a maintain option.
  • The results are displayed in a pop-up window.

10.1 Database Options
The following are the Database options available.

  • Allow: The database can be connected by users.
  • Disallow: The database cannot be connected by users.
  • Analyze Database: Collect the database statistics.
  • Vacuum Database: Garbage-collect and optionally analyze the database.
  • Reindex Database: All indexes in the database should be recreated.

10.2 Materialized  / Table View Options

The following are the Materialized/Table view options available.

  • Analyze Tables / Analyze Materialized Views: Gather statistics on the table’s contents.
  • Vacuum Tables / Vacuum Materialized Views: Garbage-collect and optionally analyze the table.
  • Reindex Tables / Reindex Materialized Views: Recreate all indexes of the table.

11. Conclusion:

We have now successfully completed reviewing the concepts of Navicat for PostgreSQL. We hope this blog is very useful to the readers and have comprehended all its features.



Source link