1. What is a generator?

• Ordinary function: return a value to the caller. After returning the value to the caller, the function will die, that is, it will be destroyed.

• Generator function: yield ("generates") a value to the caller. After yield ("generates") a value, the function is still alive. The caller will then generate the second value, the third value and the fourth value when necessary...

It's clearly a generator. Why mention functions? This is because most generators are implemented as functions.

Programming comes from life: the magic steamed stuffed bun shop

Uncle Wang opened a steamed stuffed bun shop downstairs. Don't underestimate this steamed stuffed bun shop. This steamed stuffed bun shop has two magical steamers. As long as you put the steamer on the steaming rack, you can produce steamed stuffed buns by yourself.

Little A and little B go to eat steamed stuffed buns at the same time. Little B ordered 50 steamed buns, Uncle Wang used A magical steamer to steam 50 for little B at once, and the 50 steamed buns were loaded in 50 small bowls. After loading, Uncle Wang removed the steam rack, so little B began to sit down and eat steamed buns.

Xiao A also bought 50 steamed stuffed buns, but he told Uncle Wang that you put my steamed stuffed buns in the steamer and I only eat one at A time. So Uncle Wang gave Xiao A a small bowl. Every time Xiao A ate A steamed stuffed bun, he went to the steamer to get A steamed stuffed bun. When the steamer was opened by Xiao A, A steamed stuffed bun was produced for him.

Here:

• Small A: generator function caller
• Small B: caller of ordinary function
• Xiao A's steamer: generator function (after Xiao A takes A steamed stuffed bun, he continues to put it on the steaming rack to prepare the remaining 49 steamed stuffed buns. The function remains in the state. You can record how many have been taken and how many are left)
• Xiao B's steamer: ordinary function (after taking 50 steamed stuffed buns for Xiao B, Uncle Wang directly collects them, and the function is destroyed. If you want to eat steamed stuffed buns, Uncle Wang needs to put the steamer on the steamer rack again)
• Small A eats in A small bowl: occupied memory size (1KB)
• Small B eats in 50 small bowls: occupied memory size (50KB)
• Uncle Wang: CPU

def simple_generator():
    x=1
    yield

genrator = simple_generator()  # If the yield keyword is used in the function, a generator object will be returned
print(type(genrator))  # <class 'generator'>

As always, look at what's in the generator object

print(dir(genrator))
[..., '__iter__', '__next__'...]  # See our old friend again... Two brothers in the iterator

What conclusions can we draw here?

The generator is also an iterator, which has the function of iterator. The generator is a special iterator

Friends who are not familiar with iterators can see the contents of iterators

2. Create generator

2.1 pass the yield keyword

def simple_generator():
    x=1
    yield x  # The first time you call next(), the execution stops here and returns x

genrator = simple_generator()

print(genrator)   
# <generator object simple_generator at 0x7f9e02077660>

print(type(genrator))  
# <class 'generator'>

If a function definition contains the yield keyword, the function is no longer an ordinary function, but a generator

2.2 generator expression

generator = (i for i in range(10))
print(generator)

A generator can be created by changing [] of the list derivation to ()

What's the difference between a generator and a list? Let's take a very intuitive example

_list = [i for i in range(10)]

print("Take out the first steamed stuffed bun:",_list[0])  # Take out the first steamed stuffed bun: 0
print("Take out the second steamed stuffed bun:",_list[1])  # Take out the second steamed stuffed bun: 1
print("Take out the third steamed stuffed bun:",_list[2])  # Take out the third steamed stuffed bun: 2

for i in _list:
    print("Take out the steamed stuffed bun serial number:",i)

Take out steamed stuffed bun serial number: 0
 Take out the steamed stuffed bun serial number: 1
 Take out the steamed stuffed bun serial number: 2
 Take out the steamed stuffed bun serial number: 3
 Take out the steamed stuffed bun serial number: 4
 Take out the steamed stuffed bun serial number: 5
 Take out the steamed stuffed bun serial number: 6
 Take out the steamed stuffed bun serial number: 7
 Take out the steamed stuffed bun serial number: 8
 Take out the steamed stuffed bun serial number: 9

generator = (i for i in range(10))

print("Take out the first steamed stuffed bun:",next(generator))  # Take out the first steamed stuffed bun: 0
print("Take out the second steamed stuffed bun:",next(generator))  # Take out the second steamed stuffed bun: 1
print("Take out the third steamed stuffed bun:",next(generator))  # Take out the third steamed stuffed bun: 2

for i in generator:
    print("Take out the steamed stuffed bun serial number:",i)

Take out the steamed stuffed bun serial number: 3   # This is a little different from the list. The list starts from 0 every time, and the generator can only start from the number that has been obtained
 Take out the steamed stuffed bun serial number: 4
 Take out the steamed stuffed bun serial number: 5
 Take out the steamed stuffed bun serial number: 6
 Take out the steamed stuffed bun serial number: 7
 Take out the steamed stuffed bun serial number: 8
 Take out the steamed stuffed bun serial number: 9

Compare the buns above:

Xiao B gets 50 steamed stuffed buns at one time. Each steamed stuffed bun is placed in a bowl. Assuming that the bowl is numbered, Xiao B can take any steamed stuffed bun through the number. Xiao B can arrange and combine the steamed stuffed buns at will. Xiao B also likes to count steamed stuffed buns. Xiao B always counts how many steamed stuffed buns he has, and can repeat the plural number.

But little A's situation is different. He has only one bowl and can only hold one at A time. After taking No. 1 steamed stuffed bun, if he wants to take No. 2 steamed stuffed bun, he can only throw away or eat No. 1 steamed stuffed bun. Little A can't count the steamed stuffed buns yet. He can only record how many steamed stuffed buns he has taken

• Little B takes steamed stuffed buns arbitrarily by number: list index value
• Small A takes No. 1 steamed stuffed bun and then takes No. 2: take the value through the next() function
• Small A can't take No. 1 after taking No. 2 (No. 1 has been lost / eaten): the generator can only be executed once
• Small B-number buns can be repeated many times, and each time can start from No. 1: for... In... Can be repeated many times, and each time can start from the index of 0
• Small A can only start counting from the currently obtained steamed stuffed buns. Once it is finished, it can no longer be counted: for... In... Can only be counted once. What number is currently obtained, and it will be traversed from this number

Conclusion: the generator saves the algorithm. Each time next() is called, it calculates the value of the next element until the last element is calculated. When there are no more elements, it throws the StopIteration error.

3. yield keyword

The keyword yield is an abstract concept.

Or steamed stuffed bun shop:

Uncle Wang puts the steamer on the steaming rack and starts steaming steamed buns. Every time Xiao A opens the steamer cover, the steamer will pinch A steamed steamed bun to him on the spot and automatically close the steamer cover.

• Uncle Wang: CPU
• Steamers: generator functions
• Steam rack: memory space
• Steamer shelf: loading function
• Open the steamer lid: execute the next() method
• Give the package to small A: yield (generate) A value to the caller
• Close the steamer cover: the function exits (can also be understood as pause)
• Open the cover next time: execute the next() method again, starting from the last yield, and exit when the next yield is encountered

def make_baozi(xx):
    return xx

def simple_generator():
    print("It's the first time to make steamed stuffed buns filled with pork and cabbage")
    formulation = "Pork, cabbage"
    x_zhu = make_baozi(formulation)
    yield x_zhu    
    # Open the lid for the first time, make pork, cabbage and steamed stuffed bun and return it to you. Pause, wait until you finish eating, and record that I have given you pork, cabbage and steamed stuffed bun
    # In the next execution, the above one will not be executed again, but will be executed from this keyword

    print("Making steamed stuffed buns with barbecued pork for the second time")
    formulation = "Barbecued pork"
    x_cha = make_baozi(formulation)
    yield x_cha  # Open the lid for the second time, make the barbecued buns and return them to you. Pause until you finish eating, and record that I have given you pork, cabbage and barbecued buns

    print("Making steamed stuffed buns with corn for the third time")
    formulation = "Corn"
    x_yu = make_baozi(formulation)
    yield x_yu  # Open the lid for the first time, make corn buns and return them to you. Pause until you finish eating, and record that I have given you pork and cabbage buns, barbecued buns and corn buns


genrator = simple_generator()

print(next(genrator))
# The first steamed stuffed bun with pork and cabbage
# Pork, cabbage

print(next(genrator))
# Take steamed stuffed bun with barbecued pork for the second time
# Barbecued pork

print(next(genrator)) 
# Take the steamed stuffed bun with corn for the third time
# Corn

print(next(genrator))
Traceback (most recent call last):
  File "/app/util-python/test.py", line 36, in <module>
    print(next(genrator))
StopIteration

We can understand yield as the pause key of the function, and the next() function is the start key.

While pausing, the value will also be returned to you. When the next time starts, continue the execution from the place where it was suspended last time.

3.1 yield from

The yield from syntax is added to PEP 380 in Python version 3.3. Yield from can directly return each data in the iteratable object as the result of the generator

def simple_generator():
    a = [1,2,3]
    yield a

genrator = simple_generator()
print(genrator.__next__())
# [1,2.3]

def simple_generator():
    a = [1,2,3]
    yield from a

genrator = simple_generator()
print(genrator.__next__())  # 1
print(genrator.__next__())  # 2
print(genrator.__next__())  # 3

4. Generator method

Generator is a kind of iterator. Generator has three more methods than iterator: send(), close(), throw()

4.1 send

When the generator is suspended, a value is passed to the generator

def simple_generator():
    a = "test"
    a = yield a
    yield a

genrator = simple_generator()
print(genrator.send("dd"))

Traceback (most recent call last):
  File "/app/util-python/test.py", line 12, in <module>
    genrator.send("dd")
TypeError: can't send non-None value to a just-started generator

The above usage is wrong. Why? Because our generator has not started yet, we need to start the generator first.

Method 1 to start the generator:

print(genrator.send(None))

How to start the generator 2:

print(genrator.__next__())

After the generator starts, try again:

def simple_generator():
    a = "test"
    
    # The first startup will be executed here. After the pause, you can pass parameters to this keyword through send
    a = yield a
    print("Code block to be executed next time")
    yield a

genrator = simple_generator()
print(genrator.__next__())   # Printing: Tests
print(genrator.send("new value"))  # Print: new value

To sum up: this method can send a parameter to the generator, but the generator must be started first, that is, it must execute to the first yield keyword, and then pause in this keyword. At this time, press the yield of the pause key to accept the value of external send

4.2 throw()

Throws a specified exception when the execution of the generator function is suspended

def simple_generator():
    a = "Start execution"
    try:
        yield a
    except ValueError:
        print("Exception thrown in caught")

    b = "Execute the second yield"
    yield b

genrator = simple_generator()
print(genrator.__next__())   
# Execute to yield a, so this should be print: start execution

print(genrator.throw(ValueError))  
# Start execution from yield a and throw a ValueError exception. If the thrown exception is handled, it will be executed to yield b. otherwise, the exception will be thrown directly and the program will stop
# So the result here should print: execute the second yield

You can compare it with the following code to deepen your understanding

def simple_generator():
    a = "Start execution"
    try:
        yield a
        raise ValueError
    except ValueError:
        print("Exception thrown in caught")

    b = "Preparing for the second yield"
    yield b

genrator = simple_generator()
print(genrator.__next__())   
# Execute to yield a, so this should be print: start execution

print(genrator.__next__())  
# Execute from yield a down to raise ValueError, throw a ValueError exception, and then execute to yield b

4.3 close()

Throw a GeneratorExit exception to the generator, which means that the generator life cycle ends.

def simple_generator():
    a = "Start execution"

    try:
        yield a
    except ValueError:
        print("Exception thrown in caught")
    
    except GeneratorExit:
        print("Generator exit")

    b = "Execute the second yield"
    yield b


genrator = simple_generator()
print(genrator.__next__())
genrator.close()

The final result of the above code:

Traceback (most recent call last):
  File "/app/util-python/test.py", line 23, in <module>
    genrator.close()
RuntimeError: generator ignored GeneratorExit

Because the generator has executed the genrator.close() method and thrown the GeneratorExit exception, the subsequent statements executed by the generator method cannot have a yield statement, otherwise RuntimeError will be generated

So you need to remove the following two lines of code. Or execute genrator__ next__ () execute genrator.close() and let the function close all the yield

b = "Execute the second yield"
yield b

5. Implement Fibonacci sequence

def fib(max):
    n = 0
    a = 0
    b = 1
    while n < max:
        yield b
        a, b = b, a+b
        n+=1


r = fib(10)
for i in r:
    print(i)
1
1
2
3
5
8
13
21
34
55

Friends love to click below to pay attention to the official account.