# [Python learning tutorial] derivation and generator

When we extract or process a batch of iterative data (such as list or dictionary) and finally get a new list or dictionary, the derivation is a very concise expression.

List derivation

For example, there is a batch of data

```data = [
{'name': 'Zhang San', 'gender': 'male',  'age': 12},
{'name': 'Li Si', 'gender': 'female',  'age': 10},
{'name': 'Wang Wu', 'gender': 'male',  'age': 20},
{'name': 'Zhao Liu', 'gender': 'male',  'age': 11},
{'name': 'Zhou Qi', 'gender': 'female',  'age': 16},
{'name': 'Sun Ba', 'gender': 'male',  'age': 13},
]
```

We want to extract the name s in the data to form a new list. The general operation is as follows.

```names = []  # Define an empty list

for item in data:  # Traversal data
name = item['name']  # Extract name in each row
names.append(name)  # Append to list
```

If the derivation is used, the form is as follows.

```names = [item['name'] for item in data]     # Traverse the data, extract the name in each item, and generate a new list
```

data processing

When extracting data, we can also process each item of data. Suppose we need to add the string 'Name:' before each name. This can be done.

```names = ['full name: '+item['name'] for item in data]
```

'Name:' + item ['name'] is the data of each item

Data filtering

Similarly, we can also filter the data. For example, as long as we are older than 12, we can use if to filter later

```names = [item['name'] for item in data if item['age']>12]
```

Multiple cycles

The derivation also supports multiple loops, such as

```'''
No one answers the problems encountered in learning? Xiaobian created a Python exchange of learning QQ Group: 531509025
Look for like-minded friends to help each other,There are also good videos and tutorials in the group PDF e-book!
'''
for x in range(1,5)
if x > 2
for y in range(1,4)
if y < 3
x*y
```

It is expressed as follows by derivation

```[x*y for x in range(1,5) if x > 2 for y in range(1,4) if y < 3]
```

Batch execution operation

Since derivation is a circular operation, we can also use derivation to perform some similar operations in batch, such as:

```def step1(driver):
print('Step 1)

def step2(driver):
print('Step 2)

def step3(driver):
print('Step 3)
```

We can put the function name in a list and execute it using a derivation loop

```steps = [step1, step2, step3]   # Function name list

[step(driver) for step in steps]  # We don't need variable reception, we just need it to execute circularly
```

Dictionary derivation

When we need to traverse a batch of data and finally get a dictionary, we can also use the dictionary derivation, such as:

```'''
No one answers the problems encountered in learning? Xiaobian created a Python exchange of learning QQ Group: 531509025
Look for like-minded friends to help each other,There are also good videos and tutorials in the group PDF e-book!
'''
data = [
{'name': 'Zhang San', 'gender': 'male',  'age': 12},
{'name': 'Li Si', 'gender': 'female',  'age': 10},
{'name': 'Wang Wu', 'gender': 'male',  'age': 20},
{'name': 'Zhao Liu', 'gender': 'male',  'age': 11},
{'name': 'Zhou Qi', 'gender': 'female',  'age': 16},
{'name': 'Sun Ba', 'gender': 'male',  'age': 13},
]
```

Suppose we want to get a dictionary like {'Zhang San': 12, 'Li Si': 10,...}, the dictionary derivation is as follows:

```persons = {item['name']: item['age'] for item in data}
```

Dictionary derivation also supports if filtering and other operations.

generator

The generator is actually an object containing initial data and derivation rules. For example, we can easily write all odd numbers within 1w because I only need to remember to add 2 each time from 1.

This is the case with generators. Corresponding to a large amount of data or data in CSV/Excel files, the generator can save a lot of memory. For example, csv.Reader(f) is a generator that only saves the current position and the method to read the next line of data.

When you need to traverse, it reads you one line of data at a time.
As an example of list derivation,

```data = [
{'name': 'Zhang San', 'gender': 'male',  'age': 12},
{'name': 'Li Si', 'gender': 'female',  'age': 10},
{'name': 'Wang Wu', 'gender': 'male',  'age': 20},
{'name': 'Zhao Liu', 'gender': 'male',  'age': 11},
{'name': 'Zhou Qi', 'gender': 'female',  'age': 16},
{'name': 'Sun Ba', 'gender': 'male',  'age': 13},
]
names = [item['name'] for item in data]
```

We change the brackets in the list to parentheses to get a generator

```names2 = (item['name'] for item in data)
```

Note: the generator is different from the derivation. The loop in it is not executed immediately. It will be executed only when you traverse the generator

```'''
No one answers the problems encountered in learning? Xiaobian created a Python exchange of learning QQ Group: 531509025
Look for like-minded friends to help each other,There are also good videos and tutorials in the group PDF e-book!
'''
for name in names:  # New list generated by traversing list derivation
print(name)

for name in names2:  # Traverse a generator
print(name)
```

The two printing results are the same. The generator saves more memory and runs only when traversing.

Tags: Python

Posted on Fri, 10 Sep 2021 16:25:49 -0400 by spookztar