The book continues with some cold and hot knowledge about classes and their methods. This article will focus on another important element in the class - methods. Like the previous article, it uses various magical examples to restore a different Python from the perspective of principle and mechanism. Before reading this article, it is recommended to read the contents of the previous article: Python Popular Science Series -- classes and methods (Part I)
The nature of object methods
When it comes to object-oriented programming, we should be familiar with the concept of method. Actually Part I As mentioned in, in Python, the essence of a method is a field. By assigning an executable object to the current object, a method can be formed, and an attempt is made to manually create an object.
However, if you have a better understanding of Python or a closer observation, you will find that in fact, methods can be called in the following ways
class T: def __init__(self, x, y): self.x = x self.y = y def plus(self, z): return self.x + self.y + z t = T(2, 5) t.plus(10) # 17 T.plus(t, 10) # 17, the same as t.plus(10)
Yes, it's the usage of T.plus(t, 10), which doesn't seem to be seen in other object-oriented languages. It seems a little puzzling. Don't worry, let's do another experiment
def plus(self, z): return self.x + self.y + z class T: def __init__(self, x, y): self.x = x self.y = y plus = plus t = T(2, 5) print(t) print(plus) print(T.plus) print(t.plus) # <__main__.T object at 0x7fa58afa7630> # <function plus at 0x7fa58af95620> # <function plus at 0x7fa58af95620> # <bound method plus of <__main__.T object at 0x7fa58afa7630>>
In this program, the plus function is defined separately and introduced as a field in class T. If you look at the above output, you will find the fact that t.plus and t.plus are exactly the same object, but t.plus is not the same. According to the analysis in the previous article, the former is obvious, but t.plus has become a thing called method. What's the matter? Let's continue the experiment, and then move on to the previous program
from types import MethodType print(type(t.plus), MethodType) # <class 'method'> <class 'method'> assert isinstance(t.plus, MethodType)
You will find that the legendary method was originally the object types.MethodType. Now that we have this clue, let's continue to read the source code of types. Methodtype. Some contents of the source code are not visible, but only these are found (Python version here is 3.9.6)
class MethodType: __func__: _StaticFunctionType __self__: object __name__: str __qualname__: str def __init__(self, func: Callable[..., Any], obj: object) -> None: ... def __call__(self, *args: Any, **kwargs: Any) -> Any: ...
Sorry, we can't find the official document here. The of types Library file In the MethodType section, there is only one line of overview text without substantive content, so you have to turn to the source code. If there are serious documents or instructions found by readers, you are welcome to post them in the comment area. But from this point of view, there is still a key discovery - this__ init__ Method has something. From the perspective of name and type, func should be a function and obj should be an arbitrary object. Let's think again. From the perspective of logical elements, what are the necessary factors for t.plus to operate? The answer is obvious:
- Running logic, in general, is the actually running function plus
- The running body, generally speaking, the object t separated by dots before the method
Up to this point, the answer is ready to come out. However, in the spirit of rigorous science, further verification is needed. We need to try to disassemble the t.plus to see what's in it (follow the above procedure)
print(set(dir(t.plus)) - set(dir(plus))) # {'__self__', '__func__'}
print(t.plus.__func__) # <function plus at 0x7fa58af95620>
print(t.plus.__self__) # <__main__.T object at 0x7fa58afa7630>
First, in the first line, turn the dir result into a set to see which fields are owned by t.plus but not by t.plus. Sure enough, just two fields --__ self__ And__ func__ . Then output the values of these two fields respectively, and it is found that t.plus__ func__ It's the plug defined before, and t.plus__ self__ Is instantiated t.
At this step, it is basically consistent with our conjecture, only one final verification is missing. I still remember the manually created object in the previous article. Yes, let's use MethodType to build it again more scientifically and more in line with the actual code behavior. The procedure is as follows
from types import MethodType
class MyObject(object):
pass
if __name__ == '__main__':
t = MyObject() # the same as __new__
t.x = 2 # the same as __init__
t.y = 5
def plus(self, z):
return self.x + self.y + z
t.plus = MethodType(plus, t) # a better implement
print(t.x, t.y) # 2 5
print(t.plus(233)) # 240
print(t.plus)
# <bound method plus of <__main__.MyObject object at 0x7fbbb9170748>>
The running results are consistent with those before, and are completely consistent with the objects implemented in the conventional way, and this t.plus is the kind of method seen in the previous experiment. So far, the essence of object method in Python has been very clear - object method is an executable object based on the original function and the current object, which is implemented through the types.MethodType class.
Extended thinking 1: Based on the above analysis, why does T.plus(t, 10) have an equivalent operation effect to t.plus(10)?
Extended thinking 2: why is the first parameter at the beginning of the object method self, and it is actually passed in from the second parameter? What is the internal principle of MethodType objects when they are executed?
Welcome to the comments section!
Class method and static method
Having finished object methods, let's look at two other common methods - class methods and static methods. The first is the simplest example
class T: def __init__(self, x, y): self.x = x self.y = y def plus(self, z): return self.x + self.y + z @classmethod def method_cls(cls, suffix): return str(cls.__name__) + suffix @staticmethod def method_stt(content): return ''.join(content[::-1])
Where method_cls is a class method, method_stt is a static method, which should be familiar to everyone. That's not much nonsense. Let's take a look at this method first_ What exactly is CLS (program continues above)
print(T.method_cls) # <bound method T.method_cls of <class '__main__.T'>> t = T(2, 3) print(t.method_cls) # <bound method T.method_cls of <class '__main__.T'>>
It looks familiar, right, right - whether it's t.method on class t_ CLS, or t.method on object t_ CLS are all types.MethodType objects discussed in the previous chapter, and they are the same object. Next, let's look at its internal structure (the program continues above)
print(T.method_cls.__func__) # <function T.method_cls at 0x7f78d86fe2f0> print(T.method_cls.__self__) # <class '__main__.T'> print(T) # <class '__main__.T'> assert T.method_cls.__self__ is T
Among them__ func__ This is the original method_cls function, and__ self__ Class object T. Therefore, it is not difficult to find the fact that the essence of class method is a method object that takes the current class object as the main object. In other words, class methods are essentially homologous with object methods. The only difference is that this self is called cls, and its value is changed to the current class object.
After reading class methods, the next step is static methods. First, as before, look at method_ Actual content of STT
print(T.method_stt) # <function method_stt at 0x7fd64fa70620> t = T(2, 3) print(t.method_stt) # <function method_stt at 0x7fd64fa70620>
This result is unexpected, but it is perfectly logical to think about it -- the essence of a static method is a native function attached to classes and objects. In other words, whether it's t.method_stt or t.method_stt, the original method is actually obtained_ STT function.
Extended thinking 3: why is the subject in the class method named cls instead of self? What's the meaning?
Extended thinking 4: if the cls parameter in the class method is renamed self, will it affect the normal operation of the program? Why?
Extended thinking 5: one of the most common applications of class methods is to build factory functions, such as T.new_instance, which can be used to quickly create instances with different characteristics. In Python, the class itself has a constructor, so what are the similarities and differences and division of labor between class factory methods and constructors? Please talk about your views through the analogy and practical construction of other languages.
Welcome to the comments section!
Magic method
For readers who have studied C + +, they should know that there is a special kind of function that starts with operator, and their effect is operator overloading. In fact, there are similar features in Python. For example, let's take an example to see how addition operations are overloaded
class T:
def __init__(self, x, y):
self.x = x
self.y = y
def __add__(self, other):
print('Operating self + other ...')
if isinstance(other, T):
return T(self.x + other.x, self.y + other.y)
else:
return T(self.x + other, self.y + other)
def __radd__(self, other):
print('Operating other + self ...')
return T(other + self.x, other + self.y)
def __iadd__(self, other):
print('Operating self += other ...')
if isinstance(other, T):
self.x += other.x
self.y += other.y
else:
self.x += other
self.y += other
return self
t1 = T(2, 3)
t2 = T(8, -4)
t3 = t1 + t2
print(t3.x, t3.y)
t4 = t1 + 10
print(t4.x, t4.y)
t5 = -1 + t2
print(t5.x, t5.y)
t1 += 20
print(t1.x, t1.y)
The output results are as follows
Operating self + other ... 10 -1 Operating self + other ... 12 13 Operating other + self ... 7 -5 Operating self += other ... 22 23
A set of simple explanations can be made for the above examples:
- __ add__ It is a conventional addition operation, that is, it will enter when t = a + b is executed__ add__ Method, where self is a, other is b, and the return value is t.
- __ radd__ Is the added operation, that is, it will enter when t = b + a is executed__ radd__ Method, where self is a, other is b, and the return value is t.
- __ iadd__ It is a self addition operation, that is, it will enter when a += b is executed__ iadd__ Method, where self is a before the operation, other is b, and the return value is a after the operation.
Among them, the conventional addition operation is not difficult to understand, and the addition self operation is not difficult to understand, but the added operation may be slightly difficult to understand. In fact, we can see from the example t5 = -1 + t2 in the above code that - 1, as an int type object, does not support conventional addition operations on T-type objects, and Python does not provide a mechanism to overload native types like Ruby. At this time, if you need to support addition operations such as - 1 + t2, you need to use the function of the body on the right__ radd__ method.
There are actually many examples of the three methods mentioned in the above examples, and these methods start and end with two underscores. They have a common name - Magic method. Of course, the most direct application of magic method is to support all kinds of arithmetic operators. Let's see what arithmetic operations are supported
| Magic Methods | Structural diagram | explain | |
|---|---|---|---|
| add | self + other | addition | Conventional addition operation |
| radd | other + self | Additive operation | |
| iadd | self += other | Self addition operation | |
| sub | self - other | subtraction | Conventional subtraction |
| rsub | other - self | Subtracted operation | |
| isub | self -= other | Self subtraction operation | |
| mul | self * other | multiplication | Conventional multiplication |
| rmul | other * self | Multiplicative operation | |
| imul | self *= other | Self multiplication operation | |
| matmul | self @ other | Matrix multiplication | Conventional matrix multiplication |
| rmatmul | other @ self | Matrix multiplication | |
| imatmul | self @= other | Matrix multiplication | |
| truediv | self / other | Common Division | Conventional division |
| rtruediv | other / self | Ordinary divide operation | |
| itruediv | self /= other | Ordinary self division operation | |
| floordiv | self // other | to be divisible by | Regular division operation |
| rfloordiv | other // self | Divisible operation | |
| ifloordiv | self //= other | Self dividing operation | |
| mod | self % other | Surplus | Conventional remainder operation |
| rmod | other % self | Remainder operation | |
| imod | self %= other | Self remainder operation | |
| pow | self ** other | Power | Conventional power operation |
| rpow | other ** self | Multiplicative operation | |
| ipow | self **= other | Self power operation | |
| and | self & other | Arithmetic and | General arithmetic operation |
| rand | other & self | Applied to arithmetic | |
| iand | self &= other | Self arithmetic operation | |
| or | self | other | Arithmetic or | General arithmetic or operation |
| ror | other | self | By arithmetic or operation | |
| ior | self |= other | Self arithmetic or operation | |
| xor | self ^ other | Arithmetic XOR | Conventional arithmetic XOR operation |
| rxor | other ^ self | Arithmetic XOR operation | |
| ixor | self ^= other | Autoarithmetic XOR operation | |
| lshift | self << other | Arithmetic shift left | General arithmetic shift left operation |
| rlshift | other << self | Arithmetic shift left operation | |
| ilshift | self <<= other | Self arithmetic shift left operation | |
| rshift | self >> other | Arithmetic shift right | Normal arithmetic shift right operation |
| rrshift | other >> self | Arithmetic shift right operation | |
| irshift | self >>= other | Self arithmetic shift right operation | |
| pos | +self | Take positive | Positive operation |
| neg | -self | Reverse | Inverse operation |
| invert | ~self | Arithmetic negation | Arithmetic inversion |
| eq | self == other | Size comparison | Equal to comparison operation |
| ne | self != other | Not equal to comparison operation | |
| lt | self < other | Less than comparison operation | |
| le | self <= other | Less than or equal to comparison operation | |
| gt | self > other | Greater than comparison operation | |
| ge | self >= other | Greater than or equal to comparison operation |
It can be seen that common arithmetic operations are available. However, there are still some things that cannot be overloaded by magic, including but not limited to (as of press time, the latest version of Python is 3.10.0):
- Ternary operation, i.e. xxx if xxx else xxx
- Logical and, logical or, logical non operations, i.e. xxx and yyy and xxx or yyy and not xxx
In addition, there are some common functional magic methods:
| Magic Methods | Structural diagram | explain | |
|---|---|---|---|
| getitem | self[other] | Index operation | Index query |
| setitem | self[other] = value | Index assignment | |
| delitem | del self[other] | Index deletion | |
| getattr | self.other | Attribute operation | Property acquisition |
| setattr | self.other = value | Attribute assignment | |
| delattr | del self.other | Property deletion | |
| len | len(self) | length | Get length |
| iter | for x in self: pass | enumeration | enumerable object |
| bool | if self: pass | authenticity | Determination of authenticity |
| call | self(*args, **kwargs) | function | Run object |
| hash | hash(self) | Hash | Get hash value |
Of course, there are some functional things that cannot be modified by magic methods, such as:
- Object identifier, i.e. id(xxx)
In this way, the magic method is not magical, and the functions are still very complete. As long as the collocation is reasonable, it can play a very amazing effect. What is the essence of this method? In fact, it is also very simple - it is a method with special semantics. For example, in the example of the above addition operation, it can also be run in this way
t1 = T(2, 3) t2 = T(8, -4) t3 = t1.__add__(t2) print(t3.x, t3.y) # Operating self + other ... # 10 -1
T1 above__ add__ (T2) is actually the real form of t1 + t2, and Python's object system packages these magic methods and binds them with special syntax and uses to form a rich object operation mode.
Extended thinking 6: in arithmetic operation, what is the relationship between conventional magic method, passive operation magic method and self operation magic method? When there is more than one set of matching patterns, which will actually be executed? Please try it by experiment.
Extended thinking 7: why can't ternary operation and logical operation be overloaded by magic methods? What technical barriers may exist? And what kind of problems may be caused by open overloading?
Extended thinking 8: why can't object identifier operations be overloaded by magic methods? What is the nature of object identifiers? What kind of problems may be caused by open overloading?
Extended thinking 9: in the Python libraries you have used, which use magic methods to overload operators and other functions? Specifically talk about its application scope and mode.
Extended thinking 10: consider various arithmetic operations such as addition, subtraction, multiplication and division in numpy and torch libraries, including matrix (tensor) and matrix operations, matrix to value operations, and value to matrix operations. How can they be simple and easy to use in the Python language environment? Please give your analysis by flipping through the documentation or reading the source code.
Extended thinking 11:__ matmul__ On which types of objects can the operation support @ operation? In numpy and torch libraries, @ is used as the operator to operate the matrix (tensor). Which operation function is the equivalent of the operation result?
Welcome to the comments section!
The nature of object properties
In Python classes, there is another existence similar to but different from methods - object properties. For example
class T:
def __init__(self, x):
self.__x = x
@property
def x(self):
print('Access x ...')
return self.__x
@x.setter
def x(self, value):
print(f'Set x from {self.__x} to {value} ...')
self.__x = value
@x.deleter
def x(self):
print('Delete x\'s value ...')
self.__x = None
t = T(2)
print(t.x)
t.x = 233
del t.x
# Access x ...
# 2
# Set x from 2 to 233 ...
# Delete x's value ...
Accessing T.x will enter the first getter function, assigning a value to T.x will enter the second setter function, and if you try to delete T.x, you will enter the third delete function, which is obvious for object t. However, in order to study the principle, let's look at the actual content of T. x located on class T (the code continues above)
print(T.x) # <property object at 0x7faf16853db8>
We can see that T.x is a property object. Next, let's take a look at the structure contained in it
print(set(dir(T.x)) - set(dir(lambda: None)))
print(T.x.fget)
print(T.x.fset)
print(T.x.fdel)
# {'fget', '__delete__', 'deleter', 'fdel', '__set__', '__isabstractmethod__', 'getter', 'setter', 'fset'}
# <function T.x at 0x7f39d32f41e0>
# <function T.x at 0x7f39d32f4268>
# <function T.x at 0x7f39d32f42f0>
You can see that T.x has more parts than ordinary function objects, which are basically divided into get, set and del related parts, and T.x.fget, T.x.fset and T.x.fdel point to three different functions respectively. Based on the current information, especially these names, it is very close to the correct answer. For validation, let's try to manually create an attribute and add it to the class, as shown below
def xget(self):
print('Access x ...')
return self.xvalue
def xset(self, value):
print(f'Set x from {self.xvalue} to {value} ...')
self.xvalue = value
def xdel(self):
print('Delete x\'s value ...')
self.xvalue = None
class T:
def __init__(self, x):
self.xvalue = x
x = property(xget, xset, xdel)
t = T(2)
print(t.x)
t.x = 233
del t.x
# Access x ...
# 2
# Set x from 2 to 233 ...
# Delete x's value ...
It can be seen that the above example runs completely normally. Therefore, in fact, the property object is a support object__ get__ , __ set__ , __ delete__ The special objects of the three magic methods. Since there are many contents involved in these three magic methods, the follow-up may be devoted to one issue. In short, it can be understood that by making such an assignment on the class, the attribute of the instantiated object can be accessed, assigned and deleted. This is the essence of object attributes in Python.
Extended thinking 12: how to use the property class to construct a property that can only be read and written and cannot be deleted? And how to construct read-only properties?
Extended thinking 13: what are the purposes of getter, setter and delete methods in the property object?
Welcome to the comments section!
Follow up notice
This paper focuses on the analysis of various mechanisms and characteristics of the method from the perspective of principle. After these two popular science articles on Python classes and methods, the basic concepts and mechanisms have been basically described. On this basis, the third bullet of treevaluate will also be launched soon, including the following main contents:
- The tree method and class method will be based on the function tree in the second bullet of treevaluate, combined with the discussion on the essence of the method in this article.
- Tree operation, function tree based on arithmetic type magic method, will be explained and displayed with examples.
- Based on the application of tree operation and function tree based on functional magic method, its high ease of use is displayed after explanation.
In addition, welcome to OpenDILab's open source project:
And several of my own open source projects (some are still under development or improvement):