An article explaining in detail the role of the __new__ method in Python

Preface

What is the function of the __new__ method of the class construction method in Python?

Some method names and attribute names in Python classes are preceded and followed by __ double underscores. Such methods and attributes usually belong to Python's special methods and special attributes. Implement special functions by overriding these methods or calling these methods directly. Today let’s talk about the application scenarios of the construction method __new__ in actual programs.

We know the common initialization __init__ method and can rewrite it to implement the initialization logic we want. Recently, we have encountered a type of problem in the actual business development process, such as the implementation of the data resource loading and caching mechanism. The construction method in the magic method is used. Among them, __init__() and __new__ are the constructors of the object. Reasonable use will effectively improve program performance. .

I hope that everyone can deeply understand and use it flexibly based on their own business needs to make the code more elegant.

1. Introduction to the __new__ method

Next, we will gradually elaborate on the existence of the __ new __ method in the class initialization process through examples!

1. Initialize data loading parsing class instance

class Solution(object):
 def __init__(self, name=None,data=None):
 self.name = name
 self.data = data
 #初始化加载数据
 self.xml_load(self.data)
 def xml_load(self,data):
 print("初始化init",data)
 def Parser(self):
 print("解析完成finish",self.name)
a = Solution(name="A111",data=10)
a.Parser()
b = Solution(name="A112",data=20)
b.Parser()
# print(a)与 print(b)返回了类的名称和对象的地址
print(a)
print(b)
# 可以使用内置函数id()查看python对象的内存地址
print(id(a))
print(id(b))
初始化init 10
解析完成finish A111
初始化init 20
解析完成finish A112
<__main__.Solution object at 0x0000024A3AF28D48>
<__main__.Solution object at 0x0000024A3B055C48>
2517839809864
2517841042504

Note:

1), Code instantiation class process

Generally use the __init__() method Initialize an instance of a class. When a class is instantiated in code, the first call executed is the __new__() method. When the __new__() method is not redefined in the defined class, Python will call the parent by default. The __new__() method of the class is used to construct the instance. The new method is to first create a space, then create an instantiated object each time, and then use the opened space to store the instantiated object; when creating an instantiated object again , and then use the new method to open up a space to store the instantiated object. Note that only classes that inherit object have this method.

2), memory address and object can be converted to each other

#通过_ctypes的api进行对内存地址的对象
import _ctypes
obj = _ctypes.PyObj_FromPtr(id(a))
#打印出来通过内存地址寻找到的对象
print(obj)

print(id(a)) and print(id(b)) print out the memory address (decimal) ,print(a) and print(b) return the name of the class and the address of the object, but they are not the same. Each time a class is instantiated, a different object address is created and allocated. Therefore, the address reference of the class object returned during the code instantiation process is also different.

2. Initializing data loading and overriding the new method to parse class instances

class Solution:
 """
 注：new方法是为实例化对象创建空间的方法，现在new方法被改写，没有将实例化对象引用返回给python的解释器
 无法为实例化对象创建空间存储，所以运行代码会报错。也没有完成初始化操作。
 """
 def __new__(cls, *args, **kwargs):
 print("对象创建空间")
 cls.instance = super().__new__(cls)
 print(cls.instance)
 # return cls.instance #若未返回实例对象引用，实例化方法将报错：AttributeError: 'NoneType' object has no attribute 'Parser'
 def __init__(self,name,data):
 self.name = name
 self.data = data
 self.xml_load(self.data)
 def xml_load(self,data):
 print("初始化init", data)
 def Parser(self):
 print("解析完成finish",self.data)
a = Solution("A111",10)
a.Parser()
print(id(a))

Note:

1), the difference between the __init__() method and the __new__() method

__new__() method is used to create an instance. It will be called first before the class is instantiated. It is a class method and a static method. The __init__() method is used to initialize the instance. This method is called after the instance object is created. It is a method of the instance object and is used to set some initial values ​​​​of the class instance object.

If the __init__() method and the __new__() method appear in the class at the same time, the __new__() method will be called first and then the __init__() method. The __new__() method is the first step in creating an instance. After execution, the instance of the created class needs to be returned. Otherwise, an error will be reported and the __init__() method cannot be executed. Among them, the __init__() method will not return any information.

2), rewriting the __new__() method

def __new__(cls, *args, **kwargs):
 print(cls)# cls 代表的是Solution这个类本身<class'__ main __.Solution'>
 cls.instance = super().__new__(cls)# object().__ new __()
 print(cls.instance)
 return cls.instance

super() and object.__new__(cls) are both calling the new method of the parent class, and the new method of the parent class must be Space can only be opened up by returning to the function, so return must be added. The execution order of the code is: first execute the new method, then execute the init method, and finally other methods.

2. Singleton Pattern

The original definition of the singleton pattern appeared in "Design Patterns": "Ensure that a class has only one instance and provide a global access point to access it."

The use of singletons is mainly when it is necessary to ensure that only one instance can be accessed globally, such as the output of system logs, the task manager of the operating system, etc.

1. How to implement singleton mode using new method?

class Solution:
 # 1、记录第一个被创建对象的引用，代表着类的私有属性
 _instance = None # 静态变量 存储在类的命名空间里的
 def __init__(self,name,data):
 self.name = name
 self.data = data
 self.xml_load(self.data)
 def __new__(cls, *args, **kwargs):
 # 2.判断该类的属性是否为空；对第一个对象没有被创建，我们应该调用父类的方法，为第一个对象分配空间
 if cls._instance == None:
 # 3.把类属性中保存的对象引用返回给python的解释器
 cls._instance = object.__new__(cls)# 3
 return cls._instance
 # 如果cls._instance不为None,直接返回已经实例化了的实例对象
 else:
 return cls._instance# 必须把地址返回给new方法，让它有存储空间
 def xml_load(self,data):
 print("初始化init",self.name,data)
 def Parser(self):
 print("解析完成finish",self.name)
a = Solution("A11",10)#第一次开辟一个对象空间地址，后面创建都是在该地址上进行的
a.Parser()
b = Solution("A12",20)#b把a覆盖掉
b.Parser()
print(id(a))
print(id(b))
# 内存地址，而且它们的内存地址都是一样的
print(a.name)
print(b.name)

初始化init A11 10
解析完成finish A11
初始化init A12 10
解析完成finish A12
2465140199816
2465140199816
A12
A12

Note:

1). The singleton mode always has only one space, and this space is always reused.

First define the private attribute _instance of a class to record the reference of the first created object. If cls._instance is None, it means that the class has not been instantiated yet, instantiate the class and return instance object.

Through the following data test, it can be seen that the final printouts of print(obj.name, obj.data) are A12. When "A11" is printed for the first time, the attribute is empty, and a space is opened up by executing the if statement. Store this attribute; the space has been opened since the second time, execute the else statement, directly return "A12" to the original space, and overwrite the previous cover data.

def task(id,data):
 obj = Solution("{0}".format(id), "{0}".format(data))
 print(obj.name, obj.data)
import threading
ID=["A11","A12","A13","A14","A12"]
DATA=[10,20,30,40,20]
for i in range(5):
 t = threading.Thread(target=task(ID[i],DATA[i]), args=[i, ])
 t.start()

<__main__.Solution object at 0x00000221B2129148>
初始化init A11 10
A11 10
初始化init A12 20
A12 20
初始化init A13 30
A13 30
初始化init A14 40
A14 40
初始化init A12 20
A12 20

2), Another implementation method of singleton mode

def __new__(cls,*args,**kwargs):
 # hasattr查询目标并判断有没有，not1==1返回的是False
 # if语句后面的
 # not 条件整体为True时，执行cls.instance = object....代码
 # if语句后面的
 # not 条件整体为False时，执行return代码
 if not hasattr(cls,"instance"): # hasattr查、判断的作用
 cls.instance = object.__new__(cls)
 return cls.instance

2. How to control the class to only execute the initialization method once?

The above realizes the reuse of singleton mode object space, but sometimes we want to load the initialization process only once to avoid frequent requests that waste system resources (such as database connection request data).

class Solution:
 #定义类变量
 # 记录第一个被创建对象的引用，代表着类的私有属性
 _instance = None
 #记录是否执行过初始化动作
 init_flag = False
 def __init__(self,name,data):
 self.name = name
 self.data = data
 #使用类名调用类变量,不能直接访问。
 if Solution.init_flag:
 return
 self.xml_load(self.data)
 # 修改类属性的标记
 Solution.init_flag = True
 def __new__(cls, *args, **kwargs):
 # 判断该类的属性是否为空；对第一个对象没有被创建，我们应该调用父类的方法，为第一个对象分配空间
if cls._instance == None:
 # 把类属性中保存的对象引用返回给python的解释器
 cls._instance = object.__new__(cls)
 return cls._instance
 #如果cls._instance不为None,直接返回已经实例化了的实例对象
 else:
return cls._instance
 def xml_load(self,data):
 print("初始化init",self.name,data)
 def Parser(self):
 print("解析完成finish",self.name)
a = Solution("A11",10)#第一次实例化对象地址，后面创建都是在该地址上进行的
a.Parser()
b = Solution("A12",20)#b把a覆盖掉
b.Parser()
print(id(a))
print(id(b))
print(a.name)
print(b.name)

初始化init A11 10
解析完成finish A11
解析完成finish A12
2280855720328
2280855720328
A12
A12

Note:

1). The initialization process is only loaded once in singleton mode.

这时候我们在类空间中再添加一个init_flag属性来记录是否已经执行过初始化操作即可实现加载一次初始化过程。从以上两次实例化过程结果来看，对象引用地址不变，结果被最后一次实例化数据覆盖且初始化init只被打印一次。

2）、单例模式下一次资源加载注意点

单例模式下控制类仅进行一次初始化过程适用于资源一次性加载进缓存的过程，对于多进程应用可采用多例模式实现。

三、多例模式

多个实例对象空间引用地址完全独立，从而保持避免不同请求资源不被占用。将同一个对象请求归为同一个实例。

class Solution:
 ##定义类实例化对象字典，即不同的实例对象对应不同的对象空间地址引用
 _loaded = {}
 def __init__(self,name,data):
 self.name = name
 self.data = data
 self.xml_load(self.data)
 def __new__(cls, name,*args):
 if cls._loaded.get(name) is not None:
 client = cls._loaded.get(name)
 print(f"已经存在访问对象 {name}")
 print(client)
 return client
 # 把类属性中保存的对象引用返回给python的解释器
 print(f"正在创建访问对象 {name}")
 client = super().__new__(cls)
 # 为该类实例name添加一个空间对象地址引用
 print(client)
 cls._loaded[name] = client
 return client
 def xml_load(self,data):
 print("初始化init",self.name,data)
 def Parser(self):
 print("解析完成finish",self.name)
if __name__ == '__main__':
 print("多例模式实例")
 a = Solution("A11",10)
 a.Parser()
 b = Solution("A11",10)
 b.Parser()
 c = Solution("A12", 20)
 c.Parser()
 print(f"{a is b}")
 print(a.name)
 print(b.name)
 print(c.name)

注：

1）、多例模式始终具有多个空间，不同空间完全独立。

我们在类空间中定义类实例化对象字典，即建立不同的实例对象和对象空间地址引用键值对，从而实现多例模式。通过类字典判断实例对象是否创建，节省创建的成本。

2）、多例模式测试过程

当创建相同的实例对象name="A11"时，程序首先在实例池中搜索cls._loaded.get(name)，若存在则直接返回已创建的实例对象空间。多例模式完美的实现了不同访问对象具体不同的实例化对象地址。

多例模式实例
正在创建访问对象 A11
<__main__.Solution object at 0x000001C105AA5EC8>
初始化init A11 10
解析完成finish A11
已经存在访问对象 A11
<__main__.Solution object at 0x000001C105AA5EC8>
初始化init A11 10
解析完成finish A11
正在创建访问对象 A12
<__main__.Solution object at 0x000001C105AA5F88>
初始化init A12 20
解析完成finish A12
True
A11
A11
A12

3）、多例模式下缓冲机制的实现

进一步优化多例模式初始化过程，比如读取文件或者数据库时仅进行一次初始化加载。

class Solution:
 ##定义类实例化对象字典，即不同的实例对象对应不同的对象空间地址引用
 _loaded = {}
 def __new__(cls, name,data,*args):
 if cls._loaded.get(name) is not None:
 client = cls._loaded.get(name)
 print(f"已经存在访问对象 {name}")
 print(client)
 return client
 print(f"正在创建访问对象 {name}")
 # 把类属性中保存的对象引用返回给python的解释器
 client = super().__new__(cls)
 print(client)
 # 为该类实例name添加一个空间对象地址引用
 cls._loaded[name] = client
 client._init_db(name,data)
 return client
 def _init_db(self,name,data):
 self.name = name
 self.data = data
 self.xml_load(self.data)
 def xml_load(self,data):
 print("初始化init",self.name,data)
 def Parser(self):
 print("解析完成finish",self.name)
if __name__ == '__main__':
 print("多例模式实例-缓存")
 a = Solution("A11",10)
 a.Parser()
 b = Solution("A11",10)
 b.Parser()
 c = Solution("A12", 20)
 c.Parser()
 print(f"{a is b}")
 print(a.name)
 print(b.name)
 print(c.name)

多例模式实例
正在创建访问对象 A11
<__main__.Solution object at 0x0000024198989148>
初始化init A11 10
解析完成finish A11
已经存在访问对象 A11
<__main__.Solution object at 0x0000024198989148>
解析完成finish A11
正在创建访问对象 A12
<__main__.Solution object at 0x00000241989891C8>
初始化init A12 20
解析完成finish A12
True
A11
A11
A12

注：多例模式下多个实例化对象均只进行一次初始化过程。

重写__new__方法中每个实例对象创建后绑定初始化_init_db（）方法执行一次，后面遇到同一个实例对象将不会发生什么，直接返回已创建的实例对象。从测试结果来看，创建相同的实例对象name="A11"时，第二次将略过初始化数据加载过程，很好的实现了缓存机制。

__new__方法总结

本文结合项目背景详细介绍了__new__方法实现单例模式和多例模式以及缓存机制的实现！

1、__new__ 方法是在类创建实例的时候自动调用的。

2、 实例是通过类里面的 __ new __ 方法是在类创建出来的。

3、 先调用__new__ 方法创建实例，再调用 __ init __方法初始化实例。

4、 __new__ 方法，后面的括号里面的cls代表的是类本身。

5、__new__ 方法，判断类属性为空就去开辟空间，否则复用原来的地址。

更多的特殊方法比如1、自我描述方法：__repr__2、析构方法：__del__ 3、列出对象所有属性（包括方法）名：__dir__4、__dict__属性：查看对象内部所有属性名和属性值组成的字典5、__getattr____setattr__等。

当然还有metaclass类的__new__方法，可以动态修改程序中的一批类，这个功能在开发一些基础性的框架时非常有用，可以使用metaclass为某一批需要通用功能的类添加方法，若有合适实例场景后续分享。

The above is the detailed content of An article explaining in detail the role of the __new__ method in Python. For more information, please follow other related articles on the PHP Chinese website!