__slots__

经典场景:http://tech.oyster.com/save-ram-with-python-slots/

使用了slots让内存占用从 25.5GB 降到了 16.2GB

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
from __future__ import print_function
import resource


class A(object):
def __init__(self):
self.a = 'string'
self.b = 10
self.c = True


class B(object):
__slots__ = ['a', 'b', 'c']
def __init__(self):
self.a = 'string'
self.b = 10
self.c = True


def test(cls):
mem_init = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss
l = []
for i in range(500000):
l.append(cls())
mem_final = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss
del l
print('Class: {}:\n'.format(getattr(cls, '__name__')))
print('Initial RAM usage: {:14,}'.format(mem_init))
print(' Final RAM usage: {:14,}'.format(mem_final))
print('-' * 20)


if __name__ == '__main__':
import sys
test(globals()[sys.argv[1].upper()])

执行上面的代码:

1
2
3
4
5
6
7
8
9
10
11
12
❯ python mem_test.py a
Class: A:

Initial RAM usage: 4,890,624
Final RAM usage: 200,454,144
--------------------

❯ python mem_test.py b
Class: B:

Initial RAM usage: 4,919,296
Final RAM usage: 60,235,776

发现使用 slots 可以让内存使用减少 3.5 倍左右, 不同场景下效果不同,这只在大量实例下的效果,万级别以下可以不使用,根据实际业务来确定是否使用__slots__

__slots__的缺陷

  1. 每个继承的子类都要重新定义一遍slots
  2. 实例只能包含哪些在slots定义的属性,这对写程序的灵活性有影响,比如你由于某个原因新网给 instance 设置一个新的属性,比如 instance.a = 1, 但是由于 a 不在slots里面就直接报错了,你得不断地去修改slots或者用其他方法迂回的解决
  3. 实例不能有弱引用(weakref)目标,否则要记得把weakref放进slots

说明一下weakref

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
In [2]: %pycat ref_example.py
from weakref import ref

class A(object):
__slots__ = ['b']
def __init__(self):
self.b = 1


class B(object):
__slots__ = ['b', '__weakref__']
def __init__(self):
self.b = 1

In [3]: from ref_example import *

In [4]: a = A()

In [5]: r = ref(a)
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-6-75a6d689c8b3> in <module>()
----> 1 r = ref(a)

TypeError: cannot create weak reference to 'A' object

In [6]: b = B()

In [7]: r = ref(b)

In [8]: r
Out[8]: <weakref at 0x109199578; to 'B' at 0x10919f890>

如果未在__slots__中添加弱引用魔法函数__weakref__,则会报错。

weakref模块提供了对象的弱引用,普通引用会增加对象的引用计数,垃圾回收就不会回收。

可以参照:https://learnku.com/docs/pymotw/weakref-a-weak-reference-to-an-object/3372