ERROR: Process pool report error: Can‘t pickle
注意:如果出現(xiàn)
PicklingError: Can't pickle <type 'instancemethod'>: attribute lookup __builtin__.instancemethod failed
數(shù)據(jù)序列化:
- https://zhidao.baidu.com/question/525917268.html
- https://www.liaoxuefeng.com/wiki/001374738125095c955c1e6d8bb493182103fac9270762a000/00138683221577998e407bb309542d9b6a68d9276bc3dbe000
應(yīng)該是說(shuō)一個(gè)數(shù)據(jù)結(jié)構(gòu),比如二叉樹之類,序列化以后會(huì)變成一個(gè)char數(shù)組或者一個(gè)string字符串這樣,方便你存到文件里面或者通過(guò)網(wǎng)絡(luò)傳輸。然后要恢復(fù)的時(shí)候就是“反序列化”,把文件里讀出來(lái)/從網(wǎng)絡(luò)收到的char數(shù)組或者string恢復(fù)成一棵二叉樹或者其他什么東西。主要就是方便保存
可以被序列化的類型有:
https://zhidao.baidu.com/question/619353578954760372.html
* None,True 和 False;
* 整數(shù),浮點(diǎn)數(shù),復(fù)數(shù);
* 字符串,字節(jié)流,字節(jié)數(shù)組;
* 包含可pickle對(duì)象的tuples,lists,sets和dictionaries;
* 定義在module頂層的函數(shù):
* 定義在module頂層的內(nèi)置函數(shù);
* 定義在module頂層的類;
* 擁有dict()或setstate()的自定義類型;
https://stackoverflow.com/questions/8804830/python-multiprocessing-pickling-error
The problem is that the pool methods all use a queue.Queue to pass tasks to the worker processes. Everything that goes through the queue.Queue must be pickable, and foo.work is not picklable since it is not defined at the top level of the module.
It can be fixed by defining a function at the top level
大意是類中的方法不能被序列化,而進(jìn)程中的參數(shù)或者函數(shù)必須被序列化,所以報(bào)錯(cuò)
解決:
import multiprocessingdef func(x):return x*xclass someClass(object):def __init__(self,func):self.f = funcdef go(self):pool = multiprocessing.Pool(processes=4)#result = pool.apply_async(self.f, [10])#print result.get(timeout=1)print pool.map(self.f, range(10))a=someClass(func) a.go()defining a function at the top level
將進(jìn)程需要調(diào)用的函數(shù)變成定義在module頂層的函數(shù)
解決過(guò)程中又出現(xiàn)的問題:
1. con = multiprocessing.Process(target=self.connect, args=(k, metrics, request, return_dict, ))
同樣是PicklingError: Can’t pickle錯(cuò)誤
原因是:
雖然metrics的類型是dict但是里面的具體內(nèi)容是
'vmware_vm_net_transmitted_average': <prometheus_client.core.GaugeMetricFamily object at 0x7161650>
也會(huì)報(bào)PicklingError: Can’t pickle
可能是里里面的<prometheus_client.core.GaugeMetricFamily object at 0x7161650>不能被序列化
總結(jié)
以上是生活随笔為你收集整理的ERROR: Process pool report error: Can‘t pickle的全部?jī)?nèi)容,希望文章能夠幫你解決所遇到的問題。
- 上一篇: P3121 [USACO15FEB]审查
- 下一篇: 简单API接口签名验证