前言

我通過👇復(fù)寫了模型的state_dict方法，具體就是給其增加了dynamic_mask（字典類型，里面是tensor），allocated_neuron_num（整型）。

def state_dict(self, destination=None, prefix='', keep_vars=False):state_dict = super().state_dict(destination, prefix, keep_vars)state_dict['model.dynamic_mask'] = gloVar.dynamic_maskstate_dict['model.allocated_neuron_num'] = gloVar.allocated_neuron_numreturn state_dict

結(jié)果報(bào)錯：

File "/data3/syxu/sparsenmt_exp/sparsenmt/fairseq/fairseq/models/distributed_fairseq_model.py", line 58, in DistributedFairseqModelwrapped_model = DistributedDataParallel(File "/data3/syxu/anaconda3/envs/torch18/lib/python3.8/site-packages/torch/nn/parallel/distributed.py", line 580, in __init__self._sync_params_and_buffers(authoritative_rank=0)File "/data3/syxu/anaconda3/envs/torch18/lib/python3.8/site-packages/torch/nn/parallel/distributed.py", line 597, in _sync_params_and_buffersself._distributed_broadcast_coalesced(File "/data3/syxu/anaconda3/envs/torch18/lib/python3.8/site-packages/torch/nn/parallel/distributed.py", line 1334, in _distributed_broadcast_coalesceddist._broadcast_coalesced( TypeError: _broadcast_coalesced(): incompatible function arguments. The following argument types are supported:1. (process_group: torch._C._distributed_c10d.ProcessGroup, tensors: List[at::Tensor], buffer_size: int, src: int = 0) -> None

解決

不使用報(bào)錯中顯示的DistributedDataParallel。根據(jù)文檔，這個在fairseq中體現(xiàn)為–ddp-backend參數(shù)。
報(bào)錯時–ddp-backend=pytorch_ddp（默認(rèn)），改為legacy_ddp、no_c10d都不會再報(bào)錯。

參考

https://fairseq.readthedocs.io/en/latest/command_line_tools.html
https://blog.csdn.net/j___t/article/details/104368597

總結(jié)

以上是生活随笔為你收集整理的[fairseq] 报错：TypeError: _broadcast_coalesced(): incompatible function arguments的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯，歡迎將生活随笔推薦給好友。