pytorch问题索引
20210623
RuntimeError: expected device cpu and dtype Float but got device cpu and dtype Bool ()
調(diào)試到對應(yīng)的位置 查看具體哪個(gè)變量的類型不對
20210622
https://blog.csdn.net/weixin_41041772/article/details/109820870
通俗理解torch.distributed.barrier()工作原理
20210614
https://blog.csdn.net/scut_salmon/article/details/82414730
optimizer.zero_grad()意思是把梯度置零,也就是把loss關(guān)于weight的導(dǎo)數(shù)變成0.
torch代碼解析 為什么要使用optimizer.zero_grad()
https://www.zhihu.com/question/303070254/answer/573504133
每個(gè)batch的梯度是單獨(dú)計(jì)算的 是按批次更新的 而不是按全部數(shù)據(jù)更新的
https://www.cnblogs.com/wanghui-garcia/p/11399053.html
pytorch torch.nn 實(shí)現(xiàn)上采樣——nn.Upsample
torch.nn.Module.apply(fn)
https://blog.csdn.net/m0_46653437/article/details/112649816
the parameters of a model (see also torch-nn-init).將一個(gè)函數(shù)fn遞歸地應(yīng)用到模塊自身以及該模塊的每一個(gè)子模塊(即在函數(shù).children()中返回的子模塊).該方法通常用來初始化一個(gè)模型中的參數(shù)(另見torch-nn-init部分的內(nèi)容).
https://www.cnblogs.com/wmc258/p/14539677.html
torch.nn.init.normal(tensor, mean=0, std=1)
從給定均值和標(biāo)準(zhǔn)差的正態(tài)分布N(mean, std)中生成值,填充輸入的張量或變量參數(shù):
tensor – n維的torch.Tensor
mean – 正態(tài)分布的均值
std – 正態(tài)分布的標(biāo)準(zhǔn)差
torch.nn.Init.normal_()的用法
20210609
https://www.zhihu.com/question/288350769
torch.manual_seed(1)
https://blog.csdn.net/zziahgf/article/details/78489562
計(jì)算 Log-Sum-Exp
https://www.zhihu.com/question/402741396
clone()
https://blog.csdn.net/kdongyi/article/details/108180250
Pytorch中contiguous()函數(shù)理解
相當(dāng)于深拷貝
self.optimizer = self._decay_learning_rate(epoch=epoch - 1, init_lr=self.config.learning_rate)#直接改寫為adam# self.optimizer=torch.optim.Adam(self.optimizer.param_groups,lr=self.config.learning_rate)self.optimizer=torch.optim.Adam(self.model.parameters,lr=self.config.learning_rate)
設(shè)置優(yōu)化器
https://blog.csdn.net/qq_40367479/article/details/82530324
decay_rate是衰減指數(shù),可設(shè)為略小于1的值,比如0.98。global_step是當(dāng)前的迭代輪數(shù),decay_steps是你想要每迭代多少輪就衰減的度量值,可叫作衰減速度。比如decay_steps = 1000,意味著當(dāng)global_step達(dá)到1000時(shí)就給learning_rate乘上0.98的一次方,達(dá)到2000就乘上0.98的二次方,以此類推,不斷縮小學(xué)習(xí)率。
另一種方法
# 自己寫的學(xué)習(xí)率衰減函數(shù)def _decay_learning_rate(self, epoch, init_lr):"""lr decay Args:epoch: int, epoch init_lr: initial lr"""lr = init_lr / (1 + self.config.lr_rate_decay * epoch)for param_group in self.optimizer.param_groups:param_group['lr'] = lrreturn self.optimizer
學(xué)習(xí)率衰減(learning rate decay)
https://blog.csdn.net/jining11/article/details/103825447
pytorch加載模型時(shí)使用map_location來在CPU、GPU間輾轉(zhuǎn)騰挪
https://blog.csdn.net/qq_39852676/article/details/106326580
pytorch中model.to(device)和map_location=device的區(qū)別
20210608
https://blog.csdn.net/xu380393916/article/details/97280035
pytorch 中的 forward 的使用與解釋
https://blog.csdn.net/u011501388/article/details/84062483
PyTorch之前向傳播函數(shù)forward
https://blog.csdn.net/qq_41375609/article/details/102891831
hook
標(biāo)量沒有維度
列表有維度
20210422
解決安裝torchvision自動(dòng)更新torch到最新版本
不使用pip install torchvision0.4.0
使用pip install --no-deps torchvision0.4.0
完美解決安裝完torchvision之后繼續(xù)更新torch到1.2.0版本(目前最新版)。
20210420
https://zhuanlan.zhihu.com/p/108342960
pytorch項(xiàng)目
20210128
https://blog.csdn.net/kyle1314608/article/details/113758100
pytorch常用代碼
RuntimeError: [enforce fail at ..\c10\core\CPUAllocator.cpp:72] data. DefaultCPUAllocator: not enough memory: you tried to allocate 4792320 bytes. Buy new RAM!
報(bào)這個(gè)錯(cuò)的時(shí)候 可以更改bert預(yù)訓(xùn)練模型的json 參數(shù)
比如少用幾層 每層少用幾個(gè)頭
也就是少用一些參數(shù)
tf_model.h5,model.ckpt.index
和
pytorch_model.bin
是并列關(guān)系 用pytorch 就用下面
用tf就用上面的訓(xùn)練模型
20201219
https://blog.csdn.net/kyle1314608/article/details/111405214
tensor和模型 保存與加載 PyTorch
20201210
RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn
原因是變量沒有設(shè)置為require_grad=True 沒有設(shè)置梯度更新為True
或者叫梯度不更新的問題
import torch
from torch import Variable
x=torch.randn(3)
x=Variable(x,requires_grad=True)#生成變量
print(x)#輸出x的值
y=x*2
y.backward(torch.FloatTensor([1,0.1,0.01]))#自動(dòng)求導(dǎo)
print(x.grad)#求對x的梯度
通過上面把任何變量設(shè)置需要梯度更新
https://blog.csdn.net/kyle1314608/article/details/110959432
numpy轉(zhuǎn)tensor 并有梯度更新
dim 維度的方向
https://www.cnblogs.com/jiangkejie/p/9981707.html
detach
https://pytorch-cn.readthedocs.io/zh/latest/package_references/torch-autograd
https://ptorch.com/docs/2/developer-docs
說明書 手冊 指南
20201202
https://blog.csdn.net/kyle1314608/article/details/110489180
數(shù)據(jù)類型轉(zhuǎn)換
https://blog.csdn.net/kyle1314608/article/details/110486424
#測試
aa=torch.zeros([2,2])
bb=torch.zeros([2,3])
cc=torch.cat((aa,bb),1)
#測試
torch 拼接
某個(gè)矩陣列數(shù)要增加 且填充為零 padding
loss = -(target * logsigmoid(input) + (1 - target) * logsigmoid(-input))
RuntimeError: expected device cpu and dtype Float but got device cpu and dtype Long
把target 和 input 由 target.long() 改成 target.float()
dataTar = torch.from_numpy(dataTar)
TypeError: can’t convert np.ndarray of type numpy.object_. The only supported types are: float64, float32, float16, int64, int32, int16, int8, uint8, and bool.
把ndarray里面字符型的元素轉(zhuǎn)換為數(shù)值
Python報(bào)錯(cuò)xxx.whl is not a supported wheel on this platform
linux 現(xiàn)在安裝pytorch版本和本地安裝的python版本不一致
bug提示某個(gè)對象的某種屬性無效了,一般情況下是因?yàn)檐浖姹镜淖兏斐傻?br /> 降低軟件版本,或者把屬性改成當(dāng)前的屬性
tensor 轉(zhuǎn) array 再轉(zhuǎn)list
pred=np.array(pred).tolist()
RuntimeError: Assertion cur_target 0 cur_target n_classes failed
網(wǎng)絡(luò)分類標(biāo)簽應(yīng)該跟真實(shí)分類標(biāo)簽一樣多
分類標(biāo)簽必須從零開始
分類標(biāo)簽必須連續(xù) 1 2 34 等
https://blog.csdn.net/qq_28418387/article/details/97971328
pytorch 構(gòu)建tensor
https://blog.csdn.net/qq_42146630/article/details/98784062
Pytorch手寫數(shù)字識(shí)別(一)
https://blog.csdn.net/Lee_lg/article/details/103901632
Python模塊問題:ImportError: cannot import name ‘PILLOW_VERSION’ from ‘PIL’
https://blog.csdn.net/Albert201605/article/details/79893585
MNIST 數(shù)據(jù)集 轉(zhuǎn)換為CSV
https://blog.csdn.net/zhe_csdn/article/details/93655053
pytorch的Tensor變量之間的轉(zhuǎn)換
RuntimeError: Assertion `cur_target >= 0 && cur_target < n_classes’ failed. at
https://blog.csdn.net/m0_37369043/article/details/102632297
標(biāo)簽必須從零開始
https://pytorch-cn.readthedocs.io/zh/latest/package_references/Tensor/
pytorch中文文檔 重點(diǎn)
總結(jié)
以上是生活随笔為你收集整理的pytorch问题索引的全部內(nèi)容,希望文章能夠幫你解決所遇到的問題。
- 上一篇: 宿主机虚拟机文件复制 apt-get
- 下一篇: 比Momentum更快:揭开Nester