pytorch注意事项和学习笔记

pytorch注意事项和学习笔记

查看torch版本：torch.__version__
查看cuda版本：torch.version.cuda
torch.cuda.is_available() # 判断gpu是否可用

tensor和numpy相互转化

torch.from_numpy(np_data)
torch_data.numpy()

CPU和GPU的Tensor之间转换

cpu2gpu：data.cuda()
gpu2cpu：data.cpu()

设置随机种子复现实验结果

（1）Pytorch：

torch.manual_seed(seed)#为CPU设置随机种子
torch.cuda.manual_seed(seed)#为当前GPU设置随机种子
torch.cuda.manual_seed_all(seed)#为所有GPU设置随机种子

（2）Python & Numpy

random.seed(seed)
np.random.seed(seed)

（3）CUDNN
cudnn中对卷积操作进行了优化，牺牲了精度来换取计算效率。如果需要保证可重复性，可以使用如下设置:

from torch.backendsimport cudnn
cudnn.benchmark=False# if benchmark=True, deterministic will be False
cudnn.deterministic=True

用gpu训练模型：

device= torch.device("cuda:id"if torch.cuda.is_available()else"cpu")# single-gpu training
model.to(device)# multi-gpu training
model= nn.DataParallel(model) 
model.to(device)# data processing
data.to(device)
data.to(device=id)

momentum parameter：相关博客

torch.max

torch.max(a) # returns the maximum value of all elements
torch.max(a, 0) # returns the maximum value of each column and corresponding index
torch.max(a, 1) # returns the maximum value of each row and corresponding index

squeeze & unsqueeze

torch.squeeze(input, dim=None, out=None)$\to$ Tensor
$\to$dim：if given, the input will be squeezed only in this dimension
torch.unsqueeze(input, dim, out=None)$\to$ Tensor
$\to$Returns a new tensor with a dimension of size one inserted at the specified position

模型保存与加载

# save the whole network
torch.save(model,'model.pth')
model= torch.load('model.pth')# only save the parameters of network(recommend)
torch.save(model.state_dict(),'params.pth')
model.load_state_dict(torch.load('params.pth'))

GPU和CPU模型相互加载