Dropout() in PyTorch

This content originally appeared on DEV Community and was authored by Super Kai (Kazuya Ito)

*Memos:

My post explains Dropout Layer.
My post explains manual_seed().
My post explains requires_grad.

Dropout() can get the 0D or more D tensor of the zero or more elements randomly zeroed or multiplied from the 0D or more D tensor of zero or more elements as shown below:

*Memos:

The 1st argument for initialization is p(Optional-Default:0.5-Type:float): *Memos:
- It's the probability of an element to be zeroed.
- It must be 0 <= x <= 1.
The 2nd argument for initialization is inplace(Optional-Default:False-Type:bool):
- It does in-place operation.
- Keep it False because it's problematic with True.
The 1st argument is input(Required-Type:tensor of float).
The tensor's requires_grad which is False by default is not set to True by Dropout().

import torch
from torch import nn

tensor1 = torch.tensor([8., -3., 0., 1., 5., -2.])

tensor1.requires_grad
# False

torch.manual_seed(7)

dropout1 = nn.Dropout()
tensor2 = dropout1(input=tensor1)
tensor2
# tensor([16., -0., 0., 2., 10., -4.])

tensor2.requires_grad
# False

dropout1
# Dropout(p=0.5, inplace=False)

dropout1.p
# 0.5

dropout1.inplace
# False

torch.manual_seed(7)

dropout2 = nn.Dropout()
dropout2(input=tensor2)
# tensor([32., -0., 0., 4., 20., -8.])

torch.manual_seed(7)

dropout = nn.Dropout(p=0.5, inplace=False)
dropout(input=tensor1)
# tensor([16., -0., 0., 2., 10., -4.])

torch.manual_seed(7)

dropout = nn.Dropout(p=0.8)
dropout(input=tensor1)
# tensor([40., -0., 0., 0., 25., -0.])

torch.manual_seed(7)

dropout = nn.Dropout(p=0.3)
dropout(input=tensor1)
# tensor([11.4286, -0.0000, 0.0000, 1.4286, 7.1429, -2.8571])

my_tensor = torch.tensor([[8., -3., 0.],
                          [1., 5., -2.]])
torch.manual_seed(7)

dropout = nn.Dropout()
dropout(input=my_tensor)
# tensor([[16., -0., 0.],
#         [2., 10., -4.]])

torch.manual_seed(7)

dropout = nn.Dropout(p=0.8)
dropout(input=my_tensor)
# tensor([[40., -0., 0.],
#         [0., 25., -0.]])

torch.manual_seed(7)

dropout = nn.Dropout(p=0.3)
dropout(input=my_tensor)
# tensor([[11.4286, -0.0000, 0.0000],
#         [1.4286, 7.1429, -2.8571]])

my_tensor = torch.tensor([[8.], [-3.], [0.],
                          [1.], [5.], [-2.]])
torch.manual_seed(7)

dropout = nn.Dropout()
dropout(input=my_tensor)
# tensor([[16.], [-0.], [0.], [2.], [10.], [-4.]])

torch.manual_seed(7)

dropout = nn.Dropout(p=0.8)
dropout(input=my_tensor)
# tensor([[40.], [-0.], [0.], [0.], [25.], [-0.]])

torch.manual_seed(7)

dropout = nn.Dropout(p=0.3)
dropout(input=my_tensor)
# tensor([[11.4286], [-0.0000], [0.0000], [1.4286], [7.1429], [-2.8571]])

my_tensor = torch.tensor([[[8., -3., 0.],
                           [1., 5., -2.]]])
torch.manual_seed(7)

dropout = nn.Dropout()
dropout(input=my_tensor)
# tensor([[[16., -0., 0.],
#          [2., 10., -4.]]])

This content originally appeared on DEV Community and was authored by Super Kai (Kazuya Ito)

Print Share Comment Cite Upload Translate Updates

APA

Super Kai (Kazuya Ito) | Sciencx (2024-09-16T19:31:50+00:00) Dropout() in PyTorch. Retrieved from https://www.scien.cx/2024/09/16/dropout-in-pytorch/

MLA

" » Dropout() in PyTorch." Super Kai (Kazuya Ito) | Sciencx - Monday September 16, 2024, https://www.scien.cx/2024/09/16/dropout-in-pytorch/

HARVARD

Super Kai (Kazuya Ito) | Sciencx Monday September 16, 2024 » Dropout() in PyTorch., viewed ,<https://www.scien.cx/2024/09/16/dropout-in-pytorch/>

VANCOUVER

Super Kai (Kazuya Ito) | Sciencx - » Dropout() in PyTorch. [Internet]. [Accessed ]. Available from: https://www.scien.cx/2024/09/16/dropout-in-pytorch/

CHICAGO

" » Dropout() in PyTorch." Super Kai (Kazuya Ito) | Sciencx - Accessed . https://www.scien.cx/2024/09/16/dropout-in-pytorch/

IEEE

" » Dropout() in PyTorch." Super Kai (Kazuya Ito) | Sciencx [Online]. Available: https://www.scien.cx/2024/09/16/dropout-in-pytorch/. [Accessed: ]

rf:citation

» Dropout() in PyTorch | Super Kai (Kazuya Ito) | Sciencx | https://www.scien.cx/2024/09/16/dropout-in-pytorch/ |

Please log in to upload a file.

There are no updates yet.
Click the Upload button above to add an update.

You must be logged in to translate posts. Please log in or register.

Related Posts