Optimizing Tensor Initialization in PyTorch: When to Use torch.ones and torch.new_ones

• Creates a new tensor filled with ones (value 1).
• Takes a tuple or list specifying the shape of the tensor as its argument.
• Independent of existing tensors: It doesn't inherit properties from any other tensor. You define the data type (dtype) and device (CPU or GPU) explicitly if needed.

Example:

import torch

# Create a 3x4 tensor filled with ones on CPU with float32 data type
tensor = torch.ones((3, 4), dtype=torch.float32, device=torch.device('cpu'))
print(tensor)

torch.new_ones (Method)

• A method specifically for existing tensors (x).
• Creates a new tensor with the same size (size) as the existing tensor (x).
• Inherits the data type (dtype) and device from the existing tensor (x).

# Assuming you have a tensor x defined earlier
new_tensor = x.new_ones(x.size())  # Inherits dtype and device from x
print(new_tensor)

Key Differences:

When to Use Which:

• Use torch.ones when you want to create a brand new tensor filled with ones, specifying its size, data type, and device as needed.
• Use torch.new_ones when you want to create a new tensor with the same size and properties (dtype, device) as an existing tensor. This is often convenient for operations like initialization or resetting values.

import torch

# Create a 2x3 tensor filled with ones on CPU with float32 data type (using torch.ones)
tensor1 = torch.ones((2, 3), dtype=torch.float32, device=torch.device('cpu'))
print(tensor1)

# Create a 1x4 tensor filled with ones on GPU with float64 data type (using torch.ones)
tensor2 = torch.ones((1, 4), dtype=torch.float64, device=torch.device('cuda'))  # Assuming you have a GPU
print(tensor2)

This code creates two tensors:

• tensor1: A 2x3 tensor of float32 type on the CPU
• tensor2: A 1x4 tensor of float64 type on the GPU (if available)

Example 2: Using torch.new_ones with an Existing Tensor

import torch

# Create a random tensor (example for illustration)
x = torch.randn(2, 2)  # This could be any existing tensor
print(x)

# Create a new tensor with the same size as x, inheriting its dtype and device (using torch.new_ones)
new_tensor = x.new_ones(x.size())
print(new_tensor)

This code:

1. Creates a random tensor x.
2. Uses x.new_ones(x.size()) to create a new tensor new_tensor with the same size (2x2) as x.
3. Since x.new_ones is called on x, the new tensor new_tensor will inherit the same data type (dtype) and device as x.

Additional Notes:

• If you don't specify device in torch.ones, it defaults to the current device (CPU or GPU).
• torch.randn used in Example 2 is for demonstration purposes. You can use any existing tensor with torch.new_ones.

• Modifies an existing tensor to be filled with ones.
• Useful when you want to reuse an existing tensor for efficiency.

import torch

# Create an empty tensor
tensor = torch.empty(3, 4)
print(tensor)  # Initially filled with random values

# Fill the tensor with ones (modifies tensor in-place)
tensor.fill_(1)
print(tensor)

Using torch.ones_like (Similar size and properties):

• Creates a new tensor with the same size and data type (dtype) as the provided tensor, filled with ones.
• Useful when you want a new tensor with properties matching an existing one.

import torch

# Assuming you have a tensor x defined earlier
new_tensor = torch.ones_like(x)
print(new_tensor)  # Same size and dtype as x, filled with ones

Using NumPy (if you're already using NumPy):

• Create a NumPy array filled with ones.
• Convert the NumPy array to a PyTorch tensor using torch.from_numpy.
• This can be efficient if you're working with both NumPy and PyTorch.

import torch
import numpy as np

# Create a NumPy array filled with ones
np_array = np.ones((2, 3))

# Convert NumPy array to PyTorch tensor
tensor = torch.from_numpy(np_array)
print(tensor)