Indexing and slicing are essential for manipulating and accessing specific parts of tensors. PyTorch provides flexible ways to index and slice tensors to perform various operations.
Basic Indexing
You can index tensors similarly to how you index lists or NumPy arrays.
Indexing a 1D Tensor:
import torch
tensor = torch.tensor([1, 2, 3, 4, 5])
print(tensor[0]) # Output: 1
print(tensor[2]) # Output: 3
Indexing a 2D Tensor:
matrix = torch.tensor([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
print(matrix[0, 0]) # Output: 1
print(matrix[1, 2]) # Output: 6
Slicing
Slicing allows you to access a range of elements in a tensor.
Slicing a 1D Tensor
print(tensor[1:4]) # Output: tensor([2, 3, 4])
print(tensor[:3]) # Output: tensor([1, 2, 3])
print(tensor[3:]) # Output: tensor([4, 5])
Slicing a 2D Tensor:
print(matrix[:, 1]) # Output: tensor([2, 5, 8]) - All rows, second column
print(matrix[1, :]) # Output: tensor([4, 5, 6]) - Second row, all columns
print(matrix[:2, 1:]) # Output: tensor([[2, 3], [5, 6]]) - First two rows, second and third columns
Advanced Indexing
Advanced indexing techniques allow for more complex operations.
Boolean Indexing:
mask = tensor > 3
print(tensor[mask]) # Output: tensor([4, 5])
Indexing with Tensors:
indices = torch.tensor([0, 2, 4])
print(tensor[indices]) # Output: tensor([1, 3, 5])
Tensor Reshaping
Reshaping tensors is a common operation in deep learning, allowing you to change the shape of a tensor without altering its data. PyTorch provides several methods for reshaping tensors, such as view, reshape, and transpose.
Using view
The view method returns a new tensor with the same data but a different shape. The total number of elements must remain the same.
Reshape a 1D Tensor to 2D:
import torch
tensor = torch.tensor([1, 2, 3, 4, 5, 6])
reshaped_tensor = tensor.view(2, 3)
print(reshaped_tensor)
Reshape a 2D Tensor to 3D:
matrix = torch.tensor([[1, 2, 3], [4, 5, 6]])
reshaped_tensor = matrix.view(2, 1, 3)
print(reshaped_tensor)
Using reshape
The reshape method works similarly to view but can handle cases where the new shape requires a copy of the data.
Transpose a 2D Tensor:
matrix = torch.tensor([[1, 2, 3], [4, 5, 6]])
transposed_matrix = matrix.transpose(0, 1)
print(transposed_matrix)
Transpose a 3D Tensor:
tensor_3d = torch.tensor([[[1, 2], [3, 4]], [[5, 6], [7, 8]]])
transposed_tensor_3d = tensor_3d.transpose(0, 1)
print(transposed_tensor_3d)
Example of Reshaping in a Neural Network
Reshaping tensors is often used in neural network layers, such as flattening the output of a convolutional layer before passing it to a fully connected layer.
Flattening a Tensor:
conv_output = torch.randn(10, 3, 3) # Example output from a convolutional layer
flattened_output = conv_output.view(conv_output.size(0), -1)
print(flattened_output.shape) # Output: torch.Size([10, 9])
