PyTorch - Tensor

NOTE : The Pytorch version that I am using for this tutorial is as follows.

>>> print(torch.__version__)

1.0.1

Creating a Tensor

The simplest tensor is a scalar, i.e single number. There are various ways to create a scalar type tensor.

t1 = torch.tensor(3)

t2 = torch.tensor(3.)

t3 = torch.tensor(3.0)

t4 = torch.tensor(3,dtype=torch.float64)

t1, t2, t3, t4 all store a single number 3, but the data type (i.e, the size of the memory to store the numbers) is different. You can check this by printing the types of each of these tensors.

print(t1.dtype,t2.dtype,t3.dtype,t4.dtype)

==> torch.int64 torch.float32 torch.float32 torch.float64

Next, you can create a vector or matrix type of tensors as follows.

t1 = torch.tensor([1,2,3])

t2 = torch.tensor([4,5,6]);

t3 = torch.tensor([[1,2,4],

[4,5,6]]);

Scalar Operations of Vectors

t1 = torch.tensor([1,2,3])

t2 = torch.tensor([4,5,6]);

t3 = t1+t2

print(t3)

==> tensor([5, 7, 9])

t4 = t1 * t2;

print(t4)

==> tensor([ 4, 10, 18])

Dot Product of Vectors

t1 = torch.tensor([1,2,3])

t2 = torch.tensor([4,5,6]);

t3 = torch.tensor([[1,2,4],

[4,5,6]]);

t4 = torch.dot(t1,t2) # Note that it does not require to transpose one of the vectors

print(t4)

==> tensor(32)

Dot Product of Matrices (Matrix Multiplication)

t1 = torch.tensor([1,2,3])

t2 = torch.tensor([4,5,6]);

t3 = torch.tensor([[1,2,4],

[4,5,6]]);

t4 = torch.tensor([[1,2,4],

[4,5,6],

[7,8,9]]);

t5 = torch.mm(t3,t3.t())

print(t5)

==> tensor([[21, 38],

[38, 77]])

t6 = torch.mm(t3.t(),t3)

print(t6)

==> tensor([[17, 22, 28],

[22, 29, 38],

[28, 38, 52]])

t7 = torch.mm(t4,t4)

print(t7)

==> tensor([[ 37, 44, 52],

[ 66, 81, 100],

[102, 126, 157]])

t8 = torch.mm(t4.t(),t4)

print(t8)

==> tensor([[ 66, 78, 91],

[ 78, 93, 110],

[ 91, 110, 133]])

Indexing Tensor Element

t1 = torch.tensor([1,2,3])

t2 = torch.tensor([4,5,6]);

t3 = torch.tensor([[1,2,3],

[4,5,6]]);

t4 = torch.tensor([[1,2,3],

[4,5,6],

[7,8,9]]);

t5 = torch.tensor([[1,2,3,4,5],

[6,7,8,9,10],

[11,12,13,14,15]]);

print(t1[1])

==> tensor(2)

print(t3[0,1])

==> tensor(2)

print(t3[1,0])

==> tensor(4)

print(t3[0])

==> tensor([1, 2, 3])

print(t3[0,:])

==> tensor([1, 2, 3])

print(t3[:,1])

==> tensor([2, 5])

print(t5[:,[1,3]])

==> tensor([[ 2, 4],

[ 7, 9],

[12, 14]])

print(t5[[1,2],:])

==> tensor([[ 6, 7, 8, 9, 10],

[11, 12, 13, 14, 15]])

print(t5[:,torch.arange(0,3)])

==> tensor([[ 1, 2, 3],

[ 6, 7, 8],

[11, 12, 13]])

Replacing Elements

t1 = torch.tensor([1,2,3])

t2 = torch.tensor([4,5,6]);

t3 = torch.tensor([[1,2,4],

[4,5,6]]);

t4 = torch.tensor([[1,2,3],

[4,5,6],

[7,8,9]]);

t5 = torch.tensor([[1,2,3,4,5],

[6,7,8,9,10],

[11,12,13,14,15]]);

t1[1] = 10;

print(t1)

==> tensor([ 1, 10, 3])

t3[0,1] = 10;

print(t3)

==> tensor([[ 1, 10, 4],

[ 4, 5, 6]])

t3[1,0] = 10;

print(t3)

==> tensor([[ 1, 10, 4],

[10, 5, 6]])

t3[0] = torch.tensor([10,11,12]);

print(t3)

==> tensor([[10, 11, 12],

[10, 5, 6]])

t3[1] = torch.arange(5,8);

print(t3)

==> tensor([[10, 11, 12],

[ 5, 6, 7]])

t4[0,:] = torch.tensor([10,11,12]);

print(t4)

==> tensor([[10, 11, 12],

[ 4, 5, 6],

[ 7, 8, 9]])

Reshaping Dimension

t1 = torch.tensor([1,2,3,4,5,6,7,8,9,10,11,12]);

t2 = torch.tensor([[1,2,3,4,5],

[6,7,8,9,10],

[11,12,13,14,15]]);

t3 = t1.view(3,4);

print(t3)

==> tensor([[ 1, 2, 3, 4],

[ 5, 6, 7, 8],

[ 9, 10, 11, 12]])

t4 = t1.view(4,3);

print(t4)

==> tensor([[ 1, 2, 3],

[ 4, 5, 6],

[ 7, 8, 9],

[10, 11, 12]])

t5 = t2.view(1,15);

print(t5)

==> tensor([[ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]])

t6 = t2.view(1,-1);

print(t6)

==> tensor([[ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]])

t7 = t2.view(-1,1);

print(t7)

==> tensor([[ 1],

[ 2],

[ 3],

[ 4],

[ 5],

[ 6],

[ 7],

[ 8],

[ 9],

[10],

[11],

[12],

[13],

[14],

[15]])