Posted on Mar 21, 2024 • Edited on Nov 5, 2024

Functions and operators for Dot and Matrix multiplication and Element-wise calculation in PyTorch

#python #pytorch #function #matrixmultiplication

*Memos:

My post explains Matrix and Element-wise multiplication in PyTorch.
My post explains Dot and Matrix-vector multiplication in PyTorch.
My post explains matmul() and dot()
My post explains mv(), mm() and bmm().
My post explains add().
My post explains sub() and mul().
My post explains div().
My post explains remainder() and fmod().

<Dot multiplication>

dot() can multiply 1D tensors:

import torch tensor1 = torch.tensor([2, 7, 4]) # 1D tensor tensor2 = torch.tensor([6, 3, 5]) # 1D tensor  torch.dot(tensor1, tensor2) tensor1.dot(tensor2) # tensor(53)

matmul() or @ can multiply 1D or more D tensors:

import torch tensor1 = torch.tensor([2, 7, 4]) # 1D tensor tensor2 = torch.tensor([6, 3, 5]) # 1D tensor  torch.matmul(tensor1, tensor2) tensor1.matmul(tensor2) tensor1 @ tensor2 # tensor(53)

<Matrix-vector multiplication>

mv() can multiply a 2D and 1D tensor:

import torch tensor1 = torch.tensor([[2, 7, 4], [8, 3, 2]]) # 2D tensor tensor2 = torch.tensor([5, 0, 8]) # 1D tensor  torch.mv(tensor1, tensor2) tensor1.mv(tensor2) # tensor([42, 56])

matmul() or @ can multiply 1D or more D tensors:

import torch tensor1 = torch.tensor([[2, 7, 4], [8, 3, 2]]) # 2D tensor tensor2 = torch.tensor([5, 0, 8]) # 1D tensor  torch.matmul(tensor1, tensor2) tensor1.matmul(tensor2) tensor1 @ tensor2 # tensor([42, 56])

<Matrix multiplication>

mm() can multiply 2D tensors:

import torch tensor1 = torch.tensor([[2, 7, 4], [8, 3, 2]]) # 2D tensor tensor2 = torch.tensor([[5, 0, 8, 6], # 2D tensor  [3, 6, 1, 7], [1, 4, 9, 2]]) torch.mm(tensor1, tensor2) tensor1.mm(tensor2) # tensor([[35, 58, 59, 69], [51, 26, 85, 73]])

bmm() can multiply 3D tensors:

import torch tensor1 = torch.tensor([[[2, 7]], [[8, 3]]]) # 3D tensor tensor2 = torch.tensor([[[5, 9], [3, 6]], # 3D tensor  [[7, 2], [1, 4]]]) torch.bmm(tensor1, tensor2) tensor1.bmm(tensor2) # tensor([[[31, 60]], [[59, 28]]])

matmul() or @ can multiply 1D or more D tensors by dot or matrix multiplication:

import torch tensor1 = torch.tensor([[2, 7], [8, 3]]) # 2D tensor tensor2 = torch.tensor([[[[5, 9], [3, 6]], [[7, 2], [1, 4]]], [[[6, 0], [4, 6]], [[2, 9], [8, 1]]]]) # 4D tensor torch.matmul(tensor1, tensor2) tensor1.matmul(tensor2) tensor1 @ tensor2 # tensor([[[[31, 60], [49, 90]], [[21, 32], [59, 28]]], # [[[40, 42], [60, 18]], [[60, 25], [40, 75]]]])

<Element-wise calculation>

mul() or * can do multiplication with 0D or more D tensors. *mul() and multiply() are the same because multiply() is the alias of mul():

import torch tensor1 = torch.tensor([2, 7, 4]) # 1D tensor tensor2 = torch.tensor([6, 3, 5]) # 1D tensor  torch.mul(tensor1, tensor2) tensor1.mul(tensor2) tensor1 * tensor2 # tensor([12, 21, 20])

div() or / can do division with 0D or more D tensors: *Memos:
divide() is the alias of div().
true_divide() is the alias of div() with rounding_mode=None.
floor_divide() is the same as div() with rounding_mode="trunc" as long as I experimented:

import torch tensor1 = torch.tensor([2, 7, 4]) # 1D tensor tensor2 = torch.tensor([6, 3, 5]) # 1D tensor  torch.div(tensor1, tensor2) tensor1.div(tensor2) tensor1 / tensor2 # tensor([0.3333, 2.3333, 0.8000])

remainder() or % can do modulo(mod) calculation with 0D or more D tensors:

import torch tensor1 = torch.tensor([2, 7, 4]) # 1D tensor tensor2 = torch.tensor([6, 3, 5]) # 1D tensor  torch.remainder(tensor1, tensor2) tensor1.remainder(tensor2) tensor1 % tensor2 # tensor([2, 1, 4])

add() or + can do addition with 0D or more D tensors:

import torch tensor1 = torch.tensor([2, 7, 4]) # 1D tensor tensor2 = torch.tensor([6, 3, 5]) # 1D tensor  torch.add(tensor1, tensor2) tensor1.add(tensor2) tensor1 + tensor2 # tensor([8, 10, 9])

sub() or - can do subtraction with 0D or more D tensors. *sub() and subtract() are the aliases of sub():

import torch tensor1 = torch.tensor([2, 7, 4]) # 1D tensor tensor2 = torch.tensor([6, 3, 5]) # 1D tensor  torch.subtract(tensor1, tensor2) tensor1.subtract(tensor2) tensor1 - tensor2 # tensor([-4, 4, -1])