*Memos:
- My post explains Dot and Matrix-vector multiplication in PyTorch.
- My post explains the functions and operators for Dot and Matrix multiplication and Element-wise calculation in PyTorch.
- My post explains matmul() and dot()
- My post explains mv(), mm() and bmm().
- My post explains add().
- My post explains sub() and mul().
- My post explains div().
- My post explains remainder() and fmod().
<Matrix multiplication(product)>
- Matrix multiplication is the multiplication of 2D or more tensors(arrays). *The 1st operand can be a 1D tensor(array) but not the last operand.
- The rule which you must follow to do matrix multiplication is the number of the columns of
Atensor(array) must match the number of the rows ofBtensor(array).
2D tensors(arrays):
<A> <B> [[a, b, c], x [[g, h, i, j], = [[ag+bk+co, ah+bl+cp, ai+bm+cq, aj+bn+cr], [d, e, f]] [k, l, m, n], [dg+ek+fo, dh+el+fp, di+em+fq, dj+en+fr]] [o, p, q, r]] 2 rows (3) rows (3) columns 4 columns [[2, -7, 4], x [[-5, 0, 8, -4], = [[-73, 18, 22, -37], [6, 3, -1]] [9, -2, -6, 7], [-3, -7, 39, -8]] [0, 1, -9, 5]] [[2x(-5)-7x9+4x0, 2x0-7x(-2)+4x1, 2x8-7x(-6)+4x(-9), 2x(-4)-7x7+4x5] [6x(-5)+3x9-1x0, 6x0+3x(-2)-1x1, 6x8+3x(-6)-1x(-9), 6x(-4)+3x7-1x5]] In PyTorch with matmul(), mm() or @.
*Memos:
-
matmul()or@can do dot, matrix-vector or matrix multiplication with two of 1D or more D tensors. -mm()can do matrix multiplication with two of 2D tensors:
import torch tensor1 = torch.tensor([[2, -7, 4], [6, 3, -1]]) tensor2 = torch.tensor([[-5, 0, 8, -4], [9, -2, -6, 7], [0, 1, -9, 5]]) torch.matmul(input=tensor1, other=tensor2) tensor1.matmul(other=tensor2) torch.mm(input=tensor1, mat2=tensor2) tensor1.mm(mat2=tensor2) tensor1 @ tensor2 # tensor([[-73, 18, 22, -37], [-3, -7, 39, -8]]) In NumPy with matmul(), dot() or @:
*Memos:
-
matmul()or@can do dot, matrix-vector or matrix multiplication with two of 1D or more D arrays. -
dot()can do dot, matrix-vector or matrix multiplication with two of 0D or more D arrays. *dot() is basically used to multiply two of 1D arrays.
import numpy array1 = numpy.array([[2, -7, 4], [6, 3, -1]]) array2 = numpy.array([[-5, 0, 8, -4], [9, -2, -6, 7], [0, 1, -9, 5]]) numpy.matmul(array1, array2) numpy.dot(array1, array2) array1.dot(array2) array1 @ array2 # array([[-73, 18, 22, -37], [-3, -7, 39, -8]]) A 1D and 3D tensor(array):
*The 3D tensor(array) of B has three of 2D tensors(arrays) which have 2 rows and 4 columns each.
<A> <B> [a, b] x [[[c, d, e, f], = [[(ac+bg), (ad+bh), (ae+bi), (af+bj)], [g, h, i, j]], [(ak+bo), (al+bp), (am+bq), (an+br)], [[k, l, m, n], [(as+bw), (at+bx), (au+by), (av+bz)]] [o, p, q, r]], [[s, t, u, v], [w, x, y, z]]] 1 row (2) rows (2) columns 4 columns [2, -7] x [[[-5, 0, 8, -4], = [[-73, 14, 58, -57], [9, -2, -6, 7]], [21, -26, 31, -4], [[0, 1, -9, 5], [40, 60, -1, -30]] [-3, 4, -7, 2]], [[6, 2, 3, -1], [-4, -8, 1, 4]]] [[2x(-5)-7x9, 2x0-7x(-2), 2x8-7x(-6), 2x(-4)-7x7], [2x0-7x(-3), 2x1-7x4, 2x(-9)-7x(-7), 2x5-7x2], [2x6-7x(-4), 2x2-7x(-8), 2x3-7x1, 2x(-1)-7x4]] In PyTorch with matmul() or @:
import torch tensor1 = torch.tensor([2, -7]) tensor2 = torch.tensor([[[-5, 0, 8, -4], [9, -2, -6, 7]], [[0, 1, -9, 5], [-3, 4, -7, 2]], [[6, 2, 3, -1], [-4, -8, 1, 4]]]) torch.matmul(input=tensor1, other=tensor2) tensor1.matmul(other=tensor2) tensor1 @ tensor2 # tensor([[-73, 14, 58, -57], # [21, -26, 31, -4], # [40, 60, -1, -30]]) In NumPy with matmul(), dot() or @:
import numpy array1 = numpy.array([2, -7]) array2 = numpy.array([[[-5, 0, 8, -4], [9, -2, -6, 7]], [[0, 1, -9, 5], [-3, 4, -7, 2]], [[6, 2, 3, -1], [-4, -8, 1, 4]]]) numpy.matmul(array1, array2) numpy.dot(array1, array2) array1.dot(array2) array1 @ array2 # array([[-73, 14, 58, -57], # [21, -26, 31, -4], # [40, 60, -1, -30]]) <Element-wise multiplication(product)>
- Element-wise multiplication is the multiplication of 0D or more D tensors(arrays).
- The rule which you must follow to do element-wise multiplication is 2 tensors(arrays) must have the same number of rows and columns.
1D tensors(arrays):
[a, b, c] x [d, e, f] = [ad, be, cf] 1 row 1 row 3 columns 3 columns [2, -7, 4] x [6, 3, -1] = [12, -21, -4] [2x6, -7x3, 4x(-1)] [2, -7, 4] x x x [6, 3, -1] || [12, -21, -4] In PyTorch with mul() or *. *mul() or * can do element-wise multiplication with two of 0D or more D tensors:
import torch tensor1 = torch.tensor([2, -7, 4]) tensor2 = torch.tensor([6, 3, -1]) torch.mul(input=tensor1, other=tensor2) tensor1.mul(other=tensor2) tensor1 * tensor2 # tensor([12, -21, -4]) In NumPy with multiply() or *. *multiply() or * can do element-wise multiplication with two of 0D or more D arrays.
import numpy array1 = numpy.array([2, -7, 4]) array2 = numpy.array([6, 3, -1]) numpy.multiply(array1, array2) array1 * array2 # array([12, -21, -4]) 2D tensors(arrays):
[[a, b, c], x [[g, h, i], = [[ag, bh, ci], [d, e, f]] [j, k, l]] [dj, ek, fl]] 2 rows 2 rows 3 columns 3 columns [[2, -7, 4], x [[-5, 0, 8], = [[10, 0, 32], [6, 3, -1]] [9, -2, -6]] [18, 18, 5]] [[2x(-5), -7x0, 4x8] [6x9, 3x(-2) -1x(-6)]] [[2, -7, 4], [6, 3, 5]] x x x x x x [[-5, 0, 8], [9, -2, -6]] || [[10, 0, 32], [18, 18, 5]] In PyTorch with * or mul():
import torch tensor1 = torch.tensor([[2, -7, 4], [6, 3, -1]]) tensor2 = torch.tensor([[-5, 0, 8], [9, -2, -6]]) torch.mul(input=tensor1, other=tensor2) tensor1.mul(other=tensor2) tensor1 * tensor2 # tensor([[-10, 0, 32], [54, -6, 6]]) In NumPy with * or multiply():
import numpy array1 = numpy.array([[2, -7, 4], [6, 3, -1]]) array2 = numpy.array([[-5, 0, 8], [9, -2, -6]]) numpy.multiply(array1, array2) array1 * array2 # array([[-10, 0, 32], [54, -6, 6]]) 
Top comments (0)