oneDNN operators need to support 0D Tensor #51364
Description
Backgroud
Paddle plan to support 0D Tensor for API fully.
For 0D Tensor, first describe its concept from a mathematical point of view:
- 0D Tensor represents a scalar Tensor, which corresponds to Numpy's 0D array, which can be expressed as np.array (10.), with shape of [], dimension of 0, and size of 1.
- 1D Tensor represents a vector Tensor, which corresponds to the Numpy's 1D array. If there is only one element, it can be expressed as np.array ([10.]), with shape of [1], dimension of 1, and size of 1.
Scalar Tensor is easily confused with vector Tensor with shape [1]. Although the number of elements are the same, the mathematical definition is completely different. If shape=[1] is forcibly used to represent shape=[], the scalar Tensor and vector Tensor cannot be distinguished. There is a serious diff with the existing mathematical semantics and the general calculation rules in the industry.
Feature Description
At present, many APIs have supported 0D in CPU/GPU/XPU/NPU, but there are still some onednn operators that may not support it.
In total, the following 35 oneDNN kernels involve the usage of 0D. There may not be many problems, or there may be a few public problems, or some kernels themselves support them already. And it's need to add thses 0D usage test in OpTest. Like this: #51265 .
| kernel | 0D Usage need to be supported |
|---|---|
| abs | element-wise unary, 0D->0D |
| elu | element-wise unary, 0D->0D |
| exp | element-wise unary, 0D->0D |
| gelu | element-wise unary, 0D->0D |
| hardswish | element-wise unary, 0D->0D |
| leaky_relu | element-wise unary, 0D->0D |
| mish | element-wise unary, 0D->0D |
| relu | element-wise unary, 0D->0D |
| relu6 | element-wise unary, 0D->0D |
| sigmoid | element-wise unary, 0D->0D |
| sqrt | element-wise unary, 0D->0D |
| swish | element-wise unary, 0D->0D |
| tanh | element-wise unary, 0D->0D |
| softplus | element-wise unary, 0D->0D |
| transpose | 0D transpose return itself, perm must be [] |
| cast | element-wise unary, 0D->0D |
| clip | element-wise unary, 0D->0D |
| scale | element-wise unary, 0D->0D |
| expand | 0D can be expanded to 0D or higher dimensions |
| softmax | when input 0D, outputs is 0D and value is 1.0, grad is 0D and value is 0.0 |
| log_softmax | when input 0D, outputs is 0D and value is 0.0, grad is 0D and value is 0.0 |
| prelu | x and alpha can be 0D, and output is 0D |
| reshape | []/[1]/[1, 1]/[1, 1, 1] can be reshape to each other |
| stack | stack multiple 0D to 1D |
| shape | input can be 0D Tensor, output is [] with shape [0] |
| add | element-wise binary, should support broadcast. 0D+0D->0D; 0D+ND->ND; ND+0D->ND |
| substract | element-wise binary, should support broadcast. 0D+0D->0D; 0D+ND->ND; ND+0D->ND |
| multiply | element-wise binary, should support broadcast. 0D+0D->0D; 0D+ND->ND; ND+0D->ND |
| divide | element-wise binary, should support broadcast. 0D+0D->0D; 0D+ND->ND; ND+0D->ND |
| add_n | element-wise multiary, each input can be 0D |
| kernel | 0D Usage need to be supported |
|---|---|
| reduce_max | 1) x can be 0D, axis is [], and output is 0D; 2) x is ND, and output is 0D |
| reduce_min | 1) x can be 0D, axis is [], and output is 0D; 2) x is ND, and output is 0D |
| reduce_mean | 1) x can be 0D, axis is [], and output is 0D; 2) x is ND, and output is 0D |
| reduce_sum | 1) x can be 0D, axis is [], and output is 0D; 2) x is ND, and output is 0D |
| squeeze | compression dimension. The input and output can be 0D, depending on the specific compression dimension. such as 1D.squeeze->0D, 0D.squeeze->0D. |
| full | when shape=[], will output a 0D Tensor |
| gaussian | when shape=[], will output a 0D Tensor |
Step1： First of all, it's need to add 0D case in MKLDNN OpTest Like https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/fluid/tests/unittests/test_elementwise_add_op.py#L112-L133 .
About how to write 0D unittest case, Maybe you can refer to the https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/fluid/tests/unittests/test_zero_dim_tensor.py and imitate it into MKLDNN OpTest.
Step2： If the 0D case in OpTest fails, it's need to modify the oneDNN kernel to support 0D case. If success, nothing else will do and only need to push code.
#51265 just try, because onednn full kernel failures when shape=[].
Due to the release of 2.5rc on 2022/4, 0D is expected to be released in this paddle version. It's hoped to support 0D of oneDNN kernel above table in that time. Thank all of you very much.