Skip to content

Conversation

@hanlintang
Copy link
Contributor

@hanlintang hanlintang commented Apr 30, 2025

PR Category

User Experience

PR Types

Bug fixes

Description

  1. frobenius_norm支持0-size tensor;
  2. 增加matrix_norm的OpTest;
  3. fix paddle3.0.0下需要enable_static才能通过的单测

运行单元测试显示:

test/legacy_test/test_norm_all.py::TestFrobeniusNormOp::test_check_grad PASSED [ 1%] test/legacy_test/test_norm_all.py::TestFrobeniusNormOp::test_check_output PASSED [ 3%] test/legacy_test/test_norm_all.py::TestFrobeniusNormOp2::test_check_grad PASSED [ 5%] test/legacy_test/test_norm_all.py::TestFrobeniusNormOp2::test_check_output PASSED [ 7%] test/legacy_test/test_norm_all.py::TestFrobeniusNormOpZeroSize::test_check_grad PASSED [ 9%] test/legacy_test/test_norm_all.py::TestFrobeniusNormOpZeroSize::test_check_output PASSED [ 11%] test/legacy_test/test_norm_all.py::TestFrobeniusNormOpZeroSize2::test_check_grad PASSED [ 13%] test/legacy_test/test_norm_all.py::TestFrobeniusNormOpZeroSize2::test_check_output PASSED [ 15%] test/legacy_test/test_norm_all.py::TestFrobeniusNormOpZeroSize3::test_check_grad PASSED [ 17%] test/legacy_test/test_norm_all.py::TestFrobeniusNormOpZeroSize3::test_check_output PASSED [ 19%] test/legacy_test/test_norm_all.py::TestFrobeniusNormOpZeroSize4::test_check_grad PASSED [ 21%] test/legacy_test/test_norm_all.py::TestFrobeniusNormOpZeroSize4::test_check_output PASSED [ 23%] test/legacy_test/test_norm_all.py::TestPnormOp::test_check_grad PASSED [ 25%] test/legacy_test/test_norm_all.py::TestPnormOp::test_check_output PASSED [ 27%] test/legacy_test/test_norm_all.py::TestPnormOp2::test_check_grad PASSED [ 29%] test/legacy_test/test_norm_all.py::TestPnormOp2::test_check_output PASSED [ 31%] test/legacy_test/test_norm_all.py::TestPnormOp3::test_check_grad PASSED [ 33%] test/legacy_test/test_norm_all.py::TestPnormOp3::test_check_output PASSED [ 35%] test/legacy_test/test_norm_all.py::TestPnormOp4::test_check_grad PASSED [ 37%] test/legacy_test/test_norm_all.py::TestPnormOp4::test_check_output PASSED [ 39%] test/legacy_test/test_norm_all.py::TestPnormOp5::test_check_grad PASSED [ 41%] test/legacy_test/test_norm_all.py::TestPnormOp5::test_check_output PASSED [ 43%] test/legacy_test/test_norm_all.py::TestPnormOp6::test_check_grad PASSED [ 45%] test/legacy_test/test_norm_all.py::TestPnormOp6::test_check_output PASSED [ 47%] test/legacy_test/test_norm_all.py::TestPnormOpZeroSize::test_check_grad PASSED [ 49%] test/legacy_test/test_norm_all.py::TestPnormOpZeroSize::test_check_output PASSED [ 50%] test/legacy_test/test_norm_all.py::TestPnormOpZeroSize2::test_check_grad PASSED [ 52%] test/legacy_test/test_norm_all.py::TestPnormOpZeroSize2::test_check_output PASSED [ 54%] test/legacy_test/test_norm_all.py::TestPnormOpZeroSize3::test_check_grad PASSED [ 56%] test/legacy_test/test_norm_all.py::TestPnormOpZeroSize3::test_check_output PASSED [ 58%] test/legacy_test/test_norm_all.py::TestPnormOpZeroSize4::test_check_grad PASSED [ 60%] test/legacy_test/test_norm_all.py::TestPnormOpZeroSize4::test_check_output PASSED [ 62%] test/legacy_test/test_norm_all.py::TestPnormOp_Fp16::test_check_grad PASSED [ 64%] test/legacy_test/test_norm_all.py::TestPnormOp_Fp16::test_check_output PASSED [ 66%] test/legacy_test/test_norm_all.py::TestPnormOp2_Fp16::test_check_grad PASSED [ 68%] test/legacy_test/test_norm_all.py::TestPnormOp2_Fp16::test_check_output PASSED [ 70%] test/legacy_test/test_norm_all.py::TestPnormOp3_Fp16::test_check_grad PASSED [ 72%] test/legacy_test/test_norm_all.py::TestPnormOp3_Fp16::test_check_output PASSED [ 74%] test/legacy_test/test_norm_all.py::TestPnormOp4_Fp16::test_check_grad PASSED [ 76%] test/legacy_test/test_norm_all.py::TestPnormOp4_Fp16::test_check_output PASSED [ 78%] test/legacy_test/test_norm_all.py::TestPnormOp5_Fp16::test_check_grad PASSED [ 80%] test/legacy_test/test_norm_all.py::TestPnormOp5_Fp16::test_check_output PASSED [ 82%] test/legacy_test/test_norm_all.py::TestPnormOp6_Fp16::test_check_grad PASSED [ 84%] test/legacy_test/test_norm_all.py::TestPnormOp6_Fp16::test_check_output PASSED [ 86%] test/legacy_test/test_norm_all.py::TestPnormBF16Op::test_check_grad PASSED [ 88%] test/legacy_test/test_norm_all.py::TestPnormBF16Op::test_check_output PASSED [ 90%] test/legacy_test/test_norm_all.py::API_NormTest::test_basic FAILED [ 92%] test/legacy_test/test_norm_all.py::API_NormTest::test_dygraph PASSED [ 94%] test/legacy_test/test_norm_all.py::API_NormTest::test_errors PASSED [ 96%] test/legacy_test/test_norm_all.py::API_NormTest::test_name PASSED [ 98%] py::TestMatrixNormZeroSizeTensorTensor::test_matrix_norm_zero_size FAILED [100%] ================================================== FAILURES ================================================== __________________________________________ API_NormTest.test_basic ___________________________________________ self = <legacy_test.test_norm_all.API_NormTest testMethod=test_basic> def test_basic(self): keep_dims = {False, True} for keep in keep_dims: > check_fro_static( self, p='fro', axis=[-2, -1], shape_x=[2, 3, 4], dtype="float32", keep_dim=keep, ) test/legacy_test/test_norm_all.py:739: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ test/legacy_test/test_norm_all.py:576: in check_fro_static data = paddle.static.data(name="X", shape=shape_x, dtype=dtype) ../../miniconda3/envs/test_paddle/lib/python3.12/site-packages/decorator.py:235: in fun return caller(func, *(extras + args), **kw) ../../miniconda3/envs/test_paddle/lib/python3.12/site-packages/paddle/base/wrapped_decorator.py:40: in __impl__ return wrapped_func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ('X', [2, 3, 4], 'float32', 0), kwargs = {} def __impl__(*args: _InputT.args, **kwargs: _InputT.kwargs) -> _RetT: assert ( > not in_dygraph_mode() ), f"In PaddlePaddle 2.x, we turn on dynamic graph mode by default, and '{func.__name__}()' is only supported in static graph mode. So if you want to use this api, please call 'paddle.enable_static()' before this api to enter static graph mode." E AssertionError: In PaddlePaddle 2.x, we turn on dynamic graph mode by default, and 'data()' is only supported in static graph mode. So if you want to use this api, please call 'paddle.enable_static()' before this api to enter static graph mode. ../../miniconda3/envs/test_paddle/lib/python3.12/site-packages/paddle/base/framework.py:748: AssertionError 

目前给fro-norm写的单测都能通过,basic_test也能通过了

test/legacy_test/test_norm_all.py::TestPnormBF16Op::test_check_output PASSED [ 90%] test/legacy_test/test_norm_all.py::API_NormTest::test_basic PASSED [ 92%] test/legacy_test/test_norm_all.py::API_NormTest::test_dygraph PASSED [ 94%] test/legacy_test/test_norm_all.py::API_NormTest::test_errors PASSED [ 96%] test/legacy_test/test_norm_all.py::API_NormTest::test_name PASSED 

剩下的报错都是nuclear-norm调用svd的错误,需要后续再调整svd以完整支持matrix-norm。

Issue: #69908

@HydrogenSulfate

@paddle-bot
Copy link

paddle-bot bot commented Apr 30, 2025

你的PR提交成功,感谢你对开源项目的贡献!
请关注后续CI自动化测试结果,详情请参考Paddle-CI手册
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

@paddle-bot paddle-bot bot added the contributor External developers label Apr 30, 2025
Comment on lines 30 to 49
auto xdim = x.dims();

if (x.numel() == 0) {
std::set<int> axis_set;
for (int ax : axis.GetData()) {
if (ax < 0) {
ax += xdim.size();
}
axis_set.insert(ax);
}

std::vector<int64_t> out_dims_vec;
for (int i = 0; i < xdim.size(); ++i) {
if (axis_set.find(i) == axis_set.end()) {
out_dims_vec.push_back(xdim[i]);
} else if (keep_dim) {
out_dims_vec.push_back(1);
}
}
out->Resize(phi::make_ddim(out_dims_vec));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

形状推导已经在InferMeta中进行过了,可以使用out->dims()来获得,out已经是正确的形状了,不需要resize了,可以参考https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/dev_guides/api_contributing_guides/new_cpp_op_cn.html

Comment on lines 30 to 49
bool reduce_all,
DenseTensor* out) {
if (x.numel() == 0) {
auto dim_x = x.dims();
std::set<int> axis_set;
for (auto ax : dims.GetData()) {
if (ax < 0) ax += dim_x.size();
axis_set.insert(ax);
}

std::vector<int64_t> out_dims_vec;
for (int i = 0; i < dim_x.size(); ++i) {
if (axis_set.count(i) == 0) {
out_dims_vec.push_back(dim_x[i]);
} else if (keep_dim) {
out_dims_vec.push_back(1);
}
}

// 正确调用 Full 初始化
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

建议删掉注释


class API_NormTest(unittest.TestCase):
def test_basic(self):
paddle.enable_static()
Copy link
Contributor

@fangfangssj fangfangssj May 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

建议把paddle.enable_static()更换为with static_guard(),static_guard在utils.py中

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

谢谢,已经按照建议进行修改

@hanlintang
Copy link
Contributor Author

hanlintang commented May 1, 2025

按照@fangfangssj 的建议进行了如下修改:

  1. 删除了paddle/phi/kernels/impl/frobenius_norm_kernel_impl.h的resize;
  2. 删除paddle/phi/kernels/gpu/frobenius_norm_kernel.cu的注释;
  3. test/legacy_test/test_norm_all.py中的test_basic使用with static_guard():,并且去掉了之前commit加入的因为svd没法跑通的matrix_norm的单测,保留fro-norm的单测。
    以上修改已经重新编译并且通过本地的test_norm_all的全部单测。

测试结果:

test/legacy_test/test_norm_all.py::TestFrobeniusNormOp::test_check_grad PASSED [ 2%] test/legacy_test/test_norm_all.py::TestFrobeniusNormOp::test_check_output PASSED [ 4%] test/legacy_test/test_norm_all.py::TestFrobeniusNormOp2::test_check_grad PASSED [ 6%] test/legacy_test/test_norm_all.py::TestFrobeniusNormOp2::test_check_output PASSED [ 8%] test/legacy_test/test_norm_all.py::TestFrobeniusNormOpZeroSize::test_check_grad PASSED [ 10%] test/legacy_test/test_norm_all.py::TestFrobeniusNormOpZeroSize::test_check_output PASSED [ 12%] test/legacy_test/test_norm_all.py::TestFrobeniusNormOpZeroSize2::test_check_grad PASSED [ 14%] test/legacy_test/test_norm_all.py::TestFrobeniusNormOpZeroSize2::test_check_output PASSED [ 16%] test/legacy_test/test_norm_all.py::TestFrobeniusNormOpZeroSize3::test_check_grad PASSED [ 18%] test/legacy_test/test_norm_all.py::TestFrobeniusNormOpZeroSize3::test_check_output PASSED [ 20%] test/legacy_test/test_norm_all.py::TestFrobeniusNormOpZeroSize4::test_check_grad PASSED [ 22%] test/legacy_test/test_norm_all.py::TestFrobeniusNormOpZeroSize4::test_check_output PASSED [ 24%] test/legacy_test/test_norm_all.py::TestPnormOp::test_check_grad PASSED [ 26%] test/legacy_test/test_norm_all.py::TestPnormOp::test_check_output PASSED [ 28%] test/legacy_test/test_norm_all.py::TestPnormOp2::test_check_grad PASSED [ 30%] test/legacy_test/test_norm_all.py::TestPnormOp2::test_check_output PASSED [ 32%] test/legacy_test/test_norm_all.py::TestPnormOp3::test_check_grad PASSED [ 34%] test/legacy_test/test_norm_all.py::TestPnormOp3::test_check_output PASSED [ 36%] test/legacy_test/test_norm_all.py::TestPnormOp4::test_check_grad PASSED [ 38%] test/legacy_test/test_norm_all.py::TestPnormOp4::test_check_output PASSED [ 40%] test/legacy_test/test_norm_all.py::TestPnormOp5::test_check_grad PASSED [ 42%] test/legacy_test/test_norm_all.py::TestPnormOp5::test_check_output PASSED [ 44%] test/legacy_test/test_norm_all.py::TestPnormOp6::test_check_grad PASSED [ 46%] test/legacy_test/test_norm_all.py::TestPnormOp6::test_check_output PASSED [ 48%] test/legacy_test/test_norm_all.py::TestPnormOpZeroSize::test_check_grad PASSED [ 50%] test/legacy_test/test_norm_all.py::TestPnormOpZeroSize::test_check_output PASSED [ 52%] test/legacy_test/test_norm_all.py::TestPnormOpZeroSize2::test_check_grad PASSED [ 54%] test/legacy_test/test_norm_all.py::TestPnormOpZeroSize2::test_check_output PASSED [ 56%] test/legacy_test/test_norm_all.py::TestPnormOpZeroSize3::test_check_grad PASSED [ 58%] test/legacy_test/test_norm_all.py::TestPnormOpZeroSize3::test_check_output PASSED [ 60%] test/legacy_test/test_norm_all.py::TestPnormOpZeroSize4::test_check_grad PASSED [ 62%] test/legacy_test/test_norm_all.py::TestPnormOpZeroSize4::test_check_output PASSED [ 64%] test/legacy_test/test_norm_all.py::TestPnormOp_Fp16::test_check_grad PASSED [ 66%] test/legacy_test/test_norm_all.py::TestPnormOp_Fp16::test_check_output PASSED [ 68%] test/legacy_test/test_norm_all.py::TestPnormOp2_Fp16::test_check_grad PASSED [ 70%] test/legacy_test/test_norm_all.py::TestPnormOp2_Fp16::test_check_output PASSED [ 72%] test/legacy_test/test_norm_all.py::TestPnormOp3_Fp16::test_check_grad PASSED [ 74%] test/legacy_test/test_norm_all.py::TestPnormOp3_Fp16::test_check_output PASSED [ 76%] test/legacy_test/test_norm_all.py::TestPnormOp4_Fp16::test_check_grad PASSED [ 78%] test/legacy_test/test_norm_all.py::TestPnormOp4_Fp16::test_check_output PASSED [ 80%] test/legacy_test/test_norm_all.py::TestPnormOp5_Fp16::test_check_grad PASSED [ 82%] test/legacy_test/test_norm_all.py::TestPnormOp5_Fp16::test_check_output PASSED [ 84%] test/legacy_test/test_norm_all.py::TestPnormOp6_Fp16::test_check_grad PASSED [ 86%] test/legacy_test/test_norm_all.py::TestPnormOp6_Fp16::test_check_output PASSED [ 88%] test/legacy_test/test_norm_all.py::TestPnormBF16Op::test_check_grad PASSED [ 90%] test/legacy_test/test_norm_all.py::TestPnormBF16Op::test_check_output PASSED [ 92%] test/legacy_test/test_norm_all.py::API_NormTest::test_basic PASSED [ 94%] test/legacy_test/test_norm_all.py::API_NormTest::test_dygraph PASSED [ 96%] test/legacy_test/test_norm_all.py::API_NormTest::test_errors PASSED [ 98%] test/legacy_test/test_norm_all.py::API_NormTest::test_name PASSED [100%] 
@luotao1 luotao1 added the HappyOpenSource 快乐开源活动issue与PR label May 8, 2025
Comment on lines 33 to 47
auto dim_x = x.dims();
std::set<int> axis_set;
for (auto ax : dims.GetData()) {
if (ax < 0) ax += dim_x.size();
axis_set.insert(ax);
}

std::vector<int64_t> out_dims_vec;
for (int i = 0; i < dim_x.size(); ++i) {
if (axis_set.count(i) == 0) {
out_dims_vec.push_back(dim_x[i]);
} else if (keep_dim) {
out_dims_vec.push_back(1);
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里的形状推到应该可以去掉,执行这个函数之前,在ReduceIntArrayAxisInferMetaBase中应该已经计算完,并放到out->dims()里了

Copy link
Contributor Author

@hanlintang hanlintang May 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

谢谢,已经按照建议修改,在本地重新编译并通过了全部单测

Copy link
Contributor

@HydrogenSulfate HydrogenSulfate left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@HydrogenSulfate HydrogenSulfate merged commit 4762f6a into PaddlePaddle:develop May 14, 2025
46 of 47 checks passed
GITD245 pushed a commit to GITD245/Paddle that referenced this pull request May 14, 2025
…0-size tensor (PaddlePaddle#72570) * [Tensor] Fix frobenius_norm to support 0-size tensor * [Tensor] Drop resize of fro-norm * [Tensor] Import utils with non-relative path * drop resize in gpu
@hanlintang hanlintang deleted the matrix_norm branch May 14, 2025 10:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

contributor External developers HappyOpenSource 快乐开源活动issue与PR

4 participants