Skip to content

test_lod_tensor_array_ops fails in the TeamCity CI build. #8351

@kavyasrinet

Description

@kavyasrinet

I got an email mentioning that the TeamCity CI failed while executing the test_lod_tensor_array_ops test. Before the PR got merged, all tests passed successfully and my code doesn't touch this test case either.

Has anyone else experienced this issue ?
Following is the stack trace for reference:

206/370 Test #198: test_selected_rows .............................. Passed 2.89 sec Start 199: test_lod_tensor_array_ops 207/370 Test #199: test_lod_tensor_array_ops .......................***Failed 2.75 sec .....E. [18:31:18]====================================================================== [18:31:18]ERROR: test_lod_tensor_to_array_level_2 (__main__.TestCPULoDTensorArrayOps) [18:31:18]---------------------------------------------------------------------- [18:31:18]Traceback (most recent call last): [18:31:18] File "test_lod_tensor_array_ops.py", line 117, in test_lod_tensor_to_array_level_2 [18:31:18] expect_max_len=3) [18:31:18] File "test_lod_tensor_array_ops.py", line 148, in main [18:31:18] exe.run(program, feed={'x': tensor}, scope=scope) [18:31:18] File "/paddle/build/python/build/lib-python/paddle/v2/fluid/executor.py", line 290, in run [18:31:18] self.executor.run(program.desc, scope, 0, True, True) [18:31:18]EnforceNotMet: holder_ should not be null [18:31:18]Tensor not initialized yet when Tensor::place() is called. at [/paddle/paddle/framework/tensor.h:113] [18:31:18]PaddlePaddle Call Stacks: [18:31:18]0 0x7f4c9b6d07ccp paddle::platform::EnforceNotMet::EnforceNotMet(std::__exception_ptr::exception_ptr, char const*, int) + 572 [18:31:18]1 0x7f4c9b6d1591p paddle::framework::Tensor::place() const + 209 [18:31:18]2 0x7f4c9b7e5986p paddle::framework::Vector<unsigned long>::CopyToCPU() const + 70 [18:31:18]3 0x7f4c9b93b4c0p paddle::framework::Vector<unsigned long>::resize(unsigned long) + 544 [18:31:18]4 0x7f4c9c22a322p paddle::framework::Vector<unsigned long>::Vector(unsigned long, unsigned long const&) + 178 [18:31:18]5 0x7f4c9c22a85ep void std::vector<paddle::framework::Vector<unsigned long>, std::allocator<paddle::framework::Vector<unsigned long> > >::_M_emplace_back_aux<int, int>(int&&, int&&) + 158 [18:31:18]6 0x7f4c9c227d98p paddle::framework::AppendLoD(std::vector<paddle::framework::Vector<unsigned long>, std::allocator<paddle::framework::Vector<unsigned long> > >*, std::vector<paddle::framework::Vector<unsigned long>, std::allocator<paddle::framework::Vector<unsigned long> > > const&) + 2152 [18:31:18]7 0x7f4c9c0d5bb6p paddle::operators::LoDTensorToArrayOp::Run(paddle::framework::Scope const&, boost::variant<paddle::platform::CUDAPlace, paddle::platform::CPUPlace, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_> const&) const + 2022 [18:31:18]8 0x7f4c9b7a1b32p paddle::framework::Executor::Run(paddle::framework::ProgramDesc const&, paddle::framework::Scope*, int, bool, bool) + 1490 [18:31:18]9 0x7f4c9b6ecd23p void pybind11::cpp_function::initialize<pybind11::cpp_function::initialize<void, paddle::framework::Executor, paddle::framework::ProgramDesc const&, paddle::framework::Scope*, int, bool, bool, pybind11::name, pybind11::is_method, pybind11::sibling>(void (paddle::framework::Executor::*)(paddle::framework::ProgramDesc const&, paddle::framework::Scope*, int, bool, bool), pybind11::name const&, pybind11::is_method const&, pybind11::sibling const&)::{lambda(paddle::framework::Executor*, paddle::framework::ProgramDesc const&, paddle::framework::Scope*, int, bool, bool)#1}, void, paddle::framework::Executor*, paddle::framework::ProgramDesc const&, paddle::framework::Scope*, int, bool, bool, pybind11::name, pybind11::is_method, pybind11::sibling>(pybind11::cpp_function::initialize<void, paddle::framework::Executor, paddle::framework::ProgramDesc const&, paddle::framework::Scope*, int, bool, bool, pybind11::name, pybind11::is_method, pybind11::sibling>(void (paddle::framework::Executor::*)(paddle::framework::ProgramDesc const&, paddle::framework::Scope*, int, bool, bool), pybind11::name const&, pybind11::is_method const&, pybind11::sibling const&)::{lambda(paddle::framework::Executor*, paddle::framework::ProgramDesc const&, paddle::framework::Scope*, int, bool, bool)#1}&&, void (*)(paddle::framework::Executor*, paddle::framework::ProgramDesc const&, paddle::framework::Scope*, int, bool, bool), pybind11::name const&, pybind11::is_method const&, pybind11::sibling const&)::{lambda(pybind11::detail::function_call&)#3}::_FUN(pybind11::detail::function_call) + 579 [18:31:18]10 0x7f4c9b6eaa64p pybind11::cpp_function::dispatcher(_object*, _object*, _object*) + 1236 [18:31:18]11 0x4c37edp PyEval_EvalFrameEx + 31165 [18:31:18]12 0x4b9ab6p PyEval_EvalCodeEx + 774 [18:31:18]13 0x4c16e7p PyEval_EvalFrameEx + 22711 [18:31:18]14 0x4b9ab6p PyEval_EvalCodeEx + 774 [18:31:18]15 0x4c16e7p PyEval_EvalFrameEx + 22711 [18:31:18]16 0x4c136fp PyEval_EvalFrameEx + 21823 [18:31:18]17 0x4b9ab6p PyEval_EvalCodeEx + 774 [18:31:18]18 0x4d55f3p [18:31:18]19 0x4a577ep PyObject_Call + 62 [18:31:18]20 0x4bed3dp PyEval_EvalFrameEx + 12045 [18:31:18]21 0x4b9ab6p PyEval_EvalCodeEx + 774 [18:31:18]22 0x4d54b9p [18:31:18]23 0x4eebeep [18:31:18]24 0x4a577ep PyObject_Call + 62 [18:31:18]25 0x548253p [18:31:18]26 0x4c15bfp PyEval_EvalFrameEx + 22415 [18:31:18]27 0x4b9ab6p PyEval_EvalCodeEx + 774 [18:31:18]28 0x4d55f3p [18:31:18]29 0x4a577ep PyObject_Call + 62 [18:31:18]30 0x4bed3dp PyEval_EvalFrameEx + 12045 [18:31:18]31 0x4b9ab6p PyEval_EvalCodeEx + 774 [18:31:18]32 0x4d54b9p [18:31:18]33 0x4eebeep [18:31:18]34 0x4a577ep PyObject_Call + 62 [18:31:18]35 0x548253p [18:31:18]36 0x4c15bfp PyEval_EvalFrameEx + 22415 [18:31:18]37 0x4b9ab6p PyEval_EvalCodeEx + 774 [18:31:18]38 0x4d55f3p [18:31:18]39 0x4a577ep PyObject_Call + 62 [18:31:18]40 0x4bed3dp PyEval_EvalFrameEx + 12045 [18:31:18]41 0x4b9ab6p PyEval_EvalCodeEx + 774 [18:31:18]42 0x4d54b9p [18:31:18]43 0x4eebeep [18:31:18]44 0x4a577ep PyObject_Call + 62 [18:31:18]45 0x548253p [18:31:18]46 0x4c15bfp PyEval_EvalFrameEx + 22415 [18:31:18]47 0x4c136fp PyEval_EvalFrameEx + 21823 [18:31:18]48 0x4c136fp PyEval_EvalFrameEx + 21823 [18:31:18]49 0x4b9ab6p PyEval_EvalCodeEx + 774 [18:31:18]50 0x4d54b9p [18:31:18]51 0x4eebeep [18:31:18]52 0x4ee7f6p [18:31:18]53 0x4aa9abp [18:31:18]54 0x4c15bfp PyEval_EvalFrameEx + 22415 [18:31:18]55 0x4b9ab6p PyEval_EvalCodeEx + 774 [18:31:18]56 0x4eb30fp [18:31:18]57 0x4e5422p PyRun_FileExFlags + 130 [18:31:18]58 0x4e3cd6p PyRun_SimpleFileExFlags + 390 [18:31:18]59 0x493ae2p Py_Main + 1554 [18:31:18]60 0x7f4d32440830p __libc_start_main + 240 [18:31:18]61 0x4933e9p _start + 41 [18:31:18] [18:31:18] [18:31:18]---------------------------------------------------------------------- 

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions