Remove unreferenced variables from ProgramDesc in prune()

[kexinzhao]

Fixes #7417

By following similar test procedure as in #7097, we get the testing result as follows:

FLAGS_dirname: /paddle/python/paddle/v2/fluid/tests/book/recognize_digits_mlp.inference.model WARNING: Logging before InitGoogleLogging() is written to STDERR I0125 23:58:51.136490 29702 inference.cc:26] loading model from /paddle/python/paddle/v2/fluid/tests/book/recognize_digits_mlp.inference.model/__model__ I0125 23:58:51.136664 29702 inference.cc:32] program_desc_str's size: 1351 I0125 23:58:51.136858 29702 inference.cc:78] parameter's name: fc_1.b_0 I0125 23:58:51.136916 29702 inference.cc:78] parameter's name: fc_0.b_0 I0125 23:58:51.136937 29702 inference.cc:78] parameter's name: fc_0.w_0 I0125 23:58:51.136955 29702 inference.cc:78] parameter's name: fc_1.w_0 I0125 23:58:51.136973 29702 inference.cc:78] parameter's name: fc_2.w_0 I0125 23:58:51.136991 29702 inference.cc:78] parameter's name: fc_2.b_0 W0125 23:58:51.137250 29702 init.cc:56] 'GPU' is not supported, Please re-compile with WITH_GPU option I0125 23:58:51.137315 29702 scope.cc:48] Create variable fc_2.b_0 I0125 23:58:51.137331 29702 executor.cc:99] Create Variable fc_2.b_0 global, which pointer is 0x2024d70 I0125 23:58:51.137348 29702 scope.cc:48] Create variable fc_1.b_0 I0125 23:58:51.137359 29702 executor.cc:99] Create Variable fc_1.b_0 global, which pointer is 0x2024d90 I0125 23:58:51.137369 29702 scope.cc:48] Create variable fc_0.b_0 I0125 23:58:51.137399 29702 executor.cc:99] Create Variable fc_0.b_0 global, which pointer is 0x2025000 I0125 23:58:51.137413 29702 scope.cc:48] Create variable fc_2.w_0 I0125 23:58:51.137421 29702 executor.cc:99] Create Variable fc_2.w_0 global, which pointer is 0x2025080 I0125 23:58:51.137432 29702 scope.cc:48] Create variable fc_0.w_0 I0125 23:58:51.137441 29702 executor.cc:99] Create Variable fc_0.w_0 global, which pointer is 0x202fdd0 I0125 23:58:51.137451 29702 scope.cc:48] Create variable fc_1.w_0 I0125 23:58:51.137465 29702 executor.cc:99] Create Variable fc_1.w_0 global, which pointer is 0x202ff20 I0125 23:58:51.137636 29702 executor.cc:126] Op(load), inputs:{}, outputs:{Out[fc_1.b_0[64]({})]}. I0125 23:58:51.137719 29702 executor.cc:126] Op(load), inputs:{}, outputs:{Out[fc_0.b_0[128]({})]}. I0125 23:58:51.138204 29702 executor.cc:126] Op(load), inputs:{}, outputs:{Out[fc_0.w_0[784, 128]({})]}. I0125 23:58:51.138305 29702 executor.cc:126] Op(load), inputs:{}, outputs:{Out[fc_1.w_0[128, 64]({})]}. I0125 23:58:51.138361 29702 executor.cc:126] Op(load), inputs:{}, outputs:{Out[fc_2.w_0[64, 10]({})]}. I0125 23:58:51.138411 29702 executor.cc:126] Op(load), inputs:{}, outputs:{Out[fc_2.b_0[10]({})]}. I0125 23:58:51.139745 29702 feed_fetch_method.h:28] SetFeedVariable name=feed index=0 I0125 23:58:51.139770 29702 scope.cc:48] Create variable feed I0125 23:58:51.139794 29702 scope.cc:48] Create variable fc_1.tmp_1 I0125 23:58:51.139806 29702 executor.cc:104] Create Variable fc_1.tmp_1 locally, which pointer is 0x2035410 I0125 23:58:51.139817 29702 scope.cc:48] Create variable fc_0.tmp_0 I0125 23:58:51.139827 29702 executor.cc:104] Create Variable fc_0.tmp_0 locally, which pointer is 0x2030560 I0125 23:58:51.139837 29702 scope.cc:48] Create variable fc_1.tmp_0 I0125 23:58:51.139847 29702 executor.cc:104] Create Variable fc_1.tmp_0 locally, which pointer is 0x2030260 I0125 23:58:51.139856 29702 scope.cc:48] Create variable fc_0.tmp_2 I0125 23:58:51.139868 29702 executor.cc:104] Create Variable fc_0.tmp_2 locally, which pointer is 0x20357f0 I0125 23:58:51.139881 29702 scope.cc:48] Create variable fc_0.tmp_1 I0125 23:58:51.139889 29702 executor.cc:104] Create Variable fc_0.tmp_1 locally, which pointer is 0x2035900 I0125 23:58:51.139899 29702 scope.cc:48] Create variable fc_1.tmp_2 I0125 23:58:51.139909 29702 executor.cc:104] Create Variable fc_1.tmp_2 locally, which pointer is 0x2035a50 I0125 23:58:51.139919 29702 executor.cc:99] Create Variable fc_1.b_0 global, which pointer is 0x2024d90 I0125 23:58:51.139928 29702 scope.cc:48] Create variable fc_2.tmp_1 I0125 23:58:51.139938 29702 executor.cc:104] Create Variable fc_2.tmp_1 locally, which pointer is 0x2035b60 I0125 23:58:51.139946 29702 executor.cc:99] Create Variable feed global, which pointer is 0x2024cd0 I0125 23:58:51.139953 29702 scope.cc:48] Create variable fetch I0125 23:58:51.139962 29702 executor.cc:99] Create Variable fetch global, which pointer is 0x2035c70 I0125 23:58:51.139972 29702 scope.cc:48] Create variable fc_2.tmp_2 I0125 23:58:51.139981 29702 executor.cc:104] Create Variable fc_2.tmp_2 locally, which pointer is 0x2035d10 I0125 23:58:51.139991 29702 scope.cc:48] Create variable fc_2.tmp_0 I0125 23:58:51.140000 29702 executor.cc:104] Create Variable fc_2.tmp_0 locally, which pointer is 0x2035e20 I0125 23:58:51.140010 29702 executor.cc:99] Create Variable fc_0.b_0 global, which pointer is 0x2025000 I0125 23:58:51.140019 29702 executor.cc:99] Create Variable fc_0.w_0 global, which pointer is 0x202fdd0 I0125 23:58:51.140028 29702 executor.cc:99] Create Variable fc_1.w_0 global, which pointer is 0x202ff20 I0125 23:58:51.140038 29702 scope.cc:48] Create variable x I0125 23:58:51.140048 29702 executor.cc:104] Create Variable x locally, which pointer is 0x2035f30 I0125 23:58:51.140064 29702 executor.cc:99] Create Variable fc_2.w_0 global, which pointer is 0x2025080 I0125 23:58:51.140072 29702 executor.cc:99] Create Variable fc_2.b_0 global, which pointer is 0x2024d70 I0125 23:58:51.140100 29702 feed_op.cc:44] Feed Var feed's 0 column to var x I0125 23:58:51.140112 29702 tensor_util.h:36] Copy 1, 784 from CPUPlace to CPUPlace I0125 23:58:51.140149 29702 executor.cc:126] Op(feed), inputs:{X[feed[-1]({{}})]}, outputs:{Out[x[1, 784]({})]}. I0125 23:58:51.140187 29702 mul_op.cc:36] mul operator x.shape=1, 784 y.shape=784, 128 x_num_col_dims=1 y_num_col_dims=1 I0125 23:58:51.140211 29702 operator.cc:488] expected_kernel_key:data_type[5]:data_layout[ANY_LAYOUT]:place[CPUPlace]:library_type[PLAIN] I0125 23:58:51.152216 29702 executor.cc:126] Op(mul), inputs:{X[x[1, 784]({})], Y[fc_0.w_0[784, 128]({})]}, outputs:{Out[fc_0.tmp_0[1, 128]({})]}. I0125 23:58:51.152279 29702 operator.cc:488] expected_kernel_key:data_type[5]:data_layout[ANY_LAYOUT]:place[CPUPlace]:library_type[PLAIN] I0125 23:58:51.152340 29702 executor.cc:126] Op(elementwise_add), inputs:{X[fc_0.tmp_0[1, 128]({})], Y[fc_0.b_0[128]({})]}, outputs:{Out[fc_0.tmp_1[1, 128]({})]}. I0125 23:58:51.152426 29702 operator.cc:488] expected_kernel_key:data_type[5]:data_layout[ANY_LAYOUT]:place[CPUPlace]:library_type[PLAIN] I0125 23:58:51.152494 29702 executor.cc:126] Op(relu), inputs:{X[fc_0.tmp_1[1, 128]({})]}, outputs:{Out[fc_0.tmp_2[1, 128]({})]}. I0125 23:58:51.152621 29702 mul_op.cc:36] mul operator x.shape=1, 128 y.shape=128, 64 x_num_col_dims=1 y_num_col_dims=1 I0125 23:58:51.152664 29702 operator.cc:488] expected_kernel_key:data_type[5]:data_layout[ANY_LAYOUT]:place[CPUPlace]:library_type[PLAIN] I0125 23:58:51.152767 29702 executor.cc:126] Op(mul), inputs:{X[fc_0.tmp_2[1, 128]({})], Y[fc_1.w_0[128, 64]({})]}, outputs:{Out[fc_1.tmp_0[1, 64]({})]}. I0125 23:58:51.152817 29702 operator.cc:488] expected_kernel_key:data_type[5]:data_layout[ANY_LAYOUT]:place[CPUPlace]:library_type[PLAIN] I0125 23:58:51.152873 29702 executor.cc:126] Op(elementwise_add), inputs:{X[fc_1.tmp_0[1, 64]({})], Y[fc_1.b_0[64]({})]}, outputs:{Out[fc_1.tmp_1[1, 64]({})]}. I0125 23:58:51.152909 29702 operator.cc:488] expected_kernel_key:data_type[5]:data_layout[ANY_LAYOUT]:place[CPUPlace]:library_type[PLAIN] I0125 23:58:51.152951 29702 executor.cc:126] Op(relu), inputs:{X[fc_1.tmp_1[1, 64]({})]}, outputs:{Out[fc_1.tmp_2[1, 64]({})]}. I0125 23:58:51.152990 29702 mul_op.cc:36] mul operator x.shape=1, 64 y.shape=64, 10 x_num_col_dims=1 y_num_col_dims=1 I0125 23:58:51.153020 29702 operator.cc:488] expected_kernel_key:data_type[5]:data_layout[ANY_LAYOUT]:place[CPUPlace]:library_type[PLAIN] I0125 23:58:51.153084 29702 executor.cc:126] Op(mul), inputs:{X[fc_1.tmp_2[1, 64]({})], Y[fc_2.w_0[64, 10]({})]}, outputs:{Out[fc_2.tmp_0[1, 10]({})]}. I0125 23:58:51.153143 29702 operator.cc:488] expected_kernel_key:data_type[5]:data_layout[ANY_LAYOUT]:place[CPUPlace]:library_type[PLAIN] I0125 23:58:51.153270 29702 executor.cc:126] Op(elementwise_add), inputs:{X[fc_2.tmp_0[1, 10]({})], Y[fc_2.b_0[10]({})]}, outputs:{Out[fc_2.tmp_1[1, 10]({})]}. I0125 23:58:51.153374 29702 operator.cc:488] expected_kernel_key:data_type[5]:data_layout[ANY_LAYOUT]:place[CPUPlace]:library_type[PLAIN] I0125 23:58:51.153532 29702 executor.cc:126] Op(softmax), inputs:{X[fc_2.tmp_1[1, 10]({})]}, outputs:{Out[fc_2.tmp_2[1, 10]({})]}. I0125 23:58:51.153575 29702 tensor_util.h:36] Copy 1, 10 from CPUPlace to CPUPlace I0125 23:58:51.153626 29702 fetch_op.cc:62] Fetch variable fc_2.tmp_2 to fetch I0125 23:58:51.153672 29702 executor.cc:126] Op(fetch), inputs:{X[fc_2.tmp_2[1, 10]({})]}, outputs:{Out[fetch[-1]({{}})]}. I0125 23:58:51.153836 29704 scope.cc:33] Destroy variable x I0125 23:58:51.154021 29704 scope.cc:33] Destroy variable fc_2.tmp_0 I0125 23:58:51.154045 29704 scope.cc:33] Destroy variable fc_2.tmp_2 I0125 23:58:51.154078 29704 scope.cc:33] Destroy variable fc_2.tmp_1 I0125 23:58:51.154228 29704 scope.cc:33] Destroy variable fc_1.tmp_2 I0125 23:58:51.154316 29704 scope.cc:33] Destroy variable fc_0.tmp_1 I0125 23:58:51.154392 29704 scope.cc:33] Destroy variable fc_1.tmp_1 I0125 23:58:51.154435 29704 scope.cc:33] Destroy variable fc_0.tmp_0 I0125 23:58:51.154466 29704 scope.cc:33] Destroy variable fc_1.tmp_0 I0125 23:58:51.154597 29704 scope.cc:33] Destroy variable fc_0.tmp_2 I0125 23:58:51.154650 29702 feed_fetch_method.h:51] Fetch fetch with index 0 shape= 1, 10 I0125 23:58:51.154670 29702 scope.cc:33] Destroy variable fetch I0125 23:58:51.154847 29702 scope.cc:33] Destroy variable feed I0125 23:58:51.154989 29702 scope.cc:33] Destroy variable fc_2.b_0 I0125 23:58:51.155105 29702 scope.cc:33] Destroy variable fc_1.b_0 I0125 23:58:51.155323 29702 scope.cc:33] Destroy variable fc_0.b_0 I0125 23:58:51.155359 29702 scope.cc:33] Destroy variable fc_1.w_0 I0125 23:58:51.155376 29702 scope.cc:33] Destroy variable fc_0.w_0 I0125 23:58:51.155407 29702 scope.cc:33] Destroy variable fc_2.w_0 dims_i: 1 10 result: 0.106794 0.1657 0.0748172 0.0881855 0.119018 0.0405249 0.111904 0.103682 0.0953393 0.0940347 

[Xreki]

Maybe, there is no need for an extra map. We can directly use var map of input ProgramDesc. We can use input.FindVar(name)->Proto() to get the proto::VarDesc.

[kexinzhao]

There are two different classes one is framework::proto::xxxDesc, one is framework::xxxDesc. The latter is basically a wrapper about the former.

For this prune function, the input is proto::ProgramDesc, so we only have limited functionality provided by the protobuf library itself. After you compile the framework.proto, you will get framework.pb.h + cc, which lists all the available functions for proto::ProgramDesc and etc.

So I don't think we can use input.FindVar() here.

[kexinzhao]

Later, when we are going to implement out transpiler function for inference optimization on the c++ side after loading a inference desc from file or buffer, we can use your suggestion, as in that case, we can take a framework::ProgramDesc as input.

@Xreki what do you think?

[Xreki]

For this prune function, the input is proto::ProgramDesc, so we only have limited functionality provided by the protobuf library itself.

You are right. Sorry for didn't notice that.

[Xreki]

input -> input_var? Because the first argument of this function is named input.

[kexinzhao]

Done

[Xreki]

output -> output_var? Because the second argument of this function is named output.

[kexinzhao]

Done

[Xreki]

LGTM

[Xreki]

For this prune function, the input is proto::ProgramDesc, so we only have limited functionality provided by the protobuf library itself.

You are right. Sorry for didn't notice that.