add new API: paddle.clone;Tensor.element_size;nn.utils.parameters_to_vector

[zhwesky2010]

PR types

New features

PR changes

APIs

Describe

1.New API: paddle.clone

Returns a copy of input Tensor. It will always have a Tensor copy.

Tn addition, the OP provides gradient propagation.

import paddle x = paddle.ones([2], stop_gradient=False) clone_x = paddle.clone(x) y = clone_x**3 y.backward() print(clone_x.grad) # [3] print(x.grad) # [3] 

2.New API: paddle.Tensor.element_size

Returns the size in bytes of an element in the Tensor.

import paddle x = paddle.static.data(name='x', shape=[3, 2], dtype='bool') x.element_size() # 1 x = paddle.static.data(name='x', shape=[3, 2], dtype='int16') x.element_size() # 2 x = paddle.static.data(name='x', shape=[3, 2], dtype='float16') x.element_size() # 2 x = paddle.static.data(name='x', shape=[3, 2], dtype='float32') x.element_size() # 4 x = paddle.static.data(name='x', shape=[3, 2], dtype='float64') x.element_size() # 8 

3.New API：paddle.nn.utils.parameters_to_vector

Flatten parameters to a 1-D Tensor.

import paddle linear = paddle.nn.Linear(10, 15) paddle.nn.utils.parameters_to_vector(linear.parameters()) # 1-D Tensor: [165] 

4.New API：paddle.nn.utils.vector_to_parameters

Transform a Tensor with 1-D shape to the parameters.

import paddle linear1 = paddle.nn.Linear(10, 15, paddle.ParamAttr(paddle.nn.initializer.Constant(3.))) vec = paddle.nn.utils.parameters_to_vector(linear1.parameters()) linear2 = paddle.nn.Linear(10, 15) # copy weight of linear1 to linear2 paddle.nn.utils.vector_to_parameters(vec, linear2.parameters()) # weight: Tensor(shape=[10, 15], dtype=float32, place=CUDAPlace(0), stop_gradient=False, # [[3. , ..., 3. ], # [..., ..., ...], # [3. , ..., 3. ]]) 

[paddle-bot-old]

Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

[TCChenlong]

这两句描述不够清楚，可以在改一下

[zhwesky2010]

Done

[TCChenlong]

Tn addition -> In addition

[zhwesky2010]

Done, thx

[TCChenlong]

LGTM

[Xreki]

能否用1个split代替多个slice？

[zhwesky2010]

Done，已去掉多个slice

[Xreki]

为什么这里需要显式使用inplace接口呢？

[zhwesky2010]

core.ops.reshape2_是inplace机制，core.ops.reshape2是view机制。使用view机制的OP是写死的，目前有4个：

inplace机制下：输入、输出是同一个var。
view机制下：需要创建一个输出var，但输出var会和输入var是share_buffer_with的，所以两者都没有成本。
这里指定用inplace版本的，写法会简单些，不需要创建一个临时输出var

[Xreki]

vector_to_parameters可能会比较慢

[zhwesky2010]

已去掉assign，改成最简化的写法：

parameters_to_vector：使用1个concat OP + 完全inplace的reshape_ OP
vector_to_parameters：使用1个split OP + 完全inplace的reshape_ OP

目前全部使用inplace版的reshape_ OP，每个API实质上只有1个concat或split OP的耗时，性能可以得到保证了

[TCChenlong]

LGTM

[luotao1]

LGTM for CMakeLists.txt