Skip to content

Move embedding to phi #39901

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 14 commits into from
Mar 22, 2022
Merged

Move embedding to phi #39901

merged 14 commits into from
Mar 22, 2022

Conversation

@phlrain
Copy link
Collaborator

@phlrain phlrain commented Feb 24, 2022

PR types

Breaking changes

PR changes

OPs

Describe

move embedding to phi
@paddle-bot-old
Copy link

paddle-bot-old bot commented Mar 8, 2022

Sorry to inform you that ab1042f's CIs have passed for more than 7 days. To prevent PR conflicts, you need to re-run all CIs manually.
zyfncg
zyfncg reviewed Mar 18, 2022
View reviewed changes
paddle/phi/kernels/cpu/embedding_grad_kernel.cc Outdated
namespace phi {

template <typename T, typename Context>
struct LookupTableV2GradCPUFunctor {
Copy link
Contributor

@zyfncg zyfncg Mar 18, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

在phi下面LookupTableV2要不要都统一成Embedding？
Copy link
Collaborator Author

@phlrain phlrain Mar 20, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done
paddle/phi/kernels/cpu/embedding_grad_kernel.cc Outdated
// paddings makes no sense and we don't deal with it in backward.
{
auto* d_output = &out_grad_;
// auto d_table = weight_grad_;
Copy link
Contributor

@zyfncg zyfncg Mar 18, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

注释可以删除
Copy link
Collaborator Author

@phlrain phlrain Mar 20, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done
paddle/phi/kernels/cpu/embedding_grad_kernel.cc Outdated
ids_data[i],
N,
phi::errors::InvalidArgument(
"Variable value (input) of OP(fluid.layers.embedding) "
Copy link
Contributor

@zyfncg zyfncg Mar 18, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fluid.layers.embedding->paddle.nn.functional.embedding
Copy link
Collaborator Author

@phlrain phlrain Mar 20, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done
paddle/phi/kernels/cpu/embedding_grad_kernel.cc Outdated
ids_data[i],
0,
phi::errors::InvalidArgument(
"Variable value (input) of OP(fluid.layers.embedding) "
Copy link
Contributor

@zyfncg zyfncg Mar 18, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fluid.layers.embedding->paddle.nn.functional.embedding
Copy link
Collaborator Author

@phlrain phlrain Mar 20, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done
paddle/phi/kernels/cpu/embedding_grad_kernel.cc
};

template <typename T, typename Context>
void EmbeddingSparseGradKernel(const Context& ctx,
Copy link
Contributor

@zyfncg zyfncg Mar 18, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个Kernel可以放到selected_rows下
Copy link
Collaborator Author

@phlrain phlrain Mar 20, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

selected rows的拆分，单独用一个pr来做
paddle/phi/kernels/cpu/sparse_weight_embedding_grad_kernel.cc

namespace phi {

template <typename T, typename Context>
Copy link
Contributor

@zyfncg zyfncg Mar 18, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个文件可以放到selected_rows目录下
Copy link
Collaborator Author

@phlrain phlrain Mar 20, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

selected rows的拆分，单独用一个pr来做
paddle/phi/kernels/cpu/sparse_weight_embedding_grad_kernel.cc Outdated
#include "paddle/phi/kernels/funcs/embedding_util.h"

#include "paddle/fluid/framework/convert_utils.h"
#include "paddle/fluid/framework/data_type.h"
Copy link
Contributor

@zyfncg zyfncg Mar 18, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

引用phi下的data_type.h
Copy link
Collaborator Author

@phlrain phlrain Mar 20, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done
paddle/phi/kernels/cpu/sparse_weight_embedding_grad_kernel.cc Outdated
ids_data[i],
0,
phi::errors::InvalidArgument(
"Variable value (input) of OP(fluid.layers.embedding) "
Copy link
Contributor

@zyfncg zyfncg Mar 18, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fluid.layers.embedding->paddle.nn.functional.embedding
Copy link
Collaborator Author

@phlrain phlrain Mar 20, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done
paddle/phi/kernels/cpu/sparse_weight_embedding_kernel.cc Outdated
auto output_t = out_;
int64_t row_width = table_t.value().dims()[1];
const auto* table = table_t.value().template data<T>();
auto* output = output_t->template mutable_data<T>(dev_ctx_.GetPlace());
Copy link
Contributor

@zyfncg zyfncg Mar 18, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

使用dec_ctx.Alloc
Copy link
Collaborator Author

@phlrain phlrain Mar 20, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done
paddle/phi/kernels/cpu/sparse_weight_embedding_kernel.cc
Comment on lines +49 to +50
auto input_data_type =
paddle::framework::TransToProtoVarType(table_t.value().dtype());
Copy link
Contributor

@zyfncg zyfncg Mar 18, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里可以不转ProtoVarType，直接用dtype
Copy link
Collaborator Author

@phlrain phlrain Mar 20, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done
zyfncg
zyfncg previously approved these changes Mar 20, 2022
View reviewed changes
chenwhql
chenwhql reviewed Mar 21, 2022
View reviewed changes
paddle/phi/kernels/cpu/sparse_weight_embedding_grad_kernel.cc Outdated
} else if (input.dtype() == phi::DataType::INT64) {
functor.template apply<int64_t>();
} else {
PADDLE_THROW("emebdding input only support int32 and int64");
Copy link
Contributor

@chenwhql chenwhql Mar 21, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这不符合报错规范，需要加报错类型
Copy link
Collaborator Author

@phlrain phlrain Mar 21, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已修改
paddle/phi/kernels/cpu/sparse_weight_embedding_grad_kernel.cc Outdated
} else if (input.dtype() == phi::DataType::INT64) {
functor.template apply<int64_t>();
} else {
PADDLE_THROW("emebdding input only support int32 and int64");
Copy link
Contributor

@chenwhql chenwhql Mar 21, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

同上
Copy link
Collaborator Author

@phlrain phlrain Mar 21, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已修改
paddle/phi/kernels/gpu/embedding_grad_kernel.cu Outdated
} else if (input.dtype() == phi::DataType::INT64) {
functor.template apply<int64_t>();
} else {
PADDLE_THROW("emebdding input only support int32 and int64");
Copy link
Contributor

@chenwhql chenwhql Mar 21, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

同上，共性问题，麻烦都check一下
paddle/phi/kernels/sparse_weight_embedding_grad_kernel.h
namespace phi {

template <typename T, typename Context>
void SparseWeightEmbeddingGradKernel(const Context& ctx,
Copy link
Contributor

@chenwhql chenwhql Mar 21, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SelectedRows相关的尽快移到selected_rows目录
@phlrain phlrain closed this Mar 21, 2022
@phlrain phlrain reopened this Mar 21, 2022
chenwhql
chenwhql approved these changes Mar 22, 2022
View reviewed changes
Copy link
Contributor

@chenwhql chenwhql left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

如评论，selected rows kernel后续再调整下目录和形式
@phlrain
Copy link
Collaborator Author

phlrain commented Mar 22, 2022

如评论，selected rows kernel后续再调整下目录和形式

TODO已记录
XiaoguangHu01
XiaoguangHu01 approved these changes Mar 22, 2022
View reviewed changes
Copy link
Contributor

@XiaoguangHu01 XiaoguangHu01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
@phlrain phlrain merged commit 0331cfd into develop Mar 22, 2022
@chenwhql chenwhql deleted the move_embedding_to_phi branch March 22, 2022 14:30
@Liangwei-0521
Copy link

Liangwei-0521 commented Jul 26, 2023

Traceback (most recent call last):
File "D:\rank_v1.2\quant_predictor_2.py", line 191, in
results.extend(predictor.predict(batch_data, tokenizer))
File "D:\rank_v1.2\quant_predictor_2.py", line 134, in predict
self.predictor.run()
NotImplementedError: (Unimplemented) There are no kernels which are registered in the lookup_table_v2 operator.
[Hint: Expected kernels_iter != all_op_kernels.end(), but received kernels_iter == all_op_kernels.end().] (at C:\home\workspace\Paddle\paddle\fluid\framework\operator.cc:1895)
[operator < lookup_table_v2 > error

embedding量化后，导入静态模型报错。
@Liangwei-0521
Copy link

Liangwei-0521 commented Jul 26, 2023

embedding量化代码
def _quant_embeddings(input_prefix):

# input_prefix: 静态模型路径 # 最终量化的模型路径： output_filename_prefix = "rank_quant_emb" paddle.enable_static() place = paddle.set_device('cpu') exe = paddle.static.Executor(place) main_program, feed_target_names, fetch_targets = paddle.static.load_inference_model(input_prefix, exe) config = { "quantize_op_types": ["lookup_table_v2"], "lookup_table_v2": { "quantize_type": "log", 'quantize_bits': 8, 'dtype': 'int8' }} quant_emb_program = quant.quant_embedding(main_program, place, config) input_dir = os.path.dirname(input_prefix) paddle.fluid.io.save_inference_model( input_dir, feed_target_names, fetch_targets, exe, quant_emb_program, model_filename=output_filename_prefix + ".pdmodel", params_filename=output_filename_prefix + ".pdiparams", export_for_deployment=True, program_only=False, )

if name == 'main':
_quant_embeddings(input_prefix='./checkpoint/quant')
@zyfncg
Copy link
Contributor

zyfncg commented Jul 27, 2023

Traceback (most recent call last): File "D:\rank_v1.2\quant_predictor_2.py", line 191, in results.extend(predictor.predict(batch_data, tokenizer)) File "D:\rank_v1.2\quant_predictor_2.py", line 134, in predict self.predictor.run() NotImplementedError: (Unimplemented) There are no kernels which are registered in the lookup_table_v2 operator. [Hint: Expected kernels_iter != all_op_kernels.end(), but received kernels_iter == all_op_kernels.end().] (at C:\home\workspace\Paddle\paddle\fluid\framework\operator.cc:1895) [operator < lookup_table_v2 > error

embedding量化后，导入静态模型报错。

@1998-Chen 这个问题在issue里提下吧，在PR下面不太好跟踪
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

6 participants

@phlrain @Liangwei-0521 @zyfncg @chenwhql @XiaoguangHu01