[FlashMask] Add FlashMask for Qwen2

[DrownFish19]

PR types

New features

PR changes

Others

Description

Add FlashMask for Qwen2.

对齐验证步骤：

1. 修改shuffle为False
   

   PaddleNLP/paddlenlp/trainer/trainer.py

   Lines 1326 to 1345 in 0e96b0f

   	def _get_train_sampler(self) -> Optional[paddle.io.Sampler]:
   	if self.train_dataset is None or not has_length(self.train_dataset):
   	return None
   	if self.args.world_size <= 1:
   	return paddle.io.BatchSampler(
   	dataset=self.train_dataset,
   	shuffle=True,
   	batch_size=self.args.per_device_train_batch_size,
   	drop_last=self.args.dataloader_drop_last,
   	)
   	return DistributedBatchSampler(
   	self.train_dataset,
   	batch_size=self.args.per_device_train_batch_size,
   	shuffle=True,
   	num_replicas=self.args.dataset_world_size,
   	rank=self.args.dataset_rank,
   	drop_last=self.args.dataloader_drop_last,
   	)

2. 验证指令

2.1 验证单卡与流水线并行的一致性

# pipeline parallel python -m paddle.distributed.launch \ --devices 6,7 \ run_finetune.py \ config/qwen/sft_argument.json \ --model_name_or_path Qwen/Qwen2-0.5B \ --gradient_accumulation_steps 1 \ --zero_padding true \ --flash_mask true \ --pipeline_parallel_degree 2

# data parallel python -m paddle.distributed.launch \ --devices 7 \ run_finetune.py \ config/qwen/sft_argument.json \ --model_name_or_path Qwen/Qwen2-0.5B \ --gradient_accumulation_steps 1 \ --zero_padding true \ --flash_mask true \ --pipeline_parallel_degree 1

对齐验证结果：

• modeling + flashmask vs. modeling_pp + flashmask 前4步训练loss无diff，后续逐渐出现随机误差，误差范围在1e-3，符合bf16精度。

2.2 验证flashmask开关前后正确性
验证指令

# pipeline parallel python -m paddle.distributed.launch \ --devices 6,7 \ run_finetune.py \ config/qwen/sft_argument.json \ --model_name_or_path Qwen/Qwen2-0.5B \ --gradient_accumulation_steps 1 \ --zero_padding true \ --flash_mask true \ --pad_to_max_length true --pipeline_parallel_degree 2

数据类型为bf16时，训练loss误差范围为1e-3
数据类型为fp16时，训练loss误差范围为1e-4

[paddle-bot]

Thanks for your contribution!

[codecov]

Codecov Report

Attention: Patch coverage is 20.37037% with 43 lines in your changes missing coverage. Please review.

Project coverage is 52.62%. Comparing base (220cc95) to head (b4a0ba5).
Report is 263 commits behind head on develop.

Files with missing lines	Patch %	Lines
paddlenlp/transformers/qwen2/modeling_pp.py	9.52%	38 Missing ⚠️
paddlenlp/transformers/qwen2/modeling.py	58.33%	5 Missing ⚠️

Additional details and impacted files

@@ Coverage Diff @@ ## develop #9264 +/- ## =========================================== - Coverage 52.84% 52.62% -0.22%  =========================================== Files 661 661 Lines 107783 107365 -418 =========================================== - Hits 56955 56501 -454  - Misses 50828 50864 +36 

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[lugimzzz]

后续验证flashmask和fa2，可以直接用加上,能够所有loss逐位对齐即可
export FLAGS_cudnn_deterministic=1
export FLAGS_embedding_deterministic=1

[lugimzzz]

LGTM

[ZHUI]

todo：attn_mask_startend_row_indices 这个参数可以加内置函数里面吧

[ZHUI]

PaddleNLP/paddlenlp/trainer/trainer.py

Line 1978 in 2b975b1

ret = tuple([inputs.pop(k) for k in keys if k in inputs])

会有None是不是？

[lugimzzz]

LGTM