add auto_parallel context_parallel strategy

[danleifeng]

Before submitting

• Lint code. If there are lint issues, please format the code first.

# Install and register `pre-commit` in the project folder pip install pre-commit && pre-commit install # Process previous code files separately pre-commit run --file XXXX.py

• Add test cases into tests folder. If there are codecov issues, please add tests cases first.

PR types

New features

PR changes

Models

Description

新增自动并行contextparallel策略，此pr包含基础api和中层api的llama模型适配，3000steps收敛对齐非cp策略
ps：paddle需基于最新3.1.0之后的（6.20之后的develop）

[paddle-bot]

Thanks for your contribution!

[codecov]

Codecov Report

❌ Patch coverage is 8.33333% with 176 lines in your changes missing coverage. Please review.
✅ Project coverage is 46.76%. Comparing base (78741f9) to head (cb225b3).
⚠️ Report is 61 commits behind head on develop.

Files with missing lines	Patch %	Lines
paddlenlp/transformers/llama/modeling_network.py	6.89%	108 Missing ⚠️
paddlenlp/transformers/llama/modeling_auto.py	6.81%	41 Missing ⚠️
paddlenlp/transformers/context_parallel_utils.py	14.28%	12 Missing ⚠️
paddlenlp/transformers/model_utils.py	0.00%	9 Missing ⚠️
paddlenlp/trainer/auto_trainer.py	0.00%	3 Missing ⚠️
paddlenlp/trainer/training_args.py	50.00%	3 Missing ⚠️

Additional details and impacted files

@@ Coverage Diff @@ ## develop #10722 +/- ## =========================================== - Coverage 46.80% 46.76% -0.05%  =========================================== Files 802 802 Lines 133728 133882 +154 =========================================== + Hits 62597 62605 +8  - Misses 71131 71277 +146 

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[jeff41404]

LGTM

[ZHUI]

这个需要加 try-catch吗?

[danleifeng]

paddle3.1已包含：https://github.com/PaddlePaddle/Paddle/blob/v3.1.0/python/paddle/distributed/auto_parallel/ring_attention.py#L22