Merge branch 'PaddlePaddle:develop' into dev_add_tokenizer_fast

Author: DrownFish19

README.md

@@ -72,6 +72,7 @@ Unified Checkpoint 大模型存储格式在模型参数分布上支持动态扩
 | [LLaMA](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/llm/config/llama) | facebook/llama-7b, facebook/llama-13b, facebook/llama-30b, facebook/llama-65b |
 | [LLama2](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/llm/config/llama) | meta-llama/Llama-2-7b, meta-llama/Llama-2-7b-chat, meta-llama/Llama-2-13b, meta-llama/Llama-2-13b-chat, meta-llama/Llama-2-70b, meta-llama/Llama-2-70b-chat |
 | [LLama3](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/llm/config/llama) | meta-llama/Meta-Llama-3-8B, meta-llama/Meta-Llama-3-8B-Instruct, meta-llama/Meta-Llama-3-70B, meta-llama/Meta-Llama-3-70B-Instruct |
+| [LLama3.1](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/llm/config/llama) | meta-llama/Meta-Llama-3.1-8B, meta-llama/Meta-Llama-3.1-8B-Instruct, meta-llama/Meta-Llama-3.1-70B, meta-llama/Meta-Llama-3.1-70B-Instruct, meta-llama/Meta-Llama-3.1-405B, meta-llama/Meta-Llama-3.1-405B-Instruct, meta-llama/Llama-Guard-3-8B |
 | [Baichuan](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/llm/config/baichuan) | baichuan-inc/Baichuan-7B, baichuan-inc/Baichuan-13B-Base, baichuan-inc/Baichuan-13B-Chat |
 | [Baichuan2](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/llm/config/baichuan) | baichuan-inc/Baichuan2-7B-Base, baichuan-inc/Baichuan2-7B-Chat, baichuan-inc/Baichuan2-13B-Base, baichuan-inc/Baichuan2-13B-Chat |
 | [Bloom](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/llm/config/bloom) | bigscience/bloom-560m, bigscience/bloom-560m-bf16, bigscience/bloom-1b1, bigscience/bloom-3b, bigscience/bloom-7b1, bigscience/bloomz-560m, bigscience/bloomz-1b1, bigscience/bloomz-3b, bigscience/bloomz-7b1-mt, bigscience/bloomz-7b1-p3, bigscience/bloomz-7b1, bellegroup/belle-7b-2m |
@@ -95,6 +96,7 @@ Unified Checkpoint 大模型存储格式在模型参数分布上支持动态扩
 | Llama | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
 | Llama2 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
 | Llama3 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
+| Llama3.1 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
 | Qwen | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
 | Qwen1.5 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
 | Qwen2 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |

csrc/generation/test_tune_cublaslt_gemm.py

@@ -0,0 +1,25 @@
+# Copyright (c) 2024 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from paddlenlp_ops import tune_cublaslt_gemm
+import paddle
+
+M_tensor = paddle.to_tensor([1024])
+K_tensor = paddle.to_tensor([1024, 2048])
+N_tensor = paddle.to_tensor([4096, 8192])
+
+Dtype = "int8"
+Path = "./search.csv"
+
+tune_cublaslt_gemm(M_tensor, K_tensor, N_tensor, Dtype, True, False, Path)