This repository was archived by the owner on Oct 25, 2024. It is now read-only.
Upgrade jblas #165
Merged
Upgrade jblas #165
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
...sion_for_transformers/backends/neural_engine/graph/jblas/jblas/jit_blas_weight_compression.h Show resolved Hide resolved
luoyu-intel force-pushed the graph-int8-quantization branch 2 times, most recently from c9146ed to a31b888 Compare Co-Authored-By: Ding, Yi1 <yi1.ding@intel.com> Co-Authored-By: Wang, Zhe1 <zhe1.wang@intel.com> rebase with main add fusion support for llama add fp8 ffn_silu fusion update to jblas 8ad9853 fix hasISA issue fix gcc9 compile fix bug of fp16 weight's quant fix 4bit size add fusion support for gemm_add enable ffn_gelu_add sync jblas, pass compilation fix gcc error fix bug. remove lm_head from non_quant fix mha Signed-off-by: Ding, Yi1 <yi1.ding@intel.com> sync with QBits updates. fix f4 scale
luoyu-intel force-pushed the graph-int8-quantization branch from a31b888 to 7c92d1a Compare 
DDEle reviewed Aug 24, 2023
intel_extension_for_transformers/backends/neural_engine/graph/jblas/CMakeLists.txt Show resolved Hide resolved
Remove the high gcc version requirement. auto-fusion: depends on weight type and runtime ISA support. Signed-off-by: luoyu-intel <yu.luo@intel.com> Co-Authored-By: Ding, Yi1 <yi1.ding@intel.com> Co-Authored-By: Wang, Zhe1 <zhe1.wang@intel.com>
…tel-extension-for-transformers into graph-int8-quantization
lvliang-intel pushed a commit that referenced this pull request Aug 24, 2023
add fusion support for llama add fp8 ffn_silu fusion fix hasISA issue fix gcc9 compile fix bug of fp16 weight's quant fix 4bit size add fusion support for gemm_add enable ffn_gelu_add sync jblas, pass compilation fix gcc error fix bug. remove lm_head from non_quant fix mha sync with QBits updates. fix f4 scale Synchronize jblas code. Remove the high gcc version requirement. auto-fusion: depends on weight type and runtime ISA support. --------- Signed-off-by: luoyu-intel <yu.luo@intel.com> Co-authored-by: Ding, Yi1 <yi1.ding@intel.com> Co-authored-by: Wang, Zhe1 <zhe1.wang@intel.com> Signed-off-by: lvliang-intel <liang1.lv@intel.com>
airMeng reviewed Nov 10, 2023
| static GemmKernel kernel; | ||
| assert(cd->AVX512F()); | ||
| packedw = kernel.getWeightPtr()->compressWeightTranspose(n, k, f32ptr, k, params.block_size, type); | ||
| if (params.scale_dtype == quant_sdtype::fp32) { |
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments. Suggestions cannot be applied while the pull request is queued to merge. Suggestion cannot be applied right now. Please check back later.
Type of Change
Upgrade jblas: a more flexible dynamic-quant interface.
Description
How has this PR been tested?
gptj-6B and llama-7B model.