Skip to content

Commit da4d7a3

Browse files
ZailiWangDiweiSun
andauthored
remove req.txt in LLM inference example env_activate (#3848)
* remove req.txt in LLM inference example env_activate * CI fail debug logs * disable exit on error since the error is expected * CI script will take care of `set` commands * remove KMP BARRIER env for testing; remove torchscript in README.md * restore KMP BARRIER env variables --------- Co-authored-by: DiweiSun <105627594+DiweiSun@users.noreply.github.com>
1 parent de626b7 commit da4d7a3

File tree

3 files changed

+1
-27
lines changed

3 files changed

+1
-27
lines changed

examples/cpu/inference/python/python-scripts/README.md

Lines changed: 0 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,6 @@ We provided inference examples for eager mode as well as graph mode, in which th
2323
Eager mode is the default execution mode in PyTorch, the codes are executed in a “define-by-run” paradigm, so it is flexible, interactive and easy to debug.
2424
On the other hand, in graph mode the codes are executed in “define-and-run” paradigm, which means the building of the entire computation graph is required before running the function.
2525
During the graph compilation process, optimizations like layer fusion and folding are applied, and the compiled graphs are more friendly for backend optimizations, leading to accelerated execution.
26-
TorchScript and TorchDynamo are the 2 graph compiling tools that PyTorch\* provides.
2726

2827
From numerical precision perspective, we provided inference examples for [BFloat16](#bfloat16) and [INT8 quantization](#int8) in addition to the default [Float32](#float32) precision.
2928
Low-precision approaches including [Automatic Mixed Precision (AMP)](https://pytorch.org/docs/stable/amp.html) and [quantization](https://pytorch.org/docs/stable/quantization.html) are commonly used in PyTorch\* to improve performance.
@@ -44,12 +43,6 @@ Running ResNet50 inference in eager mode:
4443
python resnet50_eager_mode_inference_fp32.py
4544
```
4645

47-
Running ResNet50 inference in TorchScript mode:
48-
49-
```bash
50-
python resnet50_torchscript_mode_inference_fp32.py
51-
```
52-
5346
Running ResNet50 inference in TorchDynamo mode:
5447

5548
```bash
@@ -62,12 +55,6 @@ Running BERT inference in eager mode:
6255
python bert_eager_mode_inference_fp32.py
6356
```
6457

65-
Running BERT inference in TorchScript mode:
66-
67-
```bash
68-
python bert_torchscript_mode_inference_fp32.py
69-
```
70-
7158
Running BERT inference in TorchDynamo mode:
7259

7360
```bash
@@ -82,12 +69,6 @@ Running ResNet50 inference in eager mode:
8269
python resnet50_eager_mode_inference_bf16.py
8370
```
8471

85-
Running ResNet50 inference in TorchScript mode:
86-
87-
```bash
88-
python resnet50_torchscript_mode_inference_bf16.py
89-
```
90-
9172
Running ResNet50 inference in TorchDynamo mode:
9273

9374
```bash
@@ -100,12 +81,6 @@ Running BERT inference in eager mode:
10081
python bert_eager_mode_inference_bf16.py
10182
```
10283

103-
Running BERT inference in TorchScript mode:
104-
105-
```bash
106-
python bert_torchscript_mode_inference_bf16.py
107-
```
108-
10984
Running BERT inference in TorchDynamo mode:
11085

11186
```bash

examples/cpu/llm/inference/requirements.txt

Lines changed: 0 additions & 1 deletion
This file was deleted.

examples/cpu/llm/tools/env_activate.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -58,7 +58,7 @@ if [ ${MODE} == "inference" ]; then
5858
export PATH=${PATH}:${JAVA_HOME}/bin
5959
cd ${MODE}
6060

61-
python -m pip install -r requirements.txt
61+
python -m pip install transformers==4.51.3
6262
if [ -f prompt.json ]; then
6363
rm -f prompt.json
6464
fi

0 commit comments

Comments
 (0)