Skip to content

Commit 6f39984

Browse files
authored
Update README.md
1 parent a415bb4 commit 6f39984

File tree

1 file changed

+3
-10
lines changed

1 file changed

+3
-10
lines changed

README.md

Lines changed: 3 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -46,19 +46,12 @@ and has open-sourced the training framework.
4646

4747
## Evaluation
4848

49-
We provide the code for evaluating LLaDA-8B-Base using the [lm evaluation harness](https://github.com/EleutherAI/lm-evaluation-harness) library. Please refer to the `./evaluation` directory.
49+
We use two evaluation methods: conditional likelihood estimation and conditional generation. For the base model, conditional likelihood estimation is applied to specific metrics and conditional generation to the rest. For the Instruct model, conditional generation is used for all metrics.
5050

51-
To run the evaluation, you need to install the lm evaluation harness library
51+
We implement conditional likelihood estimation using the [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness) library, while conditional generation is performed with an internal library, as lm-evaluation-harness lacks support for certain metrics (i..e, HumanEval-FIM). For details, see Appendix B.5 of our paper.
5252

53-
```angular2html
54-
pip install lm_eval==0.4.5
55-
```
56-
57-
Then, replace the `<path_to_llada_base_model>` in `eval.sh` with your LLaDA model path and run:
53+
We provide the code for evaluation using the open-source library lm-evaluation-harness. To begin, please install `lm_eval==0.4.5` and refer to `evaluation/eval.sh` for the specific commands.
5854

59-
```angular2html
60-
bash ./evaluation/eval.sh
61-
```
6255

6356
## FAQ
6457
Here, we address some common questions about LLaDA.

0 commit comments

Comments
 (0)