Skip to content

Commit f3322d1

Browse files
rohan-paulrohan-paul
committed
updating
1 parent e9fea6b commit f3322d1

File tree

3 files changed

+400
-7
lines changed

3 files changed

+400
-7
lines changed

‎LlaMa-2/LlaMa-2-FineTuning.ipynb‎

Lines changed: 23 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -21,20 +21,36 @@
2121
"metadata": {},
2222
"source": [
2323
"# From source. For running the examples in the repository\n",
24-
"# Clone the repository and install it with pip:\n",
25-
"\n",
26-
"git clone https://github.com/lvwerra/trl.git\n",
27-
"cd trl/\n",
28-
"pip install .\n",
24+
"# Clone the repository and install it with pip:"
25+
]
26+
},
27+
{
28+
"cell_type": "code",
29+
"execution_count": null,
30+
"metadata": {},
31+
"outputs": [],
32+
"source": [
33+
"!git clone https://github.com/lvwerra/trl.git\n",
2934
"\n",
35+
"!cd trl/\n",
3036
"\n",
31-
"python examples/scripts/sft_trainer.py \\\n",
37+
"! pip install ."
38+
]
39+
},
40+
{
41+
"cell_type": "markdown",
42+
"metadata": {},
43+
"source": [
44+
"```py\n",
45+
"!python examples/scripts/sft_trainer.py \\\n",
3246
" --model_name meta-llama/Llama-2-7b-hf \\\n",
3347
" --dataset_name timdettmers/openassistant-guanaco \\\n",
3448
" --load_in_4bit \\\n",
3549
" --use_peft \\\n",
3650
" --batch_size 4 \\\n",
37-
" --gradient_accumulation_steps 2"
51+
" --gradient_accumulation_steps 2\n",
52+
"\n",
53+
"```"
3854
]
3955
}
4056
],

‎NLP/Compute_Metric_Method_for_LLM_FineTuning/compute_metrics_in_an_LLM_FineTuning_Code_FINAL.ipynb‎

Lines changed: 323 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,323 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"metadata": {},
6+
"source": [
7+
"🚀 **The all IMPORTANT compute_metrics() method implemented in all LLM (Large Language Model) FineTuning Project**🚀🔥\n"
8+
]
9+
},
10+
{
11+
"cell_type": "markdown",
12+
"metadata": {},
13+
"source": [
14+
"---------------------\n",
15+
"\n",
16+
"👉 **How does the `tokenizer.batch_decode` really work**\n",
17+
"\n",
18+
"👉 The `tokenizer.batch_decode` method is a part of the tokenizer object provided by Hugging Face's transformers library. It's used to convert tokenized sequences back into human-readable text, performing the inverse operation of the tokenization process.\n",
19+
"\n",
20+
"👉 When you tokenize a piece of text, the tokenizer converts each word or subword into a numerical representation, known as a token. For example, the sentence \"I love AI\" might be tokenized into something like `[72, 1801, 1789]`. The tokenizer maintains a mapping from these numerical tokens back to the original words or subwords.\n",
21+
"\n",
22+
"👉 The `tokenizer.batch_decode` function takes as input a batch of these token sequences (each sequence is a list of integers) and decodes them into their original textual form. If the `skip_special_tokens` flag is set to `True`, it will ignore special tokens like padding, start of sentence, end of sentence, etc., when converting back to text.\n",
23+
"\n",
24+
"Here is a basic example:\n",
25+
"\n",
26+
"```python\n",
27+
"from transformers import AutoTokenizer\n",
28+
"\n",
29+
"# Initialize a tokenizer\n",
30+
"tokenizer = AutoTokenizer.from_pretrained(\"bert-base-uncased\")\n",
31+
"\n",
32+
"# Suppose we have the following tokenized input\n",
33+
"input_ids = tokenizer.encode(\"Hello, I'm a data scientist.\", return_tensors='pt')\n",
34+
"\n",
35+
"print(f\"Tokenized Input: {input_ids}\")\n",
36+
"# Output: tensor([[ 101, 7592, 1010, 1045, 1005, 1049, 1037, 2951, 7155, 1012, 102]])\n",
37+
"\n",
38+
"# Now, we decode it back to the original text\n",
39+
"decoded_text = tokenizer.batch_decode(input_ids, skip_special_tokens=True)\n",
40+
"\n",
41+
"print(f\"Decoded Text: {decoded_text}\")\n",
42+
"# Output: [\"hello, i'm a data scientist.\"]\n",
43+
"```\n",
44+
"\n",
45+
"In this example, the tokenizer first encoded the input sentence into a sequence of tokens. Then, using the `batch_decode` method, the token sequence was decoded back to the original text."
46+
]
47+
},
48+
{
49+
"cell_type": "markdown",
50+
"metadata": {},
51+
"source": [
52+
"----------------\n",
53+
"\n",
54+
" **Why does the code replaces any instance of `-100` with `50256`**\n",
55+
"\n",
56+
" 👉 In the given code snippet, `-100` is a special token ID that is often used to represent tokens that should be ignored when computing the loss during model training. These might be extra padding tokens, or tokens that are not part of the target sequence in a sequence-to-sequence model.\n",
57+
"\n",
58+
"👉 During the process of training a language model, not all tokens in the output sequence are always relevant. For instance, we may have padded sequences to a particular length with irrelevant tokens or we may want to mask certain tokens. In these scenarios, we assign a token ID of `-100` to those tokens we wish the model to ignore when it computes the loss. \n",
59+
"\n",
60+
"👉 However, when it's time to evaluate the model, the `-100` tokens cannot be decoded back into text, since it's not a valid token ID. So, before decoding the sequence, these tokens must be replaced with a valid token ID.\n",
61+
"\n",
62+
"👉 The ID `50256` is chosen here because it corresponds to the padding token in the tokenizer being used. When `tokenizer.batch_decode` is called with the argument `skip_special_tokens=True`, it will automatically ignore these padding tokens, so they don't show up in the final decoded text. This is why any `-100` tokens are replaced with `50256` before decoding the predicted sequences.\n",
63+
"\n",
64+
"Here is a small pseudo-example to illustrate this process:\n",
65+
"\n",
66+
"```python\n",
67+
"preds = [[1, 2, -100, 3], [2, -100, 1, 4]]\n",
68+
"# These are the predicted sequences. -100 represents tokens to be ignored\n",
69+
"\n",
70+
"for idx in range(len(preds)):\n",
71+
" for idx2 in range(len(preds[idx])):\n",
72+
" if preds[idx][idx2]==-100:\n",
73+
" preds[idx][idx2] = 50256\n",
74+
"\n",
75+
"print(preds)\n",
76+
"# Output: [[1, 2, 50256, 3], [2, 50256, 1, 4]]\n",
77+
"# Now all -100 tokens are replaced with 50256 (padding token) and can be properly ignored when decoding\n",
78+
"```\n"
79+
]
80+
},
81+
{
82+
"cell_type": "markdown",
83+
"metadata": {},
84+
"source": [
85+
"--------------\n",
86+
"\n",
87+
"**What is the significance of the below line here**\n",
88+
"\n",
89+
"`labels = np.where(labels != pad_tok, labels, tokenizer.pad_token_id)`\n",
90+
"\n",
91+
"👉 The `np.where` function works like this: `np.where(condition, x, y)`. It checks the `condition` for each element in the array. If the condition is `True`, it selects the corresponding element from `x`; if the condition is `False`, it selects the corresponding element from `y`.\n",
92+
"\n",
93+
"The condition is `labels != pad_tok`. It checks each element in the `labels` array and sees if it's not equal to `pad_tok`. `pad_tok` is the padding token, which is typically used to fill in sequences to a consistent length for batch processing. If the condition is `True` (the label is not a padding token), it keeps the original label. If the condition is `False` (the label is a padding token), it replaces that label with `tokenizer.pad_token_id`.\n",
94+
"\n",
95+
"👉 This operation ensures that all instances of the padding token in the labels are represented with the padding token ID specific to the tokenizer being used. This is important for the subsequent decoding of the labels into text, as the padding tokens will be correctly ignored during the decoding process."
96+
]
97+
},
98+
{
99+
"cell_type": "code",
100+
"execution_count": null,
101+
"metadata": {},
102+
"outputs": [],
103+
"source": [
104+
"def compute_metrics(eval_preds):\n",
105+
" preds, labels = eval_preds\n",
106+
" if isinstance(preds, tuple):\n",
107+
" preds = preds[0]\n",
108+
" for idx in range(len(preds)):\n",
109+
" for idx2 in range(len(preds[idx])):\n",
110+
" if preds[idx][idx2]==-100:\n",
111+
" preds[idx][idx2] = 50256\n",
112+
" decoded_preds = tokenizer.batch_decode(preds, skip_special_tokens=True)\n",
113+
" # Replace -100 in the labels as we can't decode them.\n",
114+
" labels = np.where(labels != pad_tok, labels, tokenizer.pad_token_id)\n",
115+
" decoded_labels = tokenizer.batch_decode(labels, skip_special_tokens=True)\n",
116+
"\n",
117+
" # Some simple post-processing\n",
118+
" decoded_preds, decoded_labels = postprocess_text(decoded_preds, decoded_labels)\n",
119+
"\n",
120+
" result = metric.compute(predictions=decoded_preds, references=decoded_labels, use_stemmer=True)\n",
121+
" \n",
122+
" # The results of the Rouge metric are then multiplied by 100 and rounded to four decimal places.\n",
123+
" result = {k: round(v * 100, 4) for k, v in result.items()}\n",
124+
" prediction_lens = [np.count_nonzero(pred != tokenizer.pad_token_id) for pred in preds]\n",
125+
" result[\"gen_len\"] = np.mean(prediction_lens)\n",
126+
" return result"
127+
]
128+
},
129+
{
130+
"cell_type": "markdown",
131+
"metadata": {},
132+
"source": [
133+
"==============\n",
134+
"\n",
135+
"FULL SOURCE CODE\n",
136+
"\n",
137+
"=============="
138+
]
139+
},
140+
{
141+
"cell_type": "code",
142+
"execution_count": null,
143+
"metadata": {},
144+
"outputs": [],
145+
"source": [
146+
"# Source - https://github.com/artidoro/qlora/issues/157\n",
147+
"import pandas as pd\n",
148+
"import os\n",
149+
"from transformers import AutoTokenizer, AutoModelForSeq2SeqLM, BitsAndBytesConfig\n",
150+
"import torch\n",
151+
"from peft import LoraConfig, get_peft_model, prepare_model_for_int8_training, TaskType, prepare_model_for_kbit_training\n",
152+
"from transformers import DataCollatorForSeq2Seq\n",
153+
"import evaluate\n",
154+
"import nltk\n",
155+
"import numpy as np\n",
156+
"from nltk.tokenize import sent_tokenize\n",
157+
"from transformers import Seq2SeqTrainer, Seq2SeqTrainingArguments\n",
158+
"from datasets import Dataset, DatasetDict\n",
159+
"import argparse\n",
160+
"import pickle\n",
161+
"import json\n",
162+
"\n",
163+
"parser = argparse.ArgumentParser(description='Options')\n",
164+
"parser.add_argument('--dataset_dir', default='data', type=str, help=\"folder in which the dataset is stored\")\n",
165+
"parser.add_argument('--output_dir', default=\"lora-instructcodet5p\", type=str, help=\"output directory for the model\")\n",
166+
"parser.add_argument('--results_dir', default=\"results\", type=str, help=\"where the results should be stored\")\n",
167+
"args = parser.parse_args()\n",
168+
"\n",
169+
"nltk.download(\"punkt\")\n",
170+
"tokenized_dataset = DatasetDict.load_from_disk(args.dataset_dir)\n",
171+
"# Metric\n",
172+
"metric = evaluate.load(\"rouge\")\n",
173+
"pad_tok = 50256\n",
174+
"token_id=\"Salesforce/instructcodet5p-16b\"\n",
175+
"\n",
176+
"tokenizer = AutoTokenizer.from_pretrained(token_id)\n",
177+
"# helper function to postprocess text\n",
178+
"def postprocess_text(preds, labels):\n",
179+
" preds = [pred.strip() for pred in preds]\n",
180+
" labels = [label.strip() for label in labels]\n",
181+
"\n",
182+
" # rougeLSum expects newline after each sentence\n",
183+
" preds = [\"\\n\".join(sent_tokenize(pred)) for pred in preds]\n",
184+
" labels = [\"\\n\".join(sent_tokenize(label)) for label in labels]\n",
185+
"\n",
186+
" return preds, labels\n",
187+
"\n",
188+
"def compute_metrics(eval_preds):\n",
189+
" preds, labels = eval_preds\n",
190+
" if isinstance(preds, tuple):\n",
191+
" preds = preds[0]\n",
192+
" for idx in range(len(preds)):\n",
193+
" for idx2 in range(len(preds[idx])):\n",
194+
" if preds[idx][idx2]==-100:\n",
195+
" preds[idx][idx2] = 50256\n",
196+
" decoded_preds = tokenizer.batch_decode(preds, skip_special_tokens=True)\n",
197+
" # Replace -100 in the labels as we can't decode them.\n",
198+
" labels = np.where(labels != pad_tok, labels, tokenizer.pad_token_id)\n",
199+
" decoded_labels = tokenizer.batch_decode(labels, skip_special_tokens=True)\n",
200+
"\n",
201+
" # Some simple post-processing\n",
202+
" decoded_preds, decoded_labels = postprocess_text(decoded_preds, decoded_labels)\n",
203+
"\n",
204+
" result = metric.compute(predictions=decoded_preds, references=decoded_labels, use_stemmer=True)\n",
205+
" result = {k: round(v * 100, 4) for k, v in result.items()}\n",
206+
" prediction_lens = [np.count_nonzero(pred != tokenizer.pad_token_id) for pred in preds]\n",
207+
" result[\"gen_len\"] = np.mean(prediction_lens)\n",
208+
" return result\n",
209+
"\n",
210+
"def get_dict(predicts):\n",
211+
" d = {}\n",
212+
" for num in range(len(tokenized_dataset['test'])):\n",
213+
" pred = tokenizer.decode([n for n in predicts[0][num] if n!=50256 and n!=-100])[1:]\n",
214+
" d[num+1] = {'Question':tokenizer.decode([n for n in tokenized_dataset['test'][num]['input_ids'] if n!=50256]),\n",
215+
" 'Ground truth solution':tokenizer.decode([n for n in tokenized_dataset['test'][num]['labels'] if n!=50256]),\n",
216+
" 'Prediction': pred if pred else None}\n",
217+
" return d\n",
218+
"\n",
219+
"def find_all_linear_names(model):\n",
220+
" cls = torch.nn.Linear\n",
221+
" lora_module_names = set()\n",
222+
" for name, module in model.named_modules():\n",
223+
" if isinstance(module, cls):\n",
224+
" names = name.split('.')\n",
225+
" lora_module_names.add(names[0] if len(names) == 1 else names[-1])\n",
226+
"\n",
227+
"\n",
228+
" if 'lm_head' in lora_module_names:\n",
229+
" lora_module_names.remove('lm_head')\n",
230+
" return list(lora_module_names)\n",
231+
"\n",
232+
"\n",
233+
"def main():\n",
234+
" device = 'cuda'\n",
235+
"\n",
236+
" # huggingface hub model id\n",
237+
" model_id=\"instructcodet5p-16b\"\n",
238+
" if not os.path.exists(model_id):\n",
239+
" model_id=token_id\n",
240+
" bnb_config = BitsAndBytesConfig(\n",
241+
" load_in_4bit=True,\n",
242+
" bnb_4bit_use_double_quant=True,\n",
243+
" bnb_4bit_quant_type=\"nf4\",\n",
244+
" bnb_4bit_compute_dtype=torch.bfloat16\n",
245+
" )\n",
246+
" # load model from the hub\n",
247+
" model = AutoModelForSeq2SeqLM.from_pretrained(model_id,\n",
248+
" # torch_dtype=torch.bfloat16,\n",
249+
" low_cpu_mem_usage=True,\n",
250+
" trust_remote_code=True, decoder_start_token_id=1, pad_token_id=pad_tok, device_map=\"auto\", quantization_config=bnb_config)\n",
251+
" modules = find_all_linear_names(model)\n",
252+
" # Define LoRA Config\n",
253+
" lora_config = LoraConfig(\n",
254+
" r=16,\n",
255+
" lora_alpha=32,\n",
256+
" target_modules=modules,\n",
257+
" lora_dropout=0.05,\n",
258+
" bias=\"none\",\n",
259+
" task_type=TaskType.SEQ_2_SEQ_LM\n",
260+
" )\n",
261+
" model = prepare_model_for_kbit_training(model, False)\n",
262+
"\n",
263+
" # add LoRA adaptor\n",
264+
" model = get_peft_model(model, lora_config)\n",
265+
" model.print_trainable_parameters()\n",
266+
"\n",
267+
" # we want to ignore tokenizer pad token in the loss\n",
268+
" label_pad_token_id = pad_tok\n",
269+
" # Data collator\n",
270+
" data_collator = DataCollatorForSeq2Seq(\n",
271+
" tokenizer,\n",
272+
" model=model,\n",
273+
" label_pad_token_id=label_pad_token_id,\n",
274+
" pad_to_multiple_of=8\n",
275+
" )\n",
276+
" output_dir=args.output_dir\n",
277+
"\n",
278+
" training_args = Seq2SeqTrainingArguments(\n",
279+
" output_dir=output_dir,\n",
280+
" per_device_train_batch_size=1,\n",
281+
" # per_device_eval_batch_size=1,\n",
282+
" predict_with_generate=True,\n",
283+
" weight_decay=0.05,\n",
284+
" # warmup_steps=200,\n",
285+
" fp16=False, # Overflows with fp16\n",
286+
" learning_rate=1e-4,\n",
287+
" num_train_epochs=5,\n",
288+
" logging_dir=f\"{output_dir}/logs\",\n",
289+
" logging_strategy=\"epoch\",\n",
290+
" report_to=\"tensorboard\",\n",
291+
" push_to_hub=False,\n",
292+
" # generation_max_length=200,\n",
293+
" optim=\"paged_adamw_8bit\",\n",
294+
" lr_scheduler_type = 'constant'\n",
295+
" )\n",
296+
"\n",
297+
" # Create Trainer instance\n",
298+
" trainer = Seq2SeqTrainer(\n",
299+
" model=model,\n",
300+
" args=training_args,\n",
301+
" data_collator=data_collator,\n",
302+
" train_dataset=tokenized_dataset[\"train\"],\n",
303+
" # eval_dataset=tokenized_dataset[\"validation\"],\n",
304+
" # compute_metrics=compute_metrics,\n",
305+
" )\n",
306+
"\n",
307+
" # train model\n",
308+
" train_result = trainer.train()\n",
309+
"\n",
310+
"if __name__ == '__main__':\n",
311+
" main()"
312+
]
313+
}
314+
],
315+
"metadata": {
316+
"language_info": {
317+
"name": "python"
318+
},
319+
"orig_nbformat": 4
320+
},
321+
"nbformat": 4,
322+
"nbformat_minor": 2
323+
}

0 commit comments

Comments
 (0)