Skip to content

Commit 90c2a92

Browse files
authored
Append better skip layer example
1 parent 69eb736 commit 90c2a92

File tree

1 file changed

+15
-6
lines changed

1 file changed

+15
-6
lines changed

β€ŽREADME.mdβ€Ž

Lines changed: 15 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -8,8 +8,6 @@
88
----
99
### ⭐ Update 23-FEB-2025
1010

11-
![cats-compare](https://github.com/user-attachments/assets/cf354db7-6928-4cd6-923c-173d0a683501)
12-
1311
- Add ability to skip layers in Text & Vision Encoder for generating images
1412
- Counting from the back of the transformer, -1 = last, -2 = penultimate, etc.
1513
- Examples:
@@ -24,14 +22,11 @@ python clip-generate.py --deterministic --make_anti --manu_vit --manu_txt --mode
2422
- For all models. Default OpenAI-ViT-B/32:
2523
- `python clip-generate.py --deterministic --make_anti --manu_vit -set_vit 1 --set_txt 2`
2624

27-
![cats-final](https://github.com/user-attachments/assets/05413481-cda7-4f62-a032-465ef863216b)
28-
2925
πŸ€– Also recommended: (layer 20 (of 0-23, vision), layer 11 (of 0-11, text):
3026
```
3127
python clip-generate.py --deterministic --batch_size 16 --augs_cp 32 --make_anti --manu_vit --manu_txt --model_name "OpenAI-ViT-L/14" --set_vit 4 --set_txt 1
3228
```
33-
![recommended](https://github.com/user-attachments/assets/36993de5-d0ce-476d-a0e8-574f0e1638b8)
34-
29+
![cats-compare](https://github.com/user-attachments/assets/cf354db7-6928-4cd6-923c-173d0a683501)
3530
----
3631
### ⭐ First commit 21-FEB-2025
3732

@@ -84,3 +79,17 @@ python clip-generate.py --model_name "OpenAI-ViT-L/14" "mymodels/finetune.pt" --
8479
- `python clip-generate.py --help` for a quick review.
8580

8681
![example-of-all](https://github.com/user-attachments/assets/f11ab1e2-898d-4c9b-bc2d-5045aee4a9c1)
82+
-----
83+
84+
## Skip Text Encoder layers until just plugging the first layer into projection - a 1-layer #CLIP text encoder!
85+
86+
- ViT-B/32: *fails*
87+
- ViT-L/14: Relentlessly just makes something else.πŸ¦ΎπŸ€–
88+
- Banana Cat incomprehensible, make: M + 🍟🀑 and πŸ•‘πŸ’₯πŸšΆπŸŽ‘πŸ‘βš½οΈπŸ§¦πŸŒŸπŸ”’
89+
90+
![relentless-large](https://github.com/user-attachments/assets/f22c079c-2d23-4d1c-a99c-321a9ade00d5)
91+
92+
## A striking difference in complexity for a 12 layer ViT vs. 24 layer ViT:
93+
94+
![comparisons-final](https://github.com/user-attachments/assets/f8f78ca9-2c2c-48c2-9c61-3a3361bf0129)
95+

0 commit comments

Comments
Β (0)