! pip install git+https://github.com/xinyu1205/recognize-anything.git

Looking in indexes: https://mirrors.aliyun.com/pypi/simple Collecting git+https://github.com/xinyu1205/recognize-anything.git  Cloning https://github.com/xinyu1205/recognize-anything.git to /tmp/pip-req-build-j5jd7kf3  Running command git clone --filter=blob:none --quiet https://github.com/xinyu1205/recognize-anything.git /tmp/pip-req-build-j5jd7kf3

Resolved https://github.com/xinyu1205/recognize-anything.git to commit 7cb804a8609e9f4b1a50b7f31436d2df40bb9481  Preparing metadata (setup.py) ... ?25ldone ?25hWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv [notice] A new release of pip is available: 23.0.1 -> 25.0.1 [notice] To update, run: pip install --upgrade pip

[  {  "conversations": [  {  "from": "system",  "value": "你是一个导游，请生动有趣地回答游客提出的问题"  },  {  "from": "human",  "value": "给我讲讲这个东西<image>"  },  {  "from": "gpt",  "value": "各位游客朋友们，大家好！\n\n现在映入大家眼帘的，是一件极为珍贵的文物——**神面纹玉戚**。它可不是一般的宝贝，而是来自遥远的新石器时代的一件精美玉器哦！\n\n这件宝贝现藏于山西博物院，它的尺寸相当精致：高20.6厘米，宽13.1厘米，厚仅0.4厘米，真是薄如蝉翼，工艺精湛！\n\n1964年，它在黎城县后庄村广志山出土，仿佛穿越时空，向我们诉说着古老的故事。\n\n大家仔细看，这块玉戚采用了线刻工艺，一侧雕刻着神人的半侧面头像，头戴华美的冠饰，披拂着飘逸的长发，眼眉清晰可辨，仿佛在凝视着我们；而另一侧则是一个方形台座上置神人冠饰的形象，显得神秘而威严。\n\n要知道，玉戚在古代可是象征军事统帅权力的神器，握有它，就如同掌握了千军万马！\n\n大家不妨多停留片刻，感受一下这件文物散发出的古老而神秘的气息吧！\n\n谢谢大家的聆听，接下来我们将继续参观其他精彩展品，敬请期待！"  }  ],  "images": [  "images/instance_1579398113581395972.jpg"  ]  } ]

import os region = os.getenv('DATAWORKS_REGION')

!wget https://dataworks-notebook-{region}.oss-{region}-internal.aliyuncs.com/public-datasets/Image_Tagging_GPU/Qwen2-VL-History.zip !mkdir data && unzip -q Qwen2-VL-History.zip -d ./data

--2025-03-13 16:03:20-- https://dataworks-notebook-cn-shanghai.oss-cn-shanghai-internal.aliyuncs.com/public-datasets/Image_Tagging_GPU/Qwen2-VL-History.zip 正在解析主机 dataworks-notebook-cn-shanghai.oss-cn-shanghai-internal.aliyuncs.com (dataworks-notebook-cn-shanghai.oss-cn-shanghai-internal.aliyuncs.com)... 100.118.102.33, 100.118.102.36, 100.118.102.35, ... 正在连接 dataworks-notebook-cn-shanghai.oss-cn-shanghai-internal.aliyuncs.com (dataworks-notebook-cn-shanghai.oss-cn-shanghai-internal.aliyuncs.com)|100.118.102.33|:443... 已连接。 已发出 HTTP 请求，正在等待回应... 200 OK 长度： 7436378 (7.1M) [application/zip] 正在保存至: ‘Qwen2-VL-History.zip’ Qwen2-VL-History.zi 100%[===================>] 7.09M --.-KB/s 用时 0.1s 2025-03-13 16:03:21 (50.4 MB/s) - 已保存 ‘Qwen2-VL-History.zip’ [7436378/7436378])

!mkdir models !wget -P ./models https://dataworks-notebook-{region}.oss-{region}-internal.aliyuncs.com/public-datasets/Image_Tagging_GPU/ram_plus_swin_large_14m.pth

--2025-03-13 16:03:21-- https://dataworks-notebook-cn-shanghai.oss-cn-shanghai-internal.aliyuncs.com/public-datasets/Image_Tagging_GPU/ram_plus_swin_large_14m.pth 正在解析主机 dataworks-notebook-cn-shanghai.oss-cn-shanghai-internal.aliyuncs.com (dataworks-notebook-cn-shanghai.oss-cn-shanghai-internal.aliyuncs.com)... 100.118.102.34, 100.118.102.35, 100.118.102.33, ... 正在连接 dataworks-notebook-cn-shanghai.oss-cn-shanghai-internal.aliyuncs.com (dataworks-notebook-cn-shanghai.oss-cn-shanghai-internal.aliyuncs.com)|100.118.102.34|:443... 已连接。 已发出 HTTP 请求，正在等待回应...

200 OK 长度： 3010210801 (2.8G) [application/octet-stream] 正在保存至: ‘./models/ram_plus_swin_large_14m.pth’ ram_plus_swin_large 100%[===================>] 2.80G 109MB/s 用时 21s 2025-03-13 16:03:42 (136 MB/s) - 已保存 ‘./models/ram_plus_swin_large_14m.pth’ [3010210801/3010210801])

!wget https://dataworks-notebook-{region}.oss-{region}-internal.aliyuncs.com/public-datasets/Image_Tagging_GPU/bert-base-uncased.zip !unzip -q bert-base-uncased.zip -d ./models

--2025-03-13 16:03:42-- https://dataworks-notebook-cn-shanghai.oss-cn-shanghai-internal.aliyuncs.com/public-datasets/Image_Tagging_GPU/bert-base-uncased.zip 正在解析主机 dataworks-notebook-cn-shanghai.oss-cn-shanghai-internal.aliyuncs.com (dataworks-notebook-cn-shanghai.oss-cn-shanghai-internal.aliyuncs.com)... 100.118.102.33, 100.118.102.36, 100.118.102.35, ... 正在连接 dataworks-notebook-cn-shanghai.oss-cn-shanghai-internal.aliyuncs.com (dataworks-notebook-cn-shanghai.oss-cn-shanghai-internal.aliyuncs.com)|100.118.102.33|:443... 已连接。 已发出 HTTP 请求，正在等待回应... 200 OK 长度： 3194230164 (3.0G) [application/zip] 正在保存至: ‘bert-base-uncased.zip’ bert-base-uncased.z 100%[===================>] 2.97G 154MB/s 用时 21s 2025-03-13 16:04:03 (146 MB/s) - 已保存 ‘bert-base-uncased.zip’ [3194230164/3194230164])

├─models │ │─ ram_plus_swin_large_14m.pth │ │─ bert-base-uncased │ │ │─coreml │ │ │ │─... │ │ │─config.json │ │ │─model.safetensors │ │ │─pytorch_model.bin │ │ │─...

from collections import Counter from datasets import Image import os import numpy as np import torch from ram.models import ram_plus from ram.transform import get_transform import logging logging.basicConfig(level=logging.INFO,  format='%(asctime)s:%(name)s:%(levelname)s:%(message)s') IMAGE_BASE_DIR = './data' def load_image(path):  img_feature = Image()  img = img_feature.decode_example(img_feature.encode_example(path))  img = img.convert('RGB')  return img class ImageTaggingMapper(object):  """Generate image tags.  """  def __init__(self,  image_filed: str = 'images',  tag_field_name: str = 'image_tags',  *args,  **kwargs):  """  Initialization method.  :param image_filed: the field name of images.  :param tag_field_name: the field name to store the tags.  :param args: extra args  :param kwargs: extra args  """  super().__init__(*args, **kwargs)  self.image_filed = image_filed  logging.info('Loading recognizeAnything model...')  self.model = ram_plus(  pretrained='./models/ram_plus_swin_large_14m.pth',  text_encoder_type='./models/bert-base-uncased',  image_size=384,  vit='swin_l',  threshold=0.68)  self.device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')  self.model.to(self.device).eval()   self.transform = get_transform(image_size=384)  self.tag_field_name = tag_field_name  def process_single(self, sample):  # 检查标签是否已生成  if self.tag_field_name in sample:  return sample  # 如果图像不存在，则返回空标签  if self.image_filed not in sample or not sample[self.image_filed]:  sample[self.tag_field_name] = np.array([[]], dtype=np.str_)  return sample  # 加载图像  image_paths = sample[self.image_filed]   image_tags = []  for img_path in image_paths:  img_path = os.path.join(IMAGE_BASE_DIR, img_path) # 获取图像的路径  image = load_image(img_path)  image_tensor = torch.unsqueeze(self.transform(image), dim=0).to(self.device)  with torch.no_grad():  tags, _ = self.model.generate_tag(image_tensor)  words = [word.strip() for word in tags[0].split('|')]  word_count = Counter(words)  sorted_word_list = [item for item, _ in word_count.most_common()]  image_tags.append(np.array(sorted_word_list, dtype=np.str_))  sample[self.tag_field_name] = image_tags  return sample

from datasets import load_dataset data_path = './data/train.json' tag_data_path = './out_tag_data.json' # 加载数据 dataset = load_dataset('json', data_files=data_path) # 初始化打标算子 image_tagging_op = ImageTaggingMapper() # 数据集打标 dataset = dataset.map(function=image_tagging_op.process_single) # 保存打标结果，在原数据文件中新增一个tag field保存打标结果 dataset['train'].to_json(tag_data_path, force_ascii=False)

2025-03-13 16:04:36,406:root:INFO:Loading recognizeAnything model...

-------------- ./models/ram_plus_swin_large_14m.pth -------------- load checkpoint from ./models/ram_plus_swin_large_14m.pth vit: swin_l

Creating json from Arrow format: 0%| | 0/1 [00:00<?, ?ba/s]

349288

!head -n 1 ./out_tag_data.json

{"conversations":[{"from":"system","value":"你是一个文博工作人员，请严谨认真地回答用户提出的问题"},{"from":"human","value":"<image>给我讲讲这个东西"},{"from":"gpt","value":"# 文物名称\n神面纹玉戚\n# 时代\n新石器时代\n# 类别\n玉器\n# 收藏机构\n山西博物院\n# 尺寸\n高20.6厘米，宽13.1厘米，厚0.4厘米\n# 出土时间\n1964年\n# 出土地点\n黎城县后庄村广志山\n# 特征描述\n线刻，一侧为神人半侧面头像，头戴冠饰，披拂长发，眼眉清晰可辨；一侧为一方形台座上置神人冠饰的形象，神秘威严\n# 象征意义\n象征军事统帅权力"}],"images":["images\/instance_1579398113581395972.jpg"],"image_tags":[["artifact","hole","metal","rust","writing"]]}

import matplotlib.pyplot as plt from PIL import Image, ImageDraw, ImageFont import os import json import random TAG_FONT_SIZE = 30 FONT_PATH = "./DingTalk JinBuTi.ttf" # 使用字体文件 TAG_COLOR = "red" # 可视化函数 def visualize_with_tags(image_path, tags):  try:  # 加载图片  img = Image.open(image_path)  draw = ImageDraw.Draw(img)   # 加载字体并设置大小  try:  font = ImageFont.truetype(FONT_PATH, TAG_FONT_SIZE)  except Exception as e:  print('load default font')  font = ImageFont.load_default() # 默认字体可能无法缩放   # 在图片左上角添加标签  tag_text = "标签：\n" + ", \n".join(tags)  draw.text((10, 10), tag_text, fill=TAG_COLOR, font=font)   # 显示图片  plt.figure(figsize=(10, 8))  plt.imshow(img)  plt.axis("off")  plt.title(os.path.basename(image_path))  plt.show()   except FileNotFoundError:  print(f"图片文件不存在：{image_path}")  except Exception as e:  print(f"处理图片时发生错误：{str(e)}") # 加载数据 with open(tag_data_path, 'r') as f:  data_list = f.readlines()  random.shuffle(data_list) # 有一些连续相同的图像，shuffle 显示不同的图像   # 批量可视化  for item in data_list[:3]: # 显示3个样本  item = json.loads(item)  images = item['images']  tags = item['image_tags']  for i in range(len(images)):  image_path = os.path.join(IMAGE_BASE_DIR, images[i])  tag = item['image_tags'][i]  print(f"图片路径：{images[i]}")  print(f"标注标签：{tags[i]}")   visualize_with_tags(image_path, tag)

图片路径：images/instance_1579398113589784578.jpg 标注标签：['calligraphy', 'ink', 'manuscript', 'mark', 'pen', 'scroll', 'text', 'write', 'writing']

图片路径：images/instance_1579398113581395972.jpg 标注标签：['artifact', 'hole', 'metal', 'rust', 'writing']

图片路径：images/instance_1586990758474346497.jpg 标注标签：['architecture', 'building', 'city', 'pillar', 'entrance', 'palace', 'place', 'plaza', 'red', 'shrine', 'sky', 'structure', 'temple', 'worship']