简体中文

Pillar Service is a multifunctional service platform based on FastAPI, supporting image caption generation, text translation, and more. It is suitable for rapid integration and extension of AI inference services.

• FastAPI-based, fast response, and easy to extend
• Supports image captioning (JoyCaption) and text translation
• Model cache and memory management
• Detailed health check and logging

• Python 3.11
• Conda (Anaconda or Miniconda recommended)
• NVIDIA GPU and driver (for GPU acceleration)
• Dependencies listed in requirements.txt

# 1. Create and activate conda environment conda create -n pillar_env python=3.11 conda activate pillar_env # 2. Configure Tsinghua conda-forge mirror (recommended for China) conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge conda config --set show_channel_urls yes # 3. Install CUDA dependencies conda install -y cudatoolkit cudnn # 4. Install PyTorch (example: CUDA 12.4) conda install -y pytorch torchvision torchaudio pytorch-cuda=12.4 -c pytorch -c nvidia # 5. Install Python dependencies pip install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple # 6. Install bitsandbytes and accelerate pip install bitsandbytes --prefer-binary -i https://pypi.tuna.tsinghua.edu.cn/simple pip install accelerate -i https://pypi.tuna.tsinghua.edu.cn/simple # 7. Check CUDA availability python -c "import torch; print('[INFO] torch.cuda.is_available():', torch.cuda.is_available())" # 8. Start the service run_server.bat

Pillar_Service/ ├── server/ │ ├── main.py # FastAPI entry point │ ├── api/ # API routes and service management │ └── ... ├── Pillar/ │ ├── dto/ # Data Transfer Objects (DTO) │ ├── service/ # Business service implementations │ └── util/ # Utilities and constants ├── requirements.txt # Python dependencies ├── run_server.bat # Startup script (Windows) └── README.md # Project documentation 

• /translate
  Text translation API, POST request, parameter: text.
• /joycaption/generate
  Image caption generation API, POST request, parameters: image and related generation options.
• /health
  Health check API, GET request.

For detailed parameters and return formats, please refer to code comments or API documentation.

• Slow or failed dependency installation: Use Tsinghua, USTC, or other China mirrors for pip/conda.
• bitsandbytes device not supported: Make sure you have an NVIDIA GPU, CUDA driver, and PyTorch CUDA version.
• For other issues, please refer to the documentation or submit an issue.

Feel free to open an issue or contact us if you have any questions!