A lightweight, Spring Boot–friendly plugin framework to orchestrate complex AI workflows — integrating multiple LLMs, tools, and MCP (Model Context Protocol) services with zero boilerplate.

• 🪶 Lightweight Architecture – No external AI frameworks. Uses custom HTTP clients instead of heavy solutions like LangChain4j.
• 📦 Smaller JAR Size – Reduced by 15–20 MB due to minimal dependencies.
• ⚡ Faster Startup – No auto-configuration overhead, ensuring quick boot times.
• 🔧 Direct API Control – Fully optimized request/response handling with Java HttpClient.
• 🌱 Spring Boot Friendly – Plug-and-play integration with any Spring Boot application.
• 🚀 High Performance – Efficient, low-latency orchestration with native HTTP API calls.

• 🚀 Features
• 📁 Project Structure
• 🛠️ Quick Start
• 🧠 Local LLM Setup with Ollama
• 🌐 API Usage
• 🏗️ Architecture
• 🔌 Workflow Types & Step Types
• ⚡ Plug & Play Workflow Creation
• 📊 Real-World Plug-and-Play Workflow Examples
• ⚙️ Advanced Configurations
• 🧪 CLI Mode
• 🧰 Logging
• 🤝 Contributing
• 📄 License
• 👨‍💻 Author
• 🆘 Support
• 🔗 Related Projects

• 🧠 Multi-LLM Support: OpenAI, Anthropic, and Ollama with custom HTTP clients
• ⚙️ Dynamic Workflow Planning: AI-driven step orchestration
• 🔌 MCP Protocol Support: HTTP, WebSocket & Async clients
• 🛠️ Tool Integration: Register & invoke custom tools seamlessly
• 🔄 Schema Transformation: JSON schema-based data conversion
• 🖥️ CLI & Web Modes: Dual execution interfaces
• 🧩 Intelligent Transformations: LLM-powered data format conversions
• 🪶 Lightweight Architecture: No external AI framework dependencies
• ⚡ High Performance: Direct HTTP API calls with Java HttpClient

SpringAIOrchestrator/ ├── ai-orchestrator-bom/ # Bill of Materials & Core Framework │ └── ai-orchestrator-core/ # Core orchestration engine ├── examples/ # Example workflow implementations │ ├── ai-travel-workflow/ # Travel planning workflow example │ └── ai-resume_builder-workflow/ # Resume building workflow example │ └── ai-document_summarizer-workflow/ # Summaring the document and classification ├── LICENSE # MIT License ├── README.md # This documentation └── USAGE.md # Detailed usage guide 

• ☕ Java 21+
• 📦 Maven 3.8+
• 🔑 API Keys (OpenAI / Anthropic / Ollama)
• 🌐 Internet connection for LLM API calls
• 🧠Local Ollama Installation

The AI Orchestrator Framework can run entirely offline using Ollama — a lightweight local runtime for open-source LLMs.
This allows you to test workflows locally without API keys, network dependencies, or cloud costs.

Install Ollama on your system:

MacOS:

brew install ollama

Linux

curl -fsSL https://ollama.com/install.sh | sh

Windows

• Download the latest installer from Ollama Download
• Run the installer and follow the on-screen instructions.

• The core requirement for using any Ollama-based model in your models.yml is having the Ollama server running locally on the default host and port: http://localhost:11434.
• Verify Service: Open your terminal and run ollama. The service typically starts automatically in the background. You can verify it's running by checking http://localhost:11434 in your browser.
• Using Quantized Models Locally : Quantized models are compressed versions of larger LLMs (e.g., Llama 2, Mistral, Gemma), optimized for performance on consumer hardware with less VRAM or RAM. Ollama models are quantized by default (typically to 4-bit, or q4). You use the model tag to select the specific quantization level if needed, but for most models, the default tag is already a quantized version.

The local alias uses the default quantized llama2 model pulled from Ollama:

models: - alias: "local" provider: "ollama" modelName: "llama2" baseUrl: "http://localhost:11434" enabled: true

For models that are too large to run locally on your personal machine, Ollama Cloud Models provide a seamless solution. They utilize remote, high-performance GPU hardware while still being accessed through your local http://localhost:11434 endpoint — so your application code doesn’t need any changes.

Cloud models require an Ollama account. Sign in from your terminal:

ollama signin

# Example: Pulling a cloud model ollama pull qwen3-coder:480b-cloud

The example below shows how to configure the default alias to use the cloud-offloaded qwen3-coder:480b-cloud model.

models: - alias: "default" provider: "ollama" modelName: "qwen3-coder:480b-cloud" # This model is offloaded to Ollama Cloud baseUrl: "http://localhost:11434" enabled: true - alias: "transformer" provider: "ollama" modelName: "qwen3-coder:480b-cloud" baseUrl: "http://localhost:11434" enabled: true

1. Clone the repository

git clone <repository-url> cd SpringAIOrchestrator/spring-ai-orchestrator

2. Build the framework

cd ai-orchestrator-bom mvn clean install

3. Set environment variables

# Required LLM API Keys export OPENAI_API_KEY="your-openai-key" export ANTHROPIC_API_KEY="your-anthropic-key" export OLLAMA_BASE_URL="http://localhost:11434" # Application Configuration export SERVER_PORT=8282 export CLI_MODE=false export APP_NAME="ai-orchestrator" # Optional: Planner Configuration export PLANNER_FALLBACK=true export PLANNER_MAX_ITERATIONS=5 export PLANNER_TARGET_RANK=10

4. Run an example workflow

cd ../examples/ai-travel-workflow mvn spring-boot:run # Or run resume workflow cd ../ai-resume_builder-workflow mvn spring-boot:run

# Plan a trip to Tokyo curl -X POST "http://localhost:8282/api/workflows/run/travel-planner" \ -H "Content-Type: application/x-www-form-urlencoded" \ -d "prompt=Plan a 5-day trip to Tokyo in March with budget of $3000" # Business trip planning curl -X POST "http://localhost:8282/api/workflows/run/travel-planner" \ -H "Content-Type: application/x-www-form-urlencoded" \ -d "prompt=Business trip to London for 3 days, need flights and hotel recommendations"

# Create Java developer resume curl -X POST "http://localhost:8282/api/workflows/run/resume-builder" \ -H "Content-Type: application/x-www-form-urlencoded" \ -d "prompt=Create resume for Senior Java Developer with 5 years Spring Boot experience" # Create data scientist resume curl -X POST "http://localhost:8282/api/workflows/run/resume-builder" \ -H "Content-Type: application/x-www-form-urlencoded" \ -d "prompt=Build resume for Data Scientist role with Python and ML expertise"

curl -X GET "http://localhost:8282/api/workflows/list" # Response example: { "travel-planner": "travel-planner - AI-powered travel planning with weather, flights, and cost analysis", "resume-builder": "AI Resume Builder - Create a professional resume from user inputs and job descriptions" }

The AI Orchestrator Framework uses a lightweight, custom architecture:

• 🧠 Planner LLM – Generates and ranks intelligent execution plans
• 🔄 Transformer LLM – Handles format conversions between steps
• 🔗 Custom HTTP Clients – Direct API integration without external AI frameworks
• 🪶 Zero Dependencies – No LangChain4j or similar heavy frameworks

┌─────────────────────────────────────────────────────────────────────────┐ │ AI Orchestrator Framework │ └─────────────────────────────────────────────────────────────────────────┘ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ User Input │ │ REST API │ │ CLI Mode │ └──────┬───────┘ └──────┬───────┘ └──────┬───────┘ │ │ │ └─────────────────┼─────────────────┘ │ ┌─────────▼─────────┐ │ Workflow Controller│ └─────────┬─────────┘ │ ┌──────────▼──────────┐ │ Dynamic Engine (AI) │ └──────────┬──────────┘ │ ┌─────────────▼─────────────┐ │ 🧠 Planner + 🔄 Transformer │ └─────────────┬─────────────┘ │ ┌─────────────▼─────────────┐ │ Step Execution Engine │ └────────────────────────────┘ 

• 🔁 Sequential – Predefined step order
• 🧠 Dynamic – AI-planned execution order

• 🤖 LLM – Language model interactions
• 🔧 Tool – Custom annotated methods
• 🌐 MCP – External service calls (HTTP, WebSocket, Async)

Create a new AI workflow in 5 minutes 🚀

1. Add Dependency

<dependency> <groupId>codetumblr.net.ai.orchestrator</groupId> <artifactId>ai-orchestrator-core</artifactId> <version>1.0-SNAPSHOT</version> </dependency>

2. Create Application

@SpringBootApplication @ComponentScan({"codetumblr.net.ai.orchestrator.core", "your.package"}) public class MyWorkflowApplication { public static void main(String[] args) { SpringApplication.run(MyWorkflowApplication.class, args); } }

3. Define Models (models.yml)

models: - alias: "default" provider: "openai" model: "gpt-4" apiKey: "${OPENAI_API_KEY}"

4. Create Workflow (workflows/my-workflow.yaml)

id: "document-analyzer" executionMode: "dynamic" steps: - id: "extract_content" type: "tool" parameters: tool: "document_extractor" - id: "analyze_sentiment" type: "llm" parameters: model: "default" prompt: "Analyze sentiment: {input}"

5. Add Tools

@Component public class DocumentTools { @AiTool(name = "document_extractor") public String extractText(String filePath) { return "Extracted content..."; } }

6. Run the Workflow

mvn spring-boot:run

The AI Workflow Orchestrator comes with ready-to-use, production-grade workflow examples. These demonstrate how to combine LLMs, tools, and MCP services into powerful pipelines with minimal setup.

• 🛫 Travel Planner – Automatically plan detailed, multi-day travel itineraries with flights, weather, and cost analysis.
• 📄 Resume Builder – Generate tailored, professional resumes directly from a job description or skill set.
• 📜 Document Summarizer – Extract, summarize, classify, and store metadata from documents (PDF, DOCX, etc.) using a Dynamic Workflow.

• 🧠 Intelligent Chat Agent – Build a context-aware, tool-invoking assistant.
• 📈 Financial Report Analyzer – Parse CSVs, extract key metrics, and generate business summaries.
• 🧪 Code Review Assistant – Analyze pull requests and suggest improvements using code-aware LLMs.

💡 Tip: Each example is fully modular and can be customized or extended by simply editing the workflow YAML file — no code changes required.

The framework supports typical Spring Boot web controllers to enable simple file upload and dynamic prompt generation for complex workflows like Document Summarizer.

How it Works User uploads a file (PDF, DOCX, etc.) via the web interface.

A Spring Controller reads the file, Base64-encodes the binary content.

The Controller dynamically constructs a detailed Planner Prompt containing the Base64 content and file metadata.

The user can review/edit the prompt before it's submitted to the Orchestrator's API for execution. The LLM Planner then uses the Base64 content to route to the initial MCP step (extract_text).

Run the Web Example To run the web application for the Document Summarizer:

cd examples/ai-document_summarizer-workflow mvn spring-boot:run

Access the application in your browser: http://localhost:8383/

# application.yml server: port: ${SERVER_PORT:8282} spring: application: name: ${APP_NAME:ai-orchestrator-travel_planner} shell: interactive: enabled: ${CLI_MODE:false} command: quit: enabled: true history: enabled: false script: enabled: false main: allow-circular-references: true web-application-type: servlet devtools: restart: enabled: false livereload: enabled: false boot: admin: enabled: false logging: config: classpath:log4j2.xml ai: orchestrator: workflows: path: ${WORKFLOWS_PATH:classpath*:workflows/*-workflow.yaml} models: description-truncate-length: ${DESC_TRUNCATE_LENGTH:30} path: classpath:${MODELS_PATH:/models.yml} planner: fallback-enabled: ${PLANNER_FALLBACK:true} step-separator: ${STEP_SEPARATOR:,} max-iterations: ${PLANNER_MAX_ITERATIONS:5} target-rank: ${PLANNER_TARGET_RANK:10} prompt: path: ${PLANNER_PROMPT_PATH:} transformer: prompt: path: ${TRANSFORMER_PROMPT_PATH:} mcp: clients: path: classpath:${MCP_CLIENTS_PATH:/mcp-clients.yml}

# mcp-clients.yml clients: - name: "weather-services-api" endpoint: "${MCP_WEATHER_ENDPOINT:http://localhost:8282}" service: "weather_forecast" protocol: "${MCP_WEATHER_PROTOCOL:http}" timeout: "${MCP_WEATHER_TIMEOUT:5000}" # Configurable timeout auth: type: "${MCP_WEATHER_AUTH_TYPE:none}" # none, basic, oauth2, api-key, jwt apiKey: "${MCP_WEATHER_API_KEY:}" apiKeyHeader: "${MCP_WEATHER_API_KEY_HEADER:X-API-Key}" enabled: "${MCP_WEATHER_ENABLED:true}" - name: "flight-services-api" endpoint: "${MCP_FLIGHT_ENDPOINT:http://localhost:8282}" service: "search_flights" protocol: "${MCP_FLIGHT_PROTOCOL:http}" timeout: "${MCP_FLIGHT_TIMEOUT:5000}" # Configurable timeout auth: type: "${MCP_FLIGHT_AUTH_TYPE:none}" enabled: "${MCP_FLIGHT_ENABLED:true}"

# models.yml config: defaultMaxTokens: 1000 promptTruncateLength: 50 descriptionTruncateLength: 30 autoRegisterForPlugins: true models: - alias: "default" provider: "ollama" modelName: "qwen3-coder:480b-cloud" baseUrl: "http://localhost:11434" temperature: 0.7 maxTokens: 2000 enabled: true - alias: "openai" provider: "openai" modelName: "gpt-4o-mini" apiKey: "${OPENAI_API_KEY}" temperature: 0.7 maxTokens: 4000 enabled: true - alias: "planner" provider: "ollama" modelName: "qwen2.5-coder:14b-instruct-q3_K_L" baseUrl: "http://localhost:11434" temperature: 0.3 maxTokens: 1500 enabled: true - alias: "transformer" provider: "ollama" modelName: "qwen3-coder:480b-cloud" baseUrl: "http://localhost:11434" temperature: 0.1 maxTokens: 1000 enabled: true

Enable interactive CLI execution:

export CLI_MODE=true mvn spring-boot:run or CLI_MODE=true mvn spring-boot:run or java -DCLI_MODE=true -jar ai-travel-workflow-1.0-SNAPSHOT.jar

# List workflows lw # Interactive workflow selection interactive i run-interactive # Run specific workflow run --workflow-id travel-planner --prompt "Plan weekend trip to Paris" run-workflow --workflow-id resume-builder --prompt "Create resume for DevOps engineer" # Quick run by number quick --number 1 q --number 2 # Show help help h ? # Logging info log-info logs debug-help # Exit CLI exit quit bye

The framework uses log4j2.xml for logging configuration. Logging is configured via:

logging: config: classpath:log4j2.xml

Log files are written to the logs/ directory in the application root.

# Planner Configuration (All Configurable) ai: orchestrator: planner: max-iterations: 5 # Maximum planning iterations target-rank: 10 # Target plan quality rank fallback-enabled: true # Enable fallback when planning fails # MCP Client Timeouts (Per Client) clients: - name: "weather-services-api" timeout: "5000" # 5 seconds (configurable) - name: "flight-services-api" timeout: "5000" # 5 seconds (configurable)

• LLM Planning Timeout: 120 seconds
• Planning Session Timeout: 60 seconds
• LLM Client Timeouts: 5 minutes request, 30 seconds connection
• Schema Caching: Automatic using ConcurrentHashMap
• HTTP Client: Java 21 HttpClient with connection pooling

export SERVER_PORT=8282 export CLI_MODE=false export APP_NAME="ai-orchestrator"

export OPENAI_API_KEY="sk-your-openai-key" export ANTHROPIC_API_KEY="sk-ant-your-anthropic-key" export OLLAMA_BASE_URL="http://localhost:11434"

export PLANNER_FALLBACK=true export PLANNER_MAX_ITERATIONS=5 export PLANNER_TARGET_RANK=10 export STEP_SEPARATOR="," export PLANNER_PROMPT_PATH="" export TRANSFORMER_PROMPT_PATH=""

export WORKFLOWS_PATH="classpath*:workflows/*-workflow.yaml" export MODELS_PATH="/models.yml" export MCP_CLIENTS_PATH="/mcp-clients.yml"

export PLANNER_LOG_LEVEL=INFO export ENGINE_LOG_LEVEL=INFO export WORKFLOW_LOG_LEVEL=INFO export MCP_SERVICE_LOG_LEVEL=INFO export SCHEMA_LOG_LEVEL=INFO

# Check workflow status curl -X GET "http://localhost:8282/api/workflows/list" # Verify environment variables echo $OPENAI_API_KEY echo $ANTHROPIC_API_KEY echo $OLLAMA_BASE_URL

1. Enable Debug Logging

   export PLANNER_LOG_LEVEL=DEBUG export ENGINE_LOG_LEVEL=DEBUG export WORKFLOW_LOG_LEVEL=DEBUG 

Refer log4j2.xml - For different component logging levels

 2. **Check Log Files** ```bash tail -f logs/ai-orchestrator-**.log 

3. Validate Configuration

   # Check YAML syntax cat src/main/resources/models.yml cat src/main/resources/mcp-clients.yml

We ❤️ contributions!

1. Fork the repo
2. Create a feature branch
3. Add Javadoc & tests
4. Submit a PR 🚀

This project is licensed under the MIT License. See the LICENSE file for details.

Ravinderjeet Singh Nagpal
Building next-generation AI orchestration systems.

• No External AI Frameworks - Custom HTTP clients instead of LangChain4j
• Smaller JAR Size - Reduced by ~15-20MB without heavy dependencies
• Faster Startup - No auto-configuration overhead
• Direct API Control - Optimized request/response handling

• 🔑 Verify environment variable configuration
• 📜 Validate YAML and schema syntax
• 📖 See USAGE.md for detailed examples
• 🐛 Enable debug logging for detailed traces

• Spring Boot – Java Backend Framework
• Model Context Protocol – MCP Specification

⭐ Star this repo if you find it helpful! Your support keeps this project growing 🚀