This project demonstrates Retrieval-Augmented Generation (RAG) using Embabel Agent with Apache Lucene for vector storage and Spring Shell for interaction.

API Key: Set at least one LLM provider API key as an environment variable:

# For OpenAI (GPT models) export OPENAI_API_KEY=sk-... # For Anthropic (Claude models) export ANTHROPIC_API_KEY=sk-ant-...

The model configured in application.yml determines which key is required. The default configuration uses OpenAI.

Java: Java 21+ is required.

1. Set your API key (see above)
2. Run the shell:

   ./scripts/shell.sh

3. Ingest a document:

   ingest 

4. Start chatting:

   chat 

Run the shell script to start Embabel under Spring Shell:

./scripts/shell.sh

You can also run the main class, com.embabel.examples.ragbot.RagShellApplication, directly from your IDE.

# Start the shell ./scripts/shell.sh # Ingest a document ingest https://example.com/document # View what was indexed chunks # Chat with the RAG-powered assistant chat > What does this document say about X? # Clear the index when done zap

┌─────────────────────────────────────────────────────────────────────────────┐ │ Spring Shell │ │ │ │ > chat │ │ > What penalties apply to social media platforms? │ └─────────────────────────────────────┬───────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────────────────────┐ │ AgentProcess │ │ │ │ Starts when chat begins. Manages conversation state and action dispatch. │ │ Listens for triggers (UserMessage) and invokes matching @Action methods. │ └─────────────────────────────────────┬───────────────────────────────────────┘ │ UserMessage triggers ▼ ┌─────────────────────────────────────────────────────────────────────────────┐ │ @Action: ChatActions.respond() │ │ │ │ Fired on each user message. Uses Ai interface to build request: │ │ context.ai() │ │ .withLlm(...) │ │ .withReference(toolishRag) ◄── ToolishRag added as LLM tool │ │ .withTemplate("ragbot") │ │ .respondWithSystemPrompt(conversation, ...) │ └─────────────────────────────────────┬───────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────────────────────┐ │ Ai Interface │ │ │ │ • Renders system prompt from Jinja template │ │ • Packages ToolishRag as tool definition for LLM │ │ • Sends request to LLM provider (OpenAI / Anthropic) │ └─────────────────────────────────────┬───────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────────────────────┐ │ LLM (GPT / Claude) │ │ │ │ Receives prompt + tool definitions. Decides to call tools as needed: │ │ │ │ "I need to search for penalty information..." │ │ │ │ │ ▼ │ │ ┌─────────────────────────────────────────────────────────────────────┐ │ │ │ Tool Call: vectorSearch("penalties social media platforms") │ │ │ └─────────────────────────────────┬───────────────────────────────────┘ │ │ │ │ └─────────────────────────────────────┼───────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────────────────────┐ │ ToolishRag → LuceneSearchOperations │ │ │ │ • Converts query to embedding vector │ │ • Searches ./.lucene-index for similar chunks │ │ • Returns relevant content to LLM │ └─────────────────────────────────────┬───────────────────────────────────────┘ │ ▼ LLM generates final response using retrieved context │ ▼ Response sent to user 

Flow Summary:

1. User types chat → AgentProcess starts and manages the session
2. User sends a message → triggers @Action(trigger = UserMessage.class)
3. ChatActions.respond() builds request via Ai interface, adding ToolishRag with .withReference()
4. Ai packages prompt + tool definitions, sends to LLM
5. LLM decides to call a ToolishRag tool to search for relevant content
6. The ToolishRag tool queries Lucene index, returns matching chunks to LLM
7. LLM generates response using retrieved context → sent back to user
8. Loop continues for each new message until user exits

RAG is configured in RagConfiguration.java:

@Bean LuceneSearchOperations luceneSearchOperations( ModelProvider modelProvider, RagbotProperties properties) { var embeddingService = modelProvider.getEmbeddingService(DefaultModelSelectionCriteria.INSTANCE); var luceneSearchOperations = LuceneSearchOperations .withName("docs") .withEmbeddingService(embeddingService) .withChunkerConfig(properties.chunkerConfig()) .withIndexPath(Paths.get("./.lucene-index")) .buildAndLoadChunks(); return luceneSearchOperations; }

Key aspects:

• Lucene with disk persistence: The vector index is stored at ./.lucene-index, surviving application restarts
• Embedding service: Uses the configured ModelProvider to get an embedding service for vectorizing content
• Configurable chunking: Content is split into chunks with configurable size (default 800 chars), overlap (default 50 chars), and optional section title inclusion

Chunking properties can be configured via application.yml:

ragbot: chunker-config: max-chunk-size: 800 overlap-size: 100

The chatbot is created in ChatConfiguration.java:

@Bean Chatbot chatbot(AgentPlatform agentPlatform) { return AgentProcessChatbot.utilityFromPlatform(agentPlatform); }

The AgentProcessChatbot.utilityFromPlatform() method creates a chatbot that automatically discovers all @Action methods in @EmbabelComponent classes. Any action with a matching trigger becomes eligible to be called when appropriate messages arrive.

Chat actions are defined in ChatActions.java:

@EmbabelComponent public class ChatActions { private final ToolishRag toolishRag; private final RagbotProperties properties; public ChatActions(SearchOperations searchOperations, RagbotProperties properties) { this.toolishRag = new ToolishRag( "sources", "Sources for answering user questions", searchOperations); this.properties = properties; } @Action(canRerun = true, trigger = UserMessage.class) void respond(Conversation conversation, ActionContext context) { var assistantMessage = context.ai() .withLlm(properties.chatLlm()) .withReference(toolishRag) .withTemplate("ragbot") .respondWithSystemPrompt(conversation, Map.of( "properties", properties )); context.sendMessage(conversation.addMessage(assistantMessage)); } }

Key concepts:

1. @EmbabelComponent: Marks the class as containing agent actions that can be discovered by the platform

2. @Action annotation:

   • trigger = UserMessage.class: This action is invoked whenever a UserMessage is received in the conversation
   • canRerun = true: The action can be executed multiple times (for each user message)

3. ToolishRag as LLM reference:

   • Wraps the SearchOperations (Lucene index) as a tool the LLM can use
   • When .withReference(toolishRag) is called, the LLM can search the RAG store to find relevant content
   • The LLM decides when to use this tool based on the user's question

4. Response flow:

   • User sends a message (triggering the action)
   • The action builds an AI request with the RAG reference
   • The LLM may call the RAG tool to retrieve relevant chunks
   • The LLM generates a response using retrieved context
   • The response is added to the conversation and sent back

Chatbot prompts are managed using Jinja templates rather than inline strings. This is best practice for chatbots because:

• Prompts grow complex: Chatbots require detailed system prompts covering persona, guardrails, objectives, and behavior guidelines
• Separation of concerns: Prompt engineering can evolve independently from Java code
• Reusability: Common elements (guardrails, personas) can be shared across different chatbot configurations
• Configuration-driven: Switch personas or objectives via application.yml without code changes

The template system separates two concerns:

• Objective: What the chatbot should accomplish - the task-specific instructions and domain expertise (e.g., analyzing legal documents, answering technical questions)
• Voice: How the chatbot should communicate - the persona, tone, and style of responses (e.g., formal lawyer, Shakespearean, sarcastic)

This separation allows mixing and matching. You could have a "legal" objective answered in the voice of Shakespeare, Monty Python, or a serious lawyer - without duplicating the legal analysis instructions in each persona template.

src/main/resources/prompts/ ├── ragbot.jinja # Main template entry point ├── elements/ │ ├── guardrails.jinja # Safety and content restrictions │ └── personalization.jinja # Dynamic persona/objective loader ├── personas/ # HOW to communicate (voice/style) │ ├── clause.jinja # Serious legal expert │ ├── shakespeare.jinja # Elizabethan style │ ├── monty_python.jinja # Absurdist humor │ └── ... └── objectives/ # WHAT to accomplish (task/domain) └── legal.jinja # Legal document analysis 

The main template ragbot.jinja composes the system prompt from reusable elements:

{% include "elements/guardrails.jinja" %} {% include "elements/personalization.jinja" %}

The personalization.jinja template dynamically includes persona and objective based on configuration:

{% set persona_template = "personas/" ~ properties.voice().persona() ~ ".jinja" %} {% include persona_template %} {% set objective_template = "objectives/" ~ properties.objective() ~ ".jinja" %} {% include objective_template %}

Templates are invoked using .withTemplate() and passing bindings:

context.ai() .withLlm(properties.chatLlm()) .withReference(toolishRag) .withTemplate("ragbot") .respondWithSystemPrompt( conversation, Map.of( "properties", properties ));

The properties object (a Java record) is accessible in templates. Jinjava supports calling record accessor methods with properties.voice().persona() syntax for nested records.

To create a new persona, add a .jinja file to prompts/personas/ and reference it by name in application.yml. See Configuration Reference for all available settings.

This section walks through creating a new chatbot configuration from scratch, using a film critic example.

The objective defines what the chatbot should accomplish. Create a new file at:

src/main/resources/prompts/objectives/discuss_films.jinja 

Example content based on existing objectives:

Answer questions about classic cinema and film history in a clear and engaging manner. The tools available to you access a curated collection of film reviews and criticism. You must always use these tools to find answers, as your general knowledge will not extend to everything in the collection and these tools allow you to find detailed analysis if you try hard enough. Always back up your points with direct quotes from the film criticism sources. You may find that the result from one tool call leads to a search for another tool, e.g. a result mentioning "as discussed in the analysis of Citizen Kane..." might lead to a search for "Citizen Kane analysis". DO NOT RELY ON GENERAL KNOWLEDGE unless you are certain a better answer is not in the provided sources.

The persona defines how the chatbot communicates. Create a new file at:

src/main/resources/prompts/personas/film_critic.jinja 

Example content based on existing personas:

Your name is Cinephile. You are a passionate film critic with deep knowledge of cinema history. You want to share your love of films with others and help them appreciate the art of filmmaking. You speak with enthusiasm about cinematography, direction, and storytelling.

After creating the files, your prompts directory should look like:

src/main/resources/prompts/ ├── ragbot.jinja ├── elements/ │ ├── guardrails.jinja │ └── personalization.jinja ├── personas/ │ ├── clause.jinja │ ├── music-guide.jinja │ ├── film_critic.jinja # NEW │ └── ... └── objectives/ ├── legal.jinja ├── music.jinja ├── discuss_films.jinja # NEW └── ... 

The ToolishRag description in ChatActions.java helps the LLM understand what content is available. Update the constructor to describe your new content:

public ChatActions( SearchOperations searchOperations, RagbotProperties properties) { this.toolishRag = new ToolishRag( "sources", "Film reviews and criticism: Classic cinema analysis and reviews", // Updated description searchOperations) .withHint(TryHyDE.usingConversationContext()); this.properties = properties; }

The description should briefly explain what content the RAG store contains, helping the LLM make better decisions about when and how to search.

Use the ingest-directory command to load a directory of markdown or text files:

# Start the shell ./scripts/shell.sh # Ingest a directory of film reviews (markdown or text files) ingest-directory /path/to/film-reviews # Verify content was indexed chunks

The ingest-directory command recursively processes all .md and .txt files in the specified directory, chunking them for vector storage.

Finally, update your configuration to use the new objective and persona:

ragbot: voice: persona: film_critic # References personas/film_critic.jinja max-words: 50 objective: discuss_films # References objectives/discuss_films.jinja chat-llm: model: gpt-4.1-mini temperature: 0.3 # Slightly creative for engaging film discussion

Restart the application after making these changes:

All configuration is externalized in application.yml, allowing behavior changes without code modifications.

ragbot: # RAG chunking settings chunker-config: max-chunk-size: 800 # Maximum characters per chunk overlap-size: 100 # Overlap between chunks for context continuity # LLM model selection and hyperparameters chat-llm: model: gpt-4.1-mini # Model to use for chat responses temperature: 0.0 # 0.0 = deterministic, higher = more creative # Voice controls HOW the chatbot communicates voice: persona: clause # Which persona template to use (personas/*.jinja) max-words: 30 # Hint for response length # Objective controls WHAT the chatbot accomplishes objective: legal # Which objective template to use (objectives/*.jinja) embabel: agent: shell: # Redirect logging during chat sessions redirect-log-to-file: true

When redirect-log-to-file: true, console logging is redirected to a file during chat sessions, providing a cleaner chat experience. Logs are written to:

logs/chat-session.log 

To monitor logs while chatting, open a separate terminal and tail the log file:

tail -f logs/chat-session.log

This is useful for debugging RAG retrieval, seeing which chunks are being returned, and monitoring LLM API calls.

To change the chatbot's personality, simply update the persona value:

ragbot: voice: persona: shakespeare # Now responds in Elizabethan English

To use a different LLM:

ragbot: chat-llm: model: gpt-4.1 # Use the larger GPT-4.1 instead temperature: 0.7 # More creative responses

No code changes required - just restart the application.