Download as PDF, PPTX
Uploaded byBuhake Sindi
AI in Java - MCP in Action, Langchain4J-CDI, SmallRye-LLM, Spring AI
This is the presentation I gave with regards to AI in Java, and the work that I have been working on. I've showcased Model Context Protocol (MCP) in Java, creating server-side MCP server in Java. I've also introduced Langchain4J-CDI, previously known as SmallRye-LLM, a CDI managed too to inject AI services in enterprise Java applications. Also, honourable mention: Spring AI.
AI in Java - MCP in Action, Langchain4J-CDI, SmallRye-LLM, Spring AI
- 1.
- 2. AGENDA Model Context Protocol(MCP) in action Langchain4J-CDI – build enterprise AI agents in Java Demo Q&A
- 3.
- 4. MODEL CONTEXT PROTOCOL (MCP) •Introduced by Anthropic in the late 2024. • Considered as the “USB-C for all AI apps and agents.” • An open standard to enable Large Language Models (LLMs) and other AI systems to securely request and consume contextual data (files, databases, APIs, tools) from external sources in a uniform, provider-agnostic way. • Client and server context delivery are 2-way interactions through RPC-style protocol (JSONRPC v2.0 standards) • Official SDKs: Python, Typescript, Java, C#, Kotlin, Swift • Official Documentation: https://modelcontextprotocol.io/
- 5. MODEL CONTEXT PROTOCOL (MCP) Generalarchitecture: • Host: AI program / interface that want to access data through MCP. • Client: MCP protocol client that creates 1-to-1 connection to the MCP server. • Server: The MCP feature provider that exposes MCP features and capabilities. Supported transport: • MCP Server: STDIO, HTTP SSE + streamable • MCP Client: STDIO, HTTP SSE + streamable Getting started • Get the official SDK. • Run the MCP server locally. • Integrate the MCP client to your MCP host (e.g., Claude) • Test and prompt.
- 6. MCP CONFIGURATION ON CLAUDEDESKTOP
- 7. PROBLEMS! How do Javadevelopers create “API” type MCP features, deploy it, host it, and make it discoverable for other MCP hosts?
- 8. CURRENTLY… • Heavy supportfor Typescript • MCP Host connects to MCP server by running host command on- device. • MCP server runs on MCP host device as a standalone server application. • No support to connect to remote HTTP server directly (requires 3rd party connector) • Official Java SDK was donated by Spring with asynchronous protocol that uses Reactor Mono/Flux. • No way to use official MCP Java SDK to integrate with Server-side Java technologies, such as Jakarta REST without writing some “hacks”.
- 9. SOLUTION FEATURES… • MCPclient and server must Java compliant. • Fully asynchronous: support for CompletableFuture (no more Mono/Flux) • Support for virtual thread and various executors (e.g, Jakarta EE ManagedExecutorService). • MCP server must be deployable in an application server, exposing MCP services. • Support for both Servlet and Jakarta REST. • MCP features should follow the enterprise Java development principle (e.g., annotation registration and discovery).
- 10. MCP FEATURES SPI. •@Tool / @ToolArgument – For registering tools with its arguments (when available). • @Resource – for registering resources. • @ResourceTemplate – for registering resource templates. • @Prompt / @PromptArgument – for registering prompt with its arguments (if available).
- 11.
- 12. SINDI-AI-MCP* Java implementation ofthe MCP protocol. • Implementation of the equivalent official Typescript MCP protocol. • Java JDK compliant – JDK 17 and higher • Fully asynchronous calls • Virtual thread – default executor • Source code: https://github.com/sinditech/sindi-ai-mcp MCP server: • Jakarta EE 10 and higher. • Current implementation: Servlet and Jakarta REST as server-side transport. • Source code: https://github.com/sinditech/sindi-ai-mcp- server
- 13. FOR DEVELOPERS... • Writea POJO class and annotate their method with MCP SPI annotations in order for the service registry to identify and register it as MCP feature. • Developers can use MCP client of choice to interface to with MCP server. • Langchain4J also support MCP (via client only). You need to configure it as a ToolProvider.
- 14.
- 15. MCP CONFIGURATION ON CLAUDEDESKTOP…
- 16. // TODO... • Makethis server pluggable through CDI. • Introduce to the Eclipse Jakarta EE / Microprofile committers for the possibility of adopting MCP into enterprise Java ecosystem (adopt it with Microprofile AI project? )
- 17.
- 18. LANGCHAIN4J ● Initially, inspiredby Langchain (Python). Now, it’s independent. ● De-facto, accepted Java library to integrate your Java applications with various LLMs. ● Provide a unified API model to interface with various LLMs (takes the pain of learning all LLM Apis) ● A toolbox that provides abstractions for various common features (prompt engineering, function calling, RAG, chat memory management, etc.). ● Integrates with variety of vector stores & document loaders. ● Current version release: langchain4j-1.0.0 with their AI providers as 1.0.0-beta5. ● Repo: https://github.com/langchain4j/langchain4j
- 19. LANGCHAIN4J AI SERVICES ●High-level LLM API ● Hides the low-level components & complexities of interacting with LLMs.
- 20.
- 21.
- 22. THIS IS TOOFAMILIAR! Java developers work with enterprise Java APIs through interfaces.
- 23. CONTEXT… • API tointerface a service, repository and/or various providers. • Repository API (@Repository) • REST Client (@RegisterRestClient) • CDI managed. • Why not make AI Services pluggable via CDI?
- 24.
- 25. INTRODUCTION… • Previously SmallRye-LLM •https://github.com/smallrye/smallrye-llm • Officially donated to Langchain4J • CDI capable • @RegisterAIService (CDI stereotype). AI Agent is @RequestScoped by default. • Jakarta EE 10 and higher, • JDK 17 support and higher. • Runs on any Microprofile compliant Application server. • Tested on Wildfly, GlassFish, OpenLiberty, Quarkus, Helidon • Source repo: https://github.com/langchain4j/langchain4j-cdi
- 26. ADDITIONAL FEATURES… • Configurable(Microprofile Config) • Low-level components are supported: ChatModel, ChatMesage, ChatMemory, EmbeddingStore, Embedding, tools • Fault Tolerance • @Timeout, @Fallback, @Retry, @CircuitBreaker, etc. • Ensure you disable maxRetries on Langchain4J (maxRetries=0) when using @Retry • Observability (MicroProfile Telemetry 2.0) • OpenTelemetry Semantic Convention for Generative AI Spans. • OpenTelemetry Semantic Convention for Generative AI Metrics.
- 27. CDI CONFIGURATION… @RegisterAIService. Namesare provided in order to tie the AI service component to the AI agent.
- 28. AI SERVICE INTEGRATION… @RegisterAIService.Registers AI Service, ready for injection (@Inject).
- 29.
- 30. PUTTING IT ALLTOGETHER
- 31. OTHER RESOURCE… • Blogpost: https://deventerpriselog.blogspot.com/2025/04/introducing- smallrye-llm-injecting.html • All examples are found on GitHub.
- 32.
- 33. SPRING AI: FORSPRING DEVELOPERS • Providing AI integration in a familiar Spring ecosystem design principle. • Unified, Spring-friendly API for integrating with diverse AI model providers, allowing developers to connect enterprise data and existing APIs with large language models (LLMs) and other AI services without introducing unnecessary complexity. • Out of the box, Spring AI supports chat (completions), text-to- image, text-to-speech, and speech-to-text models, enabling rich AI capabilities in a unified framework. • Official Documentation: https://docs.spring.io/spring-ai/reference/
- 34. BUHAKE SINDI • Enterprisesolutions expert & consultant • Close to 2 decades in tech, full-stack enterprise developer, DevOps, engineering. • Jakarta EE Ambassador • Full Member of the Java Community Process (MCP) • Jakarta EE / Microprofile member ● IT geek / Runner / Blogger / Tech Speaker ● Started Devoxx4Kids South Africa (NPO) to get kids excited to code
- 35. THANK YOU Buhake Sindi X:@EliteGentleman Social media: buhake.sindi https://linktr.ee/theEliteGentleman