TDC - The Developers Conference - Creating GenAI Apps in Java with SD4J and the ONNX Runtime

TDC - The Developers Conference - Creating GenAI Apps in Java with SD4J and the ONNX Runtime

• 1.
  Confidential - OracleRestricted Confidential - Oracle Restricted Criando Aplicativos GenAI em Java com SD4J e ONXX Juarez Barbosa Junior @juarezjunior Sr Principal Java Developer Evangelist 12 de Junho de 2024 CentroSul – Florianópolis, SC, Brasil T D C F LO R I PA
• 2.
  Juarez Barbosa Junior SeniorPrincipal Java Developer Evangelist • Coming from Dublin, Ireland • Developer since 1995 (Java ☕️ 1997) • 28 years of experience in SW Engineering & DevRel • Microsoft, Oracle, IBM, Nokia, Unisys, Accenture, startups • Microsoft Azure Developer Relations Lead • IBM Watson Tech Evangelist & Cloud Rockstar • IBM Mobile Tech Evangelist & Global Thought Leader • Nokia Developers Global Champion • Java, Python, Cloud, DevOps, SRE, Cloud-native, IoT, AI, Blockchain, Rust • Speaker at conferences • Oracle CloudWorld, Oracle Code, Microsoft Ignite, Microsoft TechX, jPrime, JCON, DevConf.cz, GeeCon, DevOpsDays, DeveloperWeek, DevOps Institute, CloudLand, DWX, The Developer’s Conference (TDC), Sec4Dev, JSNation, NodeConf, Conf42, Shift Conf, Global Azure, Open-Source Lisbon, CodeFrenzy, Mêlée Numérique, React Summit, Test.js Summit, Porto TechHub Conf, Pyjamas, JUGs, meetups, etc. @juarezjunior @juarezjunior
• 3.
  Confidential - OracleRestricted Confidential - Oracle Restricted Workshop Agenda • Stable Diffusion - Concepts • Stable Diffusion for Java (SD4J) by Oracle Labs • ONNX and ONNX Runtime Extensions • SD4J – Demo + Workshop • Oracle Database 23ai • AI / ML with Oracle Cloud • Technical References Copyright © 2024, Oracle and/or its affiliates
• 4.
  Confidential - OracleRestricted Confidential - Oracle Restricted Stable Diffusion - Concepts • In a nutshell • Stable Diffusion is a technique under the umbrella of GenAI used to generate images primarily from textual descriptions. • A diffusion model predicts the probability distribution of images using a diffusion process. • The diffusion process iteratively applies noise to an image, gradually increasing the noise level and then reversing the process to reconstruct the original image - denoising with guidance, aligning with the textual description. • One of the advantages of Stable Diffusion is its ability to control the diversity and quality of generated images through the noise level parameter. • The noise intensity allows users to adjust the image quality and diversity trade-offs. Copyright © 2024, Oracle and/or its affiliates
• 5.
  Confidential - OracleRestricted Confidential - Oracle Restricted Stable Diffusion - Concepts • In a nutshell • Stable Diffusion finds applications in various domains, such as art, design, and entertainment, by providing image synthesis, augmentation, and manipulation. • It can run on regular computers with good graphics cards, making it more accessible. • Most projects are Open-Source, allowing developers to experiment and fine- tune the model to meet specific needs. Copyright © 2024, Oracle and/or its affiliates
• 6.
  Confidential - OracleRestricted Confidential - Oracle Restricted Stable Diffusion for Java • SD4J • A Java implementation of Stable Diffusion (modified port of a C# implementation) running on top of ONNX Runtime • Created by Oracle Labs (R&D) as OSS • It targets ONNX Runtime 1.14 • It supports using CPUs and CUDA GPUs • It supports SD v1.5, SD v2 and SDXL style models • It provides a reusable class (com.oracle.labs.mlrg.sd4j.SD4J) with a clear contract (interface) to allow the creation of image generation pipelines Copyright © 2024, Oracle and/or its affiliates
• 7.
  Confidential - OracleRestricted Confidential - Oracle Restricted ONNX - Open Neural Network Exchange • ONNX Format • An open-source format designed to represent machine learning models. • Expressed as a Protocol Buffers (protobuf) file containing a computational graph representing the model. • Models trained in one framework can be converted to the ONNX format and used in another. • The ONNX ecosystem includes a wide range of tools and libraries for model conversion, optimization, inference, and visualization. • While ONNX can represent models for training, its primary focus is on inference, which is the process of using a trained model to make predictions on new data. • Promotes interoperability (frameworks, tools, runtime environments). • Also promotes standardization, efficiency (HW), and ecosystem development. Copyright © 2024, Oracle and/or its affiliates
• 8.
  Confidential - OracleRestricted Confidential - Oracle Restricted ONNX - Open Neural Network Exchange • ONNX Runtime • The ONNX Runtime is an optimized inference engine, offering efficient execution of ONNX models with low latency and high throughput. • It leverages hardware acceleration capabilities, including GPU acceleration through CUDA, and optimizations for CPUs. • It provides tools for quantizing and optimizing ONNX models to improve performance and reduce memory footprint, making them more suitable for deployment on resource- constrained devices. • It is also cross-platform, as the ONNX Runtime supports multiple operating systems, including Windows, Linux, and macOS, making it versatile for deployment across different target environments. Copyright © 2024, Oracle and/or its affiliates
• 9.
  Confidential - OracleRestricted Confidential - Oracle Restricted Copyright © 2023, Oracle and/or its affiliates. All rights reserved Experimente o poder da IA gratuitamente na nuvem Conheça os recursos de IA e ML da Oracle Cloud com o Modo Gratuito. São US$ 500 em créditos para você usar durante 30 dias – uma condição EXCLUSIVA para visitantes do TDC Floripa!
• 10.
  Confidential - OracleRestricted Confidential - Oracle Restricted New | Oracle Database 23ai Next-Generation Converged Database Services Over 300 major new features plus thousands of enhancements Available on OCI and Azure Major focus on • AI for Data features • Developer/Analyst features • Mission-Critical features Addresses data management pain points that have frustrated customers forever Copyright © 2024, Oracle and/or its affiliates
• 11.
  Confidential - OracleRestricted Confidential - Oracle Restricted Oracle Database 23ai - AI for Data Algorithmic AI AI Vector Search Augmented Generative AI (LLMs) Distributed AI AI Storage AI Developer Tools Copyright © 2024, Oracle and/or its affiliates
• 12.
  Confidential - OracleRestricted Confidential - Oracle Restricted Oracle Database 23ai - Dev for Data JSON Relational Duality Views Property Graph Views JavaScript Stored Procedures Data Intent Language Lock-free Consistent Updates, Long-running Transactions True Cache Copyright © 2024, Oracle and/or its affiliates
• 13.
  Confidential - OracleRestricted Confidential - Oracle Restricted Oracle Database 23ai - Mission-Critical Data RAFT Replication for Globally Distributed Database In-Database Firewall Real-Time SQL Plan Management RAC, Exadata, Data Guard Simplicity and Scalability Analytic SQL Simplicity and Scalability Priority Transactions Copyright © 2024, Oracle and/or its affiliates
• 14.
  Confidential - OracleRestricted Confidential - Oracle Restricted Free | Oracle Database 23ai For Developers Oracle Always Free ADB Available on OCI ADB Free Container Image Available for download Oracle Database Free Available as RPM, Docker Image, VBox VM Copyright © 2024, Oracle and/or its affiliates | Confidential - Internal Very Low Cost Supported Developer Edition of ADB, BaseDB, ExaDB-D, and ExaDB-C@C
• 15.
  Confidential - OracleRestricted Confidential - Oracle Restricted SD4J – Demo + Workshop • Creating GenAI Apps in Java with SD4J (Stable Diffusion for Java) and the ONNX Runtime • https://juarezjunior.medium.com/b72385c59329 Copyright © 2024, Oracle and/or its affiliates
• 16.
  Confidential - OracleRestricted Confidential - Oracle Restricted Copyright © 2024, Oracle and/or its affiliates Technical References • SD4J – xxx • ONNX Format and Runtime - xxx • Oracle AI Vector Search - https://rb.gy/sp41uk • Oracle AI Vector Search for Java Developers with the Oracle Database 23ai - https://rb.gy/7ivwxr • Java / JDBC – https://www.oracle.com/jdbc • Introduction to Oracle JDBC Driver Support for Virtual Threads - https://bit.ly/3UlNJWP • Developing an Oracle JDBC app with GraalVM Native Image - https://rb.gy/iy3sgh • Getting Started with Reactive Relational Database Connectivity and the Oracle R2DBC Driver - https://rb.gy/42dnz5 • Getting Started with the Java library for Reactive Streams Ingestion (RSI) - https://bit.ly/3rEiRnC • Introduction to JDBC Reactive Extensions with the Oracle Database — Developer Release – https://rb.gy/qxlrbx • Pipelined Database Operations - https://rb.gy/iy3sgh
• 17.
  Juarez Junior @juarezjunior