Constrained Optimization with Genetic Algorithms and Project Bonsai

DATA SATURDAY #10 Sofia, Oct 09th Another PoV on Data Autonomous Systems and Project Bonsai Ivelin Andreev

Speaker Bio • Software Architect @ o 19+ years professional experience • Microsoft Azure MVP • External Expert Eurostars-Eureka • External Expert InnoFund Denmark, RIF Cyprus • Business Interests o Web Development, SOA, Integration o IoT, Machine Learning, Computer Intelligence o Security & Performance Optimization • Contact ivelin.andreev@icb.bg www.linkedin.com/in/ivelin www.slideshare.net/ivoandreev

Agenda Agenda • Evolutionary and Genetic Algorithms • Autonomous vs Automated System • Bonsai Project • Bonsai Platform Components • Basic Concepts and Technical Overview • Demo

Constrained Optimization Def: Maximize/Minimize a cost function within constraints (i.e. time, money, fuel) Evolutionary Algorithms • Heuristics-based problem solving through mimicking the behavior of living creatures to avoid exhaustive solution application Pros • Solve global optimization problems • Scalable to higher dimensional problems • Flexible and adaptable to many problem types • Multiple “best” solutions Cons • Vulnerable to bad initial configuration • Premature convergence to a local extremum may not reach even close to global extremum

Genetic Algorithms Def: Тype of EA that uses crossover and mutations to search the solution space. Key Principle: Mating healthier parents shall lead to healthier offspring. • Population – set of current solutions • Individual – a solution • Fitness – criterion to compare generations/solutions (f(x) = 5x^2) • Variation – modification of population to generate new solutions • Chromosome – a set of features describing the individual • Gene – Each part of the chromosome, encoded by numbers

Simple/Sample Case Background • Thousands of heating devices • Current state, Usage history • Power price/hour 1 day ahead Objective • Cost optimization of heating Limitations • Heater has maximum energy @80°C • Charge has to be >= daily consumption • Computation scalability Solution • Time series forecasting • Genetic algorithms

Autonomous Systems? Def: Intelligent systems that operate in highly dynamic PHYSICAL environments by sense, plan and act based on changing environment variables • To apply Real World… o Millions of simulations o Assessment in real world required o Simulation needs to resemble physical world as close as possible. o A bad reward function = The effect Reacting starts from gathering information about the environment. Interpreted sensor information and translated to actionable information based on reward. Once the action plan is in progress, evaluate proximity of the objective.

• Autonomy is moving ML from bits to atoms • Levels of Autonomy L0 Humans In complete control L1 Assistance with subtasks L2 Occasional, human specifies intent L3 Limited, human fallback L4 Full control, human supervises L5 Full autonomy, human is absent Autonomous, not Automated • Automated Systems o Execute a (complex) “script” o Apply long term process changes (i.e. react to wear) o Apply short-term process changes (clean, diagnose) o No handling of uncoded decisions • Autonomous Systems o Aim at objectives w/o human intervention o Monitor and sense the environment o Suits dynamic processes and changing conditions • Disadvantages o Human trust and explainability (Consumer acceptance) o What if the computer breaks (Technology and infrastructure) o Security and Responsibility (Policy and legislation)

• Industrial robotics and manufacturing • Motion control • Smart buildings • Process control and automation • Bonsai emerges as a general-purpose, deep reinforcement learning platform • Microsoft acquire Bonsai in move to build ‘brains’ for autonomous systems • Released for public preview in May 2020 • The simplest and richest toolchain for AS o Machine teaching abstracting o Automatic model generation o Host APIs for simulator integration and inference Some History https://blogs.microsoft.com/blog/2018/06/20/microsoft-to-acquire-bonsai-in-move-to-build-brains-for-autonomous-systems/

Project Bonsai in the Spotlight • Low code end-end platform that speeds AI-powered automation development • Integrate realistic simulation • Train adaptable brains • Export and consume brains • Cost? o Completely free in preview • Azure Resources o Container Registry (brains and simulators) o Container Instance (running simulations) o Storage (uploaded ZIPped simulators) o Monitor (logs of training and assessment)

Accelerators and Setting up • Solution accelerators designed for specific industrial optimization problems • Pricing o Project Bonsai (Free for now) o Simulator software (AnyLogic, MATLAB) o Azure Services (VM D2_v4, public IP, 128GiB SSD, ACR) ~90EUR/month • Supported Use Cases o Chemical reactor optimization (Simulink - Continuous Stirred Tank Reactor) o Building energy management (Simulink – House Thermal Model) o Factory logistics (AnyLogic – Factory Floor) • Templates did not work ☺, but after report fix is in progress

Windfarm Energy Output Optimization

Simulator as Data Source • Simulator – replicate real world for realistic training environment • Supported Simulations o AnyLogic, MATLAB, VP Link or API (REST, Python) • Minimum requirements o Initial state (often random) o Iterates over time o Responds to external actions • Good simulation requirements o Good fidelity to match real world o Visualization or data output for training assessment o Well defined environment state (parameters) o Discrete actions for the brain to affect state o Clear criteria for failure and success state

Simulators • Option 1: Off the shelf software o Pros: Thousands of ready code and samples; Highly graphical o Cons: Steep learning curve; Resource consuming; Coding still required o (Small Business, Academia, Free Trial) o (Enterprise, Academic, Free Matlab Online Trial) • Option 2: Custom simulators o API Client SDK in Python, C#, Java, TypeScript o Build as Docker image o Push to AZ Container Registry (managed = scalability) o Consume from Bonsai portal using “Add Simulator” o Pros: Complete control; no upfront costs o Cons: A Developer approach; Hard to code some existing simulations o Sample: https://github.com/microsoft/cartpole-py AI Agent Action Simulator State Reward

Bonsai Components – Training Engine • Architect o Proposes the configuration of learning algorithms and topologies that have the best chance at learning o 3 deep learning reinforcement learning algorithms with heuristics params • Instructor o Configures the learner in real time to follow the training plan of the architect • Learner o Training: train and evaluate response o Inference: Initiate the trained system and respond to request • Predictor o A trained brain stored in prediction mode o HTTP API endpoint

Bonsai Components - Brain • AI models that can optimize the systems they have been trained for • Supports multiple versions • Versions reflect changes in training curriculum • Any version is exportable outside the platform • Goal Objectives o Encapsulate intention-to-reward function and simplify the work of operator o Avoid – avoid a region o Reach - go to target ASAP o Drive – go to target ASAP and stay near o Maximize – push target as high as possible o Minimize – push target as low as possible

Iteration: Concept Graph View Agent Action The brain output on every iteration Observable State Passed to brain on every iteration Simulator State Output by simulator on every iteration Concept What the brain is trying to learn Concept Goals What the brain has to achieve State Transformation Implicit conversion of sim state Simulator Simulator used for concept training

Inkling Language • Orig: Fictious language spoken by Nintendo Splatoon human-squid inklings • Now: Proprietary language defines what and how you want to teach the AI • Statically typed language o Compiler detects and reports errors • Starts with Version • Data Types o Primitives: Number (integer, float), String with ranges and enumerations o Collections: Arrays, Structures, User defined types inkling "2.0“ # Comments like that type Coordinate {X: number, Y: number} number<0 .. 6 step 2> # represents {0,2,4,6}

Concept – What to learn • Concept o Component of a brain that must be learned, programmed or imported o Sample: Ensure hot water is available in a boiler while heating in low load hours • Imported Concept o Compatible models trained on other platforms (ONNX, TensorFlow) • Programmed Concept o Defined using the programmed keyword followed by a function definition or reference. • Learned Concept o Specify a curriculum that defines how the AI should be taught concept ConceptName(Input1, Input2): OutputType { # Concept definition } function doJob() {…} concept ConceptName (Input): OutputType { programmed doJob }

Curriculum – How to learn • Defines how the training engine should train the AI within concept o Rule: 1 curriculum per concept, each concept with curriculum • Defines lessons for staged teaching plan and control training episodes run • Training stops when: o Stopped manually o When training is no longer improving o Limits reached (i.e. training time) • Source o Data source for teaching the concept (Simulator) • Parameters Parameter Default Description EpisodeIterationLimit 1000 Iterations per training episode TotalIterationLimit 50,000,000 Total iterations for the concept NoProgressIterationLimit 250’000 Iterations allowed w/o improvement LessonRewardThreshold None Minimum reward value for success LessonSuccessThreshold 0.9 Min success rate to complete lesson LessonAssessmentWindow 30 Number of episodes per assessment

Goal Keyword • High level specification of what we want AI to learn o The Bonsai engine determines appropriate reward functions • Objectives o Avoid o Reach o Drive o Maximize o Minimize • Goal Ranges using Goal # namespace Curriculum { … goal (s:SimState){ avoid Fall: Math.Abs(state.angle) in Goal.RangeAbove(MaxAngle) drive SmallAngle: Math.Abs(state.angle) in Goal.Range(0, MaxAngle/6) }} Parameter Description Goal.Range(X, Y) Values between X and Y, inclusive Goal.RangeAbove(X) Values greater than or equal to X Goal.RangeBelow(X) Values less than or equal to X Goal.Sphere(X, R) Values within line with radius R Goal.Sphere([X, Y], R) Values within 2D circle with radius R Goal.Sphere([X, Y, Z], R) Values within 3D sphere with radius R

Simulator Keyword • Virtual environment to simulate the behaviour of a real-world physical env. • Simulator has observable state information - the state at a given moment o Typically sensor measurements • Note: Inkling does not contain simulators code but defines interaction with the simulator to train a concept. • Simulator statement o Describes the interface to the simulator program o ActionType – action input type o ConfigType – configuration input type o StateType – state output type • Packages (optional) o references a simulator package registered for use in the cloud simulator MySimulator(action: ActionType, config: ConfigType): StateType {}

Takeaways o MS Innovation Tech Hub (News, concepts, recent developments) • https://innovation.microsoft.com/en-us/autonomous-systems o Bonsai Portal • https://preview.bons.ai/ o Community Forum • https://techcommunity.microsoft.com/t5/project-bonsai/ct-p/ProjectBonsai o Accelerators • https://docs.microsoft.com/en-us/autonomous-systems/bonsai-accelerators/what-are-bonsai-accelerators o Fixed Accelerator Sample • https://techcommunity.microsoft.com/t5/project-bonsai/accelerators-failed-to-deploy-image-is-missing/m-p/2460816 o End-end Walkthrough – Project Bonsai & AnyLogic • https://www.youtube.com/watch?v=-iTSZaG3lg8 • https://microsoft.github.io/moab/tutorials/2-robustness/index.html o Simulator Sample & Bonsai SDK • https://github.com/microsoft/cartpole-py • https://github.com/microsoft/microsoft-bonsai-api

Upcoming Events jsTalks (Bulgaria), 2021 November 19-20 http://jstalks.net/ Azure MVP Unplugged, 5th Edition October 21 Global AI Back Together Bulgaria 2021 October 19

Constrained Optimization with Genetic Algorithms and Project Bonsai

More Related Content

What's hot

Similar to Constrained Optimization with Genetic Algorithms and Project Bonsai

More from Ivo Andreev

Recently uploaded

Constrained Optimization with Genetic Algorithms and Project Bonsai