Applying AI to Small Robotics Systems

Today, Science Robotics has published our work on the first drone performing fully #neuromorphic vision and control for autonomous flight! 🥳 Deep neural networks have led to amazing progress in Artificial Intelligence and promise to be a game-changer as well for autonomous robots 🤖. A major challenge is that the computing hardware for running deep neural networks can still be quite heavy and power consuming. This is particularly problematic for small robots like lightweight drones, for which most deep nets are currently out of reach. A new type of neuromorphic hardware draws inspiration from the efficiency of animal eyes 👁 and brains 🧠. Neuromorphic cameras do not record images at a fixed frame rate, but instead have the pixels track the brightness over time, sending a signal only when the brightness changes. These signals can now be sent to a neuromorphic processor, in which the neurons communicate with each other via binary spikes, simplifying calculations. The resulting asynchronous, sparse sensing and processing promises to be both quick and energy efficient! 🔋 In our article, we investigated how a spiking neural network (#SNN) can be trained and deployed on a neuromorphic processor for perceiving and controlling drone flight 🚁. Specifically, we split the network in two. First, we trained an SNN to transform the signals from a downward looking neuromorphic camera to estimates of the drone’s own motion. This network was trained on data coming from our drone itself, with self-supervised learning. Second, we used an artificial evolution 🦠🐒🚶♂️ to train another SNN for controlling a simulated drone. This network transformed the simulated drone’s motion into motor commands such as the drone’s orientation. We then merged the two SNNs 👩🏻🤝👩🏻 and deployed the resulting network on Intel Labs’ neuromorphic research chip "Loihi". The merged network immediately worked on the drone, successfully bridging the reality gap. Moreover, the results highlight the promises of neuromorphic sensing and processing: The network ran 10-64x faster 🏎💨 than a comparable network on a traditional embedded GPU and used 3x less energy. I want to first congratulate all co-authors at TU Delft | Aerospace Engineering: Federico Paredes Vallés, Jesse Hagenaars, Julien Dupeyroux, Stein Stroobants, and Yingfu Xu 🎉 Moreover, I would like to thank the Intel Labs' Neuromorphic Computing Lab and the Intel Neuromorphic Research Community (#INRC) for their support with Loihi (among others Mike Davies and Yulia Sandamirskaya). Finally, I would like to thank NWO (Dutch Research Council), the Air Force Office of Scientific Research (AFOSR) and Office of Naval Research Global (ONR Global) for funding this project. All relevant links can be found below. Delft University of Technology, Science Magazine #neuromorphic #spiking #SNN #spikingneuralnetworks #drones #AI #robotics #robot #opticalflow #control #realitygap

Artificial Intelligence can look intimidating - “black‑box” algorithms, pricey hardware, teams of PhDs. Yet remarkable results are possible with modest gear and a bit of curiosity. Take 17 year old Ben Choi. Instead of implanting electrodes in the brain (a procedure that can cost hundreds of thousands of dollars), he placed postage stamp sized sensors on the skin of the forearm. These sensors pick up the tiny electrical pulses that our brains send to muscles signals so small they’re measured in microvolts. Here’s where AI enters the story: - Signal capture: The surface sensors record raw voltage changes every few milliseconds. - Pattern learning: A lightweight machine learning model (think of a mini neural network running on a laptop) studies those voltage patterns and learns to match them with the user’s intended hand motions - open, close, rotate, and so on. - Robotic action: A 3D printed arm receives the AI’s instructions and moves accordingly, almost in real time. Because everything runs on off the shelf parts - an Arduino microcontroller, free Python libraries, and affordable hobby grade motors - Ben kept the parts bill under US$300. That price point matters: sophisticated prosthetics and assistive robots typically run well into five or six figures, placing them out of reach for many people who need them most. Projects like this shows that: - Open source tools lower barriers: Frameworks such as TensorFlow, PyTorch, and Scikit‑learn put advanced algorithms a few commands away. - Community knowledge compounds: Tutorials, discussion boards, and hobbyist forums mean you rarely start from scratch. Yes, AI raises legitimate concerns - bias, misuse, security. But it also unlocks practical solutions that improve lives: smarter medical devices, safer vehicles, more intuitive home tech. Have you seen other low cost, high impact AI projects? #innovation #technology #future #management #startups

Brain Size Doesn't Predict Intelligence, Movement Does Current AI systems require massive computational resources to achieve basic pattern recognition tasks that insects accomplish with brains smaller than sesame seeds. University of Sheffield researchers discovered that bees use flight movements and body wiggles to help their brains learn and recognize visual patterns with remarkable accuracy, achieving 96-98% accuracy versus 60% for stationary observation. Organizations developing AI systems have focused on scaling up neural networks rather than optimizing how information gets processed. Bee-inspired computational models demonstrate that intelligence emerges from dynamic interaction between brains, bodies, and environments rather than pure processing power, suggesting a fundamentally different approach to building efficient AI. The computational model shows that just 16 lobula neurons prove sufficient for complex pattern discrimination tasks, while bees scanning only the lower half of patterns dramatically outperform stationary observation methods. This reveals that active perception through movement creates sparse, decorrelated neural responses where specific neurons activate only for particular visual features. Robotics applications could benefit enormously from this bio-inspired approach, enabling systems that actively shape their sensory input through movement rather than passively processing massive datasets. Future robots can become smarter and more efficient by using movement to gather information, rather than relying on massive computing power. Implementation challenges include developing movement strategies that optimize information gathering while maintaining energy efficiency. The research suggests that intelligence requires embodied interaction with environments, not just larger neural networks or faster processors. Bio-inspired AI represents a pathway toward more sustainable artificial intelligence that achieves better performance with dramatically reduced computational requirements, fundamentally changing how we approach machine learning architecture design. 🔗https://lnkd.in/ePcWpYK5