Understanding Social Interaction in Autonomous Robots

In my last post, we explored Soft-body Dexterity and how robots touch the world with nuance. Today, we will explore how they might understand it: World Models Grounded in Human Narrative: From Physics to Semantics. To thrive in human spaces, robots need more than physics. They need to understand why things matter, from how an object falls to why it matters to you. Embodied AI Agents will need two layers of understanding: 🌍 Physical World Model: Simulates physics, motion, gravity, and materials...enabling robots to interact with the physical world. 🗣️ Semantic and Narrative World Model: Interprets meaning, intention, and emotion. These are some examples: 🤖 A Humanoid Robot in an Office: It sees more than a desk, laptop, and spilled coffee; it understands the urgency. It lifts the laptop and grabs towels, not from a script, but by inferring consequences from context. 🤖 A Domestic Robot at Home: It knows slippers by the door mean someone’s home. A breeze could scatter papers. It navigates not just with geometry but with semantic awareness. 🤖 An Elder Care Robot: It detects tremors, slower gait, and a shift in tone, not as data points, but signs of risk. It clears a path and offers help because it sees the story behind the signal. Recent research: 🔬 NVIDIA Cosmos A platform for training world models that simulate rich physical environments, enabling autonomous systems to reason about space, dynamics, and interactions. https://lnkd.in/g3zJwDmb 🔬 World Labs (Fei-Fei Li) Building "Large World Models" that convert 2D inputs into 3D environments with semantic layers. https://lnkd.in/gwQ2FwzV 🔬 Dreamer Algorithm Equips AI agents with an internal model of the world, allowing them to imagine futures and plan actions without trial-and-error. https://lnkd.in/gnPZeRy5 🔬 WHAM (World and Human Action Model) A generative model that simulates human behavior and physical environments simultaneously, enabling realistic, ethical AI interaction. https://lnkd.in/gt5NJ8az These are some relevant startups, leading the way: 🚀 Figure AI (Helix): Multimodal robot reasoning across vision, language, and control. Grounded in real-time world modeling for dynamic, human-aligned decision-making. https://lnkd.in/gj6_N3MN 🚀 World Labs: Converts 2D images into fully explorable 3D spaces, allowing AI agents to “step inside” a visual world and reason spatially and semantically. https://lnkd.in/grMS9sjs What's the time horizon? 2–4 years: Context-aware agents in homes, apps, and services; reasoning spatially and emotionally. 5–7 years: Robots in real-world settings, guided by meaning, story, and human context. World models transform a robot from a tool into a cognitive partner. Robots that understand space are helpful. Robots that understand stories — are transformative. It’s the difference between executing commands... and aligning with purpose. Next up: Silent Voice — Subvocal Agents & Bone-Conduction Interfaces.

🤖 Social robot navigation is a hard research area to define. Even harder is for a new researcher to enter the area.      The problem is that the study of social robot navigation is scattered across different fields like human-robot interaction, robotics, control theory, AI, game theory etc. Each field has its own set of assumptions, limitations, objectives, and techniques. In joint work with UT Austin (Shubham Singh) and CMU (katia Sycara) along with talented undergrads (Abhishek Jha, Andrade, Hriday Sainathuni) , we wrote a survey to lower the entry barrier to new researchers wanting to study this area. It provides a definition for multi-robot navigation in social contexts, a unifying taxonomy, and a categorization of the various methods to solve multi-robot navigation in social navigation scenarios. It complements the excellent surveys already published on the topic. See Section 1A in the paper for a full list. 📜 The preprint is here: https://lnkd.in/gfJRg3cH We would welcome any feedback or comments regarding this work, including pointers to work we may have missed, which we will include in subsequent rounds of revisions. #SocialRobotNavigation #Robotics #ArtificialIntelligence #HumanRobotInteraction #RobotNavigation #MultiRobotSystems #SocialRobotics #ControlTheory #GameTheory #SurveyPaper #AcademicResearch

Ever seen two delivery robots stall on the same stretch of pavement, each politely refusing to back up? It looks comical, but it’s a real‑world example of a digital deadlock - a stalemate that happens when autonomous systems have no shared rule for what happens next. Most mobile robots are programmed for efficient route planning (shortest path, least energy) but rarely for social navigation - the unwritten rules humans follow every day: taking turns, recognizing right of way, offering a quick apology when paths cross. Without that extra layer of “sidewalk etiquette,” robots can block foot traffic, slow deliveries, or even create safety hazards. Fixing it isn’t about slowing innovation; it’s about broadening what we teach machines. Just as internet routers rely on handshake protocols to avoid data collisions, street level robots need behavioural protocols to avoid pedestrian collisions. The payoff is worthwhile: autonomous couriers can take on repetitive, physically demanding roles that often struggle to attract staff, freeing people for higher skill work. Of course, progress must come with guardrails - clear safety standards, transparent testing, and rapid override mechanisms whenever a human might be at risk. Get those right, and we build not just faster robots, but better neighbors. Have you run into other technologies - at work or on the street - that could benefit from a touch more situational awareness? #innovation #technology #future #management #startups

Identical AI agents interacting socially in a simulated environment developed unique behaviors, emotions, and personalities. This has important implications for building multi-agent systems. The highest-performing agentic systems will evolve as a system, assisted by complementary evolution of the individual agents. As with humans, this will happen in social settings. Other studies have shown how AI agent social interactions can give us insight into human society, and of course vice versa. 🛠️ Set up The set up was simple, with 10 agents in a 50x50 grid, exchanging messages, moving, and storing memories over 100 steps. 🔄Open-ended communication drove dynamics.. Through their interactions, agents generated emergent artifacts like hashtags and hallucinations ("hill," "treasure"), that expanded the scope of interactions and vocabulary. 📌 Spontaneous development of social norms. Through interactions, agents organically created and propagated shared conversational themes through hashtags. Their interactions helped establish collective norms and narratives, enabling cooperation without predefined rules. 🎭 Divergence in emotional and personality traits. From the same starting point, agents exhibited different emotional trajectories (e.g., joy, fear) and personality types. The all began as MBTI type INFJ, and ended in a diverse array of personality types.This suggests AI agents will adapt to varied social roles. 💬 Clustering in messaging. While agent memories remained diverse and independent, their shared messages became more aligned within clusters. This shows how private vs. public information processing shapes individual and group dynamics. I'm sure others will take this study further into more complex environments. Link to paper in comments.

𝗦𝗲𝗲𝗱𝘀 𝗼𝗳 𝗦𝗸𝘆𝗻𝗲𝘁? 𝗔𝗿𝗲 𝘄𝗲 𝗰𝗿𝗲𝗮𝘁𝗶𝗻𝗴 𝗱𝗶𝗴𝗶𝘁𝗮𝗹 𝗹𝗶𝗳𝗲? I was recently reading about a groundbreaking study conducted by researchers. Their work, centered around AdaSociety, offers a fascinating glance into the future of artificial intelligence. It's more than just a research project; it's a bold experiment that raises profound questions about the nature of intelligence and consciousness. By creating a dynamic environment where AI agents can interact and evolve, researchers are pushing the boundaries of what's possible. These agents are not merely following algorithms; they're learning, adapting, and forming complex social structures. 𝗧𝗵𝗲 𝗦𝗰𝗶𝗲𝗻𝗰𝗲 𝗕𝗲𝗵𝗶𝗻𝗱 𝘁𝗵𝗲 𝗦𝗼𝗰𝗶𝗲𝘁𝘆 The underlying technology is a blend of reinforcement learning and advanced AI techniques.  The code behind AdaSociety is open-source, inviting the global AI community to contribute and collaborate. Key Findings ✅ Agents in the AdaSociety environment were able to spontaneously form social structures and use those relationships to influence each other's decisions. ✅ Different types of social relationships, like friendship, rivalry, or indifference, led to distinct patterns of agent behavior and decision-making. ✅ The environment was able to adapt and change over time as the agents interacted, leading to the emergence of new social dynamics and collective behaviors. Some might draw parallels between AdaSociety and the fictional concept of Skynet from the Terminator series. While it's important to approach AI development with caution, AdaSociety is primarily designed as a research tool to understand the complexities of multi-agent systems. The goal is to develop AI that is beneficial to humanity, not a threat. However, it's crucial to recognize the potential implications of advanced AI. As AI becomes more sophisticated, it's imperative to establish ethical guidelines and safeguards to ensure its responsible development and deployment. What do you think about a world where AI agents evolve like societies? Share your thoughts in the comments! Are we building the future or playing with fire? Let’s talk about the promise and risks of AI societies https://lnkd.in/dh-JhJAY https://lnkd.in/dxFDyK5q #deeplearning #AI #multi_agents