Advancing AI Development

Last week, I described four design patterns for AI agentic workflows that I believe will drive significant progress: Reflection, Tool use, Planning and Multi-agent collaboration. Instead of having an LLM generate its final output directly, an agentic workflow prompts the LLM multiple times, giving it opportunities to build step by step to higher-quality output. Here, I'd like to discuss Reflection. It's relatively quick to implement, and I've seen it lead to surprising performance gains. You may have had the experience of prompting ChatGPT/Claude/Gemini, receiving unsatisfactory output, delivering critical feedback to help the LLM improve its response, and then getting a better response. What if you automate the step of delivering critical feedback, so the model automatically criticizes its own output and improves its response? This is the crux of Reflection. Take the task of asking an LLM to write code. We can prompt it to generate the desired code directly to carry out some task X. Then, we can prompt it to reflect on its own output, perhaps as follows: Here’s code intended for task X: [previously generated code] Check the code carefully for correctness, style, and efficiency, and give constructive criticism for how to improve it. Sometimes this causes the LLM to spot problems and come up with constructive suggestions. Next, we can prompt the LLM with context including (i) the previously generated code and (ii) the constructive feedback, and ask it to use the feedback to rewrite the code. This can lead to a better response. Repeating the criticism/rewrite process might yield further improvements. This self-reflection process allows the LLM to spot gaps and improve its output on a variety of tasks including producing code, writing text, and answering questions. And we can go beyond self-reflection by giving the LLM tools that help evaluate its output; for example, running its code through a few unit tests to check whether it generates correct results on test cases or searching the web to double-check text output. Then it can reflect on any errors it found and come up with ideas for improvement. Further, we can implement Reflection using a multi-agent framework. I've found it convenient to create two agents, one prompted to generate good outputs and the other prompted to give constructive criticism of the first agent's output. The resulting discussion between the two agents leads to improved responses. Reflection is a relatively basic type of agentic workflow, but I've been delighted by how much it improved my applications’ results. If you’re interested in learning more about reflection, I recommend: - Self-Refine: Iterative Refinement with Self-Feedback, by Madaan et al. (2023) - Reflexion: Language Agents with Verbal Reinforcement Learning, by Shinn et al. (2023) - CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing, by Gou et al. (2024) [Original text: https://lnkd.in/g4bTuWtU ]

A recent study by Harvard and MIT concluded that today’s AI models can predict physical phenomena but can’t yet explain them. This shouldn't be surprising to anyone, because “AI for Science” today is still largely being driven by AI researchers... for other AI researchers. Even AI for Science departments at AI Research Labs are lead by people with a CS background. I say this as a physicist who experienced intense discrimination in AI research circles for years. I have even been told I couldn't "understand researche" because my PhD was in Particle Physics, not in CS... 😵💫 And now, some of those same people want to solve all of science 🤯 Just to be clear, I believe AI deserves a central spot in scientific circles as a tool for all scientists to accelerate their work; it can even be a game changer. But this is also the right time for us all to call for cross-disciplinary partnerships. If AI wants to contribute to scientific discovery, it needs to become more scientific by: 👉 Encouraging collaboration with domain experts (and treat them as peers) 👉 Build architectures inspired by physical priors, not just data trends 👉 Value explanations and not just predictions Until then, AI for Science will keep predicting planetary motion while missing the laws that govern it. #AIforScience #AIResearch #Research #Physics

I recently spoke to Gartner about what is next in #AI. Here are my thoughts: We have seen impressive progress in #llm by scaling data and compute. Will this continue to hold? Yes, I believe so, but most of those gains will be in reasoning tasks where we have precise metrics to measure uplift, as well as the ability to have synthetic data to train further, and also the freedom to trade off computation for accuracy at test time. This is seen in the recent o1 model. For reasoning tasks, we will also be able to remove hallucination when we can construct accurate verifiers that can certify every statement that #llm makes. We have been doing this in our Leandojo project for mathematical theorem proving. However, there is one area of reasoning where #llm will never be good enough: understanding the physical world. This is because language is only high-level knowledge, and cannot simulate the complex physical phenomena needed in many applications. For instance, LLMs can talk about playing tennis or look up a weather app, but they cannot internally simulate any of these processes. While images and videos can help improve their knowledge of the physical world, models like Sora learn physics by accident, and hence, still produce physically wrong outputs. How can we overcome this? By teaching AI physics from the ground up. We are building AI models that are trained in a physics-informed manner at multiple scales. They are several orders of magnitude faster than traditional simulations, and can also generate novel designs that are physically valid. You can watch some of those examples in my recent TED talk.

🔥 Why DeepSeek's AI Breakthrough May Be the Most Crucial One Yet. I finally had a chance to dive into DeepSeek's recent r1 model innovations, and it’s hard to overstate the implications. This isn't just a technical achievement - it's democratization of AI technology. Let me explain why this matters for everyone in tech, not just AI teams. 🎯 The Big Picture: Traditional model development has been like building a skyscraper - you need massive resources, billions in funding, and years of work. DeepSeek just showed you can build the same thing for 5% of the cost, in a fraction of the time. Here's what they achieved: • Matched GPT-4 level performance • Cut training costs from $100M+ to $5M • Reduced GPU requirements by 98% • Made models run on consumer hardware • Released everything as open source 🤔 Why This Matters: 1. For Business Leaders: - model development & AI implementation costs could drop dramatically - Smaller companies can now compete with tech giants - ROI calculations for AI projects need complete revision - Infrastructure planning can possibly be drastically simplified 2. For Developers & Technical Teams: - Advanced AI becomes accessible without massive compute - Development cycles can be dramatically shortened - Testing and iteration become much more feasible - Open source access to state-of-the-art techniques 3. For Product Managers: - Features previously considered "too expensive" become viable - Faster prototyping and development cycles - More realistic budgets for AI implementation - Better performance metrics for existing solutions 💡 The Innovation Breakdown: What makes this special isn't just one breakthrough - it's five clever innovations working together: • Smart number storage (reducing memory needs by 75%) • Parallel processing improvements (2x speed increase) • Efficient memory management (massive scale improvements) • Better resource utilization (near 100% GPU efficiency) • Specialist AI system (only using what's needed, when needed) 🌟 Real-World Impact: Imagine running ChatGPT-level AI on your gaming computer instead of a data center. That's not science fiction anymore - that's what DeepSeek achieved. 🔄 Industry Implications: This could reshape the entire AI industry: - Hardware manufacturers (looking at you, Nvidia) may need to rethink business models - Cloud providers might need to revise their pricing - Startups can now compete with tech giants - Enterprise AI becomes much more accessible 📈 What's Next: I expect we'll see: 1. Rapid adoption of these techniques by major players 2. New startups leveraging this more efficient approach 3. Dropping costs for AI implementation 4. More innovative applications as barriers lower 🎯 Key Takeaway: The AI playing field is being leveled. What required billions and massive data centers might now be possible with a fraction of the resources. This isn't just a technical achievement - it's a democratization of AI technology.

OpenAI's latest model consumes more electricity than Pittsburgh. Microsoft's new AI datacenter will require 1 gigawatt of power—equivalent to powering 750,000 homes. When you do the math for human-level capabilities, we'd need 9 orders of magnitude more compute than today's largest models. This isn't a problem we can solve by throwing money at it. Tech giants are turning to nuclear power and courting clean energy providers. But the laws of thermodynamics don't care about your market cap.   At a certain scale, the heat from computation becomes physically impossible to manage. For enterprise AI these constraints change everything. Companies betting their stack on LLMs need to understand this reality. There is a huge opportunity for those who find novel architectures that are radically more efficient. A few paths look promising: → Test-time compute: making models think harder during inference instead of training → State Space Models that handle longer sequences without exploding compute costs → New architectures like RWKV that slash the cost of running these systems Constraints drive innovation. The physical limits we're hitting will force the industry toward more elegant solutions. For founders building in AI this creates massive opportunities to outmaneuver larger players who are over-invested in the scaling approach. Physics is about to break up AI's scaling party.

If you are building AI agents or learning about them, then you should keep these best practices in mind 👇 Building agentic systems isn’t just about chaining prompts anymore, it’s about designing robust, interpretable, and production-grade systems that interact with tools, humans, and other agents in complex environments. Here are 10 essential design principles you need to know: ➡️ Modular Architectures Separate planning, reasoning, perception, and actuation. This makes your agents more interpretable and easier to debug. Think planner-executor separation in LangGraph or CogAgent-style designs. ➡️ Tool-Use APIs via MCP or Open Function Calling Adopt the Model Context Protocol (MCP) or OpenAI’s Function Calling to interface safely with external tools. These standard interfaces provide strong typing, parameter validation, and consistent execution behavior. ➡️ Long-Term & Working Memory Memory is non-optional for non-trivial agents. Use hybrid memory stacks, vector search tools like MemGPT or Marqo for retrieval, combined with structured memory systems like LlamaIndex agents for factual consistency. ➡️ Reflection & Self-Critique Loops Implement agent self-evaluation using ReAct, Reflexion, or emerging techniques like Voyager-style curriculum refinement. Reflection improves reasoning and helps correct hallucinated chains of thought. ➡️ Planning with Hierarchies Use hierarchical planning: a high-level planner for task decomposition and a low-level executor to interact with tools. This improves reusability and modularity, especially in multi-step or multi-modal workflows. ➡️ Multi-Agent Collaboration Use protocols like AutoGen, A2A, or ChatDev to support agent-to-agent negotiation, subtask allocation, and cooperative planning. This is foundational for open-ended workflows and enterprise-scale orchestration. ➡️ Simulation + Eval Harnesses Always test in simulation. Use benchmarks like ToolBench, SWE-agent, or AgentBoard to validate agent performance before production. This minimizes surprises and surfaces regressions early. ➡️ Safety & Alignment Layers Don’t ship agents without guardrails. Use tools like Llama Guard v4, Prompt Shield, and role-based access controls. Add structured rate-limiting to prevent overuse or sensitive tool invocation. ➡️ Cost-Aware Agent Execution Implement token budgeting, step count tracking, and execution metrics. Especially in multi-agent settings, costs can grow exponentially if unbounded. ➡️ Human-in-the-Loop Orchestration Always have an escalation path. Add override triggers, fallback LLMs, or route to human-in-the-loop for edge cases and critical decision points. This protects quality and trust. PS: If you are interested to learn more about AI Agents and MCP, join the hands-on workshop, I am hosting on 31st May: https://lnkd.in/dWyiN89z If you found this insightful, share this with your network ♻️ Follow me (Aishwarya Srinivasan) for more AI insights and educational content.

I used to be skeptical of vertical-focused startups—historically the TAM looked small next to broad horizontal plays. I’ve changed my mind. As others have written extensively, LLMs, AI Agents, and MPC lets founders push past traditional software boundaries while building deeper, defensible value. My partner Navin Chaddha wrote an excellent analysis on the case for vertical AI (link in comments). Here are the key takeaways: WHY VERTICAL AI WINS • Precision. In medicine, finance, or chips, a model that’s even slightly more accurate wins. • Fast ROI. Fine-tune on industry data with Agentic and MPC enabled workflows, deploy quickly, show savings or net-new revenue quickly. • Lower hallucinations. A narrow answer space built on clean domain data earns trust where “maybe” isn’t acceptable. • Built-in compliance. Treat regulation as a moat—design audit trails and privacy walls up-front. VERTICAL AI PLAYBOOK - Attack “hair-on-fire” use cases in a single domain; be the best, not the broadest. - Build proprietary data loops and generate synthetic data where necessary. - Start from the best open-source vertical model in your space and innovate on the last mile and integrations.  - Meet customers where they are—on-prem, VPC, or air-gapped; don’t let deployment friction slow adoption. AREAS OF INTEREST Healthcare care-coordination, underwriting and servicing, industrial maintenance, supply chain, audit, home services—all demand vertical precision today (and more). Each could be a billion-dollar company for a team pairing domain pain with AI advantage. BOTTOM LINE General LLMs are the platform. Vertical AI is where durable enterprise value accrues. We’re still in inning one—most sectors rely on manual workflows or legacy tools ripe for replacement. 

😵 Woah, there’s a full-blown paper on how you could build a memory OS for LLMs. Memory in AI systems has only started getting serious attention recently, mainly because people realized that LLM context lengths are limited and passing everything every time for complex tasks just doesn’t scale. This is a forward-looking paper that treats memory as a first-class citizen, almost like an operating system layer for LLMs. It’s a long and dense read, but here are some highlights: ⛳ The authors define three types of memory in AI systems: - Parametric: Knowledge baked into the model weights - Activation: Temporary, runtime memory (like KV cache) - Plaintext: External editable memory (docs, notes, examples) The idea is to orchestrate and evolve these memory types together, not treat them as isolated hacks. ⛳ MemOS introduces a unified system to manage memory: representation, organization, access, and governance. ⛳ At the heart of it is MemCube, a core abstraction that enables tracking, fusion, versioning, and migration of memory across tasks. It makes memory reusable and traceable, even across agents. The vision here isn't just "memory", it’s to let agents adapt over time, personalize responses, and coordinate memory across platforms and workflows. I definitely think memory is one of the biggest blockers to building more human-like agents. This looks super well thought out, it gives you an abstraction to actually build with. Not totally sure if the same abstractions will work across all use cases, but very excited to see more work in this direction! Link: https://lnkd.in/gtxC7kXj

NVIDIA is… having a moment. Their shocking revenue (and profit) growth over the last 6 months is literally unprecedented for a business of their size. But why is it happening? The obvious answer is LLM mania and the release of OpenAI's ChatGPT. But why is *that* happening now? And why does all the value seem to accurue to NVIDIA compared to all other GPU companies and suppliers? It's the perfect storm of these factors: 1. 🔋 OpenAI bet big on the transformer architecture in 2019, which just became commercially viable After Google's 2017 research paper "Attention is all you Need", the AI world discovered a new mechanism that could dramatically improve the output from language models: the Transformer. By 2019, OpenAI had bet the farm to this new (but expensive!) approach, raising billions of dollars and shifting their efforts to a Generative Pre-Trained Transformer model (GPT). Over the next 4 years, models went from terrible, to promising, to shockingly good. And plenty of other companies entered the LLM race too. 2. 💻 Turns out, LLMs need an astonishing amount of compute LLMs have a huge tradeoff. They can require hundreds of millions of dollars to train the models and huge amounts of data. NVIDIA had been working on just the right chip architecture (A100 and H100s) for the last 5 years to enable all of this. But it doesn't just take compute... 3. 🔌 LLMs also need super fast networking and on-chip memory Models like GPT-4 are so large that they can't be stored in the memory of a single GPU. They need to be spread across multiple GPUs (and multiple racks of GPUs!) in order to train. This requires high-bandwidth connection between GPUs and racks in order to all work like "one computer". In Jensen's words, "the data center is the computer." NVIDIA bought Mellanox a few years ago, and are the only provider in the market of InfiniBand networking. Infiniband is uniquely well-suited for this, with speeds of 3200 GB per *second*! Now, NVIDIA's integrated networking and high-bandwidth memory connected to the H100 GPUs make for a very nice solution. 4. 💾 NVIDIA's software stack (CUDA) is the standard for the entire ecosystem of AI development Programming for parallel execution is hard. NVIDIA has spent 15+ years building the entire software stack (compiler, programming language, dev tools, libraries, etc.) to make it doable. Their belief was that accelerated computing could happen much faster if you provide the right tools. Now, over 4 million developers use CUDA, providing a meaningful switching cost to move off of NVIDIA as an AI developer. 5. 🌕 As luck would have it, NVIDIA has reserved a huge chunk of TSMC's capacity to make chips like this. Interestingly, this actually pre-dates the AI boom (partially from crypto mining!) but either way, this advantage belongs to NVIDIA at the moment. This is just part of the story. For the rest, check out this Acquired episode! #ai https://lnkd.in/g-Aygazn

NVIDIA reported earnings yesterday, and, as is tradition, they crushed expectations, guided conservatively, and the stock promptly fell 3% because when you’re priced for perfection, even dominance is a mild disappointment. But let’s ignore the stock market tantrum for a moment and parse Jensen Huang's earnings call commentary for industry context: 🚀 AI Demand is Still in Hyper-Growth Mode. Data Center revenue surged to $35.6B (up 93% YoY). Blackwell is NVIDIA's fastest-ramping product ever—$11B in its first full quarter, not even a year after it was first announced. Jensen notes "It will be common for Blackwell clusters to start with 100,000 GPUs". 🧴 Inference is the Bottleneck. Reasoning models like OpenAI's GPT-4.5, DeepSeek AI-R1, and Grok-3 require 100x more compute per query than their early ancestors. AI is moving beyond one-shot inference to multi-step reasoning, chain-of-thought prompting, and autonomous agent workflows. Blackwell was designed for this shift, delivering 25x higher token throughput and 20x lower cost vs. Hopper. 📈 3 Scaling Laws. Jensen identified three major AI scaling trends that are accelerating demand for AI infrastructure: (1) Pretraining scaling (more data, larger models) (2) Post-training scaling (fine-tuning, reinforcement learning) (3) Inference-time scaling (longer reasoning chains, chain-of-thought AI, more synthetic data generation). 💰 Who's Buying? Cloud Service Providers (CSPs) still make up about 50% of NVIDIA's Data Center revenue, and their demand nearly doubled YoY but many enterprises are also investing in their own AI compute instead of relying solely on cloud providers 🍟 Custom Silicon and the ASIC vs. GPU Debate. Big Tech is building custom AI ASICs (Google has TPUs, Amazon has Trainium, Inferentia) to reduce dependency on NVIDIA but Jensen dismissed the notion that custom silicon would challenge NVIDIA’s dominance. GPUs remain more flexible across training, inference, and different AI models, while ASICs are often limited in their use cases. He flagged the CUDA ecosystem as a major competitive moat. 🛰️ The Next Frontier. Jensen repeatedly emphasized “agentic AI” and “physical AI” as the next major trends. The first AI boom was digital—models that generate text, images, and video. The next phase is AI that acts and interacts with the physical world. The market may worry about Nvidia's forward guidance but its hard to discount a company that controls everything from the chips to the networking (NVLink, InfiniBand), software (CUDA, TensorRT) and system-level AI solutions.