As a full-stack independent developer, I recently encountered a limitation when using AI IDEs powered by LLMs: the context window is often capped at 64K or 128K tokens. While this may seem large, it falls short for use cases involving complex or large-scale projects. As a result, LLMs often "forget" important information, and we’re forced to write overly verbose prompts, which consume a significant number of tokens and hinder productivity.
After looking into the transformer-based architecture that underlies most LLMs, I realized that the models are trained once and then essentially "frozen"—they don’t update themselves post-deployment. To address this, many current systems use RAG (Retrieval-Augmented Generation) to simulate long-term memory by segmenting and indexing knowledge externally. However, this is still fundamentally prompt-based, and doesn't constitute true memory in the sense of persistent internal learning.
At one point, I wondered whether it would be possible to continuously feed data into the model and retrain it incrementally, similar to how human memory works. But from what I’ve learned, this is nearly infeasible with current architectures. Retraining or fine-tuning models, even incrementally, is extremely costly—requiring reprocessing of the data into vector representations and adjustment of learned weights, which is not practical for most real-world applications.
So, the challenge isn’t just about cost—though that’s a big part—it’s also about how current architectures are fundamentally not designed for continuous learning or dynamic memory.
Recently, I came across an open-source project called memOS, which claims to support long-term contextual memory. At first glance, it seemed promising. However, I'm still unsure how it's implemented under the hood and whether it's genuinely different from RAG-based systems. I’d be curious to hear if others have used it and whether it truly offers long-term memory capabilities beyond traditional retrieval.
It is also clearly stated here that the API-based LLM call does not support the parameter memory function.
Reference Links：
https://memos.openmem.net/
https://charitydoing.com/building-memory-aware-ai-an-introduction-to-memos-in-memos/
Top comments (0)