RAG, Fine-Tuning & Context
86 records · 2 notes · records tagged with this as a related axis are included.
Ways to give a model what it doesn't know, right when it needs it
Once training is over, a model knows nothing about what happened afterward, or about the documents inside a company. This axis looks at three ways of closing that gap together: RAG, which finds the documents you need and attaches them to the input; fine-tuning, which adjusts the weights a little further; and widening the context itself, meaning how much you can put in at once. The three look like rivals, but real systems usually mix them, and how much to lean on each one is itself the design decision.
The point at which deep-research-style capabilities came down to an 8B open-weight model. Using citation accuracy and recall as evaluation axes matters, because in report generation the problem is not sentence quality bu
A case of the competitive focus of embedding models moving from scores to operating costs. The design in which tiers share an embedding space aims to eliminate reindexing costs — the biggest barrier to switching models i
An approach that retrieves and adapts skills from repositories instead of learning them anew for each harness, turning skills from outputs tied to a model and harness into transferable assets. Read together with ClawScan
The second vertical-industry product after ChatGPT for Financial Services (Sep 10). The pattern of selling domain search and governance layered on a general-purpose model is repeating not quarter by quarter but week by w
One more design that keeps agent memory not in a vector store but in files people can read and edit — the direction in which the industry is converging on where the canonical copy of memory should live.
This describes a real deployment that, instead of feeding hundreds of pages of design documents and constantly amended laws into one model wholesale, has a lead role set the review plan and order and then divides searchi
A case of a frontier model going beyond a general-purpose assistant to be vertically integrated into the work stack of a regulated industry; data connectors and audit logs have become product requirements as important as
A trend in which the form of contract shifts from "selling a model" to "training a dedicated model on the customer's data", with air-gapped deployment rising to a commercial requirement.
The point at which data warehouse queries became a basic agent tool without going through analysts, showing the BI tool layer being absorbed into agents.
An attempt to handle contexts in the 1 million-token class on a single node; solutions to the memory bottleneck of long-context agents are converging on a combination of sparse attention and hierarchical memory.
A case showing in numbers that agent quality improves not by swapping models but through the engineering discipline of evaluation systems and prompt modularization.
A strand that evolves memory instead of putting an entire long video into the context, breaking down RAG's separation of retrieval and generation for time-series input. In the authors' own reporting, it claims the best p
A stage at which general-purpose chatbots connect directly to regulated medical record systems. The front line of context connection (RAG, connectors) has moved from the public web into regulated data, while the safety e
A strand that, instead of the ColPali lineage of reusing generative models as retrievers, returns to a single tower designed as an encoder from the start — a claim that "encoder-only" is valid again for multimodal retrie
A strand that reorganizes agent long-term memory, which had relied on a single graph structure, into a triple structure of timeline, graph and documents, with provenance tracking and citation locks. In the authors' own r
Claude Code pricing is shown in tokens, but the real cost is GPU inference time, and the per-token price is set by model size, direction (input or output) and whether the cache is used. Input (prefill) is processed in pa
Opens a parametric knowledge-graph strand that, unlike the GraphRAG lineage, which queries the graph at retrieval time, compiles the graph into model weights in advance — while at the same time pinning down that this str
A strand that redefines the knowledge graph of agentic systems from a "flat store of extracted triples" into an object of governance that records who owns a fact, why it was adopted and how it should be used. In the auth
Organizes previously scattered research on context compression into a single theoretical framework
An attempt to absorb chunking, a RAG preprocessing step, into the embedding model itself
Quantifies Context Rot in the setting of search agents, and also shows that a fundamental fix is hard
Brings into the mainstream the self-criticism that evaluations centered on retrieval metrics have overstated the benefits of graph techniques
Empirical evidence that long-context limits also break down safety monitoring performance
A successor in the Search-R1 line. Made the number of searches itself something to learn
Showed that agentic search narrows the gap between the two approaches, updating the 2024 GraphRAG debate
The first systematic taxonomy of agentic RAG. A reference point for follow-up research
Became the default model for Free and Pro plans in claude.ai and Claude Cowork; Anthropic reported that early-access developers strongly preferred it to its predecessor and often to Opus 4.5 from November 2025
Empirical evidence that agentic RAG is not always better. A disconfirming result that grounds design choices
Part of a shift in who manages context: from the pipeline to the agent itself
The first measurement comparing the 3 providers under identical conditions since caching became a product
The primary source for the line of definitions that treats the harness as an operating system
A case in which what to remember and what to discard is learned rather than decided by people
The route of working around long-context limits through architecture rather than retrieval
A structural fix for tool definitions eating up the context
97% OCR precision at compression below 10×, about 60% at 20×. A shift in thinking: images as a medium for compressing context
A direction that shrinks context without fine-tuning, so that even small models can be used as long-horizon agents
Names four patterns: compaction, structured note-taking, sub-agents and just-in-time retrieval
Context management becomes a built-in API feature
A primary record disclosing the context-engineering principles of an agent running in real production
The report that made context rot an industry term
The turning point into a phase of testing with benchmarks when graphs actually help — the methodological forerunner of the 2026 skepticism
A commercial agent long-term memory product adopted graph representations as a core extension — graph engineering spreading beyond RAG into the memory layer
Search strategy acquired through training rather than prompting
Showed that needle-in-a-haystack scores overstate long-context ability
A leading example of putting temporal knowledge graphs forward as a first-class design element of agent memory
A sign that agentic retrieval had settled in as a category of its own
Integrated agentic retrieval into the reasoning model's inner loop
A branch that skips RAG altogether in the long-context era
The original GraphRAG authors acknowledged the cost problem themselves and turned toward deferred graph construction
The open-source camp's leading lightweight alternative to the costly indexing of Microsoft GraphRAG
Community complaints that the official implementation was heavyweight surfaced as a hackable, ultra-light reimplementation
The last of the three companies to productize prefix caching
Prepending 50–100 tokens of LLM-generated context to each chunk cut the retrieval failure rate by 35%; by 49% when combined with BM25, and by 67% with reranking added
A case of Chinese Big Tech industrializing the GraphRAG lineage for professional domains (finance, healthcare)
Solved the problem of chunking cutting off context by changing the order of embedding
Brought a sprawl of variants under one taxonomy, formalizing graph-based RAG as a research field of its own
Cache economics became visible on the price list and went on to become a key variable in agent economics
Productized the prefix cache; an internal serving optimization surfaced in the API price list
Framed the long-context vs RAG debate as a cost-performance trade-off
The point at which a research technique spread as a widely replicable implementation
Ultra-long context left the waitlist and became an option for ordinary developers
A lineage of visual document retrieval that skips the OCR and parsing pipeline
Treats RAG and long context not as rivals but as a redistribution of roles
The multi-hop retrieval branch of the GraphRAG lineage
Handles global summarization queries that local retrieval cannot answer
An early case of a graph DB vendor adopting GraphRAG as a product category — the point at which an academic term entered the commercial stack
Exposed, in a standardized way, the gap between advertised and effective context length
Context length jumped by an order of magnitude, sparking the RAG-is-dead debate
A lineage of structured indexes beyond vector similarity search
Proposed a hierarchical summary index as an alternative to chunk-level retrieval
A defensive RAG design that assumes retrieval will fail
A community test settled in as the de facto standard evaluation, establishing the recognition that the stated context length and actual recall performance differ
The context-length race entered the 200,000-token range
The turning point at which homemade RAG pipelines began to be absorbed into built-in platform features
The turning point that moved retrieval from a fixed pipeline to a decision made by the model
The prototype of the out-of-context memory management line (later the memory tool and others)
The first large-scale counterevidence to the assumption that you can simply put in a long context
Fine-tunes a 65B model on a single 48 GB GPU using 4-bit NF4, double quantization and paged optimizers
The starting gun for the long-context race; made feeding in whole documents without RAG a real option
With three large rounds in 3 weeks, the narrative of long-term memory for LLMs peaked in the capital markets
The first case of a lab putting retrieval augmentation on its own product surface; a direct demand signal for the vector DB ecosystem
Wired up LoRA and Prefix/Prompt Tuning in a few lines, laying the groundwork for fine-tuning on consumer GPUs to go mainstream
Arriving the month after LangChain, it shaped another standard vocabulary for the RAG indexing layer
Spread the standard RAG vocabulary of loader, chunking, embedding, vector store and retriever
The follow-up that brought late interaction down to practically sized indexes
Freezes the weights and trains only low-rank matrices, cutting trainable parameters to 1/10,000 for GPT-3 175B, with no increase in inference latency
The canonical text of retrieval-augmented generation, combining parametric and non-parametric memory
The starting point of the late interaction lineage, distinct from single-vector retrieval