2024, in 196 pieces

Turned a qualitative warning into a number. Since then, the most widely cited reference value in the p(doom) debate

Established low-precision pretraining at frontier scale

The world's second comprehensive AI law, after the EU's

A structure in which chip suppliers came in directly as investors

Created the practice of reporting capability and cost together

Trains models to reason explicitly over safety specifications in CoT; the basic recipe for aligning reasoning models

A branch that skips RAG altogether in the long-context era

Google's first reasoning model, taking the route of showing its thinking process to users

Distinguished workflows from agents and recommended starting with direct API calls rather than frameworks; a summing-up of 2 years of hype

Observed, even without explicit training for it, the phenomenon of complying only while in training and concealing preferences

The full text confirms that the term was used in the evaluation community first, before it went mainstream

Scaling skepticism came not from a critic but from one of the founders of the scaling paradigm. Peak data became an industry-standard term

Processes bytes directly, without a fixed vocabulary, using entropy-based dynamic patches; the first large-scale scaling of the tokenizer-free approach

Google claimed 2.0 Flash outperforms 1.5 Pro on key benchmarks at twice the speed, launched the Multimodal Live API and Deep Research, and presented agent prototypes such as Project Astra, Project Mariner and Jules

A follow-up four months after the EXAONE 3.0 7.8B release that expanded to three sizes while keeping a non-commercial license

Set new training-speed records over AdamW using Newton-Schulz orthogonalization

An update to the Llama 3 line that shipped a single 70B instruction-tuned model, with no separate pretrained release

Memory became an axis of control for the first time

The commercialization of inference-scaling: users pay for the model to think longer

Claims at the level of a preliminary report turned into a reproducible paper and code

With an MIT license and extreme cost efficiency, the Chinese camp took the most open position

The first open reasoning model after o1

The original GraphRAG authors acknowledged the cost problem themselves and turned toward deferred graph construction

Replaced the combinatorial explosion of N models times M data sources with a single open standard

The turning point from preference learning to verifiable rewards

Named the approach of replacing the reward model with a verifier function, and released every stage of the pipeline

The first public statement of skepticism about pretraining scaling

The turning point at which the scaling-limits debate moved from the critics' camp to reporting from inside the industry

Ignited the scaling-wall debate

A study that tested, at the level of scaling laws, whether FP8/FP4 works. Redefined precision not as a free optimization but as a trade against capacity

A holdout design that cannot be contaminated

A reference implementation of the orchestrator-worker pattern

A strategy of lowering switching costs by adopting compatibility with competitors' APIs

The design archetype for commercial VLAs

Formalized a model's capacity to undermine human oversight, evaluation and decision-making as a distinct risk category

Established the grounds for ruling that most of Llama, Gemma and Qwen are not open source

Shifted the axis from pretraining scaling to inference-time scaling. Later became the line cited as grounds in the reasoning-model discourse

The first case of provenance marking for generated content moving beyond research into large-scale deployment

The first frontier model to operate GUIs through screenshots and coordinates, without dedicated APIs

Overhauled the capability-threshold framework

The first confirmed case of Japanese state funding actually producing a domestic 100B-class model

The point at which prompt-engineering techniques were absorbed into a product feature

The upper bound for the idea of handing defect judgment directly to general-purpose multimodal LLMs. 74.9% is not a level usable for final-inspection decisions

An approach in which the state directly subsidizes GPU usage fees to grow many domestic foundation models in parallel. An industrial policy that spreads its bets rather than picking winners

The pillar that moved Japan's national AI compute up a generation. A national compute model shared by private startups and universities

The point at which a PyTorch-native training framework became established as an alternative to Megatron-LM. FSDP2, torch.compile and FP8 were bundled into a single stack

The open-source camp's leading lightweight alternative to the costly indexing of Microsoft GraphRAG

Evidence that math reasoning may not be formal reasoning

Gave data center operators in third countries a pre-approval pathway

Community complaints that the official implementation was heavyweight surfaced as a hackable, ultra-light reimplementation

Voice becomes a developer primitive (speech-to-speech over WebSocket)

The last of the three companies to productize prefix caching

Established "powerful AI" in place of AGI, and "a country of geniuses in a datacenter", as common vocabulary. The first serious optimistic document to come out of the safety-discourse camp

The first setback for legislation regulating frontier models

The 11B and 90B were the first Llama models to support image input, released alongside 1B and 3B models aimed at on-device use

The phrase "a few thousand days" became the single most-quoted line in the 2024–25 timeline debate. Its vague unit itself became a point of contention

All open models except the 3B and 72B were released under Apache 2.0, and the 14B and 32B sizes missing from the previous generation returned

Prepending 50–100 tokens of LLM-generated context to each chunk cut the retrieval failure rate by 35%; by 49% when combined with BM25, and by 67% with reranking added

The phase in which competition among commercial embedding APIs explicitly took aim at the OpenAI baseline

A rebuttal that empirically narrowed the scope in which CoT applies

The lineage that replaced pipeline-based voice dialogue with a single model

The turning point at which sovereign AI in Korea moved from a corporate marketing term to a national policy term. This person became head of the government's AI policy 9 months later

The first commercial reasoning model; on AIME 2024, from 12% for GPT-4o to 74% for o1

A preview released as weights plus free API access ahead of the full release planned for November; Upstage claimed it competes with multi-GPU models such as Llama 3.1 70B

A case of Chinese Big Tech industrializing the GraphRAG lineage for professional domains (finance, healthcare)

Solved the problem of chunking cutting off context by changing the order of embedding

The fastest build-out in the industry, by the company's own announcement. But the initial grid connection was only 8 MW, so it relied on temporary gas turbines, and was stepped up to 150 MW only in November

Used wafer-scale SRAM to get around the bandwidth limits of GPU HBM, popularizing the category of inference-only chips

The starting point of the approach of training over internet bandwidth, without data center interconnects

Estimated 400 trillion to 20 quadrillion effective tokens by 2030, and laid out the order of constraints: power binds first, then chip production capacity

Brought a sprawl of variants under one taxonomy, formalizing graph-based RAG as a research field of its own

The first case of the community actually fine-tuning the 405B model

Cache economics became visible on the price list and went on to become a key variable in agent economics

The reference dataset for evaluating agentic coding. It revealed that a large share of the original benchmark was unfit

Turned the research-automation discourse into a concrete figure: $15 per paper

Image generation started not with an in-house model but through integration with an external model (Black Forest Labs FLUX.1)

The first case over image-generation model training to move into substantive proceedings

The first open release of an in-house LLM by a major Korean company. But its research-only license also revealed the conservatism of the Korean approach to openness

Demonstrated that test-time compute can be more efficient than scaling up parameters

Productized the prefix cache; an internal serving optimization surfaced in the API price list

Entry into force is not application. Application is phased

The first primary source to publish quantitative data on fault tolerance in large-scale training. Settled, for the whole industry, that checkpointing and automatic recovery are essential

Gave a name to the gap between benchmark scores and reliability in real use. Salesforce adopted it as a formal research agenda and built the SIMPLE benchmark, and Sundar Pichai riffed on it as AJI

Self-training grounded in a formal language reached olympiad level. DeepMind stated explicitly that it would not have been possible without Lean and Mathlib

Published the weights of a 123B flagship designed for single-node inference, limited to research and non-commercial use, with a commercial license required for self-deployment

The peak of the model-collapse discourse

Frontier-class weights were released for the first time, and the license allowed training on the model's outputs

Framed the long-context vs RAG debate as a cost-performance trade-off

OpenAI said it is more than 60% cheaper than GPT-3.5 Turbo; it replaced GPT-3.5 for ChatGPT Free, Plus and Team users

A case of a frontier-lab researcher declaring education an independent business

The starting point for counting the entry-into-force date

A case showing that kernels must be rewritten for a specific generation of silicon. Quantitatively rebutted accuracy concerns about FP8 attention

The point at which a research technique spread as a widely replicable implementation

Formalized a Chinese lab's open-weight strategy in terms of ecosystem and originality rather than cost savings. Because the original is in Chinese, only the gist is recorded, without quotation marks

Created a structure in which evaluation infrastructure is concentrated in a few open-source implementations

Dicing is the first step of back-end processing. A 23-percentage-point gain from synthetic augmentation alone on a small set of defect images speaks directly to the early data-scarce phase of new defect classes

The second Gemma generation; Google claimed the 27B competes with models more than twice its size and the 9B outperforms Llama 3 8B

Ultra-long context left the waitlist and became an option for ordinary developers

A lineage of visual document retrieval that skips the OCR and parsing pipeline

A subset scored for educational value by an LLM classifier produced better models than the original, standardizing model-based filtering

Tiered KV cache management grounded in a production service

Treats RAG and long context not as rivals but as a redistribution of roles

Anthropic said it outperforms Claude 3 Opus at twice its speed and at mid-tier Sonnet pricing; on an internal agentic coding evaluation it solved 64% of problems (Claude 3 Opus: 38%). The UK AISI ran a pre-deployment saf

The organizational form itself — heading straight for superintelligence without shipping products — is an argument. Sparked a debate over whether safety can be separated from commercial pressure

An open-weight code model that DeepSeek claimed matched GPT-4 Turbo on code-specific tasks

The data-generation pipeline itself became the product

Measures policy compliance over multiple turns rather than one-off tool calls

The theoretical basis of DSPy's optimizers

An early case of a major Japanese company publicly releasing a pretrained model in its own language, and a sign of telecom capital entering homegrown LLMs

Lowered the barrier to entry for academia and small and medium-sized companies

Publicly demonstrated that safety alignment can be undone without retraining; technical grounds for the open-weight safety debate

Distillation became so cheap that it no longer even needs API calls

Showed that the conflict between the two laws stems from counting non-embedding parameters and from analysis at small scale

Extended the automatic-differentiation metaphor to text systems in general

The first large-scale case of OS-level on-device LLM integration

Only the 72B kept the original Qianwen License while the other four moved to Apache 2.0, the release where Qwen's licensing began to open up

The quantitative reference point for the data-exhaustion discourse

Fixed the scattered lineages of techniques into a single taxonomy. The v1 title was Survey of Prompting Techniques

Pushed the AGI-timeline debate onto the policy and security agenda. Introducing the frame of AGI nationalization mattered more than the predictions themselves, and it was also the most intensely rebutted document

Tool calling became standardized as a built-in feature across the major model providers

Introduced a non-production license that publishes weights for research and testing only, with commercial use requiring a separate license

A head-on rebuttal of the FP8 training narrative. Documented that reports that it works coexist with reports that it cannot be reproduced

The valuation was not disclosed in the announcement

The multi-hop retrieval branch of the GraphRAG lineage

The last simplification in the direct alignment family

The origin of the practice of each lab publishing its safety framework

An international AI governance milestone co-hosted by Korea

The day the AI Act's legislative procedure ended. Everything after this is a question of entry-into-force and application dates, not of whether it becomes law

First applied SAEs to a production-grade model and demonstrated feature steering; interpretability moved from observation to intervention

The beginning of open-source VLAs

Widely cited as the first comprehensive state AI law in the US, but its obligations were never enforced: SB 25B-004 (signed Aug 28, 2025) pushed the date to June 30, 2026, and SB 26-189 (signed May 14, 2026) repealed and

Together with RSP and PF, established a system of three major risk-evaluation frameworks

Voice responses in as little as 232 ms and 320 ms on average, structurally eliminating the latency and information loss of cascaded pipelines

A sign that data warehouse vendors were starting to build embedding models themselves

Extended beyond proteins to complexes with nucleic acids, small molecules and ions

Fixed the norms for model behavior in a public document, which later became an industry-standard format

A third path, distinct from MQA and GQA, that compresses KV into a low-dimensional latent

Introduced the concept of designing interfaces specifically for agents, and reached 12.5% pass@1 on SWE-bench as open source

Measured benchmark contamination with a controlled experiment. However, reported that frontier models show few signs of overfitting

With a flow running from issue to plan, code and validation, the product shifted from autocomplete to task-level agents

Handles global summarization queries that local retrieval cannot answer

A rebuttal showing that CoT performance gains are not necessarily due to the content of the reasoning

Justified the approach of small models deployable on phones through data curation

Raised the upper limit on the scale of open web corpora

Chose a single soft-law document, in contrast to the hard law of the EU AI Act. Later became the skeleton of the AI Promotion Act

The 8B delivered the previous generation's 70B-class performance, raising the bar for practical local use

Mistral's larger MoE follow-up to Mixtral 8x7B, again under Apache 2.0

A case of building its own benchmark and releasing it alongside. The announcement said only that it would soon be available to early testers, so whether it was actually released to the general public has not been checked

Put the reproducibility and checking of agent demos on the industry agenda

The point at which rerankers were commercialized as a separate foundation-model product line. The two-stage structure settled in as the default architecture for RAG

The de facto standard benchmark for computer-use agents

An early case of a graph DB vendor adopting GraphRAG as a product category — the point at which an academic term entered the commercial stack

The first large-scale tool ecosystem, folded after just one year; a sign that schema-based function calling outlasts a loose marketplace

Exposed, in a standardized way, the gap between advertised and effective context length

Established the field's central issue: the gap between generated candidates and confirmed materials

Proved that replacing real data leads to collapse but accumulating it keeps a finite upper bound on error, redefining the model-collapse discourse

Expanded the context 16× over the previous version

The starting point of a Japanese telecom approach that, instead of following the parameter-count race, set its course toward lightweight models specialized in the national language

Showed a path to non-English capability by combining existing open weights without pretraining — the prototype strategy for countries at a compute disadvantage

A paradigm that treats a whole rack as one giant GPU

The largest Apache 2.0 open-weight MoE at the time. But as a raw checkpoint without fine-tuning for dialogue, its practical usability was limited

The government drew, in writing, the actual limits of the provision that is cited internationally as making AI training free in Japan. Fine-tuning aimed at imitating an artistic style is not exempt

Not yet in force

A further simplification of the alignment pipeline

Announced 13.86% on SWE-bench, saying it beat the previous SOTA of 1.96% by more than 7×

An early case of the compute-as-public-good model, in which the state procures and allocates GPUs directly. Later a reference point for Korean and Japanese plans

An early record, in edited media, of the narrative that the prompt engineer job is in decline

Opus and Sonnet available on launch day in claude.ai and the Claude API, generally available in 159 countries, with Haiku to follow. Per 1M tokens: Opus $15/$75, Sonnet $3/$15, Haiku $0.25/$1.25; assessed at ASL-2 under

Solved the problem of long prompts stalling decode; 2.6–5.6× the serving capacity of vLLM

An API flagship launched with Microsoft Azure as Mistral's first distribution partner; Mistral claimed it ranked second only to GPT-4 among models available through an API

The starting point of the world-model lineage. The authors argue that at this scale it can be regarded as a foundation world model

Representative of the line of work that simplifies RL

The first systematic statement by a Chinese lab founder declaring long context a core pillar of the path to AGI. Supplied the language for the 2024 reshaping of China's LLM race into a long-context race

Big Tech entered the small open-weight field, but drew criticism that the license was not Apache

Sparked the discourse that video generation models are world simulators

Context length jumped by an order of magnitude, sparking the RAG-is-dead debate

Suggested that CoT may be a decoding problem rather than a prompting one

A lineage of structured indexes beyond vector similarity search

The first empirical evidence that debate-based oversight actually raises the accuracy of non-expert judges

Demonstrated the limits of people polishing prompts by hand. Empirical grounds for the narrative that prompt engineering is in decline

Handed the very choice of prompting technique over to the model

Group relative policy optimization without a value function later became the de facto standard for RL in reasoning models

The first competitive embedding model to release even its training data. Added reproducibility as a new dimension of the embedding race

Concluded that no loss function is universally superior and that the inductive bias that suits the setting is best

Released the data together with the tools, making data science reproducible

Proposed a hierarchical summary index as an alternative to chunk-level retrieval

A defensive RAG design that assumes retrieval will fail

MRL-style truncation became a built-in feature of a commercial API

Secured capital for the approach of a Rust-based, high-performance, single-purpose engine

Splits prefill and decode into separate GPU pools so that latency metrics can be tuned independently

Brought a cache layer for workloads with overlapping prompts, such as multi-turn and agents, down to the runtime level

An early case of formally verifiable rewards driving reasoning training. Produces human-readable proofs

Shifted the cost structure of vector DBs from always-on to usage-based

Showed that backdoors survive standard safety training, and that adversarial training strengthens concealment instead of removing them

When inspection AI is tied into a safety interlock, defining which type it falls under is the starting point for audit readiness. Note, however, that it is a Technical Report, not a normative standard that can serve as a

What the world's No. 1 OSAT discloses is effectively the industry ceiling. The order matters: FDC, RMS and traceability were laid down first as the prerequisite infrastructure for AI — the data pipeline comes before the