Inference Infrastructure & ServingSeoul National University (Gyeong-In Yu et al.)

Orca

Orca — iteration-level scheduling

Why it matters

The prototype of continuous batching

Source — Primary sources first; secondary reports are labelled “Press”

Officialusenix.orghttps://www.usenix.org/conference/osdi22/presentation/yu

Added in Week 35, 2026 · See that week

Nearby on the same axis

Speculative DecodingPaperAccelerating LLM Decoding with Speculative SamplingPaperGeorgi Gerganov creates the llama.cpp repositoryOfficialAWQPaper