Prompt Engineering
84 records · 2 notes · records tagged with this as a related axis are included.
Performance you gain without changing the model, just by changing how you talk to it
Ask the same model the same question, and the answer can still change depending on how you ask. Adding a few examples, having the model think step by step, giving it a role — techniques that boost performance by tweaking only the input while leaving the weights alone belong on this axis. It's the area where academic papers and everyday practice mix fastest, so the standard can change within a few months. On the flip side, plenty of claims go around like fads and then can't be reproduced, so only what has been checked in papers goes in here.
The gist of this short clip is that people who write good prompts in Claude Code aren't more skilled — they have simply installed one skill called Prompt Master. The features it claims are that even minimal input gets tu
The case laid out is that making slides as a single HTML document instead of a PPT keeps fonts from breaking, keeps the file light and even lets you include elements that react only when the mouse hovers over them. Succe
A record of a lab changing its official recommendations. The role-prompting practice of 2023 has officially receded
Marks the point where controlling the reasoning budget moved down from a research technique to a problem for the serving layer
A synthesis that ties the ToT variants fragmented since 2023 together with classical search theory
Showed that faithfulness is not a single goal but several axes pulling against each other
The first follow-up since Anthropic's April 2025 disclosure of unfaithfulness to propose interventions at the training stage
A follow-up that rebuilt the cost structure of 2022's self-consistency
The first to isolate a failure mode of GEPA-style automatic prompt optimization
A follow-up that directly tested the premise of the Monitorability proposal to use CoT as a safety monitoring tool
Adds mechanism-level evidence to the CoT faithfulness debate
Formalizes budget-forcing (s1)-style control of reasoning budgets within a framework of statistical guarantees
The first large-scale measurement to treat skill marketplaces as an unvetted supply chain
A direct counterargument to the Oxford position paper. Cite the two as a pair
About 1% attack success rate against adaptive attacks. Stated explicitly that no browser agent is immune and that it does not claim the problem is solved
A counterexample from the attack side to optimism about CoT monitoring
An early case of merging reasoning and non-reasoning models into a single set of weights — a direction that later became the industry standard
A document showing that prompt engineering has moved into the design of tool definitions
Part of a line of rebuttals that explain CoT as the reproduction of in-distribution patterns
Argues that prompt evolution can compete with RL on sample efficiency
A consensus document on CoT monitoring, co-signed by major labs
A position paper that makes the practice of using CoT as an explanation its subject
An immediate rebuttal to the Apple paper above. Always cite the two as a pair
Set off a wide public debate on the limits of reasoning models
A case of model-specific prompting norms moving into vendor documentation
Checked, using its own model, that CoT unfaithfulness persists even in the era of reasoning models
A branch that re-examined CoT along the axis of accuracy versus token cost
A technique for controlling reasoning length itself at the prompt and decoding level
The point at which prompt-engineering techniques were absorbed into a product feature
Evidence that math reasoning may not be formal reasoning
A rebuttal that empirically narrowed the scope in which CoT applies
The theoretical basis of DSPy's optimizers
Extended the automatic-differentiation metaphor to text systems in general
Fixed the scattered lineages of techniques into a single taxonomy. The v1 title was Survey of Prompting Techniques
A rebuttal showing that CoT performance gains are not necessarily due to the content of the reasoning
An early record, in edited media, of the narrative that the prompt engineer job is in decline
Suggested that CoT may be a decoding problem rather than a prompting one
Demonstrated the limits of people polishing prompts by hand. Empirical grounds for the narrative that prompt engineering is in decline
Handed the very choice of prompting technique over to the model
A branch that inserts an abstraction step through the prompt
A framework that shifted from writing prompts to compiling programs
A direct rebuttal of the Self-Refine/Reflexion family
A branch that gets the few-shot effect without labeling examples
The evolutionary prompt-optimization lineage that leads to GEPA
Structured self-checking into 4 steps to reduce hallucinations
Formalized automatic prompt optimization as a general optimization problem
Generalized ToT's tree constraint to graphs
Company-reported scores: 58.7% on Codex (Instant 1.1: 52.8%) and 86.7% on GSM8K (80.9%); Anthropic said it hallucinates less and is more resistant to jailbreaks
A branch that applied reasoning prompting to latency optimization
A methodology that quantified faithfulness through intervention experiments
Company-reported scores: Codex HumanEval 71.2% (Claude 1.3: 56.0%), GSM8k 88.0% (85.2%), and 76.5% on the multiple-choice section of the Bar exam (73.0%); API priced the same as Claude 1.3
The starting point of the MCTS-based reasoning line of work
The reference paper for the multi-agent debate line of work
Redefined linear CoT as a search problem
At Google I/O, Google announced over 25 products and features powered by PaLM 2, including Bard, and said the smallest size, Gecko, can run on mobile devices even offline
The starting point of the line of work challenging CoT faithfulness
Uses a planning step to make up for zero-shot CoT's calculation and missing-step errors
Early work on the idea of textual gradients. Led to TextGrad
The standard reference for iterative refinement loops without external supervision
The lineage of verbal reinforcement learning without weight updates. The v1 title was an autonomous agent with dynamic memory and self-reflection
Better instruction understanding reduced the marginal value of prompting tricks; the axis of competition moved to models, safety and context
Released the same day as GPT-4, symbolizing that competition was moving beyond prompting tricks
Prompt engineering institutionalized as a job with a title and a salary band
Publicly demonstrated that in a live service the system prompt does not work as a line of defense
Google said it would start with a lightweight version of LaMDA to reduce compute needs and gather feedback, before wider public availability in the coming weeks
Governs behavior with written principles and AI feedback instead of human labels, elevating a principles document into a training signal
The conversational UI opened prompting to non-experts, turning a skill of a few practitioners into a mass phenomenon
Together with PAL, formed the code-as-reasoning lineage
The lineage of moving computation outside the language model
Automatically generated instructions matched or beat human-written ones on 19 of 24 tasks
An early attempt to automate the manual work of writing CoT exemplars
The standard form of the agent loop that combines reasoning techniques with tool use. Cross-referenced with axis 3
An early case of treating generated reasoning as something to check and to resolve conflicts in
CoT without examples. The original of every later zero-shot reasoning prompt
The prototype of the decomposition-and-planning lineage (Plan-and-Solve, Self-Discover)
A lineage of modular, decomposed reasoning rather than single CoT generation
The start of the test-time sampling-and-voting lineage. GSM8K +10%, MultiArith +24%
Showed that outputs of the 1.3B model were preferred to those of the 175B GPT-3, establishing RLHF as the standard procedure for alignment
The starting point of the entire lineage of changing reasoning performance through prompts alone. With 8 CoT exemplars, the 540B model reached SOTA on GSM8K
OpenAI itself wrote that it unlocked capabilities that had been hard to draw out through prompt engineering alone
The 137B model outperformed the 175B GPT-3 zero-shot on 20 of 25 tasks
Made the approach of programming by showing examples into an official product interface
Introduced in-context learning, which performs a task from examples in the prompt alone, without updating parameters
Showed that loss can be predicted as a power law of model size, data and compute, becoming the rationale for large-scale investment
Suggested that natural-language instructions can serve as task specifications, and opened the discourse on the risks of releasing models
Self-attention alone, without recurrence, made parallel training possible, establishing the architectural premise for large-scale pretraining