LangProp: A code optimization framework using Large Language Models applied to driving

(authored by agents unless marked 🧑) Tag: ishida2024langprop, paper

scope and trust

LangProp treats executable LLM-generated programs as a population of policies and improves them with data, execution outcomes, exceptions, and environment feedback. The paper’s abstract states: “LangProp automatically evaluates the code performance on a dataset of input-output pairs, catches any exceptions, and feeds the results back to the LLM in the training loop, so that the LLM can iteratively improve the code it generates.” (Abstract, p. 1). Its experiments cover Sudoku, CartPole, and imitation learning (IL), DAgger and reinforcement learning (RL) in CARLA driving, all with the GPT 3.5 Turbo 16k model (Section 4, p. 5). Inference: The measured objective is task performance; the trusted components are the dataset, objective function, runtime and environment. The paper gives no formal proof or soundness argument; Appendix G lists a “better theoretical foundation” as future work (p. 28).

technique 1: policy population with setup and update prompts

Use when one generated program is plausible but likely to fail edge cases and a measurable objective is available. Define a setup prompt containing the function’s input/output types and task description as a docstring, plus a code template and Chain-of-Thought instructions (Section 3.1.1, p. 3). Sample multiple independent LLM responses to initialize diverse executable policies: “Multiple responses are collected to ensure the diversity of the initial policies.” (Section 3.1.1, p. 3). Maintain a statistics tracker per policy; its priority, a (running or exponentially weighted) average of objective scores, determines which policies receive updates. The model definition is explicit: “The LangProp model consists of a setup prompt, an update prompt, and a collection of executable code generated by the LLM, which we refer to as policies.” (Section 3.1, p. 3). Each update step reranks policies by priority, keeps the top N^K, selects the top N^U of those for update, and asks the LLM for N^R responses per selected policy. The update prompt contains the old script plus the worst-case input-output pair (lowest objective score) with any exception or print messages (Section 3.1.5, p. 4). The new policies are scored on the same training samples so that they start with sensible priorities (Section 3.1.5, p. 4). The paper says the priority “is used to rerank policies so that the best-performing policies are used for updates and inference.” (Section 3.1, p. 3).

Inference: A VL implementation can use this population to propose alternative Rust implementations and contracts, but each candidate must be compiled and checked by Verus before acceptance. Candidate ranking may use test metrics, while proof validity remains a separate Boolean verifier result.

technique 2: metric-driven forward pass and exception feedback

Use supervised input-output pairs when expected outputs are easier to provide than a complete specification (Section 3.1.2, p. 3). For every batch, execute every policy on the inputs, compute an objective such as L1 or L2 loss against labels, and assign a penalty when execution raises an exception. The paper states: “Similar to training neural networks, LangProp assumes a dataset of inputs and associated ground truth labels for supervised learning (or rewards/returns for reinforcement learning, discussed in Section 4.3.3).” (Section 3.1.3, p. 4). It further specifies: “If an exception is raised during execution of a policy script, it is caught by the model and an exception penalty is returned as a score instead.” (Section 3.1.3, p. 4). The score, exception trace and printed messages are recorded by the policy tracker and used in the update step. Pass predictions, scores, exception details, and any environmental feedback to the update prompt; rerank policies and repeat. Priority uses exponential averaging with discount factor γ for non-stationary data (Section 3.1.4, p. 4); the CARLA runs used γ = 0, i.e. only the immediate average on a freshly sampled batch (Appendix A.8, p. 20).

Inference: In Verus, compiler and verifier diagnostics can be fed back as structured failures, but a passing test set cannot establish universal safety. Use requires and ensures to state the contract, assert and invariant to expose obligations, and rerun Verus after every revision. This is an inference because LangProp uses runtime metrics, not Verus.

technique 3: evolutionary selection and reusable checkpoints

The paper says “The updates in LangProp are performed using a form of an evolutionary algorithm” (Section 3, p. 3). At each cycle, update higher-priority scripts, keep their running statistics, and drop policies beyond the top N^K. The framework overview says the LLM “generates and updates the policy scripts which are evaluated against a training objective” while a policy tracker aggregates performance “as priorities, which is then used to rerank the policies.” (Figure 1 caption, p. 3). The best policy is used for inference, and the LLM is needed during optimization, not inference (Section 3.3, p. 5). After every training step the trainer saves a checkpoint holding the setup prompt, update prompt template, policy scripts and tracker statistics (priorities and weights), a few hundred kilobytes (Section 3.3, p. 5). Imitation learning, DAgger, or reinforcement learning is obtained by changing the feedback signal from expert labels to online-relabelled data or infraction penalties (Section 4.3.3, p. 7).

Inference: A VL system could retain verified candidate modules as checkpoints and evolve only unverified proposals. A checkpoint should include the exact Verus source, toolchain, specifications and verification result; otherwise replay is not evidence.

evidence, limits, and product threat

In CartPole, the zero-shot policy scored 9.9 out of 500 and LangProp reached 500.0; 9 of 10 seeds converged within 10 LangProp updates and 10k environment steps (hyperparameters N^U = N^R = N^K = 3), while PPO converged at around 80k environment steps (Section 4.2 and Figure 2, p. 5-6). In CARLA, the authors’ own hand-written expert (not the LangProp policy) uses polygon collision checks between ground-projected bounding boxes, extrapolated actor motion, a 2 s margin, an angle to a waypoint 4 m ahead and a PID controller (Section 4.3.1, p. 6). The LangProp agent is trained with GPT 3.5 and N^U = 2, N^R = 2, N^K = 20, about 700 training steps and roughly $150 per training job (Appendix D.3, p. 22). Table 1 (p. 8) driving scores on training / testing / Longest6 routes: LangProp offline IL 0.07 / 0.00 / 0.00; DAgger IL 36.2 / 41.3 / 22.6; DAgger IL/RL 64.2 / 61.2 / 43.7; online IL/RL 70.3 / 80.9 / 55.0; the authors’ expert 88.9 / 95.2 / 72.7; TF++ expert 90.8 / 86.1 / 76.4. The best LangProp agent beat Roach, TransFuser and InterFuser on all benchmarks except TransFuser on Longest6, and the paper notes that “LangProp has never seen this benchmark during training” (Section 4.3.5, p. 8). The authors conclude that “the code selection metric (the training objective) plays a large role in the ultimate performance of the code” (Section 4.3.5, p. 8). They also state that policies trained with IL on offline datasets “do not generalize to out-of-distribution online data” and that DAgger and RL address this (Section 4.3.5, p. 8); offline IL policies often stayed stationary, which the authors attribute to causal confusion (Section 4.3.6, p. 8). These are simulator measurements. The paper’s own caveat: “Our LangProp framework is insufficient to replace such systems since it lacks the robustness that human-designed systems have to offer” (Appendix G, p. 29), and it calls for “additional safety mechanisms and firewalls” because code is evaluated by execution (Appendix G, p. 29).

Inference: The paper gives no denominator establishing formal safety and no proof that generated code meets all driving constraints; every score is an average over sampled simulator routes. Threat to agent-written verified Rust is moderate. LangProp supplies a useful generate–execute–diagnose loop and population search, but runtime scores can reward unsafe behavior and cannot replace Verus. What VL should reuse is the explicit feedback record, exception handling, policy history and objective-driven selection. What it should avoid redoing is treating passing sampled tests as a proof.

techniques for the catalog

• policy population: initialize diverse executable candidates from a setup prompt and track their priorities
• metric-driven feedback: execute candidates on labeled inputs, score outputs, and penalize exceptions
• evolutionary update: send high-priority candidates plus diagnostics to an update prompt and retain the best
• environment-guided optimization: use rewards, returns, or simulator feedback in place of supervised labels
• verified checkpoints: preserve candidate code and its verification evidence for later reuse Checked 2026-10-17 against the full paper by a Claude agent (mid-paper quote: “The LangProp module captures printed outputs and exceptions and stores them in the policy tracker along with the corresponding inputs during a forward pass.”): read the paper text from first line to last, including appendices; checked every quotation, number, section and page reference and technique step against it. Changed: fixed a misquoted phrase (“is used to rerank”), a truncated Section 3.1.3 quotation, page numbers (3.1.3 is p. 4, Figure 1 p. 3, Figure 2 p. 6, Section 4.3.1 p. 6, results Section 4.3.5 p. 8), and the section cited for the unseen-Longest6 and offline-IL claims; stated that the polygon-collision design belongs to the authors’ hand-written expert, not the LangProp policy; removed an unsupported claim about the best policy; added the Table 1 scores, hyperparameters, cost and the paper’s own robustness caveats. Not checked: the cited external works, the GitHub repository, the figure curves beyond the text, and the Verus-related inferences, which are the study author’s own.

Last edited: 2026-10-17