Trustworthy AI Software Engineers

(authored by agents unless marked 🧑) Tag: aleti2026trustworthy, paper (arXiv:2602.06310v2, printed on the paper’s first page; the paper’s own DOI is the placeholder 10.1145/nnnnnnn.nnnnnnn)

scope and claims

This five-page vision paper asks what makes an AI coding agent a trustworthy software engineer. It defines trustworthiness as a system property, distinct from a person’s willingness to trust. Its central proposal is selective evidence review: “we introduce the notion of evidence-centric inspection, arguing that developers should evaluate selective signals and justifications of trustworthiness rather than raw outputs.” (abstract, p. 1) Inference: for VeruLaw, a passing cargo verus result is one signal; reviewers also need traceability to requirements, explicit assumptions, process evidence, and runtime behavior.

technique: multidimensional trustworthiness model

What it is: assess configurations of dimensions rather than one score. The paper says, “no single dimension is both necessary and sufficient for trustworthiness.” (Section 3, p. 3) It identifies technical quality (correctness, reliability, performance, robustness, security, cost), transparency and accountability (transparency, explainability, traceability, auditability, reproducibility, accountability), epistemic humility (communicating uncertainty, acknowledging limitations, awareness of biases), ethical alignment (fairness, bias mitigation, privacy, regulatory compliance, safety, sustainability), and socio-technical practice (collaboration, alignment with team culture, developer experience, societal impact). The paper presents these in Figure 1 and the text of Section 3 (pp. 2-3); it gives no scoring method. What the paper specifies: dimensions apply at different levels (system, model, and output), matter to different stakeholders (developers, organisations, and society), and operate over short- and long-term horizons (New Idea #2, Section 3, p. 3). It says trustworthiness “emerges from configurations of dimensions that collectively justify reliance in a given setting” and that it should be assessed in a continuous, context-sensitive way (Section 5, Conclusion, pp. 4-5). Inference: a usable procedure is not given by the paper; a reasonable one is to identify stakeholders and context, write requirements and constraints, select evidence for each relevant dimension (including traceability from output to input, model, and process, and recorded uncertainty and limitations), and reassess after changes. Verus form: Inference: technical correctness should be a Verus contract and proof, but reliability, cost, security, and requirement alignment need separate evidence. Use requires for environmental assumptions, ensures for functional promises, invariant for repeated behavior, and proof fn for supporting facts. Do not treat a proof as evidence for unmodeled external libraries or ethical constraints. Evidence and limits: this is a conceptual framework with no dataset or experiment; Section 6 states, “There are no data available yet.” Therefore no denominator, accuracy, or effectiveness estimate supports the dimensions. Inference: dimensions are a checklist for evidence selection, not validated metrics.

technique: evidence-centric inspection

What it is: replace exhaustive review of every generated artifact with curated visibility into decisions, assumptions, and risks. “In this view, developers do not need to see everything; rather, they need to see the right things.” (Section 4, p. 4) What the paper says to show (New Idea #3, Section 4, p. 4): “signals about whether the agent’s outputs align with requirements and constraints, whether key decisions are grounded in defensible reasoning, and whether uncertainty, assumptions, and potential failure modes are explicitly communicated.” New Idea #4 adds distilling agent activity into concise representations of key decisions, assumptions, and risks, traceability from artifacts back to requirements, inputs, and intermediate steps, and surfacing uncertainty, alternative solutions, or known limitations; it also mentions automated trustworthiness oracles [18] as a possible way to generate checks for robustness, fairness, or security (p. 4). The paper frames this as a “selective visibility problem” because agentic engineers operate through “multi-step processes involving planning, tool use, intermediate decisions, and iterative refinement” (Section 4, pp. 3-4). Inference: the terms “evidence package” and letting the reviewer expand from a signal to a trace are this study’s reading, not the paper’s wording. Verus form: Inference: a VL evidence package should include the requirement-to-contract link, exact requires assumptions, ensures and loop invariants, proof compile output, verifier/toolchain versions, changed dependencies, and any external_body or trusted boundary. A proof result is evidence of the stated contract, not evidence that the contract captures the user’s intent. Evidence and limits: the proposal has no user study or measured review reduction. Inference: a concise dashboard can reduce review fatigue only if its signals are faithful and complete enough to expose missing assumptions; the paper does not establish that.

technique: multi-level verification and validation

What it is: extend V&V beyond final code. The paper states, “This introduces additional layers of verification, including process-level verification, which concerns whether the agent’s planning and tool use are appropriate, and epistemic verification, which concerns whether the agent accurately represents its own uncertainty and limitations.” (Section 4, p. 4) How to do it: the paper says developers must reason about “the correctness of outputs” and “the soundness of the agent’s decision-making processes and its interpretation of human intent”; verification gains process-level and epistemic layers; validation “extends beyond outputs to include whether the agent’s behaviour remains aligned with evolving requirements and constraints”; V&V becomes “a multi-level activity spanning outputs, processes, and context”; traceability lets developers “reconstruct how a particular outcome was produced” (Section 4, p. 4). Traditional testing and static analysis “remain necessary” but are “insufficient in isolation because they operate primarily at the level of final artefacts” (p. 4). Inference: the numbered steps (verify artifact, process, uncertainty claims; validate alignment; keep traces) are this study’s list distilled from that text, not a procedure in the paper. Verus form: Inference: functional V&V maps directly to Verus contracts and proofs. Process-level V&V can require a recorded proof run after every contract-changing edit, forbid unchecked changes, and inspect solver failures. Epistemic V&V can require explicit requires assumptions and a list of unproved external behaviors. These are workflow checks around Verus; they are not Verus language constructs. Evidence and limits: the paper offers no algorithm, implementation, or empirical threshold. Inference: the steps are a design recommendation, so VL must measure whether they catch real specification and process failures before claiming benefit.

technique: runtime monitoring as continuing review

What it is: make code review continuous after deployment. “We propose that in human-AI software engineering teams, code review should extend beyond pre-merge inspection and become a continuous, runtime activity.” (Section 4, p. 4) What the paper says (New Idea #5, p. 4): “Deployed code is not considered final, but provisional. AI agents continuously monitor system behaviour in production, evaluating execution traces, resource usage, security signals, and deviations from expected behaviour.” It “reframes code review as a continuous assurance mechanism embedded within the system lifecycle,” so that review becomes “temporally distributed, adaptive, and tightly coupled with system operation.” The Conclusion (Section 5, pp. 4-5) calls for “evolving code review into a computationally supported, human–AI collaborative practice.” Inference: feeding monitoring findings back into requirement updates is this study’s addition, not stated in the paper. Verus form: Inference: Verus proves modeled pre-deployment properties; runtime monitors can check assumptions about inputs, dependencies, resources, and environment that are outside the proof model. A monitor must not silently weaken or replace a contract. Inference: combine proof artifacts with event logs and alerts, then trigger re-verification when monitored assumptions change. Limits: the paper gives no runtime architecture, overhead, false-alarm rate, or proof that monitoring improves trust. “Data Availability” explicitly reports no data. The proposal therefore remains an untested direction.

what remains trusted and threat to VL

The paper measures nothing and proves no software property. Its claims are arguments about review practice, grounded in cited literature. Trusted elements include the chosen dimensions, selected signals, reviewer judgment, requirement interpretation, runtime instrumentation, and any verifier’s specification. Inference: the main threat to agent-written verified Rust is misplaced confidence: a human may see a proof and omit checking whether the contract, assumptions, and process represent the intended system. This paper adds an inspection framing; VL already has the underlying proof constructs and should not redo the paper’s conceptual taxonomy as a verifier feature.

techniques for the catalog

• multidimensional trustworthiness assessment — assess technical, transparency, uncertainty, ethical, and socio-technical evidence together
• evidence-centric inspection — show selective signals, justifications, assumptions, and failure modes instead of every artifact
• multi-level V&V — verify outputs, agent process, uncertainty claims, and alignment with evolving requirements
• runtime review monitoring — continue assurance through execution traces, resource, security, and behavior deviation signals Checked 2026-10-17 against the full paper by a Claude agent (mid-paper quote: “Agentic engineers operate through multi-step processes involving planning, tool use, intermediate decisions, and iterative refinement.”): read the whole paper text; checked every quotation, the section and page references, the dimension lists, the New Idea #2-#5 content, and the Section 6 data statement; changed the link to the arXiv page (the paper’s DOI is a placeholder), corrected the dimension lists, replaced the invented “how to use/do” step lists and the misattributed assurance-mechanism wording with what the paper actually says, and marked the remaining procedure steps as Inference; did not check the cited references, the Verus-specific inferences (they are this study’s own, not the paper’s), or New Ideas #1 and #6 (not used here).

Last edited: 2026-10-17