Inconsistent and unclear reporting from coding agents regarding their actions and modifications, leading to potential misunderstandings and errors in code manag

Signals over time

2 observed signals across 1 sources, tracked for 8 days. Confidence: low.

What people are saying Observed

• “A couple of months ago, I gave access to my coding agent to do work on my repos. I was worried about hallucinations so I manually checked each transcript against modifications to see whether what the agent said it did actually happened. I found that in many cases, my coding agent either failed to disclose what it actually did or found workarounds to things like failed tests instead of telling me upfront. That led me to the idea for Rashomon. Rashomon is an independent execution recorder for coding agents that notifies users whenever there is a discrepancy between what the agent said it did and what we capture as a ground truth. It aims to capture items like shell commands, exit codes, file writes, tool calls, tests, and subagents. We build a timeline of the events but we do not store any prompts, responses, file contents, or tool outputs. A question we got often has been how is this different than a diff. A diff gives you a look at the final result and what changed while what we aim to provide is visibility into the execution that led to it. Currently, we do not capture network codes, prompts, responses file contents, and tool outputs. We plan to implement some of those based on fe”

  Hacker News · complaint

• “A couple of months ago, I gave access to my coding agent to do work on my repos. I was worried about hallucinations so I manually checked each transcript against modifications to see whether what the ge t said it did actually happened. I found that in many cases, my coding agent either failed to disclose what it actually did or found workarounds to things like failed tests instead of telling me upfront. That led me to the idea for Rashomon. Rashomon is an independent execution recorder for coding agents that notifies users whenever there is a discrepancy between what the agent said it did and what we capture as a ground truth. It aims to capture items like shell commands, exit codes, file writes, tool calls, tests, and subagents. We build a timeline of the events but we do not store any prompts, responses, file contents, or tool outputs. Currently, we do not capture network codes, prompts, responses file contents, and tool outputs. We plan to implement some of those based on feedback recently, but please let us know whether you would find that usefull. To get started, you can install it is as a Claude Code plugin or from the command line. I would appreciate feedback on the solution”

  Hacker News · request