AI is good at reading a lot of code quickly, connecting related files, and suggesting places a reviewer should inspect. But a suggestion is not a verdict. A model can miss project context, misunderstand intent, or produce a convincing explanation for a problem that is not there.

AI can be fluent and still be wrong

NIST calls this risk confabulation: generative AI can produce content that is confidently stated but false. That matters in code review because a finding can sound precise, include a line number, and still misunderstand how the code behaves.

A factuality benchmark, not a code review score

60.8%

of GPT 4o answers were graded incorrect on OpenAI’s SimpleQA benchmark. The test contains 4,326 short factual questions, and most were selected because they tended to trigger hallucinations in earlier models.

This is not a measure of code review accuracy, and it should not be treated as the error rate of every model or use case. It demonstrates why fluent output still needs verification.

Read the SimpleQA paper ↗

The number is useful when read narrowly. SimpleQA deliberately tests hard fact retrieval without browsing. Its results do not predict how often a code review model will be wrong. They do show that even a capable model can return an incorrect answer instead of abstaining. In a pull request, that is enough reason to treat findings as leads for a human reviewer.

What SOC 2 says about approval

SOC 2 is an attestation about an organization’s controls within a defined system and scope. It is not a blanket rule that a human must approve every AI output. It also does not certify that AI findings are accurate.

The AICPA Trust Services Criteria include CC8.1 for change management. It describes controls for authorizing, testing, approving, and implementing changes to infrastructure, data, software, and procedures. A SOC 2 Type II examination considers whether the controls in scope operated effectively during the stated period. The organization defines the specific control process; the criteria do not name AI generated review comments or require a particular person to approve each draft.

So the careful conclusion is this: SOC 2 makes controlled, approved system changes part of the criteria. If a team uses AI in its change process, it should document who or what is allowed to act, where approval happens, and what evidence is retained. Human approval of an AI drafted code review comment is a clear way to keep that decision accountable, but it is not a claim that every SOC 2 report mandates a human click on every suggestion.

Sources: AICPA Trust Services Criteria · AICPA SOC resources

Why a person should make the final call

A reviewer brings information the model may not have: the product requirement, a recent incident, a deliberate tradeoff, or a team convention that is not written in the diff. The reviewer can compare a proposed finding with tests, related changes, and the intended behavior.

  • Check the evidence. A useful finding points to the changed code and explains a concrete failure path.
  • Apply project context. Reviewers can tell a bug from an intentional change or an established pattern.
  • Own the communication. The person who accepts or edits a finding stands behind the comment sent to the author.

This is a better division of work: let AI read broadly and prepare a review path, then let a person decide what is true, important, and worth interrupting someone for.

AI drafts, people decide

Rocky is built around that boundary. It orders changed files by risk, drafts findings with their source context, and maps how changes connect. A reviewer can accept a draft, edit it, or dismiss it. Nothing written by AI is posted without human approval.

That keeps the speed benefit while preserving a clear owner for each review comment. The goal is not to make the human reviewer rubber stamp the model. It is to give them better context and a faster first pass.

Sources and further reading

  1. NIST AI 600 1, Generative Artificial Intelligence Profile. Defines confabulation as confidently stated erroneous or false content and documents risks that can arise when users trust it.
  2. Wei et al., Measuring Short Form Factuality in Large Language Models. Introduces SimpleQA and reports model results on 4,326 fact seeking questions.
  3. AICPA 2017 Trust Services Criteria, revised points of focus 2022. Includes CC8.1, the change management criterion.

Benchmark figures describe the named evaluations and models. They are not a guarantee of performance on Rocky or on any particular codebase.