Llm-as-Judge on Start AI Tools - Presented by Intent Solutionshttps://startaitools.com/tags/llm-as-judge/Recent content in Llm-as-Judge on Start AI Tools - Presented by Intent SolutionsHugoen-USIntent Solutions. All rights reserved.Wed, 08 Jul 2026 07:31:30 -0600Noise-Robust LLM-Judge Evals: Don't Sign a Coin Fliphttps://startaitools.com/posts/noise-robust-signed-llm-judge-evals/Tue, 07 Jul 2026 10:00:00 -0500https://startaitools.com/posts/noise-robust-signed-llm-judge-evals/<p>An un-seeded LLM judge is nondeterministic even at temperature 0. So a single-call binary verdict is a coin flip. And when you sign that verdict into a public transparency log, you have cryptographically attested one noisy draw as if it were ground truth.</p> <p>That is the integrity bug at the center of this post. It sits underneath every &ldquo;LLM-as-judge&rdquo; eval pipeline that gates a rollout, and it gets worse the moment you add a signature, because a <a href="https://docs.sigstore.dev/logging/overview/">Rekor entry</a> is permanent. You cannot retract a coin flip once a stranger can verify it.</p>