I will build an llm evaluation suite or audit your llm judge

Your LLM feature passed the demo. Will it still pass after the next prompt or model change?

I build evaluation suites that answer that. You get a rubric written for your use case, a test set with expected answers and, if an LLM grades your outputs, a check of how far that judge can be trusted.

What you get:

• A rubric and test set built around your real failure modes
• An LLM-as-judge audit: agreement with human ratings, chance-corrected (Cohen's kappa, Krippendorff's alpha), plus bias probes
• A CI regression gate (Premium) so every change runs against the suite

Why me: I built judgecheck, a reliability report for LLM judges that shows how 96% raw agreement can hide chance-corrected scores as low as 0.60, and coding-eval, which replays coding-agent attempts in a sandbox (20 of 21 claimed success; 5 succeeded). I evaluate frontier-model code professionally and build rubric and calibration tooling for Arabic clinical AI.

Message me with what the feature does, a few example inputs and outputs, and what a good answer looks like to you.