Evaluate stochastic outputs with deterministic metrics.
DIBER compares AI outputs against a structured ground truth and turns coverage, quality, errors, stability, logical consistency, methodology, and context exposure into reproducible, auditable metrics. No LLM-as-a-judge.
Prompt Regression · Candidate
Model-A · prompt-v2 · synthetic demo
same model · same context · prompt changed · synthetic demonstration values
A single score cannot explain why an AI system has improved.
And an evaluation performed by another model introduces an additional layer of variability. DIBER separates the underlying phenomena and keeps metric computation deterministic.
Coverage ≠ Quality
A response may address every point but handle them poorly, or be correct only on part of the problem.
A single run does not measure stability
Repeated runs let you examine the mean, dispersion, range, and confidence interval instead of mistaking a single sample for the behavior of the configuration.
Controlled A/B tests
The report distinguishes replication, comparisons with a single changed variable, and multidimensional tests.
Relational consistency
An apparently correct conclusion is not enough if it rests on a premise that is wrong, omitted, or hallucinated.
From human-defined criteria to an auditable report.
DIBER does not decide on its own what is true. It receives a ground truth and explicit classifications, then applies deterministic rules, weights, relationships, and aggregations.
Define the ground truth
Material points, importance, source documents, and semantic dependencies.
Classify each point
Separate Coverage from Status: what was addressed from how it was handled.
Calculate and compare
Per-run metrics, repeated-run statistics, transitions, methodology audit, exposure, and model/prompt matrices.
Multi-axis architecture
Not a single score: six complementary perspectives on system behavior.
1 · Coverage × Quality
Separates what was addressed from how it was handled, then combines the dimensions into usable and effective information.
2 · Repeated-run statistics
Mean, min, max, range, variance, standard deviation, and confidence interval.
3 · Relation consistency
Tracks relationships such as depends_on, supports and contradicts; the Core applies deterministic rules to relationships supported by the current logic.
4 · A/B methodology
Detects changed variables and distinguishes replication, strictly controlled, and multidimensional comparisons.
5 · Input exposure
Tracks the documents that were actually available, out-of-context points, and the exposure assessment.
6 · Model / Prompt Matrix & Behavioral Profile
Groups results by model/prompt and aggregates explicit behavioral dimensions such as tone adaptation and irony recognition when provided.
Where DIBER proves useful.
The same mechanisms can be applied to different scenarios as long as a ground truth or structured criterion is available.
Prompt regression
Compare prompt versions while keeping the model and context fixed.
Model comparison
Compare models within the same scope without reducing everything to a single score.
Repeated-run stability
Measure the mean and dispersion of a repeated configuration.
RAG / context exposure
Distinguish model errors from points that were not available in the actual context.
CI/CD quality gates
Apply reproducible thresholds to critical metrics before a release.
Manual response audit
With DIBER Free, users can define their own expected points and generate a local report without a second LLM.
Start with the result. Then, if you want, drill down to the JSON.
The full demo uses synthetic scenarios to show KPIs, A/B comparisons, Coverage × Quality, error profiles, repeated-run statistics, transitions, relationships, methodology audits, exposure, model/prompt matrices, and behavioral profiles.
Prompt v1 → Prompt v2
synthetic · prompt onlyThree ways to get started with DIBER.
Choose the level of depth you need.
Interactive Demo
Explore ready-made reports and charts based on synthetic scenarios.
Open the Demo →DIBER Free
Audit a real response in a guided workflow, entirely in the browser.
DIBER an AI response →Developer Hub
Playground, API, CLI, schema, formulas, validation, and report output.
Open the docs →Not a developer? DIBER an AI response with DIBER.
Paste the question and response, define the points you expected, classify them with guided instructions, and generate a deterministic report. You can save a local draft, import or export the audit, and download the JSON report. No LLM judge.
Integrate the Core without turning the homepage into an API reference.
The Developer Hub includes the Core Tester, installation instructions, CLI/CI-CD, API, JSON schema, Coverage/Status dictionary, formulas, configuration, validation, aggregations, and the complete report format.
import { buildExperimentReport } from '@anonipro/diber-core'; const report = buildExperimentReport({ experiment, groundTruth, evaluations }); if (report.runs[0].metrics.gt_hallucination_rate > 0.02) { throw new Error('Quality gate failed'); }
Auditable all the way down to the calculation logic.
DIBER is presented as an open-source project under the MIT License. The Developer Hub makes schemas, weights, formulas, denominators, validation, and report structure explicit, so the engine’s behavior can be inspected and reproduced.
Deterministic core
Scoring is derived from structured inputs and explicit rules, not from a second model generation.
Transparent schema
Ground truth, evaluations, and reports use documented, inspectable fields.
Portable outputs
The JSON report can be saved, compared, and used in pipelines or external tools.
Human-defined criteria
The quality of the result still depends on the quality of the ground truth and the classifications provided.