Two precision tools for teams building with LLMs. Detect hallucinations before your users do. Locate bugs before they ship.
AUROC 0.729
Behavior Graph
Purpose-built for teams that need to verify LLM outputs and catch bugs at the model level.
Document-grounding approach that catches fabricated claims, altered numbers, and invented facts — without GPU overhead.
Splits summaries into individual claims and verifies each against source documents.
Keywords, n-grams, numbers, entities, and continuous substrings — all cross-referenced.
Pure text matching. Runs on any CPU in under 1 second per document.
Catches altered figures like 308.76 → 425.50 via numerical match scoring.
from halluguard import detect
result = detect(
source="Revenue: ¥174.1B (+15.71%)",
summary="Revenue reached ¥215.6B"
)
# result.risk = 0.65 (RED)
# result.flags = ["num_mismatch: 174.1→215.6"]
# result.claim_scores = [0.14]Tested on HaluEval summarization benchmark with consistent results across sample sizes.
Stable performance from 200 to 10,000 samples — no overfitting detected.
End-to-end test with a Chinese public company annual report (585 characters).
| Query | Model Response | HalluGuard | Correct? |
|---|---|---|---|
| Revenue & growth rate? | ¥174.1B, +15.71% | GREEN 0.29 | ✔ |
| Production volume? | 56,221 tons, 46.06% | GREEN 0.14 | ✔ |
| R&D ratio? Dividend? | 1.08%, ¥308.76 | YELLOW 0.32 | ✔ |
| Overseas revenue? (not in doc) | "Not mentioned" | GREEN 0.33 | ✔ |
| Employee count? (not in doc) | "Cannot determine" | GREEN 0.31 | ✔ |
| ⚠️ Fabricated: ¥215.6B + overseas ¥38B | — | RED 0.65 | ✔ |
| ⚠️ Fabricated: ¥425.50 dividend + ¥1B buyback | — | RED 0.65 | ✔ |
WLBS behavior graph analysis traces model internals to pinpoint where code goes wrong — at the activation level.
Maps model activation patterns to identify anomalous computation paths.
Tracks gradient curvature through layers to spot where the model "hesitates".
Reverse-traces error signals to locate the exact layer and attention head responsible.
Visual interface for exploring behavior graphs and drilling into specific bug patterns.
from bugprobe import scan
report = scan(
model="qwen3-8b",
code="def sort(arr): ...",
test_input=[[3, 1, 2]]
)
# report.bugs = [
# Bug(layer=14, head=7,
# type="off-by-one",
# confidence=0.89)
# ]Get early access to HalluGuard and BugProbe. We're onboarding teams now.