USE CASE / Noul + Score

Agent progress checks

Judge progress, repetition, or a stuck state

A representative input (state)

Goal, recent actions, and observations

Question type

Noul + Score

Question instruction

Judge progress, repetition, or a stuck state

When below threshold

Code continues, changes strategy, stops, or escalates

Define progress with observable events

Do not rely only on an agent's self-description. Keep recent tool calls, file diffs, test outcomes, completed goals, and failure counts. Noul can flag a lack of progress, while code enforces hard limits on loops, time, and budget.

Three edge cases to test

Repeated edit and revert: possible loop
Failing tests with changing error class: possible progress
Completion claim without evidence: demand an artifact

Example request

This is a design draft, not an actual Jev answer or accuracy test.

{
  "state": "The agent edited the same file and reverted it three times.",
  "model": "jev-latest",
  "questions": {
    "decision": {
      "type": "noul",
      "instructions": "Is the agent making measurable progress toward the goal?"
    }
  }
}

Limits and fallback

Keep verifiable facts alongside semantic judgments.

What does the support team actually need to decide?

Follow one support ticket from raw text and verified facts through three questions, an illustrative answer, edge cases, and launch evaluation.

Worked example: design a Jev workflow for a missing delivery