Skip to content

add EvalStatus enum#59

Merged
peterj merged 4 commits intomainfrom
peterj/addevalstatus
Mar 23, 2026
Merged

add EvalStatus enum#59
peterj merged 4 commits intomainfrom
peterj/addevalstatus

Conversation

@peterj
Copy link
Copy Markdown
Contributor

@peterj peterj commented Mar 22, 2026

No description provided.

peterj added 2 commits March 22, 2026 22:50
Signed-off-by: Peter Jausovec <peter.jausovec@solo.io>
Signed-off-by: Peter Jausovec <peter.jausovec@solo.io>
Copy link
Copy Markdown
Contributor

@krisztianfekete krisztianfekete left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have to change the protocol definition as well:

class EvalResult(BaseModel):
"""Output payload expected from a custom evaluator on stdout."""
score: float = Field(ge=0.0, le=1.0)
status: Optional[str] = Field(
default=None,
description='One of "PASSED", "FAILED", "NOT_EVALUATED". Derived from score vs threshold if omitted.',
)
per_invocation_scores: list[Optional[float]] = Field(default_factory=list)
details: Optional[dict[str, Any]] = None

Signed-off-by: Peter Jausovec <peter.jausovec@solo.io>
Comment thread packages/evaluator-sdk-py/src/agentevals_evaluator_sdk/types.py Outdated
Signed-off-by: Peter Jausovec <peter.jausovec@solo.io>
@peterj peterj merged commit df8ffe5 into main Mar 23, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants