cayu.evals.results

Public Cayu API. Definitions are grouped by their public source module, and definitions from private implementation modules appear on their package’s page; adding a symbol does not move existing links. The import shown at each entry is a supported public namespace. Read the Python API overview and install the required extra.

`captured_evaluation_result_from_json`

from cayu.evals import captured_evaluation_result_from_json

Parse a bounded captured-result document without format guessing.

captured_evaluation_result_from_json(source: str) -> CapturedEvaluationResultV1

Release source · Runtime guarantees · Capability guides

`CAPTURED_EVALUATION_RESULT_MAX_BYTES`

from cayu.evals import CAPTURED_EVALUATION_RESULT_MAX_BYTES

CAPTURED_EVALUATION_RESULT_MAX_BYTES = 4 << 20

Release source · Runtime guarantees · Capability guides

`CAPTURED_EVALUATION_RESULT_SCHEMA_VERSION`

from cayu.evals import CAPTURED_EVALUATION_RESULT_SCHEMA_VERSION

CAPTURED_EVALUATION_RESULT_SCHEMA_VERSION = 1

Release source · Runtime guarantees · Capability guides

`CapturedEvaluationResultV1`

from cayu.evals import CapturedEvaluationResultV1

Immutable public result scored only from retained session evidence.

The document deliberately contains no session id, store handle, application object, credential, or historical execution authority. Its corpus revision supplies the separately persisted expectation contract; captured-only corpora deliberately contain no runnable input.

Bases: _SchemaV1PortableModel.

Fields and attributes

Defaults below are source expressions; Field(...) includes schema/validation settings and default_factory creates a fresh value. Inherited declarations are included. Constructor behavior follows the model or dataclass shown in the bases/decorators.

Name	Type	Default
`schema_version`	`Literal[1]`	`CAPTURED_EVALUATION_RESULT_SCHEMA_VERSION`
`revision`	`StrictStr`	`required`
`origin`	`Literal[EvalResultOrigin.CAPTURED_SESSION]`	`EvalResultOrigin.CAPTURED_SESSION`
`target`	`EvalResultTargetIdentityV1`	`required`
`corpus_revision`	`StrictStr`	`required`
`suite_id`	`StrictStr`	`required`
`suite_revision`	`StrictStr`	`required`
`score`	`CapturedRunScoreV1`	`required`

validate_schema_version_type (method)

CapturedEvaluationResultV1.validate_schema_version_type(value: object) -> object

validate_revisions (method)

CapturedEvaluationResultV1.validate_revisions(value: str, info) -> str

validate_suite_id (method)

CapturedEvaluationResultV1.validate_suite_id(value: str, info) -> str

copy_target (method)

CapturedEvaluationResultV1.copy_target(value: object) -> object

copy_score (method)

CapturedEvaluationResultV1.copy_score(value: object) -> object

validate_contract (method)

CapturedEvaluationResultV1.validate_contract() -> CapturedEvaluationResultV1

create (method)

CapturedEvaluationResultV1.create(*, corpus: EvalCorpusDocument, target: EvalResultTargetIdentityV1, score: CapturedRunScoreV1) -> CapturedEvaluationResultV1

Release source · Runtime guarantees · Capability guides

`eval_result_projection`

from cayu.evals import eval_result_projection

Project either immutable result origin into one comparison contract.

eval_result_projection(result: CorpusExecutionResult | CapturedEvaluationResultV1) -> EvalResultProjectionV2

Release source · Runtime guarantees · Capability guides

`EVAL_RESULT_PROJECTION_MAX_BYTES`

from cayu.evals import EVAL_RESULT_PROJECTION_MAX_BYTES

EVAL_RESULT_PROJECTION_MAX_BYTES = 4 << 20

Release source · Runtime guarantees · Capability guides

`EVAL_RESULT_PROJECTION_SCHEMA_VERSION`

from cayu.evals import EVAL_RESULT_PROJECTION_SCHEMA_VERSION

EVAL_RESULT_PROJECTION_SCHEMA_VERSION = 2

Release source · Runtime guarantees · Capability guides

`EvalResultAssertionIdentityV1`

from cayu.evals import EvalResultAssertionIdentityV1

Bases: _SchemaV1PortableModel.

Fields and attributes

Name	Type	Default
`schema_version`	`Literal[1]`	`1`
`assertion_id`	`StrictStr`	`required`
`assertion_revision`	`StrictStr`	`required`
`comparison_revision`	`StrictStr`	`required`

validate_schema_version_type (method)

EvalResultAssertionIdentityV1.validate_schema_version_type(value: object) -> object

validate_assertion_id (method)

EvalResultAssertionIdentityV1.validate_assertion_id(value: str, info) -> str

validate_assertion_revision (method)

EvalResultAssertionIdentityV1.validate_assertion_revision(value: str, info) -> str

Release source · Runtime guarantees · Capability guides

`EvalResultCaseProjectionV1`

from cayu.evals import EvalResultCaseProjectionV1

Bases: _SchemaV1PortableModel.

Fields and attributes

Name	Type	Default
`schema_version`	`Literal[1]`	`1`
`case_id`	`StrictStr`	`required`
`case_revision`	`StrictStr`	`required`
`status`	`PublishedStatus`	`required`
`score`	`StrictFloat \| None`	`Field(default=None, ge=0.0, le=1.0)`
`assertions`	`tuple[EvalResultAssertionIdentityV1, ...]`	`Field(min_length=1, max_length=EVAL_CORPUS_MAX_ASSERTIONS_PER_CASE)`
`trial_diagnostic_codes`	`tuple[EvalTrialDiagnosticCode, ...]`	`Field(default=(), max_length=EVAL_CORPUS_MAX_TRIALS, exclude_if=lambda value: not value)`

validate_schema_version_type (method)

EvalResultCaseProjectionV1.validate_schema_version_type(value: object) -> object

validate_case_id (method)

EvalResultCaseProjectionV1.validate_case_id(value: str, info) -> str

validate_case_revision (method)

EvalResultCaseProjectionV1.validate_case_revision(value: str, info) -> str

validate_assertion_order (method)

EvalResultCaseProjectionV1.validate_assertion_order(value: object, info) -> object

validate_case_contract (method)

EvalResultCaseProjectionV1.validate_case_contract() -> EvalResultCaseProjectionV1

Release source · Runtime guarantees · Capability guides

`EvalResultOrigin`

from cayu.evals import EvalResultOrigin

How one immutable published result was produced.

Bases: StrEnum.

Fields and attributes

Name	Type	Default
`CAPTURED_SESSION`	`class attribute`	`'captured_session'`
`FRESH_EXECUTION`	`class attribute`	`'fresh_execution'`

Release source · Runtime guarantees · Capability guides

`EvalResultProjectionV1`

from cayu.evals import EvalResultProjectionV1

The single comparison projection shared by captured and fresh results.

Bases: _SchemaV1PortableModel.

Fields and attributes

Name	Type	Default
`schema_version`	`Literal[1]`	`1`
`result_revision`	`StrictStr`	`required`
`origin`	`EvalResultOrigin`	`required`
`target`	`EvalResultTargetIdentityV1`	`required`
`external_target_revision`	`StrictStr \| None`	`None`
`corpus_revision`	`StrictStr`	`required`
`suite_id`	`StrictStr`	`required`
`suite_revision`	`StrictStr`	`required`
`evidence_policy_revision`	`StrictStr`	`required`
`pricing_profile_fingerprint`	`StrictStr \| None`	`None`
`uses_pricing`	`StrictBool`	`required`
`memory_attribution_support`	`Literal['unsupported']`	`'unsupported'`
`status`	`PublishedStatus`	`required`
`score`	`StrictFloat \| None`	`Field(default=None, ge=0.0, le=1.0)`
`cases`	`tuple[EvalResultCaseProjectionV1, ...]`	`Field(min_length=1, max_length=EVAL_CORPUS_MAX_CASES)`

validate_schema_version_type (method)

EvalResultProjectionV1.validate_schema_version_type(value: object) -> object

validate_revisions (method)

EvalResultProjectionV1.validate_revisions(value: str | None, info) -> str | None

validate_suite_id (method)

EvalResultProjectionV1.validate_suite_id(value: str, info) -> str

validate_case_order (method)

EvalResultProjectionV1.validate_case_order(value: object, info) -> object

validate_projection (method)

EvalResultProjectionV1.validate_projection() -> EvalResultProjectionV1

Release source · Runtime guarantees · Capability guides

`EvalResultProjectionV2`

from cayu.evals import EvalResultProjectionV2

Current comparison projection with exact trial and exposure identity.

Bases: EvalResultProjectionV1.

Fields and attributes

Name	Type	Default
`schema_version`	`Literal[2]`	`EVAL_RESULT_PROJECTION_SCHEMA_VERSION`
`result_revision`	`StrictStr`	`required`
`origin`	`EvalResultOrigin`	`required`
`target`	`EvalResultTargetIdentityV1`	`required`
`external_target_revision`	`StrictStr \| None`	`None`
`corpus_revision`	`StrictStr`	`required`
`suite_id`	`StrictStr`	`required`
`suite_revision`	`StrictStr`	`required`
`evidence_policy_revision`	`StrictStr`	`required`
`pricing_profile_fingerprint`	`StrictStr \| None`	`None`
`uses_pricing`	`StrictBool`	`required`
`memory_attribution_support`	`Literal['unsupported']`	`'unsupported'`
`status`	`PublishedStatus`	`required`
`score`	`StrictFloat \| None`	`Field(default=None, ge=0.0, le=1.0)`
`cases`	`tuple[EvalResultCaseProjectionV1, ...]`	`Field(min_length=1, max_length=EVAL_CORPUS_MAX_CASES)`
`trial_policy_revision`	`StrictStr`	`required`
`accepted_exposure_revision`	`StrictStr \| None`	`None`
`accepted_exposure_comparison_revision`	`StrictStr \| None`	`None`

validate_schema_version_type (method)

EvalResultProjectionV2.validate_schema_version_type(value: object) -> object

validate_revisions (method)

EvalResultProjectionV2.validate_revisions(value: str | None, info) -> str | None

validate_suite_id (method)

EvalResultProjectionV2.validate_suite_id(value: str, info) -> str

validate_case_order (method)

EvalResultProjectionV2.validate_case_order(value: object, info) -> object

validate_projection (method)

EvalResultProjectionV2.validate_projection() -> EvalResultProjectionV1

validate_v2_revisions (method)

EvalResultProjectionV2.validate_v2_revisions(value: str | None, info) -> str | None

validate_exposure_origin (method)

EvalResultProjectionV2.validate_exposure_origin() -> EvalResultProjectionV2

Release source · Runtime guarantees · Capability guides

`EvalResultTargetIdentityV1`

from cayu.evals import EvalResultTargetIdentityV1

Comparable public target identity without executable application authority.

Bases: _SchemaV1PortableModel.

Fields and attributes

Name	Type	Default
`schema_version`	`Literal[1]`	`1`
`target_key`	`StrictStr`	`required`
`application_release_id`	`StrictStr`	`required`
`app_manifest_schema_version`	`StrictStr`	`required`
`app_manifest_fingerprint`	`StrictStr`	`required`

validate_schema_version_type (method)

EvalResultTargetIdentityV1.validate_schema_version_type(value: object) -> object

validate_target_key (method)

EvalResultTargetIdentityV1.validate_target_key(value: str, info) -> str

validate_release_id (method)

EvalResultTargetIdentityV1.validate_release_id(value: str, info) -> str

validate_manifest_schema_version (method)

EvalResultTargetIdentityV1.validate_manifest_schema_version(value: str, info) -> str

validate_manifest_fingerprint (method)

EvalResultTargetIdentityV1.validate_manifest_fingerprint(value: str, info) -> str

from_fresh_target (method)

EvalResultTargetIdentityV1.from_fresh_target(target: EvaluationTargetIdentity) -> EvalResultTargetIdentityV1

Release source · Runtime guarantees · Capability guides

`validate_captured_result_for_corpus`

from cayu.evals import validate_captured_result_for_corpus

Bind one captured score to the exact immutable corpus it was saved with.

validate_captured_result_for_corpus(result: CapturedEvaluationResultV1, corpus: EvalCorpusDocument) -> CapturedEvaluationResultV1

Release source · Runtime guarantees · Capability guides