Skip to content

cayu.evals.scenario_authoring

Public Cayu API. Definitions are grouped by their public source module, and definitions from private implementation modules appear on their package’s page; adding a symbol does not move existing links. The import shown at each entry is a supported public namespace. Read the Python API overview and install the required extra.

compile_eval_scenario_draft

Section titled “compile_eval_scenario_draft”
from cayu.evals import compile_eval_scenario_draft

Canonicalize one authority-free draft and compute its immutable revision.

compile_eval_scenario_draft(draft: EvalScenarioDraftV2) -> EvalScenarioDocumentV2

Release source · Runtime guarantees · Capability guides

EvalScenarioDraftV2

Section titled “EvalScenarioDraftV2”
from cayu.evals import EvalScenarioDraftV2

Revision-free scenario material edited by a human or Control Plane.

A draft is never executable and is not accepted by the durable store. The framework canonicalizes it into EvalScenarioDocumentV2, computes the content revision, and then runs launch preflight against current server-owned authority.

Bases: _PortableModel.

Fields and attributes

Section titled “Fields and attributes”

Defaults below are source expressions; Field(...) includes schema/validation settings and default_factory creates a fresh value. Inherited declarations are included. Constructor behavior follows the model or dataclass shown in the bases/decorators.

Name Type Default
id StrictStr required
target_key StrictStr required
name StrictStr required
description StrictStr | None None
source EvaluationSourceIdentityV1 | None None
events tuple[ScenarioEventV2, ...] Field(min_length=1, max_length=EVAL_SCENARIO_MAX_EVENTS)
artifact_requirements tuple[ScenarioArtifactRequirementV2, ...] Field(default_factory=tuple, max_length=EVAL_SCENARIO_MAX_ARTIFACT_REQUIREMENTS)
secret_requirements tuple[ScenarioSecretRequirementV2, ...] Field(default_factory=tuple, max_length=EVAL_SCENARIO_MAX_SECRET_REQUIREMENTS)

validate_collections_are_ordered (method)

Section titled “validate_collections_are_ordered (method)”
EvalScenarioDraftV2.validate_collections_are_ordered(value: object, info) -> object

validate_ids (method)

Section titled “validate_ids (method)”
EvalScenarioDraftV2.validate_ids(value: str, info) -> str

validate_name (method)

Section titled “validate_name (method)”
EvalScenarioDraftV2.validate_name(value: str, info) -> str

validate_description (method)

Section titled “validate_description (method)”
EvalScenarioDraftV2.validate_description(value: str | None, info) -> str | None

from_scenario (method)

Section titled “from_scenario (method)”
EvalScenarioDraftV2.from_scenario(scenario: EvalScenarioDocumentV2) -> EvalScenarioDraftV2

Release source · Runtime guarantees · Capability guides

replace_eval_scenario_artifact_requirement

Section titled “replace_eval_scenario_artifact_requirement”
from cayu.evals import replace_eval_scenario_artifact_requirement

Return a new immutable scenario revision with one exact requirement replaced.

replace_eval_scenario_artifact_requirement(scenario: EvalScenarioDocumentV2, replacement: ScenarioArtifactRequirementV2) -> EvalScenarioDocumentV2

Release source · Runtime guarantees · Capability guides

validate_expected_scenario_revision

Section titled “validate_expected_scenario_revision”
from cayu.evals import validate_expected_scenario_revision

Reject stale mutation requests before any artifact or store side effect.

validate_expected_scenario_revision(scenario: EvalScenarioDocumentV2, expected_revision: str) -> EvalScenarioDocumentV2

Release source · Runtime guarantees · Capability guides