Skip to content

cayu.evals.scenario_execution

Public Cayu API. Definitions are grouped by their public source module, and definitions from private implementation modules appear on their package’s page; adding a symbol does not move existing links. The import shown at each entry is a supported public namespace. Read the Python API overview and install the required extra.

corpus_for_eval_scenario

Section titled “corpus_for_eval_scenario”
from cayu.evals import corpus_for_eval_scenario

Create the immutable one-case result contract used by a scenario run.

corpus_for_eval_scenario(scenario: EvalScenarioDocumentV2, binding: ScenarioLaunchBindingV2, target: CorpusTarget, *, project_root: Path | None=None) -> EvalCorpusDocument

Release source · Runtime guarantees · Capability guides

run_compiled_eval_scenario

Section titled “run_compiled_eval_scenario”
from cayu.evals import run_compiled_eval_scenario

Execute all scenario trials and return the ordinary corpus result shape.

async run_compiled_eval_scenario(target: CorpusTarget, compiled: CompiledCorpusSuite, scenario: EvalScenarioDocumentV2, binding: ScenarioLaunchBindingV2, *, store: EvalStore, claim: EvalRunClaim, max_concurrency: int, poll_seconds: float, manifest_project_root: Path | None=None, expected_app_manifest_fingerprint: str | None=None, expected_execution_profile: ExecutionProfileIdentity | None=None, execution_capacity: EvalExecutionCapacity | None=None, accepted_exposure: EvalSuiteRunExposureV1 | None=None) -> CorpusExecutionResult

Release source · Runtime guarantees · Capability guides

scenario_launch_settings_from_invocation

Section titled “scenario_launch_settings_from_invocation”
from cayu.evals import scenario_launch_settings_from_invocation

Reconstruct the exact public launch selection retained at admission.

scenario_launch_settings_from_invocation(invocation: EvalScenarioRunInvocation, *, max_concurrency: int, max_steps: int | None, limits: RunLimits | None, cost_budget: EvalRunCostBudget | None) -> ScenarioLaunchSettingsV2

Release source · Runtime guarantees · Capability guides

ScenarioExecutionError

Section titled “ScenarioExecutionError”
from cayu.evals import ScenarioExecutionError

Safe orchestration failure retained in bounded scenario progress.

Bases: RuntimeError.

init (method)

Section titled “init (method)”
ScenarioExecutionError.__init__(code: EvalScenarioTrialFailureCode, message: str) -> None

Release source · Runtime guarantees · Capability guides