cayu.evals.external

Public Cayu API. Definitions are grouped by their public source module, and definitions from private implementation modules appear on their package’s page; adding a symbol does not move existing links. The import shown at each entry is a supported public namespace. Read the Python API overview and install the required extra.

`external_body_content_revision`

from cayu.evals import external_body_content_revision

Return the canonical content revision of one bounded multi-file body.

external_body_content_revision(root: str | Path) -> str

Release source · Runtime guarantees · Capability guides

`external_body_file_revision`

from cayu.evals import external_body_file_revision

Return a content revision for one required file inside a body snapshot.

external_body_file_revision(root: str | Path, relative_path: str) -> str

Release source · Runtime guarantees · Capability guides

`EXTERNAL_BODY_MAX_BYTES`

from cayu.evals import EXTERNAL_BODY_MAX_BYTES

EXTERNAL_BODY_MAX_BYTES = 32 << 20

Release source · Runtime guarantees · Capability guides

`EXTERNAL_BODY_MAX_FILES`

from cayu.evals import EXTERNAL_BODY_MAX_FILES

EXTERNAL_BODY_MAX_FILES = 512

Release source · Runtime guarantees · Capability guides

`EXTERNAL_PROCESS_PROTOCOL_VERSION`

from cayu.evals import EXTERNAL_PROCESS_PROTOCOL_VERSION

EXTERNAL_PROCESS_PROTOCOL_VERSION = 'cayu.external-process.v1'

Release source · Runtime guarantees · Capability guides

`external_trial_envelope_from_request`

from cayu.evals import external_trial_envelope_from_request

Extract and remove the one trusted marker from a provider request.

external_trial_envelope_from_request(request: ModelRequest, *, expected_target_revision: str) -> tuple[ExternalTrialEnvelopeV1, ModelRequest]

Release source · Runtime guarantees · Capability guides

`EXTERNAL_TRIAL_ENVELOPE_PREFIX`

from cayu.evals import EXTERNAL_TRIAL_ENVELOPE_PREFIX

EXTERNAL_TRIAL_ENVELOPE_PREFIX = 'cayu-external-trial-v1:'

Release source · Runtime guarantees · Capability guides

`ExternalBodyReleaseV1`

from cayu.evals import ExternalBodyReleaseV1

Content identity and launch contract for one immutable packaged body.

content_revision commits the complete multi-file body. The private runtime and launch protocol are named independently so replacing the bundled runtime cannot hide behind an otherwise unchanged target identity.

Bases: _ExternalModel.

Fields and attributes

Defaults below are source expressions; Field(...) includes schema/validation settings and default_factory creates a fresh value. Inherited declarations are included. Constructor behavior follows the model or dataclass shown in the bases/decorators.

Name	Type	Default
`schema_version`	`Literal[1]`	`1`
`revision`	`StrictStr`	`required`
`content_revision`	`StrictStr`	`required`
`private_runtime_path`	`StrictStr`	`required`
`private_runtime_revision`	`StrictStr`	`required`
`launch_protocol_revision`	`StrictStr`	`required`
`entrypoint`	`tuple[StrictStr, ...]`	`Field(min_length=1, max_length=EXTERNAL_TARGET_MAX_ENTRYPOINT_PARTS)`

validate_schema_version_type (method)

ExternalBodyReleaseV1.validate_schema_version_type(value: object) -> object

validate_revisions (method)

ExternalBodyReleaseV1.validate_revisions(value: str, info) -> str

validate_private_runtime_path (method)

ExternalBodyReleaseV1.validate_private_runtime_path(value: str, info) -> str

validate_entrypoint_sequence (method)

ExternalBodyReleaseV1.validate_entrypoint_sequence(value: object) -> object

validate_entrypoint (method)

ExternalBodyReleaseV1.validate_entrypoint(value: tuple[str, ...]) -> tuple[str, ...]

validate_revision (method)

ExternalBodyReleaseV1.validate_revision() -> ExternalBodyReleaseV1

create (method)

ExternalBodyReleaseV1.create(*, content_revision: str, private_runtime_path: str, private_runtime_revision: str, launch_protocol_revision: str, entrypoint: Sequence[str]) -> ExternalBodyReleaseV1

from_directory (method)

ExternalBodyReleaseV1.from_directory(root: str | Path, *, private_runtime_path: str, launch_protocol_revision: str, entrypoint: Sequence[str]) -> ExternalBodyReleaseV1

Pin a complete body plus its separately identified bundled runtime.

Release source · Runtime guarantees · Capability guides

`ExternalProcessModelProvider`

from cayu.evals import ExternalProcessModelProvider

One-step provider façade over a reconnectable trusted target adapter.

The provider owns no scheduling or retry state. Cayu’s ordinary provider operation machinery persists the adapter’s opaque operation identity and reconnects it after process loss.

Bases: ModelProvider.

Fields and attributes

Name	Type	Default
`name`	`class attribute`	`'cayu-external-process'`
`billing_provider_name`	`str \| None`	`None`
`usage_dialect`	`UsageDialect`	`UsageDialect.AUTO`
`supports_native_structured_output`	`bool`	`False`

prepare_auxiliary_request (method)

ExternalProcessModelProvider.prepare_auxiliary_request(request: ModelRequest, *, max_output_tokens: int) -> ModelRequest

Prepare one bounded request without dispatch, credentials, or I/O.

Supporting adapters return a detached request with their native output token cap. They must preserve model/messages, reject tools and raw options, and must not hide additional provider dispatches. Transparent wrappers delegate this hook. Default unsupported keeps ordinary custom providers usable without granting them managed inference capability.

stream_deadlines (property)

ExternalProcessModelProvider.stream_deadlines() -> ProviderStreamDeadlines

Return immutable dispatch deadlines for this provider.

runtime_stream (method)

ExternalProcessModelProvider.runtime_stream(request: ModelRequest) -> AsyncIterator[ModelStreamEvent]

Open the governed model-attempt stream.

Runtime, application, example, and conformance callers must use this entrance so normalized semantic and absolute deadlines also guard opaque custom providers. Transparent provider wrappers must override this entrance and delegate to the wrapped provider’s runtime_stream; forwarding only the raw stream hook would add an opaque outer guard that cannot trust terminal evidence accepted by the wrapped adapter.

execution_profile_identity (property)

ExternalProcessModelProvider.execution_profile_identity() -> ExecutionProfileBehaviorIdentity

provider_operation_mode (property)

ExternalProcessModelProvider.provider_operation_mode() -> ProviderOperationMode

provider_operations (property)

ExternalProcessModelProvider.provider_operations() -> ProviderOperationAdapter

context_pressure_profile (property)

ExternalProcessModelProvider.context_pressure_profile() -> ModelContextPressureProfile

request_cache_policy (method)

ExternalProcessModelProvider.request_cache_policy(request: ModelRequest) -> CachePolicy | None

Describe the effective provider-visible cache breakpoints, when known.

Implementations must be deterministic, side-effect free, and perform no I/O.

request_cache_projection (method)

ExternalProcessModelProvider.request_cache_projection(request: ModelRequest) -> RequestCacheProjection | None

Project the effective cache policy and provider-selected prefix.

Providers whose message preparation can filter or normalize content should override this hook and return the exact ephemeral prefix through the marker they applied. The default preserves compatibility with policy-only provider extensions and lets the runtime derive a provider-neutral prefix. Implementations must be deterministic, side-effect free, and perform no I/O.

request_footprint_options (method)

ExternalProcessModelProvider.request_footprint_options(request: ModelRequest) -> dict[str, Any]

Project privacy-safe provider-visible options for local measurement.

Implementations must return copied JSON, omit arbitrary metadata and secret-bearing extension fields, and perform no I/O.

request_fingerprint_options (method)

ExternalProcessModelProvider.request_fingerprint_options(request: ModelRequest) -> dict[str, Any]

Project complete effective provider-visible options for local analysis.

The runtime keeps this copied value ephemeral: it supplies keyed HMAC input when configured and identifies which non-allowlisted option categories are active, but the value and its field names are never persisted. Built-in adapters override this hook so ignored namespaces, defaults, and normalized controls follow their payload-building semantics. The default conservatively treats every non-runtime request option as provider-visible. Implementations must be deterministic, side-effect free, and perform no I/O.

preflight_model_target (method)

ExternalProcessModelProvider.preflight_model_target(*, model: str) -> None

preflight_thinking (method)

ExternalProcessModelProvider.preflight_thinking(*, model: str, thinking: ThinkingConfig | None) -> None

Reject thinking controls whose semantics this adapter has not established.

Routed runs call this side-effect-free hook for every configured target and again for the actual selected request before dispatch. Adapters must preserve native effort values and their documented best-effort controls; passing preflight does not guarantee an unknown backend model accepts them. Transparent wrappers delegate the hook. The default accepts only None.

preflight_portable_messages (method)

ExternalProcessModelProvider.preflight_portable_messages(*, model: str, messages: list[Message], tools: list[dict[str, Any]]) -> None

preflight_hosted_tools (method)

ExternalProcessModelProvider.preflight_hosted_tools(*, model: str, hosted_tools: tuple[OpenAIWebSearch, ...], options: dict[str, Any]) -> None

Reject provider-hosted authority this adapter cannot honor.

This hook is side-effect free and is called before a new session is created. The conservative default rejects every hosted tool so custom providers cannot silently omit registered execution authority.

supports_targeted_tool_projection (method)

ExternalProcessModelProvider.supports_targeted_tool_projection(*, model: str, protocol: str) -> bool

Report an explicitly established provider/model projection capability.

The conservative default is unsupported. Implementations must not infer support from a model-name pattern because provider aliases and compatible endpoints can expose different wire contracts under the same model name.

preflight_targeted_tool_projection (method)

ExternalProcessModelProvider.preflight_targeted_tool_projection(*, model: str, protocol: str) -> None

Reject targeted-tool authority that this provider cannot project.

supports_tool_discovery_projection (method)

ExternalProcessModelProvider.supports_tool_discovery_projection(*, model: str, protocol: str) -> bool

Report an explicitly established provider/model discovery capability.

preflight_tool_discovery_projection (method)

ExternalProcessModelProvider.preflight_tool_discovery_projection(*, model: str, protocol: str) -> None

Reject discovery authority that this provider cannot project.

billing_identity_for_request (method)

async ExternalProcessModelProvider.billing_identity_for_request(request: ModelRequest) -> BillingIdentity | None

Resolve optional request context needed for commercial accounting.

billing_identity_for_completion (method)

ExternalProcessModelProvider.billing_identity_for_completion(identity: BillingIdentity | None, payload: dict[str, Any]) -> BillingIdentity | None

Merge provider-reported completion facts into a request identity.

preflight_native_structured_output_schema (method)

ExternalProcessModelProvider.preflight_native_structured_output_schema(json_schema: dict[str, Any]) -> None

Optionally reject a NATIVE structured-output schema this adapter’s provider API would refuse, before any model request is made.

The runtime calls this at every entrance (after the supports_native_structured_output check, before any session is created or transitioned) with a caller-owned copy of the schema; implementations must not mutate it. Adapters should raise NativeStructuredOutputSchemaInvalid with a path-specific message. The default accepts everything: adapters without provider-specific schema rules stay source-compatible.

count_input_tokens (method)

async ExternalProcessModelProvider.count_input_tokens(request: ModelRequest) -> InputTokenCountResult | None

Optionally count the input tokens for one request before submission.

Providers that need to call a remote counting endpoint should do so here. Remote counters are opt-in observability/calibration hooks, not default context-overflow enforcement. The default implementation is intentionally unavailable so existing providers remain source-compatible.

stream (method)

async ExternalProcessModelProvider.stream(request: ModelRequest)

init (method)

ExternalProcessModelProvider.__init__(*, identity: ExternalProcessTargetIdentityV1, operations: ProviderOperationAdapter) -> None

Release source · Runtime guarantees · Capability guides

`ExternalProcessTargetIdentityV1`

from cayu.evals import ExternalProcessTargetIdentityV1

Independently pinned identities for one trusted external target.

Bases: _ExternalModel.

Fields and attributes

Name	Type	Default
`schema_version`	`Literal[1]`	`1`
`revision`	`StrictStr`	`required`
`body`	`ExternalBodyReleaseV1`	`required`
`evaluator_runtime_revision`	`StrictStr`	`required`
`target_implementation_revision`	`StrictStr`	`required`
`runner_revision`	`StrictStr`	`required`
`environment_revision`	`StrictStr`	`required`
`reset_contract_revision`	`StrictStr`	`required`
`evidence_policy_revision`	`StrictStr`	`required`

validate_schema_version_type (method)

ExternalProcessTargetIdentityV1.validate_schema_version_type(value: object) -> object

validate_revisions (method)

ExternalProcessTargetIdentityV1.validate_revisions(value: str, info) -> str

copy_body (method)

ExternalProcessTargetIdentityV1.copy_body(value: object) -> object

validate_revision (method)

ExternalProcessTargetIdentityV1.validate_revision() -> ExternalProcessTargetIdentityV1

create (method)

ExternalProcessTargetIdentityV1.create(*, body: ExternalBodyReleaseV1, evaluator_runtime_revision: str, target_implementation_revision: str, runner_revision: str, environment_revision: str, reset_contract_revision: str, evidence_policy_revision: str) -> ExternalProcessTargetIdentityV1

Release source · Runtime guarantees · Capability guides

`ExternalTrialEnvelopeV1`

from cayu.evals import ExternalTrialEnvelopeV1

Runtime-authored marker carried to the trusted external provider only.

Bases: _ExternalModel.

Fields and attributes

Name	Type	Default
`schema_version`	`Literal[1]`	`1`
`protocol`	`Literal['cayu.external-process.v1']`	`EXTERNAL_PROCESS_PROTOCOL_VERSION`
`trial`	`ExternalTrialIdentityV1`	`required`
`opaque_case_ref`	`OpaqueExternalCaseRefV1 \| None`	`Field(default=None, exclude_if=lambda value: value is None)`

validate_schema_version_type (method)

ExternalTrialEnvelopeV1.validate_schema_version_type(value: object) -> object

copy_trial (method)

ExternalTrialEnvelopeV1.copy_trial(value: object) -> object

copy_opaque_case_ref (method)

ExternalTrialEnvelopeV1.copy_opaque_case_ref(value: object) -> object

message (method)

ExternalTrialEnvelopeV1.message() -> Message

Release source · Runtime guarantees · Capability guides

`ExternalTrialIdentityV1`

from cayu.evals import ExternalTrialIdentityV1

Exact native run/corpus/suite/case/trial identity fixed before dispatch.

Bases: _ExternalModel.

Fields and attributes

Name	Type	Default
`schema_version`	`Literal[1]`	`1`
`revision`	`StrictStr`	`required`
`native_run_id`	`StrictStr`	`required`
`target_key`	`StrictStr`	`required`
`target_revision`	`StrictStr`	`required`
`corpus_revision`	`StrictStr`	`required`
`suite_id`	`StrictStr`	`required`
`suite_revision`	`StrictStr`	`required`
`case_id`	`StrictStr`	`required`
`case_revision`	`StrictStr`	`required`
`trial_number`	`StrictInt`	`Field(ge=1, le=100)`

validate_schema_version_type (method)

ExternalTrialIdentityV1.validate_schema_version_type(value: object) -> object

validate_revisions (method)

ExternalTrialIdentityV1.validate_revisions(value: str, info) -> str

validate_identities (method)

ExternalTrialIdentityV1.validate_identities(value: str, info) -> str

validate_revision (method)

ExternalTrialIdentityV1.validate_revision() -> ExternalTrialIdentityV1

create (method)

ExternalTrialIdentityV1.create(*, native_run_id: str, target_key: str, target_revision: str, corpus_revision: str, suite_id: str, suite_revision: str, case_id: str, case_revision: str, trial_number: int) -> ExternalTrialIdentityV1

Release source · Runtime guarantees · Capability guides

`OpaqueExternalCaseRefV1`

from cayu.evals import OpaqueExternalCaseRefV1

Authority-free alias for case material retained by a trusted adapter.

Bases: _ExternalModel.

Fields and attributes

Name	Type	Default
`schema_version`	`Literal[1]`	`1`
`id`	`StrictStr`	`required`
`revision`	`StrictStr`	`required`

validate_schema_version_type (method)

OpaqueExternalCaseRefV1.validate_schema_version_type(value: object) -> object

validate_id (method)

OpaqueExternalCaseRefV1.validate_id(value: str, info) -> str

validate_revision (method)

OpaqueExternalCaseRefV1.validate_revision(value: str, info) -> str

Release source · Runtime guarantees · Capability guides

`with_external_trial_envelope`

from cayu.evals import with_external_trial_envelope

Return detached messages prefixed by one exact runtime-owned envelope.

with_external_trial_envelope(messages: Sequence[Message], envelope: ExternalTrialEnvelopeV1) -> list[Message]

Release source · Runtime guarantees · Capability guides