Skip to content

cayu.tools.inference

Public Cayu API. Definitions are grouped by their public source module, and definitions from private implementation modules appear on their package’s page; adding a symbol does not move existing links. The import shown at each entry is a supported public namespace. Read the Python API overview and install the required extra.

AuxiliaryInferencePolicy

Section titled “AuxiliaryInferencePolicy”
from cayu.tools import AuxiliaryInferencePolicy

Explicit application opt-in; omission grants no inference capability.

Bases: BaseModel.

Fields and attributes

Section titled “Fields and attributes”

Defaults below are source expressions; Field(...) includes schema/validation settings and default_factory creates a fresh value. Inherited declarations are included. Constructor behavior follows the model or dataclass shown in the bases/decorators.

Name Type Default
limits InferenceLimits required
purposes tuple[str, ...] Field(min_length=1, max_length=MAX_INFERENCE_PURPOSES)

validate_limits (method)

Section titled “validate_limits (method)”
AuxiliaryInferencePolicy.validate_limits(value: object) -> InferenceLimits

validate_purposes (method)

Section titled “validate_purposes (method)”
AuxiliaryInferencePolicy.validate_purposes(value: object) -> tuple[str, ...]

Release source · Runtime guarantees · Capability guides

InferenceInvoker

Section titled “InferenceInvoker”
from cayu.tools import InferenceInvoker

Optional invocation-scoped capability supplied only by the runtime.

This interface exposes neither provider clients nor identity/credential selection. A declaration or structurally matching caller object grants no runtime authority.

Bases: Protocol.

invoke (method)

Section titled “invoke (method)”
async InferenceInvoker.invoke(request: ModelRequest, *, purpose: str, limits: InferenceLimits) -> ModelResponse

Release source · Runtime guarantees · Capability guides

InferenceLimits

Section titled “InferenceLimits”
from cayu.tools import InferenceLimits

Declared token envelope and local collection/deadline bounds.

Token bounds describe a conservative request envelope, not measured usage. Accounting must retain the provider’s actual usage, including any overrun.

Bases: BaseModel.

Fields and attributes

Section titled “Fields and attributes”

Defaults below are source expressions; Field(...) includes schema/validation settings and default_factory creates a fresh value. Inherited declarations are included. Constructor behavior follows the model or dataclass shown in the bases/decorators.

Name Type Default
max_input_tokens StrictInt Field(gt=0, le=MAX_DURABLE_JSON_INTEGER)
max_output_tokens StrictInt Field(gt=0, le=MAX_DURABLE_JSON_INTEGER)
timeout_seconds float Field(gt=0, le=3600)
max_request_bytes StrictInt Field(default=1024 * 1024, gt=0, le=MAX_INFERENCE_BYTES)
max_response_bytes StrictInt Field(default=1024 * 1024, gt=0, le=MAX_INFERENCE_BYTES)

validate_timeout (method)

Section titled “validate_timeout (method)”
InferenceLimits.validate_timeout(value: object) -> float

bounded_by (method)

Section titled “bounded_by (method)”
InferenceLimits.bounded_by(ceiling: InferenceLimits) -> InferenceLimits

Validate a caller’s narrower envelope without silently clipping it.

Release source · Runtime guarantees · Capability guides