cayu.evals.memory_baseline

Public Cayu API. Definitions are grouped by their public source module, and definitions from private implementation modules appear on their package’s page; adding a symbol does not move existing links. The import shown at each entry is a supported public namespace. Read the Python API overview and install the required extra.

load_memory_retrieval_corpus

from cayu.evals import load_memory_retrieval_corpus

load_memory_retrieval_corpus(path: str | Path) -> MemoryRetrievalCorpus

Release source · Runtime guarantees · Capability guides

MEMORY_RETRIEVAL_BASELINE_SCHEMA_VERSION

from cayu.evals import MEMORY_RETRIEVAL_BASELINE_SCHEMA_VERSION

MEMORY_RETRIEVAL_BASELINE_SCHEMA_VERSION = 'cayu.memory_retrieval_baseline.v1'

Release source · Runtime guarantees · Capability guides

MEMORY_RETRIEVAL_CORPUS_SCHEMA_VERSION

from cayu.evals import MEMORY_RETRIEVAL_CORPUS_SCHEMA_VERSION

MEMORY_RETRIEVAL_CORPUS_SCHEMA_VERSION = 'cayu.memory_retrieval_corpus.v1'

Release source · Runtime guarantees · Capability guides

MemoryRetrievalAccessSpec

from cayu.evals import MemoryRetrievalAccessSpec

Application-derived access fixture; it intentionally contains no tenant model.

Bases: BaseModel.

Fields and attributes

Defaults below are source expressions; Field(...) includes schema/validation settings and default_factory creates a fresh value. Inherited declarations are included. Constructor behavior follows the model or dataclass shown in the bases/decorators.

Name	Type	Default
allowed_namespaces	tuple[str, ...]	required
required_labels	dict[str, str]	Field(default_factory=dict)
allowed_visibilities	tuple[KnowledgeVisibility, ...]	(KnowledgeVisibility.GLOBAL,)
allowed_source_types	tuple[str, ...] | None	None
allowed_source_ids	tuple[str, ...] | None	None
allowed_statuses	tuple[KnowledgeStatus, ...]	(KnowledgeStatus.ACTIVE,)
include_expired	bool	False

copy_required_labels (method)

MemoryRetrievalAccessSpec.copy_required_labels(value) -> dict[str, str]

validate_nonempty_boundaries (method)

MemoryRetrievalAccessSpec.validate_nonempty_boundaries() -> MemoryRetrievalAccessSpec

to_scope (method)

MemoryRetrievalAccessSpec.to_scope() -> KnowledgeAccessScope

Release source · Runtime guarantees · Capability guides

MemoryRetrievalBaselineMetrics

from cayu.evals import MemoryRetrievalBaselineMetrics

Bases: BaseModel.

Fields and attributes

Defaults below are source expressions; Field(...) includes schema/validation settings and default_factory creates a fresh value. Inherited declarations are included. Constructor behavior follows the model or dataclass shown in the bases/decorators.

Name	Type	Default
case_count	int	required
recall_at_k	float	required
mean_reciprocal_rank	float	required
mean_ndcg_at_k	float	required
false_injection_rate	float	required
stale_result_rate	float	required
authorization_leak_rate	float	required
citation_correctness	float	required
latency_p50_ms	float	required
latency_p95_ms	float	required
total_candidate_count	int	required
mean_candidate_count	float	required
truncated_case_count	int	required
model_facing_bytes	int	required
estimated_model_facing_tokens	int	required

Release source · Runtime guarantees · Capability guides

MemoryRetrievalBaselineResult

from cayu.evals import MemoryRetrievalBaselineResult

Bases: BaseModel.

Fields and attributes

Defaults below are source expressions; Field(...) includes schema/validation settings and default_factory creates a fresh value. Inherited declarations are included. Constructor behavior follows the model or dataclass shown in the bases/decorators.

Name	Type	Default
schema_version	Literal['cayu.memory_retrieval_baseline.v1']	required
corpus_revision	str	required
corpus_origin	Literal['hermetic_public', 'external_private']	required
backend	str	required
search_mode	KnowledgeSearchMode	required
embedding_identity	str | None	required
reranker_identity	str | None	required
configuration	dict[str, Any]	required
metrics	MemoryRetrievalBaselineMetrics	required
language_slices	tuple[MemoryRetrievalLanguageSlice, ...]	required
cases	tuple[MemoryRetrievalCaseResult, ...]	required

validate_backend (method)

MemoryRetrievalBaselineResult.validate_backend(value: str) -> str

validate_optional_component_identity (method)

MemoryRetrievalBaselineResult.validate_optional_component_identity(value: str | None, info) -> str | None

copy_configuration (method)

MemoryRetrievalBaselineResult.copy_configuration(value) -> dict[str, Any]

Release source · Runtime guarantees · Capability guides

MemoryRetrievalCase

from cayu.evals import MemoryRetrievalCase

Bases: BaseModel.

Fields and attributes

Defaults below are source expressions; Field(...) includes schema/validation settings and default_factory creates a fresh value. Inherited declarations are included. Constructor behavior follows the model or dataclass shown in the bases/decorators.

Name	Type	Default
id	str	required
trajectory_id	str	required
turn_index	int	required
language	str	required
query	str	required
namespace	str	required
access	MemoryRetrievalAccessSpec	required
relevant_entry_ids	tuple[str, ...]	required
forbidden_entry_ids	tuple[str, ...]	()
expected_source_ids	dict[str, str]	Field(default_factory=dict)
cutoff	int	5

validate_identity (method)

MemoryRetrievalCase.validate_identity(value: str, info) -> str

validate_query (method)

MemoryRetrievalCase.validate_query(value: str) -> str

validate_turn_index (method)

MemoryRetrievalCase.validate_turn_index(value) -> int

validate_cutoff (method)

MemoryRetrievalCase.validate_cutoff(value) -> int

validate_relevance (method)

MemoryRetrievalCase.validate_relevance() -> MemoryRetrievalCase

Release source · Runtime guarantees · Capability guides

MemoryRetrievalCaseResult

from cayu.evals import MemoryRetrievalCaseResult

Bases: BaseModel.

Fields and attributes

Defaults below are source expressions; Field(...) includes schema/validation settings and default_factory creates a fresh value. Inherited declarations are included. Constructor behavior follows the model or dataclass shown in the bases/decorators.

Name	Type	Default
case_id	str	required
language	str	required
selected_entry_ids	tuple[str, ...]	required
candidate_count	int	required
truncated	bool	required
recall_at_k	float	required
reciprocal_rank	float	required
ndcg_at_k	float	required
false_injection_count	int	required
stale_result_count	int	required
authorization_leak_count	int	required
citation_correct_count	int	required
citation_evaluated_count	int	required
model_facing_bytes	int	required
latency_ms	float	required

Release source · Runtime guarantees · Capability guides

MemoryRetrievalCorpus

from cayu.evals import MemoryRetrievalCorpus

Portable public or private corpus accepted by the same hermetic runner.

Bases: BaseModel.

Fields and attributes

Defaults below are source expressions; Field(...) includes schema/validation settings and default_factory creates a fresh value. Inherited declarations are included. Constructor behavior follows the model or dataclass shown in the bases/decorators.

Name	Type	Default
schema_version	Literal['cayu.memory_retrieval_corpus.v1']	required
corpus_revision	str	required
origin	Literal['hermetic_public', 'external_private']	required
entries	tuple[MemoryRetrievalCorpusEntry, ...]	required
cases	tuple[MemoryRetrievalCase, ...]	required
id_probes	tuple[MemoryRetrievalIdProbe, ...]	()

validate_revision (method)

MemoryRetrievalCorpus.validate_revision(value: str) -> str

validate_references (method)

MemoryRetrievalCorpus.validate_references() -> MemoryRetrievalCorpus

Release source · Runtime guarantees · Capability guides

MemoryRetrievalCorpusEntry

from cayu.evals import MemoryRetrievalCorpusEntry

Bases: BaseModel.

Fields and attributes

Defaults below are source expressions; Field(...) includes schema/validation settings and default_factory creates a fresh value. Inherited declarations are included. Constructor behavior follows the model or dataclass shown in the bases/decorators.

Name	Type	Default
id	str	required
text	str	required
namespace	str	required
labels	dict[str, str]	Field(default_factory=dict)
visibility	KnowledgeVisibility	KnowledgeVisibility.GLOBAL
status	KnowledgeStatus	KnowledgeStatus.ACTIVE
source_type	str | None	None
source_id	str | None	None
language	str	required
expires_at	datetime | None	None

validate_identity (method)

MemoryRetrievalCorpusEntry.validate_identity(value: str, info) -> str

validate_text (method)

MemoryRetrievalCorpusEntry.validate_text(value: str) -> str

validate_optional_identity (method)

MemoryRetrievalCorpusEntry.validate_optional_identity(value: str | None, info) -> str | None

copy_labels (method)

MemoryRetrievalCorpusEntry.copy_labels(value) -> dict[str, str]

to_entry (method)

MemoryRetrievalCorpusEntry.to_entry() -> KnowledgeEntry

Release source · Runtime guarantees · Capability guides

MemoryRetrievalIdProbe

from cayu.evals import MemoryRetrievalIdProbe

Bases: BaseModel.

Fields and attributes

Defaults below are source expressions; Field(...) includes schema/validation settings and default_factory creates a fresh value. Inherited declarations are included. Constructor behavior follows the model or dataclass shown in the bases/decorators.

Name	Type	Default
id	str	required
access	MemoryRetrievalAccessSpec	required
forbidden_entry_ids	tuple[str, ...]	required

validate_probe (method)

MemoryRetrievalIdProbe.validate_probe() -> MemoryRetrievalIdProbe

Release source · Runtime guarantees · Capability guides

MemoryRetrievalLanguageSlice

from cayu.evals import MemoryRetrievalLanguageSlice

Bases: BaseModel.

Fields and attributes

Defaults below are source expressions; Field(...) includes schema/validation settings and default_factory creates a fresh value. Inherited declarations are included. Constructor behavior follows the model or dataclass shown in the bases/decorators.

Name	Type	Default
language	str	required
case_count	int	required
recall_at_k	float	required
mean_reciprocal_rank	float	required
mean_ndcg_at_k	float	required

Release source · Runtime guarantees · Capability guides

run_memory_retrieval_baseline

from cayu.evals import run_memory_retrieval_baseline

Seed an empty store and measure the current knowledge-only memory baseline.

async run_memory_retrieval_baseline(corpus: MemoryRetrievalCorpus, store: KnowledgeStore, *, backend: str, search_mode: KnowledgeSearchMode=KnowledgeSearchMode.KEYWORD, embedding_identity: str | None=None, reranker_identity: str | None=None, configuration: dict[str, Any] | None=None) -> MemoryRetrievalBaselineResult

Release source · Runtime guarantees · Capability guides