Test the right layer

Use a small test at the layer that owns the claim. Generated projects include a credential-free suite and a scripted eval so you can change the agent before configuring a live provider.

uv run --no-sync pytest -q
uv run --no-sync cayu eval run

Four useful layers

Layer	Exercise	What it proves
Domain/tool unit	Known input directly to a tool or domain function	Schema and deterministic output
Scripted runtime	Public Cayu APIs with a scripted provider	Dispatch, effects, events and terminal behavior
Recovery integration	Separate processes using durable state	Ownership, continuation and mutation receipts
Live-model evaluation	A fresh run against a configured provider	Candidate behavior for those cases and settings

For a write-capable tool, assert the actual record or file change. For a denial, assert the tool did not run and state stayed unchanged. For an approval pause, inspect the pending action and resolve its exact identity. Avoid asserting only that the final answer contains the word “approved.”

Read the complete outcome

An async event stream can report session.failed without a Python exception becoming your caller’s only signal. Consume the stream and inspect terminal state. For bounded scripts, run_to_completion gives a typed outcome with retained events.

Test timeouts, invalid provider output, rejected policy, user interruption and ambiguous external results. Keep assertions on stable domain facts rather than incidental phrasing or generated IDs.

Use the testing API and structured-output example for deterministic repair tests.