Cards

A card is a small, typed piece of UI the agent puts in the conversation: a comparison table, a chart, a weather forecast, a list of places, link previews, a choice to make. Cards are defined once, in the @codai/cards catalog (Zod 4), and rendered natively by every codai surface — desktop, web console, phone and watch. Surfaces that cannot draw a card (Telegram, email, text-to-speech, the model's own history) use the card's text twin, so nothing is lost.

How a card travelsCopy Anchor Link

1. The agent calls one of the card tools (show_card or a typed shortcut).
2. The host validates the card against the schema, fills in the text twins and renders it. An invalid card is refused with a precise error (blocks.0.rows: …) so the model can repair its call.
3. The model only gets back a short text summary (fallback.text), never the full card, so it does not repeat the content in its reply.

As a tolerant fallback, a model that cannot call tools may emit the same JSON in a fenced block with the language codai-card; hosts render it like a tool-emitted card.

Live cardsCopy Anchor Link

A card with live: { key } can be updated after it is shown. The update is a session event card_update with { cardId, patch }: cardId is the card's live.key and patch is a JSON Merge Patch (RFC 7396) over the card — objects merge, null removes a field, and arrays such as blocks and actions are replaced whole. The patched card is validated again (size cap included) and its text twins are recomputed; an update that changes the key or produces an invalid card is ignored and the previous card stays.

Every surface redraws the card where it first appeared — no new message, no flicker — and screen readers hear one short status line per change. The watch's last card follows the update.

{ "kind": "card_update", "payload": { "cardId": "job:12", "patch": { "subtitle": "Background job · Done · 42s", "actions": [] } } }

Live run cardCopy Anchor Link

When the agent works for a while (three or more tool steps, or when it waits for your approval) and for every background job it starts, desktop and phone show a live run card: the recent steps, elapsed time, cost so far and, while it waits, the approval itself. Pause, Resume and Stop act on the run directly (they never go through the model); Allow / Deny answers the pending permission. In the web console and on another device's session page, Stop, Allow and Deny are sent as session controls; pause and resume need the device that runs the agent.

The card envelopeCopy Anchor Link

The full JSON Schema is published at packages/cards/schemas/cards.schema.json (generated from the Zod catalog; a drift test keeps them in sync, and the Kotlin types used by phone and watch are generated from the same source).

Block typesCopy Anchor Link

Every URL inside a card must be https://; images may also be data:image/* URIs. A small screen (the watch) declares the blocks it can draw; anything else is replaced by a text block built from its markdown twin, and long cards are cut to a few blocks plus a spoken summary.

ActionsCopy Anchor Link

Action ids must be unique within a card; style is primary, secondary or danger, and confirm asks before running.

Fallbacks, accessibility and provenanceCopy Anchor Link

• Text twins: when the agent omits fallback, the host builds fallback.text (markdown) and fallback.speak (one or two sentences) from the blocks. These are what the model sees and what voice surfaces read aloud.
• Accessibility: a11y.label defaults to the title (or the first block), a11y.summary to the spoken twin. Charts require a plain-language summary; gallery images require alt.
• Provenance: cards built from gateway data carry their source, attribution and fetch time, shown as a badge on the card.

Size capCopy Anchor Link

A card is at most 32 KB of JSON after the text twins are filled in. A larger card is refused with card exceeds 32768 bytes; shorten the data or split it into several cards.

Inline data:image/* payloads have their own 256 KB budget, and a visual block's source has its own 64 KB budget; neither counts towards the 32 KB. A card holds at most one visual block.

SandboxCopy Anchor Link

A visual block runs in a sandboxed frame with an opaque origin (sandbox="allow-scripts", never allow-same-origin): it cannot read the app, your cookies or storage, and its Content-Security-Policy allows no network at all (default-src 'none', inline scripts and styles, data: images and fonts only). Sources that reference http(s):// or // URLs, contain <base> or a meta refresh, or an svg that does not start with <svg are refused with an error the agent can fix. Desktop serves the document once from the app's own mcpapp: scheme with the policy as a header; the web console uses a srcdoc frame; the phone uses a hardened WebView with no bridge to the app. The watch, the minimal cards mode and text-only surfaces show the summary instead.

MapsCopy Anchor Link

A map block renders the same way everywhere, and the Google Maps key never leaves the gateway:

• Desktop and web console: a static map image from GET /v1/cards/static-map, fetched with your own sign-in (the console goes through a same-origin server proxy). Pins are lettered A–T and the list below the map uses the same letters. A grey placeholder shows while the image loads. Each map is fetched once per view (each fetch is billed $0.002).
• Phone: an interactive Google map (lite mode: markers and route, it scrolls with the chat) when the app was built with a Maps key; otherwise the same gateway static image.
• Watch: the first pin labels as text.
• Always: one row per pin that opens it in your maps app, plus Open in Google Maps and Open in OpenStreetMap. The image's alt text names the pins.
• No image (offline, quota spent, maps not configured): you get the pins with their coordinates and a note that the map image is unavailable.

An interactive Google Maps JS view on desktop and web is not enabled yet: it is waiting for a referrer-restricted JS key. Hosts can plug one in through renderInteractiveMap.

Agent toolsCopy Anchor Link

Every data card shows a provenance badge in its footer: the source (linked when it has a page), how old the data is ("updated 2 hours ago", exact date on hover) and the licence (for example MET Norway, CC BY 4.0). The watch shows a compact source · 2h.

The six local tools (ask_approval to show_changes) need no gateway call: the card is built on the device by a pure builder (buildLocalCard in @codai/cards, Kotlin twin LocalCards), with labels in your app language (English or Romanian).

The agent is told not to repeat a card's content in its text reply; at most one sentence of commentary.

PreferenceCopy Anchor Link

Each user picks how much the agent uses cards: Settings → Chat → Rich cards on the desktop, Settings → Agent → Rich cards on the phone (the watch follows the phone).

The preference is read when a chat starts, so the tool list and system prompt stay the same for the whole chat (the prompt cache keeps working); a change applies to new chats, while the way cards look changes at once. The web console has no chat-rendering preferences and always shows full cards.

Data endpointsCopy Anchor Link

The data tools call gateway endpoints that return { block, provenance }, where block is already a valid card block that can be dropped into show_card unchanged (or null when nothing matched). You can call them directly with any codai API key; the full reference is on the Cards API page.

Parcel tracking: Sameday is tracked live through its public tracking endpoint. FAN Courier, Cargus and DPD only return parcels of a customer account, so their tracking is live only on gateways that hold that carrier's contract credentials; elsewhere the card says live tracking is unavailable and links to the carrier's own tracking page. The easybox code from the carrier's SMS is never sent to the gateway: the app draws it as a QR.

Tap-to-play: transcript links open https://hub.codai.ro/transcripts/<id>?t=<seconds>; the transcript page scrolls to and highlights the utterance at that second.

Rendering a card to an imageCopy Anchor Link

POST /v1/cards/render { block, format?, width?, height?, theme?, scale? } turns a chart or qr block into an image for surfaces that cannot draw it — watch tiles and complications, phone notifications, email, Telegram. The answer is the image itself: image/png (default) or image/svg+xml, with the pixel size in x-codai-render-size. Charts are laid out with Vega-Lite and rasterised on the gateway with a bundled font (no network, no system fonts); QR codes stay black on white in both themes. width and height are 64–1200 CSS px (defaults 480 × 300; a QR is square), scale 1–3 multiplies the PNG pixels.

Not supported yet: diagram (Mermaid) and math (KaTeX) blocks — and every other block type — answer 422 render_unsupported; use the block's text twin instead. Rendering is local compute, so it is free.

LimitsCopy Anchor Link

super_admin accounts are exempt. A spent quota answers 429. All endpoints also pass your key's normal rate limit and your rules: unfurl is gated as browser_fetch, so a rule that denies page fetching applies here too, and render as cards_render. When maps are not set up on a gateway, the maps endpoints answer 503 maps_not_configured — fall back to a text answer.

BillingCopy Anchor Link

Paid Google calls are billed to your usage at Google's list price, as a row in the same session as the conversation: Places text or nearby search $0.032, geocoding $0.005, static map $0.002, route $0.010 when driving (traffic-aware) or $0.005 on foot, by bike or transit per call. The charged cost is in provenance.costMicroUsd (USD micros) and the x-codai-cost-micro-usd response header. A failed upstream call is neither billed nor counted. Weather, exchange rates, link previews, parcel tracking, e-Factura status and renders are free.

From the SDKs and MCPCopy Anchor Link

• TypeScript: client.cards.weather({ lat, lng }), .places({ q }), .geocode({ address }), .staticMap({ center }) (PNG bytes), .fx(), .route({ from, to }), .track({ awb }), .efactura(), .unfurl(url), .render({ block, format }) (image bytes) — see the TypeScript SDK.
• Python: client.cards.weather(lat, lng), .places(q=…), .geocode(address), .static_map(center=…), .fx(from_=…, to=…), .route(from_=…, to=…), .track(awb), .efactura(), .unfurl(url), .render(block, format=…) (image bytes) — see the Python SDK.
• MCP: the codai MCP server exposes cards_weather, cards_places, cards_fx, cards_unfurl, cards_product, cards_route, cards_track and cards_efactura; each returns { block, provenance, card } as structuredContent (card is the block as a ready one-block card, null when nothing matched) and the markdown twin as text. Render is not an MCP tool: MCP clients receive the card JSON and draw it themselves, or use the MCP App below.

import { Codai } from 'codai-sdk';

const codai = new Codai({ apiKey: process.env.CODAI_API_KEY! });
const { block, provenance } = await codai.cards.weather({ lat: 44.43, lng: 26.1, name: 'București' });
// block is a ready `weather` card block; provenance.source === 'met_norway'

Cards as an MCP AppCopy Anchor Link

The codai MCP server also ships the cards as an MCP App (MCP Apps 2026-01-26). Every cards_* tool carries _meta.ui.resourceUri: "ui://codai/card", and the server lists that resource (text/html;profile=mcp-app). Hosts that support MCP Apps (VS Code, Claude, ChatGPT, Cursor, the codai desktop and phone apps) load it in a sandboxed frame and draw the tool's card with the same renderer the codai apps use, in the host's theme and language. Hosts without MCP Apps ignore the field and show the markdown twin.

• The view is one self-contained HTML file: it makes no network requests and declares an empty CSP domain list (connect, resources, frames, base URI).
• It is small (about 200 KB, 65 KB compressed) and both resources/list and resources/read carry its _meta["codai/sha256"], so a host can cache it per version.
• Card buttons go back through the host: open a link (ui/open-link, the host confirms), send a prompt (ui/message), copy a value. The view never calls tools on its own.
• In VS Code the setting is chat.mcp.apps.enabled.

MCP Apps from other servers in codaiCopy Anchor Link

The desktop and phone apps also host MCP Apps from any MCP server you connect: when a tool declares a ui:// resource, the result is shown inline under the tool step in a sandboxed frame.

• Isolation: the frame has its own opaque origin (sandbox="allow-scripts", never same-origin). It cannot reach the codai app, your keys or the device bridge.
• Network: the strict CSP comes from the server's _meta.ui.csp. Only https/wss origins are accepted, never localhost or IP addresses, and the default is no network at all.
• Tool calls from the app: allowed only for the same server's tools, and only tools the server makes available to apps. Read-only tools run directly. Anything else follows your agent settings: in Autopilot (desktop) or Allow all / Autopilot (phone) it runs without asking; otherwise you get Allow / Always allow / Deny. "Always allow" is remembered for that tool the same way as for the agent's own calls (desktop: for the current chat; phone: in Settings → Agent, where you can reset it). The same-server rule, the app visibility rule and the cap of 10 calls per 10 seconds apply in every mode.
• Context for the agent (ui/update-model-context): an app can tell the agent what you are looking at (for example the item you selected). Only the latest update of each app is kept; the agent sees it once, at its next step, marked as untrusted data from a third-party app. An update never starts a turn by itself.
• Messages from the app (ui/message): an app can propose a message to send as you. You see the exact text and choose Send or Don't send; at most one proposal every 5 seconds. A sent message goes to the chat like one you typed (as a note if the agent is working).
• Permissions: links open only for https URLs. The camera, microphone and location are never granted.
• Web console: the session view in the console does not run the app. It shows the tool's result instead (text, images and the structured data behind a toggle) with a note to open the chat in the desktop or phone app for the interactive view.

A2UICopy Anchor Link

codai reads A2UI v0.9 (the current stable spec) and v1.0 (the release candidate) and turns it into a codai card. When an MCP tool result carries A2UI messages, the desktop and phone apps draw the mapped card under the tool step with the regular card renderer, so no extra renderer is needed.

A result counts as A2UI when it has an embedded resource with mimeType: "application/a2ui+json" (v1.0) or "application/json+a2ui" (v0.9) (its text is a JSON array of messages, one message or JSONL) or a structuredContent.a2ui array. The adapter replays the four server-to-client messages (createSurface, updateComponents, updateDataModel, deleteSurface), keeps one data model per surface and resolves { "path": … } bindings: absolute JSON Pointers, plus relative paths inside list templates. Each surface that is still alive and has a root component becomes one card. If the result had no text block, the model gets the card's text twin instead of raw JSON.

A surface follows the version of the message that created it. In a v1.0 surface only { "@path": … } and { "@call": … } are bindings; an object with a plain path or call key is shown as literal data, @@ keys lose one @, and unknown single-@ keys are rejected with a warning. v1.0 also lets createSurface carry components and dataModel in one message, deletes a data key when updateDataModel sets value to null, resolves @index inside list templates, uses Action.event.userMessage as the start of the button's prompt and shows Video.posterUrl as the thumbnail. The v1.0 function-call messages (callRendererFunction, agentFunctionResponse) are ignored: a card only displays.

For developers: @codai/cards/a2ui exports a2uiToCards(messages), which returns { cards, warnings }, and isA2uiPayload(x). The Kotlin twin is ro.codai.cards.A2ui. Both are checked against the same golden cases in packages/cards/fixtures/a2ui/.

Limitations. Inputs cannot be edited in a card: their values are shown, and a button sends the values the server already knows. Local functions other than openUrl, validation checks, themes and sendDataModel are ignored. Of the formatting functions, formatString, formatNumber, formatCurrency and pluralize are supported. A card holds at most 24 blocks, 6 actions and 32 KB, so anything over those caps is truncated. Every drop is reported as a warning, and the adapter never fails on unknown input.

v1.0. When A2UI v1.0 is final, the adapter will accept its envelope next to v0.9 (one version switch). The mapping table above stays the same unless the basic catalog changes. The decision record is docs/decisions/2026-10-14-a2ui-adapter.md.