# Lyrics and live jam (/docs/gateway/lyrics-jam)

codai Studio songwriting tools — write lyrics with syllable and rhyme checks, align them to a vocal clip for karaoke timing and LRC, and jam live with generated music over a WebSocket.

Three tools sit next to [audio creatives](./audio-creatives.mdx): a **lyrics writer** that checks meter and rhyme, **lyrics alignment** that times the lines to a vocal clip, and a **live jam** that streams generated music you steer in real time.

| Method | Path | Result | Billing |
| --- | --- | --- | --- |
| `POST` | `/v1/studio/lyrics` | Lyrics with a syllable and rhyme report | Tokens (`codai::lyrics`) |
| `POST` | `/v1/studio/lyrics/align` | Timed lines + LRC, applied as `setLyrics` | Free with cached words, else speech-to-text (`codai::lyrics-align`) |
| `WS` | `/v1/studio/jam` | Live PCM audio steered by prompts | Per started minute (`codai::jam`) |

## Writing lyrics [#writing-lyrics]

```bash
curl https://ai.codai.ro/v1/studio/lyrics \
  -H "Authorization: Bearer $CODAI_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: song-7" \
  -d '{
    "brief": "a summer night by the sea, two friends who meet again",
    "language": "ro",
    "genre": "pop",
    "structure": ["verse", "chorus", "verse", "chorus"],
    "syllables_per_line": 8,
    "rhyme_scheme": "AABB",
    "lines_per_section": 4
  }'
```

| Field | Default | Notes |
| --- | --- | --- |
| `brief` | required | 3–4000 characters; never logged |
| `language` | required | `ro` or `en`; Romanian uses correct diacritics (ș ț ă î â) |
| `genre`, `mood`, `title` | — | Free text (100 / 100 / 200 characters) |
| `structure` | verse, chorus, verse, chorus, bridge, chorus | 1–12 of `intro`, `verse`, `pre_chorus`, `chorus`, `bridge`, `outro`, `instrumental`, `custom` |
| `syllables_per_line` | — | 3–20; checked with a tolerance of ±1 |
| `rhyme_scheme` | `free` | `AABB`, `ABAB` or `free` |
| `lines_per_section` | `4` | 2–8 |

The writer runs **one** bounded `codai` turn with a forced tool, then checks every line with codai's own counters: syllables (Romanian rules handle diphthongs, triphthongs and hiatus; English counts vowel groups after dropping a silent -e / -es / -ed) and rhyme endings (the last word from its stressed vowel in Romanian, from its last vowel group in English, diacritics folded). When more than **30 %** of the lines miss — a syllable count outside the target ±1 or a line of an unrhymed pair — **one repair turn** names the problems and the better of the two answers is returned (`repaired: true` when the repair won).

```json
{
  "object": "studio.lyrics",
  "title": "Noapte de vară",
  "language": "ro",
  "rhyme_scheme": "AABB",
  "syllables_per_line": 8,
  "sections": [
    {
      "kind": "verse",
      "lines": [
        { "text": "Marea cântă lin sub lună", "syllables": 8, "rhyme": "una", "letter": "A", "off": false },
        { "text": "Ne-am regăsit iar împreună", "syllables": 8, "rhyme": "una", "letter": "A", "off": false }
      ]
    }
  ],
  "text": "[verse]\nMarea cântă lin sub lună\nNe-am regăsit iar împreună",
  "report": { "lines": 16, "off_lines": 1, "off_ratio": 0.063, "pairs": 8, "rhymed": 7, "repair_ratio": 0.125 },
  "repaired": false,
  "cost_micro_usd": "4210"
}
```

`text` is the whole song with one `[kind]` header per section — pass it as `lyrics_text` to the alignment call or as the `lyrics` param of [`music.generate_ace`](./media-ops.mdx).

**Billing.** The turns are token-metered under the public model `codai` (usage model `codai::lyrics`, prompt caching on). A small hold of 20 000 µUSD is placed on your wallet first (`402 insufficient_funds` when it does not fit) and settled with the real token cost. When the model cannot produce lyrics at all the answer is `422 no_lyrics`, and only the turns that ran are billed.

## Aligning lyrics to a clip (karaoke) [#aligning-lyrics-to-a-clip-karaoke]

Alignment times written lyrics against one clip of an audio (or video) creative and writes them to `doc.audio.lyrics` with a single `setLyrics` edit:

```bash
curl https://ai.codai.ro/v1/studio/lyrics/align \
  -H "Authorization: Bearer $CODAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "creative_id": "8a1c…",
    "clip_id": "vox1",
    "lyrics_text": "[verse]\nMarea cântă lin sub lună\nNe-am regăsit iar împreună",
    "language": "ro"
  }'
```

| Field | Default | Notes |
| --- | --- | --- |
| `creative_id`, `clip_id` | required | The clip's asset must be a ready `speech`, `music`, `video` or `upload` asset of yours |
| `lyrics_text` | required | Up to 5000 characters; `[verse]` / `[chorus]` header lines start sections |
| `language` | the document's `meta.language` | Transcription hint, `ro` or `en` |
| `apply` | `true` | `false` returns the timed lines and the LRC without editing the creative |
| `base_seq` | the head | Version the edit is based on; a conflicting edit is `409 stale` |
| `retranscribe` | `false` | Ignore the cached words and transcribe again (billed) |

The gateway takes the word timestamps of the clip's asset — from the cache that [captions](./creatives.mdx) and earlier alignments leave on the asset (**free**, `cached: true`), or from a fresh Whisper transcription of the clip (at most 25 MB) — maps them onto the timeline through the clip's trim, position and stretch, and matches lyric words to transcript words with a diacritics- and typo-tolerant alignment. Lines whose words were not heard are interpolated between their neighbours. The answer:

```json
{
  "object": "studio.lyrics_alignment",
  "creative_id": "8a1c…",
  "clip_id": "vox1",
  "applied": true,
  "seq": 12,
  "doc_hash": "…",
  "lines": [{ "startMs": 12040, "endMs": 14880, "text": "Marea cântă lin sub lună", "sectionId": "verse1" }],
  "lrc": "[ti:Noapte de vară]\n[00:12.04]Marea cântă lin sub lună\n…",
  "match_ratio": 0.94,
  "cached": false,
  "cost_micro_usd": "18000"
}
```

`match_ratio` is the share of lyric words matched to a heard word; below about 0.6 the timing is mostly interpolated — check the clip and the language. The LRC is also exported by the [audio render](./audio-creatives.mdx#rendering-and-exports) (`lrc: true`).

**Billing.** Cached words are free. A fresh transcription bills `codai::lyrics-align` at the speech-to-text rate — 6 000 µUSD per started minute of the clip — under a wallet hold. Alignment needs a transcription service outside the EU and is not available for EU data-residency accounts (`403 residency_unavailable`); when transcription is not configured the answer is `503 transcription_unavailable`.

## Live jam (WebSocket) [#live-jam-websocket]

`wss://ai.codai.ro/v1/studio/jam` opens a live music session on Lyria RealTime: you send weighted text prompts and a few musical controls, the gateway streams back continuous generated audio. OpenAPI cannot describe WebSockets, so the protocol is documented here.

### Connecting [#connecting]

| Client | How to authenticate |
| --- | --- |
| Server runtime | `Authorization: Bearer <API key or ephemeral token>` on the upgrade |
| Browser | Subprotocols `codai-jam` and `bearer.<ephemeral token>`; the server answers with `codai-jam` and never echoes the credential |
| Browser (fallback) | `?key=<ephemeral token>` — a raw long-lived key in the query string is refused with close `4401` |

Mint ephemeral tokens with [`POST /v1/tokens`](./embeddings-audio.mdx) (scope `realtime` or `live`). `?session_id=` is recorded on the usage row.

### Client → gateway (JSON text frames) [#client--gateway-json-text-frames]

| Message | Fields |
| --- | --- |
| `{ "type": "prompts", "prompts": [{ "text", "weight" }] }` | 1–8 prompts; `text` up to 200 characters; `weight` from −10 to 10, not 0 (negative steers away) |
| `{ "type": "config", "bpm"?, "density"?, "brightness"?, "scale"? }` | `bpm` 60–200, `density` and `brightness` 0–1, `scale` one of `C_MAJOR_A_MINOR` … `B_MAJOR_A_FLAT_MINOR` or `SCALE_UNSPECIFIED`; partial updates are merged into the last config, and a `bpm` or `scale` change resets the musical context |
| `{ "type": "play" }`, `pause`, `stop`, `reset` | Playback control |

Messages sent before the upstream is ready are queued. Anything else — invalid JSON, an unknown type, a binary frame — closes the session with `4400`.

### Gateway → client [#gateway--client]

- **Binary frames**: raw PCM16 little-endian, interleaved stereo, 48 000 Hz. Audio is dropped (not buffered) while your socket is more than 4 MB behind.
- `{ "type": "status", "state": "connecting" | "ready" | "playing" | "paused" | "stopped", "remaining_ms", "sample_rate"?, "channels"? }` — `ready` carries `sample_rate: 48000` and `channels: 2`.
- `{ "type": "filtered", "text", "reason" }` — the model's safety filter ignored a prompt.
- `{ "type": "error", "code", "message" }` — sent before a refusal or an upstream close.

### Close codes [#close-codes]

| Code | Meaning |
| --- | --- |
| `4400` | Invalid client message |
| `4401` | Missing or invalid credentials, or a raw key in the query string |
| `4402` | The 10-minute hold does not fit in your wallet (`insufficient_funds`) |
| `4403` | EU data-residency account, model not allowed for the key, or spend cap reached |
| `4408` | The 10-minute session limit was reached (`max_duration`) |
| `4409` | You already have a live jam open (`jam_busy`) — one per account |
| `4502` | The upstream music model failed or closed |
| `4503` | Live jam is not configured on this gateway (`jam_unavailable`) |

### Pricing [#pricing]

$0.03 per **started minute** of the open upstream session (usage model `codai::jam`, upstream `lyria-realtime-exp`). The 10-minute price ($0.30) is held when the session opens and settled with the minutes actually used when it closes; a session that never reaches the upstream bills nothing. Lyria RealTime is an experimental model, so this price may change. Prompts and audio are never logged.

## SDKs [#sdks]

```ts
const song = await codai.lyrics.write({ brief: 'o noapte de vară', language: 'ro', rhyme_scheme: 'AABB' });
const aligned = await codai.lyrics.align({ creative_id: id, clip_id: 'vox1', lyrics_text: song.text });

const jam = codai.jam.connect({ token }); // browser: ephemeral token
jam.on('audio', (ev) => player.push(new Int16Array(ev.data)));
jam.on('close', (ev) => console.log('closed', ev.code));
jam.prompts([{ text: 'warm lo-fi piano', weight: 1 }]);
jam.config({ bpm: 90, density: 0.4 });
jam.play();
```

```python
song = client.lyrics.write("o noapte de vară", "ro", rhyme_scheme="AABB", syllables_per_line=8)
aligned = client.lyrics.align(creative_id, "vox1", song["text"])
```

TypeScript: `lyrics.write / align`, `jam.connect` (`JamSession`: `prompts`, `config`, `play`, `pause`, `stop`, `reset`, `close`, `on`; `JAM_CLOSE_CODES`). Python: `lyrics.write / align`; the Python SDK has no WebSocket helper — use any WebSocket client with the protocol above.