Audio creatives

An audio creative (kind: "audio") is a creative whose document is a multitrack audio timeline: a song or a podcast episode. It uses the same API as every other creative — create, read, edit with ops, versions, revert, fork, lint, share — and the same document, with the timeline in doc.audio. Creating, reading and editing it is free; the files you put on it come from media operations, music and speech generation or your own uploads.

curl https://ai.codai.ro/v1/studio/creatives \
  -H "Authorization: Bearer $CODAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "kind": "audio", "title": "Episode 12" }'

The document modelCopy Anchor Link

An audio document has one format, stereo (48 kHz mixdown; its 1920×1080 canvas is only used by an optional cover / visualiser scene), at most one scene (captions only) and an optional fps. Everything that matters lives in doc.audio (schema CreativeAudio):

Milliseconds are the source of truth; bars and beats are derived from meta.bpm and meta.timeSignature.

Tracks. role is one of voice, music, sfx, drums, bass, keys, guitar, vocal, lead, pad, fx, bus, podcast_host, podcast_guest. kind is audio (default), midi (holds midiClips only) or bus (no clips; other tracks reach it through sends). fx is a chain of up to 16 effects (eq, compressor, reverb, delay, limiter, gate, deesser, pitch, chorus, saturation) with numeric params and bypass; automation envelopes target gainDb, pan or fx.<fxId>.<param>.

Clips. { id, assetId, startMs, inMs?, outMs?, gainDb?, pan?, fadeInMs?, fadeOutMs?, fadeCurve?, stretch?, pitchSemitones?, loop?, duck?, color?, generation?, locked? }. inMs / outMs are offsets into the source asset: a clip ends on the timeline at startMs + (outMs − inMs) × stretch. fadeCurve is linear, exp or s; stretch (0.25–4) keeps the pitch; generation records how a generated clip was made (public codai provider and model, prompt, seed, parent clip, op).

Ducking. Music tracks of an audio creative duck automatically under voice, vocal, podcast_host and podcast_guest clips (150 ms attack, 400 ms release, 12 dB); any clip can opt in or out with duck.

Editing with opsCopy Anchor Link

Audio creatives take every common op (setMeta, setAsset, setAudioClip, removeAudioClip, setLocale…) plus 16 audio ops, sent to POST /v1/studio/creatives/{id}/ops like any other edit:

setMeta also accepts audio (a patch of meta) and setAudioClip a trackId. In every patch null removes an optional field, so each edit has an exact inverse and can be reverted.

{
  "base_seq": 4,
  "ops": [
    { "op": "addSection", "section": { "id": "chorus1", "kind": "chorus", "startMs": 32000, "durationMs": 16000 } },
    { "op": "setFades", "clipId": "c7", "fadeInMs": 500, "fadeCurve": "s" },
    { "op": "setMaster", "master": { "targetLufs": -14, "truePeakDb": -1, "limiter": true } }
  ]
}

Mastering targetsCopy Anchor Link

meta.loudnessTarget takes a preset or a number of integrated LUFS (−40…−5):

master.targetLufs overrides the preset; the true peak defaults to −1 dBTP. To master a mixdown file, run audio.master with target_lufs −14, −16 or −23.

LintCopy Anchor Link

Lint adds five audio rules: clipping_risk (summed clip gain above 0 dB), missing_lyrics (a verse / pre-chorus / chorus / bridge without lyrics when the song has a vocal track), missing_loudness_target, audio_overlap (overlapping clips on one track) and audio_out_of_range (a clip or note outside its source or the timeline).

Media operations for audioCopy Anchor Link

The building blocks of a song or an episode come from media operations: music.generate_ace, music.repaint, music.extend and music.cover for songs, sfx.generate for effects, audio.separate_stems to split a mix into vocal / drum / bass / other tracks, audio.denoise to clean a voice, audio.to_midi to turn a melody into a MIDI clip, audio.analyze to read BPM, key and loudness into meta, and audio.master for the final loudness. Each output is a new asset you place on a track with setAudioClip; its lineage (parent_asset_id, op) stays on the asset.

Rendering and exportsCopy Anchor Link

POST /v1/studio/creatives/{id}/render on an audio creative renders the mixdown: every track, clip, MIDI note, effect, send and automation is mixed with ffmpeg, mastered to the loudness target (two-pass loudnorm + true-peak limiter) and exported. The answer is 202 with the render id; poll GET /v1/studio/renders/{id} like any render.

curl https://ai.codai.ro/v1/studio/creatives/$ID/render \
  -H "Authorization: Bearer $CODAI_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: mix-12" \
  -d '{ "formats": ["wav", "mp3"], "stems": true, "loudness": "podcast" }'

The 202 lists the planned outputs (kind: mix / stem / lrc, format, track_id), the resolved target_lufs and true_peak_db, duration_seconds and quote_micro_usd. When the render has succeeded, its data[] holds one asset per output (1-hour signed url, output, format, track_id) and qa the measured lufs, true_peak_db and clipping. Every file carries the AI mark inside it (WAV RIFF ICMT, MP3 ID3v2.4, FLAC Vorbis comment) and its lineage points at the render.

Price: $0.001 per output second ($0.06 per minute), minimum $0.01; stems: true adds 50 %. The quote is held on your wallet before dispatch (402 insufficient_funds when short) and the measured length is billed once. At most 20 minutes per render (400 too_long); a document with nothing audible is 400 empty_audio; every referenced asset must be yours and ready (400 asset_not_found).

Lyrics for the song come from the lyrics writer, and karaoke timing from lyrics alignment.

SDKsCopy Anchor Link

TypeScript: creatives.create({ kind: 'audio' }), creatives.edit(id, { baseSeq, ops }) (types CreativeAudio, CreativeOp), creatives.renderAudio(id, { formats, stems, lrc, loudness }) (types CreativeAudioRenderRequest, CreativeAudioRenderAccepted, StudioAudioRender), plus mediaOps and lyrics. Python: client.creatives.create(...), client.creatives.edit(...), client.creatives.render_audio(id, formats=, stems=, lrc=, loudness=), client.media_ops and client.lyrics.