# Media operations (/docs/gateway/media-ops)

codai Studio media operations — remove backgrounds, upscale, inpaint, separate stems, denoise, master, convert audio to MIDI and generate music or sound effects from your assets.

**Media operations** turn one Studio [asset](./assets.mdx) into new ones: cut out a subject, upscale a still, repaint a masked area, split a song into stems, clean up a voice, master a mix, convert a melody to MIDI or generate a song or a sound effect. Every op takes asset ids and strict per-op parameters, and every output is stored as a new asset in your library. The public model id is `codai-studio-media`.

| Method | Path | Result |
| --- | --- | --- |
| `GET` | `/v1/studio/media-ops` | The op catalogue: engine, mode, inputs, `available`, a list quote (free) |
| `POST` | `/v1/studio/media-ops/{op}/quote` | The exact price of a request; nothing is held or billed |
| `POST` | `/v1/studio/media-ops/{op}` | `200` with the outputs (sync ops) or `202` with a job (async ops) |
| `GET` | `/v1/studio/media-ops/jobs/{id}` | An async job; a finished one is settled when you read it |

`{op}` is the dotted op id, for example `image.remove_background` or `audio.separate_stems`.

## The ops [#the-ops]

| Op | Inputs | Output | Mode | Engine |
| --- | --- | --- | --- | --- |
| `image.remove_background` | 1 image | Cut-out PNG / WebP with transparency | sync | BiRefNet |
| `image.upscale` | 1 image | Image ×2 or ×4 (at most 64 MP) | sync | Real-ESRGAN x4plus |
| `image.inpaint` | image + PNG mask | The image with the white mask area repainted from `prompt` | sync | Gemini image edit |
| `image.erase` | image + PNG mask | The image with the masked object removed | sync | Gemini image edit |
| `image.outpaint` | 1 image | The scene extended to a new `aspect_ratio` | sync | Gemini image edit |
| `audio.separate_stems` | 1 audio | 4 stems (vocals, drums, bass, other) or 2 (vocals, accompaniment) | async | Demucs (htdemucs) |
| `audio.denoise` | 1 audio | The same audio with background noise removed | async | DeepFilterNet 3 |
| `audio.to_midi` | 1 audio | A Standard MIDI File (`kind: "midi"`) | async | Basic Pitch |
| `audio.master` | 1 audio | A loudness-normalised master (WAV or MP3) | async | ffmpeg loudnorm, 2 passes + limiter |
| `audio.analyze` | 1 audio | JSON only: BPM, key, loudness, peak | sync | librosa + pyloudnorm |
| `music.generate_ace` | 0–1 audio | A song from a style prompt and optional lyrics | async | ACE-Step 1.5 |
| `music.repaint` | 1 audio | The song with one section replaced | async | ACE-Step 1.5 |
| `music.extend` | 1 audio | The song continued by `extend_seconds` | async | ACE-Step 1.5 |
| `music.cover` | 1 audio | Your own song restyled | async | ACE-Step 1.5 |
| `sfx.generate` | none | A sound effect from a prompt | async | ACE-Step 1.5 |

Image ops accept `image`, `upload` and `frame` assets (PNG, JPEG or WebP); audio ops accept `music`, `speech`, `sfx` and `upload` assets. For `image.inpaint` and `image.erase` the **second** asset is a black-and-white PNG mask of the first: white is edited, black is kept. The mask is sent to the model as a reference image, so edits follow it closely but not pixel-exactly (the catalogue entry carries a `note` saying so).

### Parameters [#parameters]

Parameters are validated strictly per op — an unknown key is a `400`.

| Op | Params (defaults) |
| --- | --- |
| `image.remove_background` | `format`: `png` \| `webp` (`png`) |
| `image.upscale` | `scale`: `2` \| `4` (`4`) |
| `image.inpaint` | `prompt` (required), `model`, `image_size`: `1K` \| `2K` \| `4K` |
| `image.erase` | `prompt` (optional hint), `model`, `image_size` |
| `image.outpaint` | `aspect_ratio` (required: `1:1`, `2:3`, `3:2`, `3:4`, `4:3`, `4:5`, `5:4`, `9:16`, `16:9`, `21:9`, `9:21`), `prompt`, `model`, `image_size` |
| `audio.separate_stems` | `stems`: `2` \| `4` (`4`) |
| `audio.denoise` | `strength`: 0–1 (`1`) |
| `audio.to_midi` | `onset_threshold`: 0.05–0.95 (`0.5`), `min_note_ms`: 10–1000 (`58`) |
| `audio.master` | `target_lufs`: `-14` \| `-16` \| `-23` (`-14`), `true_peak_db`: −3…−0.1 (`-1`), `format`: `wav` \| `mp3` (`wav`) |
| `audio.analyze` | none |
| `music.generate_ace` | `prompt` (required, ≤ 1000 chars), `lyrics` (≤ 4000), `duration_seconds`: 10–240 (`60`), `bpm`: 40–220, `key`, `language` (`en`, `ro`, `es`, `fr`, `de`, `it`, `pt`, `ja`, `ko`, `zh`), `seed` |
| `music.repaint` | `prompt`, `lyrics`, `start_seconds`, `end_seconds` (after the start, inside the song), `seed` |
| `music.extend` | `prompt`, `lyrics`, `extend_seconds`: 5–120, `seed` |
| `music.cover` | `prompt`, `lyrics`, `strength`: 0–1 (`0.8`), `seed` |
| `sfx.generate` | `prompt` (required), `duration_seconds`: 1–30 (`5`), `seed` |

## Running an op [#running-an-op]

```bash
curl https://ai.codai.ro/v1/studio/media-ops/image.remove_background \
  -H "Authorization: Bearer $CODAI_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: cutout-42" \
  -d '{ "asset_ids": ["3f2b8c1e-0000-4000-8000-000000000001"], "params": { "format": "png" } }'
```

A **sync** op answers `200`:

```json
{
  "object": "media_op.result",
  "op": "image.remove_background",
  "status": "ready",
  "outputs": [{ "asset_id": "…", "kind": "image", "mime_type": "image/png", "url": "https://…" }],
  "analysis": null,
  "cost_micro_usd": 2640
}
```

`audio.analyze` stores nothing and returns `analysis`: `{ duration_seconds, bpm, key, key_confidence, integrated_lufs, sample_peak_dbfs }`.

## Async jobs [#async-jobs]

Stems, denoise, MIDI, mastering and every music or sound-effect op run as **jobs**. The run answers `202` with `{ "object": "media_op.job", "id", "status": "pending", … }`. Poll `GET /v1/studio/media-ops/jobs/{id}` until `status` is `ready` (with `outputs`, one asset per stem for separation) or `failed` (with `error`: `timeout`, `input_refused`, `dispatch_failed`, `store_failed` or an engine message). A job that has not produced a result after 20 minutes fails with `timeout`. Reading a finished job stores its outputs and bills it once; the background sweeper does the same when nobody polls. At most **3 async jobs** run per account at once — a fourth answers `429` with `Retry-After`.

## Pricing [#pricing]

Every op has a **quote** — the price ceiling — that is held on your wallet before anything runs; when the wallet cannot cover it the answer is `402 insufficient_funds` and nothing starts. After the op the **measured** cost is billed once and the rest of the hold is released; a failed op releases the whole hold.

- **GPU / CPU ops** (everything except the Gemini image edits) are priced by instance time on the codai Studio media service: $0.95 per hour, at least 5 seconds per op, estimated from the input length (for example `audio.separate_stems` ≈ 15 s + 0.5 s per input second). You pay the seconds the engine reports, never more than the quote.
- **`image.inpaint` / `erase` / `outpaint`** are billed like an image edit of the chosen `model` and `image_size`.

`GET /v1/studio/media-ops` lists a reference quote per op (default params, 60 s of audio or a 1024×1024 image) as `quote_micro_usd` and the wallet amount as `quote_micro_eur`; `POST …/{op}/quote` prices your real request with the same validation as the run.

## Errors [#errors]

| Status | When |
| --- | --- |
| `400` | Bad body or params, the wrong number or kind of assets, a mask that is not PNG, an upscale over 64 MP, `end_seconds` before `start_seconds` |
| `402 insufficient_funds` | The quote does not fit in your wallet |
| `404` | Unknown op id, or an asset that does not exist, is not ready or is not yours |
| `429` | Three async jobs already running, or your key's rate limit |
| `502` | The engine failed |
| `503 media_ops_unavailable` | The op's lane is not configured on this gateway (`available: false` in the catalogue); nothing is held or billed |
| `503 assets_unavailable` | Asset storage is not configured |

## AI marking and lineage [#ai-marking-and-lineage]

Every output is marked as AI-generated before it is stored: C2PA / XMP on images, RIFF INFO or ID3 tags on audio, plus the `ai-generated` object metadata. Each output also records where it came from — `parent_asset_id` (the first input), `op` and `op_inputs` (`asset_ids` + the validated `params`) — so you can always trace a stem, a master or a cut-out back to its source. Generate covers only from music you own or have the rights to.

## SDKs [#sdks]

```ts
const job = await codai.mediaOps.run('audio.separate_stems', { assetIds: [songId], params: { stems: 4 } });
const done = await codai.mediaOps.wait(job.id); // or mediaOps.runAndWait(...)
```

```python
done = client.media_ops.run_and_wait("audio.master", [mix_id], {"target_lufs": -14})
```

TypeScript: `mediaOps.list / quote / run / job / wait / runAndWait`. Python: `media_ops.list / quote / run / job / wait / run_and_wait`. Songs and podcasts built from these outputs are edited as [audio creatives](./audio-creatives.mdx) and rendered to a mastered mix with `creatives.renderAudio`; lyrics come from the [lyrics writer](./lyrics-jam.mdx).