Control an iPhone remotely
See an iPhone's screen live and drive it from the hub, the Android app or any codai agent, through the codai iPhone bridge — no jailbreak, same account only.
The codai iPhone bridge runs on the Windows PC your iPhone is paired with and drives the phone over USB or Wi-Fi (pymobiledevice3, developer image, WebDriverAgent-free tools). With remote access switched on, the bridge also connects to codai by itself — no desktop app, no inbound port — so you can control the phone from anywhere:
- Control goes through the hosts relay: the bridge is a host with caps
iphoneandvideo, and every client callsPOST /v1/hosts/{deviceId}/execwith opiphone. - Live screen goes through codai-relay: the bridge is the daemon of its own environment "iPhone <name>" and streams the screen as HEVC video.
hub / Android / agent codai gateway + relay PC with the bridge iPhone
───────────────────── ───────────────────── ────────────────── ──────
POST /v1/hosts/:dev/exec ───▶ hosts relay (SSE) ───▶ same tool code as the ─▶ tap, type,
{ op: 'iphone', … } local MCP open app…
video viewer (WebSocket) ◀─── relay stream kind `video` ◀─── DisplayService HEVC ◀─ screenSet up the bridge
Install the bridge first (see apps/iphone-bridge/README.md; it ships with the codai desktop MCP codai-iphone). Then, once:
Give it an API key
Create a key in the hub (Keys) and store it in a file next to the bridge's control.json — never in the config itself:
Set-Content -NoNewline "$env:LOCALAPPDATA\codai\iphone-bridge\api-key.txt" 'codai_…'
'{"relay": {"enabled": true, "api_key_file": "%LOCALAPPDATA%/codai/iphone-bridge/api-key.txt"}}' |
Set-Content "$env:LOCALAPPDATA\codai\iphone-bridge\config.json"Enrol the phone
& "$env:LOCALAPPDATA\codai\iphone-bridge\venv\Scripts\codai-iphone-bridge.exe" relay enrollrelay enroll creates (or reuses) the project "codai iPhone" and a BYO environment "iPhone <name>", and pins this phone's stable device id to it. It is idempotent; options --udid, --name, --base-url.
Start it with remote access
Restart the bridge, or run it with --relay (equivalent to relay.enabled: true). It registers with os=ios, caps=iphone,video and the environment id, and reconnects with backoff (1 → 30 s) on network failures. A rejected key or another bridge taking over the same environment stops the remote side until you restart it.
config.json keys under relay: enabled, api_key_file (else CODAI_API_KEY), base_url (default https://ai.codai.ro), relay_url (default: what the gateway returned at enrolment), allow_sensitive (default false) and video (default true; false = control only).
Use it
| Where | How |
|---|---|
| Hub | hub.codai.ro/devices/iphone — Remote iPhones lists the phones online; open one for the live screen and controls (tap, scroll, Home, buttons, typing). |
| Android app | Settings → Remote hosts → iPhone: the same list, a live screen you tap and drag on, and Home / Back / Lock buttons. |
| Agents | The iphone_* tools in every codai agent (desktop, Android, VS Code, headless): iphone_hosts, iphone_screen, iphone_screenshot, iphone_find, iphone_click, iphone_tap, iphone_swipe, iphone_type, iphone_key, iphone_button, iphone_home, iphone_open_app, iphone_open_url, iphone_list_apps, iphone_wait_for. Each takes an optional host (device id); the default is the only online host with cap iphone. |
| API / SDKs | POST /v1/hosts/{deviceId}/exec with { "op": "iphone", "args": { "tool": "tap", "args": { "x": 200, "y": 640 } } }; client.hosts.iphone(deviceId, 'tap', { x: 200, y: 640 }) in TypeScript, client.hosts.iphone(device_id, "tap", {"x": 200, "y": 640}) in Python. |
The answer is { ok, result: { data, image? } }: data is what the bridge tool returned and image an optional JPEG screenshot as a data URL (≤ 1.5 MB, whole result ≤ 2 MB). Tool names and arguments are those of the local codai-iphone MCP without the iphone_ prefix.
Live screen and fallback
A viewer opens a video stream on the phone's environment. The bridge starts CoreDevice DisplayService (HEVC, typically 30–55 fps), shares one session between all viewers and stops it with the last one. When the stream cannot run — another app holds the camera or microphone (error 9022), the phone does not support it, or it failed three restarts — the bridge switches automatically to JPEG screenshots at about 2 fps and the Android app shows Screenshots instead of Live. A slow viewer gets frames dropped until the next keyframe, never a closed connection.
The hub opens the relay WebSocket with a single-use ticket valid for 60 seconds, so your API key never reaches the browser.
Security
- Same user only. The gateway relays an
iphoneop only between devices of the same account; there is no sharing across users. - Sensitive tools are refused remotely. Install / uninstall and
wifi_setuprun only locally unless you setrelay.allow_sensitive: trueon the bridge. - Everything is journaled. Remote calls run the same tool code as local ones and land in the bridge journal marked
"via": "relay". - Your key stays on the PC, in the file you chose; the bridge never prints it.
Troubleshooting
| Symptom | Fix |
|---|---|
409 host_offline | The bridge is not connected with remote access: start it with --relay or relay.enabled: true. |
| No iPhone online in the hub | Same as above; it appears within a few seconds of the bridge connecting. |
| Screenshots instead of Live | DisplayService is unavailable (camera or microphone in use, unsupported); control keeps working. |
| Phone not found over Wi-Fi | Keep it awake and on the same network, or plug the cable in once and call wifi_setup locally. |
| Developer image mount fails | Close tools that hold the image mounter (Impactor, 3uTools) before starting the bridge. |
iPhone and iPad
codai for iOS is a native SwiftUI app — chat, voice, transcripts, Studio, vault approvals and real terminals into your environments, desktops, sandboxes and SSH servers.
VS Code
The codai extension for VS Code mirrors your GitHub Copilot Chat sessions to your watch, phone and desktop, so you can approve tools and answer questions from anywhere. Plus @codai in chat.