Beta. The Video API is in beta. The surface below is validated server-side with
additionalProperties: false — a request is either exactly this shape or it is rejected before any compute runs, at $0. Contracts here are stable for the beta (additions may land; nothing is removed without notice).Using these from an AI agent? The MCP server exposes every capability as a native tool — its page has the install and connect steps.
POST https://api.eachlabs.ai/v1/prediction with header Authorization: Bearer YOUR_API_KEY, body {"model": "eachlabs-video-api", "input": {"capability": "<name>", "input_url": "...", "params": {...}}}, then poll GET /v1/prediction/{id} until a terminal status. A request needs only capability, input_url (or input_urls), and params. Analysis capabilities (probe, scene_detect, silence_detect, audio_analysis) return structured JSON in output; everything else returns hosted artifact URL(s) — the exact JSON per output class is in Response shapes. See Quickstart for the input URL requirements and Billing & Limits for pricing mechanics and the error catalog. For long-tail cases the named capabilities don’t cover, a raw-ffmpeg mode with a different top-level contract is documented at run_ffmpeg, and multi-step patterns are collected under Recipes.
The 44 capabilities by category (sections below are alphabetical; keep_ranges and silence_remove are one capability under two accepted names, so the API accepts 45 capability names):
Encode defaults
Re-encoding capabilities that expose no codec knobs (crop, scale, pad, rotate, overlay, watermark, speed, reverse, fade, color, reframe, reframe_track, concat, captions, lut3d, keep_ranges/silence_remove, slideshow, crossfade, sticker_overlay, title_card) share one fixed encode contract, keyed on the container parameter:
- CRF, audio bitrate, and sample rate are left at ffmpeg’s defaults (effectively CRF 23 for x264 and 128k AAC) unless the capability exposes a
crfknob (transcode,poster,segmentandsilence_splitwithreencode: true). - Deviations:
volumeandloudnormstream-copy the video (mp4/mkv only) and touch only audio;slideshowencodes at fixed quality 23;concatresamples audio to 44.1 kHz stereo;audio_masteralways resamples to 44.1 kHz with a 320k default bitrate.
What the probe gate checks
Before compute starts, staged inputs pass through an ffprobe gate. The gate’s checks are a closed list — nothing else is probed:- Source duration — over 3600 s rejects (
input_rejected, reasonduration_over_cap). - Stream prerequisites — the capabilities below declare which streams each input must carry; a missing stream rejects at $0. Capabilities not listed declare none:
(An image counts as a video stream — still-image watermarks and overlays pass.)
3. Empty-output parameters — the per-capability zero-bill rejects noted in the tables below (
trim start at/past EOF, fade_out_start at/past EOF, stills/poster timestamps at/past EOF, keep_ranges windows entirely past EOF or under one frame, title_card/sticker_overlay show windows outside the probed duration, crossfade offset/duration exceeding the first input’s probed duration).
4. Encoder constraints — a codec the requested container cannot mux, such as segment/silence_split with reencode: true targeting webm, is checked at submit. Each is noted per capability.
Geometry is not probed. A crop window larger than the source frame, or a pad x/y offset that pushes the picture outside the canvas, passes the gate and fails at execution as exec_error (verified against production) — probe the source first when its dimensions are uncertain. If the probe itself cannot read the source, the gates that need probe data fail open and the job proceeds.
audio_analysis
Measure the audio track — integrated loudness, loudness range, true peak, sample peak, and clipping indicators — returned as structured JSON via astats+ebur128. A metadata analysis, not a media artifact. Inputs:input_url (1) · Output: analysis JSON — {"audio_analysis": {...}} in output; no uploaded artifact, zero egress
No parameters — an empty params object is valid.
- The input must carry an audio stream — a video-only input is rejected at $0 before billing.
- Any field that cannot be measured is omitted from the result rather than reported as zero: a digitally silent input returns no
true_peak_dbtpand nosample_peak_dbfs(silence has no finite dB peak), and any statistic ffmpeg reports as-inf/nanis likewise omitted. Treat a missing field as not-measurable, never as0.
audio_duck
Duck a music/program bed under a voice track: the voice drives a sidechain compressor that lowers the music while the voice is above threshold, and the same voice is then mixed on top — output carries the voice over the ducked bed, audio-only. Inputs:input_urls (2) — input_urls[0] is the music/program bed, input_urls[1] is the voice · Output: media — single audio file (mp3/wav/m4a)
- The output runs exactly as long as
input_urls[0]’s audio: a voice longer than the music is truncated at the music’s end; a shorter voice leaves the remaining music playing at full level. - The output is always audio-only and inherits
input_urls[0]’s sample rate and channel count — a videoinput_urls[0]contributes only its audio track, its video stream is discarded. Sample-rate or channel-layout mismatches never reject: the voice is silently resampled and up/down-mixed to the music’s format. - Levels are never auto-normalized — the voice and the ducked music keep their source loudness, which can clip if both are hot.
- Both inputs must carry an audio stream — an input without one is rejected at $0 before billing.
audio_master
Master the audio track to a distribution loudness target — compressor → EBU R128 loudness normalization (single-pass) → true-peak limiter — written audio-only as MP3, WAV, or M4A. The first-class path for audio-in/audio-out mastering (e.g. MP3 → −14 LUFS MP3). Inputs:input_url (1) · Output: media — single audio file (mp3/wav/m4a)
- The input must carry an audio stream — a video-only input is rejected at $0 before billing.
- A fixed gentle compressor (2:1 ratio, -18 dB threshold, 30 ms attack) is always applied before normalization and cannot be disabled — for loudness normalization without compression use loudnorm (video containers) or run_ffmpeg.
- Normalization is single-pass, so the output’s measured integrated loudness can deviate from
target_lufson short or highly dynamic inputs;ceiling_dbtpis still enforced by the final limiter. For mastering-grade accuracy use the two-pass pattern under run_ffmpeg. - The output is always resampled to 44100 Hz regardless of the source sample rate; there is no way to keep the source rate.
audio_replace
Swap or mix a video’s audio track using a second input as the audio source; video always stream-copied. Inputs:input_urls (2) — input_urls[0] is the base video, input_urls[1] is the audio source · Output: media — single file
offset_ms replaces the old client-side pre-padding recipe (adelay before upload): the server pads the decoded stream, so the ~46ms MP3 encoder-delay skew that made local pre-padding frame-inaccurate does not apply.captions
Burn a subtitle track (SRT/VTT/ASS) into the video via libass, with optional style overrides. No subtitle file yet? Generate one with a speech-to-text model first (e.g.elevenlabs-speech-to-text-scribe-v2), or use subtitle_compose to author animated captions directly from word timings.
Inputs: input_urls (2) — input_urls[0] is the video, input_urls[1] is the subtitle file (SRT/VTT/ASS) · Output: media — single file
color
Color-grade via ffmpeg eq: brightness/contrast/saturation/gamma. All-defaults is the identity grade (still re-encodes). Inputs:input_url (1) · Output: media — single file
concat
Join 2-10 inputs end-to-end, normalizing each to a common WxH canvas and 44.1kHz stereo (accepts mixed resolutions/formats). Inputs:input_urls (2-10, must equal the inputs param; joined in array order) · Output: media — single file
concat_copy
Losslessly rejoin 2-50 already-encoded, same-codec chunks via the concat demuxer with -c copy (no re-encode). Inputs:input_urls (2-50, must equal the inputs param; joined in array order) · Output: media — single file
crop
Cut a width x height rectangle out of the frame (centered unless x/y given). Inputs:input_url (1) · Output: media — single file
The window is not checked against the source frame: a
width/height (or x/y offset) that exceeds the source dimensions passes the $0 probe gate and fails at execution as exec_error (verified against production). Probe the source first when its dimensions are uncertain.crossfade
Crossfade two clips into one: the video transition startsoffset seconds into the first clip and blends into the second over duration seconds (xfade); the audio is always crossfaded at the junction of the two audio tracks over the same duration (acrossfade ignores offset).
Inputs: input_urls (2) — joined in order, first into second · Output: media — single file
- With
offsetomitted, video and audio cross at the same junction and the output runs first + second −durationseconds. When the first input’s duration cannot be probed (direct bindings), an omittedoffsetis rejected at $0 — supply it explicitly. Anoffsetearlier than the junction switches the video to the second clip early while the audio still crosses at the junction, so the audio track outlasts the video track. - Both inputs must contain a video and an audio stream — an input missing either is rejected at $0 before billing.
- The second input’s video is scaled (stretched, square pixels) to the probed size of the first input; when that size is unknown the input sizes must already match or the run fails at execution. Both inputs must share a frame rate — mismatched rates fail at execution; container timebases are normalized in-graph, so inputs with identical frame rates but different encoder timescales are safe.
- A second input shorter than
durationtruncates the transition. Video and audio are always re-encoded (audio becomes AAC for mp4/mkv or Opus for webm); nothing is stream-copied.
extract_audio
Drop the video stream and write the audio track in the requested format (STT-ready via sample_rate/mono). Inputs:input_url (1) · Output: media — single audio file (mp3/wav/m4a)
fade
Video fade-in and/or fade-out with matching audio afade. Inputs:input_url (1) · Output: media — single file
gif
Convert a clip to an animated GIF at a given fps and width (lanczos, single pass). Inputs:input_url (1) · Output: media — single output.gif
hls_ladder
Encode a multi-rendition adaptive-bitrate HLS ladder (master.m3u8 + N variant playlists/segments) in one pass. Re-encodes each rung; upscaling is not rejected. Inputs:input_url (1) · Output: bundle {"primary": master.m3u8, "artifacts": [...]} in output — master.m3u8 + stream_N.m3u8 variants + shared TS segments — see Response shapes
Omitting both
renditions and ladder applies the default 720p preset (720@3000, 480@1500, 360@1000). Supplying both is rejected at $0 — they are mutually exclusive.keep_ranges
Keep only the supplied time ranges and close the gaps into one continuous, frame-accurate output (re-encodes). Two accepted names, one capability.keep_ranges and silence_remove are the same capability under two wire names with byte-identical contracts — submit either. silence_remove is the original name (the ‘act’ half of the silence_detect → act chain); keep_ranges says what it does. Everything below applies to both.
Inputs: input_url (1) · Output: media — single file
loudnorm
Normalize audio loudness to an EBU R128 target (single-pass); video stream-copied. Inputs:input_url (1) · Output: media — single file
- The input must carry an audio stream — a video-only input is rejected at $0 before billing. An audio-only input (e.g. an
.mp3) is accepted: the-c:v copyis a no-op with no video stream present, and the output is an audio-only mp4/mkv. - The audio track is re-encoded to AAC regardless of its source codec, and the output sample rate is not preserved: loudnorm internally resamples to 192 kHz and no rate reset is applied, so the AAC output lands at 96 kHz (the highest rate AAC accepts), not the source rate.
- Normalization is single-pass, so the output’s measured integrated loudness can deviate from
target_i— especially on short or highly dynamic inputs. For mastering-grade accuracy use the two-pass pattern under run_ffmpeg. - loudnorm outputs mp4/mkv video containers only — it never emits
.mp3. For audio-in/audio-out mastering (e.g. MP3 to a -14 LUFS MP3) use audio_master, the first-class MP3/WAV/M4A path (note it always applies a fixed compressor); run_ffmpeg remains the escape hatch for custom chains and two-pass accuracy.
lut3d
Color-grade through a caller-supplied .cube 3D LUT (lut3d filter). Inputs:input_urls (2) — input_urls[0] is the video, input_urls[1] is the .cube LUT file · Output: media — single file
overlay
Composite a second video/image over a base at a named corner/center anchor, with optional margin and pre-scale. Inputs:input_urls (2) — input_urls[0] is the base video, input_urls[1] is the overlay (video or image) · Output: media — single file
package_abr
Package an input into a single-rendition CMAF HLS + DASH adaptive-bitrate bundle (clear, no DRM). Stream-copies — never re-encodes. Inputs:input_url (1) · Output: bundle {"primary": master.m3u8, "artifacts": [...]} in output — manifest.mpd + master.m3u8 + fmp4 segments — see Response shapes
package_abr_ladder
Encode a true multi-rendition CMAF HLS + DASH ladder (re-encodes every rung, any source codec). When the source height is probeable, upscaling is rejected and the default ladder is clipped to source height; when the source cannot be probed, the ladder is not clipped (fail-open, may upscale). Inputs:input_url (1) · Output: bundle {"primary": master.m3u8, "artifacts": [...]} in output — master.m3u8 + manifest.mpd over shared fmp4 segments — see Response shapes
Omitting both
renditions and ladder applies the default ladder — 1080@5000, 720@2800, 480@1200 — clipped to the source height when probeable. Supplying both is rejected at $0 — they are mutually exclusive.pad
Frame the picture inside a width x height canvas filled with a background color. Inputs:input_url (1) · Output: media — single file
x/y are not checked against the canvas: an offset that pushes the picture outside width×height passes the $0 probe gate and fails at execution as exec_error.poster
One input -> two named outputs in one pass: a transcoded H.264 MP4 (‘video’) and a still-frame JPEG (‘thumbnail’). Inputs:input_url (1) · Output: named-outputs object {"video": ..., "thumbnail": ...} in output — see Response shapes
probe
Inspect an input with ffprobe and return the JSON metadata (format + streams) as the structured result — no media artifact, no egress. Inputs:input_url (1) · Output: analysis JSON — the ffprobe document (format + streams) returned directly in output; no uploaded artifact
No parameters — an empty params object is valid.
reframe
Convert a clip to a social aspect ratio via scale+crop (fill) or scale+pad (fit). Inputs:input_url (1) · Output: media — single file at the preset resolution
reframe_track
Pan a fixed-size crop window across the frame following a subject, driven by a crop-track sidecar (sendcmd script). Inputs:input_urls (2) — input_urls[0] is the source video, input_urls[1] is the crop-track sidecar (sendcmd script) · Output: media — single file
The sidecar. input_urls[1] is a raw ffmpeg sendcmd script the engine splices in as sendcmd=f=<sidecar>,crop=W:H. One command per line:
<seconds> crop x <px>, crop y <px>;— the timestamp in seconds, then the pixel coordinates of the crop window’s top-left corner relative to the source frame.- The window is
widthxheight(fixed by params), starts centered, and values step-hold between timestamps — there is no interpolation. - The script is not parsed or validated server-side: a malformed sidecar fails at execution as a metered
exec_error(settles $0 in beta), not a pre-run reject. Validate it locally before submitting.
reverse
Play the clip backwards (video reverse + audio areverse). Buffers the whole clip — suits short inputs. Inputs:input_url (1) · Output: media — single file
rotate
Rotate the picture in fixed 90-degree clockwise steps (transpose, no interpolation). Inputs:input_url (1) · Output: media — single file
scale
Resize video to a target width/height, keeping audio. Inputs:input_url (1) · Output: media — single file
scene_detect
Detect hard cuts and return the structured scene list as the result — a metadata analysis, not a media artifact. Inputs:input_url (1) · Output: analysis JSON — {"scenes": [{time, score}]} in output (time in seconds, score in 0-1); no uploaded artifact, zero egress
segment
Split input into fixed-duration chunks (seg_000, seg_001, …) — multi-output. Withfirst_frame: true it also delivers each clip’s first frame as a companion still in the same call.
Inputs: input_url (1) · Output: media — URL array in output, one entry per chunk, index-ordered (seg_000, seg_001, …); with first_frame: true the stills are interleaved with their clips (clip, still, clip, still, … in index order) — see Response shapes
reencode: true with container: "webm" is rejected — H.264 cannot mux into webm.A
first_frame split that produces more than 999 clips fails the whole job after the split has run (the numbered names stop pairing with their stills) — surfaced as platform_drift at $0. Raise segment_seconds so the clip count stays at or under 999.silence_detect
Detect silent spans in the audio and return the structured list as the result. No silence found = valid empty result. Inputs:input_url (1) · Output: analysis JSON — {"silences": [{start, end, duration}]} in output (all fields in seconds); no artifact, zero egress
silence_split
Detect silences and split the file at them in one job —silence_detect plus the split, with hard caps, replacing the detect → compute-cuts-client-side → one-trim-per-segment chain (see Recipes).
Inputs: input_url (1) · Output: media — URL array in output, one entry per segment, index-ordered (seg_000, seg_001, …) — see Response shapes
- Each cut lands at the end of a detected silence: the silence stays attached to the preceding segment and each new segment starts where sound resumes. A source with no qualifying silence and no forced cap cut comes back as a single file.
- The input must carry an audio stream — rejected at $0 otherwise.
- When the source duration cannot be probed, the span after the last detected silence is never force-cut (fail-open): the final segment — or the whole file, when no silence qualifies — can come back longer than the cap.
- Verified in production: a 90 s source with silences ending at 13 / 22.5 / 58 / 62 / 70.5 s and
max_segment_seconds: 30returns exactly 7 segments — the same cuts as the manual recipe below, in one billed job instead of eight.
slideshow
Turn 2-50 still images into one silent video (each looped image_duration, normalized to a common canvas, optional Ken Burns zoom, concatenated). Inputs:input_urls (2-50, must equal the images param; shown in array order) · Output: media — single silent video
speed
Change playback speed by a factor (video setpts + pitch-preserving atempo chain). Inputs:input_url (1) · Output: media — single file
sticker_overlay
Composite the second input (typically a still image: a sticker, logo, or watermark) over the base video, visible only inside a timed window — overlay’s anchors plusstart/end visibility.
Inputs: input_urls (2) — input_urls[0] is the base video, input_urls[1] is the sticker (image or video) · Output: media — single file
- The window only shows or hides the sticker — an animated sticker plays on the base’s timeline from t=0, not from
start: an animation shorter thanstartshows only its frozen last frame inside the window; one that ends beforeendstays frozen on its last frame for the remainder; one that runs longer has its extra tail dropped. A still image is unaffected. - No opacity control — the sticker’s own alpha channel (e.g. a transparent PNG) is honored, and fully opaque pixels stay fully opaque. A sticker larger than the frame (natively or after
scale) is composited with the out-of-frame overflow cropped; the output always keeps the base’s resolution and duration. - Only the base’s audio is kept (a silent base still succeeds); the sticker input’s audio is discarded. Both inputs must contain a video stream (a still image counts) — an input without one is rejected at $0 before billing.
- The window checks compare against the source’s video stream duration when it disagrees with the container duration; when the source cannot be probed the window is range-checked only. Video and audio are always re-encoded (audio becomes AAC for mp4/mkv or Opus for webm); nothing is stream-copied.
stills
Extract a single frame as JPEG or PNG (first, last, or at an exact timestamp). Inputs:input_url (1) · Output: media — single image
storyboard_sprites
Sample one frame per interval, pack into one tiled sprite sheet plus a server-computed WebVTT thumbnail index. Inputs:input_url (1) · Output: named-outputs object {"sheet": ..., "vtt": ...} in output — see Response shapes
Overflow truncates silently. The sheet holds at most
columns × rows frames: any part of the video past columns × rows × interval_seconds is omitted from both the sheet and the VTT index — no error, no second sheet, no interval stretch. The defaults (5 × 5 at 2 s) cover the first 50 s. Grid cells beyond the sampled frames are left blank. Size the grid or interval to the source duration (probe it first when unknown).subtitle_compose
Word-timed, animated captions in the style popularized by TikTok and Shorts editors: you supply the transcript as individual words with millisecond timings, pick a style, and the engine authors the subtitle document server-side and burns it into the video. An optional hook — a short headline shown near the top of the frame — can run alongside the captions. No subtitle file is ever uploaded or fetched. Word timings typically come from a speech-to-text model with word-level timestamps (for exampleelevenlabs-speech-to-text-scribe-v2 on this platform); feed its word list straight into words.
Inputs: input_url (1) · Output: media — single file
Choosing a font
Anton, Archivo Black, Bangers, and Montserrat Black are heavy display faces built for short, loud, uppercase captions — the viral-video look. Inter, Montserrat, and Poppins are text faces that stay readable in longer sentences and calmer edits; the Liberation and DejaVu families are neutral defaults. For TikTok-style captions, TikTok Sans is the platform’s own typeface in its 36pt display cut — TikTok Display is an alias of it (identical rendering) — and TikTok Text is the 12pt cut, built for small caption sizes; the Medium variants are the same cuts at medium weight. Public Sans is a neutral UI-style face. The same list is available for style.font, style.hook_font, and title_card’s font.
Choosing a hook entrance
pop (the default) fades the hook in with a short scale pop. Scaling re-computes line wrapping every frame, so a hook long enough to wrap onto multiple lines can visibly re-wrap mid-animation — for long hooks prefer fade (fade-in, never re-wraps) or none (appears instantly).
Style recipes
Ready-to-pastestyle objects for the three most-requested caption looks. Everything else (timing, paging, colors) can be tuned per the parameter table above.
Bold viral caps — heavy uppercase captions, active word flips to yellow:
thumbnails
Extract one image frame every interval_seconds — multi-output. Inputs:input_url (1) · Output: media — URL array in output, one entry per frame, index-ordered (thumb_001, thumb_002, …) — see Response shapes
title_card
Draw a text title over the single input video (drawtext) with an optional show window — the text is rendered from a staged file, never interpreted. Inputs:input_url (1) · Output: media — single file
- Injection-safe by construction: the text is staged as a server-generated text file and rendered with expansion disabled, so it is drawn verbatim — quotes, colons, backslashes, and ffmpeg
%{...}expansion sequences have no effect on the command. - Omit both
startandendto show the title for the whole video. When the source duration cannot be probed, the window is range-checked only. - The input must contain a video stream — rejected at $0 before billing otherwise. Video and audio are always re-encoded (audio becomes AAC for mp4/mkv or Opus for webm); nothing is stream-copied.
transcode
Container/codec transcode with server-side normalization and defaults. Inputs:input_url (1) · Output: media — single file, container-chosen extension
trim
Cut a clip of roughlyduration seconds via stream copy — no re-encode. The cut snaps to the keyframe at or before start (playback start is trimmed via an edit list); the end can drift by up to packet granularity. For frame-accurate cuts use keep_ranges with a single ranges entry.
Inputs: input_url (1) · Output: media — single file
volume
Scale audio loudness by a linear factor; video is stream-copied untouched. Inputs:input_url (1) · Output: media — single file
watermark
Overlay a second input (image or video) onto the primary video at a pixel offset. Inputs:input_urls (2) — input_urls[0] is the base video, input_urls[1] is the watermark (image or video) · Output: media — single file
Response shapes
Every result arrives in the polled prediction’soutput field. There are exactly six shapes — five for success, determined by the capability’s output class, and one for error; the sibling fields (status, metrics, urls, input, logs) are identical across all of them (the examples below show the result-bearing fields). On success, metrics carries the billed quantities directly — billed_seconds, output_gb, tier — alongside cost and predict_time; failed predictions carry only cost and predict_time (see Billing & Limits).
1. URL array — every single-file media capability. Single-output jobs return a one-element array:
segment, silence_split, thumbnails. Same array, one URL per produced file, index-ordered (seg_000, seg_001, …). segment with first_frame: true interleaves each clip with its companion still, in index order (verified in production):
poster (video, thumbnail) and storyboard_sprites (sheet, vtt). output maps output name → URL:
hls_ladder, package_abr, package_abr_ladder. output is {primary, artifacts}: primary is the entrypoint a player opens (master.m3u8), artifacts lists every produced file in deterministic (name-sorted) order with role ∈ playlist | init | segment | media:
probe, scene_detect, silence_detect, audio_analysis (and run_ffmpeg with ffprobe). output is the JSON document itself; no artifact is uploaded. scene_detect and silence_detect arrays are ordered ascending by time:
audio_analysis wraps its measurements under an audio_analysis key (fields in audio_analysis):
status: "error". output is the error envelope: error_code carries the public code, error_message the specific reason, retryable and classification.category the handling hints, and metrics.cost what it billed. The field-by-field contract and two real captured examples (a $0 reject and an execution failure) are in the error catalog:
run_ffmpeg
Raw ffmpeg/ffprobe for long-tail cases the named capabilities don’t cover. Available to beta organizations today; access may move to per-key grants later in the beta. This section is the canonical statement of therun_ffmpeg contract — other pages link here.
Different contract. run_ffmpeg is a top-level mode, not a capability — the input carries mode, argv, input_url (or input_urls), and output_format (no capability, no params):
alimiterneedslevel=false— the default (level=true) re-normalizes after limiting and silently defeats the true-peak ceiling.- The explicit
-ar 44100is required —loudnorminternally upsamples to 192kHz, and without an explicit rate the output stays at 192kHz.
argv — staged placeholder paths are the only way to reach your media, and the only literal path a job needs is the /dev/null sink shown below. The server substitutes {input} (the staged input), {input0}, {input1}, … (indexed inputs), and {output} (the artifact path, whose extension comes from output_format).
Multiple inputs. Pass input_urls; each URL binds its index placeholder in array order. Example — replace a video’s audio track with -c copy:
-protocol_whitelist file — commands can read local files only, never the network. Every run_ffmpeg job is billed like any capability job. The argv restrictions — this is the complete list; anything not listed passes validation (and, if ffmpeg itself rejects it, fails at execution as exec_error):
argv[0]must beffmpegorffprobe.- URL schemes are an allow-list of one (
protocol_denied): any token carrying ascheme:prefix is denied unless the scheme isexpr:(ffmpeg’s expression prefix). Nested prefixes are unwrapped recursively —async:file:...-style chains are caught. - Denied tokens (
flag_denied):-protocol_whitelist(the sandbox injects its own), a bare-(stdin/stdout),movie=/amovie=filter sources (an arbitrary-file-read vector), and-safewith any value other than the literal1. - Placeholder-assembled paths reject
..traversal; raw literal path tokens are permitted inrun_ffmpeg(it compiles in escalated mode —/dev/nulland plain local paths pass validation, though only staged placeholder paths resolve to your media). -var_stream_mapvalues are checked against a strict grammar; a nonconforming value rejects asinvalid_command.- Placeholder bindings: a bound value may not begin with
-(flag_denied), may not contain..traversal, and, when spliced inside a filtergraph argument, may not contain the filtergraph metacharacters, : ; ' [ ] \.
argv[0]: "ffprobe" the JSON is returned inline in the prediction output — no media artifact.
Analysis-to-file pattern. Filters that print metadata can write their report to the artifact instead of producing media — point the filter’s file= at {output} and satisfy ffmpeg’s output requirement with the -f null /dev/null sink:
Two-pass loudnorm. Single-pass
loudnorm is dynamic and can land off-target. For mastering-grade accuracy run a first pass with loudnorm=...:print_format=json through the -f null /dev/null sink to measure, then a second pass feeding back measured_I, measured_LRA, measured_TP with linear=true — this lands within 0.5 LU of target.Recipes
Patterns verified end-to-end against production. Two of these are now single-call capabilities — the manual chains remain documented as alternatives.Clip list + first frames: segment with first_frame: true (one call)
One prediction returns the clips and each clip’s first frame, interleaved in index order — see segment. Set reencode: true for boundary-accurate frames. Verified in production: a 90 s source with {"segment_seconds": 30, "reencode": true, "first_frame": true} returned 3 clips + 3 JPGs in one job billed $0.0225 — the same output the manual chain produced across four billed predictions.
Manual alternative (segment → stills) — when you need frames at positions other than the first (position: "at" / "last"): split with segment, then run stills on each chunk — segment output URLs are directly usable as the next job’s input_url. For boundary-accurate frames set reencode: true on the segment job: stream-copy segmentation cuts only on keyframes, and measured chunk durations drifted to 33.36s / 33.33s / 23.33s against a requested 30/30/30 split.
Split on silence with a max-duration cap: silence_split (one call)
One prediction detects the silences, applies your hard cap and minimum, and delivers the segments — see silence_split. Verified in production: the worked example below returns the same 7 segments as the manual chain, in one billed job instead of eight.
Manual alternative (silence_detect → trim per segment) — when you need custom cut logic the caps don’t express:
- Run
silence_detecton the source. - Compute capped cut points client-side: cut at each silence’s
end; hard-cut when a segment would exceed your max duration; skip segments shorter than your min; cap the tail. - Submit one
trimper computed segment.
- cut at 13 (silence end), cut at 22.5 (silence end); the next silence end (58) is more than 30 s away, so hard-cut at 22.5 + 30 = 52.5;
- cut at 58, 62, 70.5 (silence ends, each within the cap); the tail 70.5 → 90 fits the cap;
- segments: 0–13, 13–22.5, 22.5–52.5, 52.5–58, 58–62, 62–70.5, 70.5–90 → 7
trimjobs.
Stream-copy
trim carries +0.00-0.17s of end drift per segment. When exact boundaries matter, use keep_ranges (re-encodes) instead of per-segment trims — or silence_split with reencode: true.