Skill v1.0.1
currentAutomated scan100/1001 files
version: "1.0.1" name: runwayml description: Generate AI videos, images, and audio with Runway API. Use when generating video from images, text-to-video, video-to-video, character performance, text-to-image, text-to-speech, sound effects, or voice processing with Runway. license: MIT compatibility: Requires internet access and a RunwayML API key (RUNWAYML_API_SECRET). metadata: author: RunwayML version: 1.0.0
Runway API
Generate AI videos, images, and audio using Runway's API. Features Runway's latest Gen-4.5 model for high-quality text-to-video and image-to-video generation, plus Gen-4 variants and third-party models from Google (Veo, Gemini) and ElevenLabs.
Recommended: Use `gen4.5` for best results - the newest and most capable video generation model.
Setup: See Installation Guide. API key goes inRUNWAYML_API_SECRETenv var.
Quick Start
Python
from runwayml import RunwayMLclient = RunwayML()# Image-to-video with latest Gen-4.5 modeltask = client.image_to_video.create(model="gen4.5",prompt_image="https://example.com/image.jpg",prompt_text="A timelapse on a sunny day with clouds flying by",ratio="1280:720",duration=10).wait_for_task_output()print(f"Video URL: {task.output[0]}")# Text-to-video (no image required)task = client.image_to_video.create(model="gen4.5",prompt_text="A serene mountain landscape at sunset with clouds drifting",ratio="1280:720",duration=10).wait_for_task_output()
Node.js
import RunwayML from "@runwayml/sdk";const client = new RunwayML();// Image-to-video with latest Gen-4.5 modelconst task = await client.imageToVideo.create({model: "gen4.5",promptImage: "https://example.com/image.jpg",promptText: "A timelapse on a sunny day with clouds flying by",ratio: "1280:720",duration: 10,}).waitForTaskOutput();console.log(`Video URL: ${task.output[0]}`);// Text-to-video (no image required)const textTask = await client.imageToVideo.create({model: "gen4.5",promptText: "A serene mountain landscape at sunset",ratio: "1280:720",duration: 10,}).waitForTaskOutput();
cURL
# Image-to-videocurl -X POST "https://api.dev.runwayml.com/v1/image_to_video" \-H "Authorization: Bearer $RUNWAYML_API_SECRET" \-H "X-Runway-Version: 2024-11-06" \-H "Content-Type: application/json" \-d '{"model": "gen4.5","promptImage": "https://example.com/image.jpg","promptText": "A timelapse on a sunny day","ratio": "1280:720","duration": 10}'# Text-to-video (no image required)curl -X POST "https://api.dev.runwayml.com/v1/image_to_video" \-H "Authorization: Bearer $RUNWAYML_API_SECRET" \-H "X-Runway-Version: 2024-11-06" \-H "Content-Type: application/json" \-d '{"model": "gen4.5","promptText": "A serene mountain landscape at sunset","ratio": "1280:720","duration": 10}'# Poll for result (use returned task id)curl "https://api.dev.runwayml.com/v1/tasks/{task_id}" \-H "Authorization: Bearer $RUNWAYML_API_SECRET" \-H "X-Runway-Version: 2024-11-06"
All Available Models
Video Generation
| Model | Input | Pricing | Use Case | |
|---|---|---|---|---|
| `gen4.5` ⭐ | Text or Image | 12 credits/sec | Newest & recommended - Best quality text/image-to-video (no audio) | |
gen4_turbo | Image | 5 credits/sec | Fast image-to-video (no audio), iteration | |
gen4_aleph | Video + Text/Image | 15 credits/sec | Video-to-video transformation (no audio) | |
act_two | Image or Video | 5 credits/sec | Character performance/motion (no audio) | |
veo3 | Text or Image | 40 credits/sec | Google Veo high-quality video with audio | |
veo3.1 | Text or Image | 40 credits/sec | Google Veo 3.1 with keyframes & audio | |
veo3.1_fast | Text or Image | 15 credits/sec | Google Veo 3.1 faster/cheaper & audio |
Image Generation
| Model | Input | Pricing | Use Case | |
|---|---|---|---|---|
gen4_image | Text + References (optional) | 5 credits/720p, 8 credits/1080p | High-quality with style transfer | |
gen4_image_turbo | Text + References (required) | 2 credits/image (any res) | Fast iteration | |
gemini_2.5_flash | Text + References | 5 credits/image | Google Gemini image gen |
Audio Generation (ElevenLabs)
| Model | Input → Output | Pricing | |
|---|---|---|---|
eleven_multilingual_v2 | Text → Speech | 1 credit/50 chars | |
eleven_text_to_sound_v2 | Text → Sound Effects | 1 credit/6 sec | |
eleven_voice_isolation | Audio → Clean Audio | 1 credit/6 sec | |
eleven_voice_dubbing | Audio → Dubbed Audio | 1 credit/2 sec | |
eleven_multilingual_sts_v2 | Speech → Speech | 1 credit/2 sec |
1 credit = $0.01. Get credits at dev.runwayml.com
Video Generation
Gen-4.5 (Text-to-Video and Image-to-Video)
The latest and most capable Runway model supporting both text-only and image-to-video generation.
Text-to-Video
Generate videos from text descriptions only:
task = client.image_to_video.create(model="gen4.5",prompt_text="A serene mountain lake at sunrise with mist rising from the water",ratio="1280:720",duration=10,seed=12345 # Optional: reproducibility).wait_for_task_output()
Text-to-Video Aspect Ratios: Landscape 1280:720 (16:9) | Portrait 720:1280 (9:16)
Image-to-Video
Animate existing images with motion:
task = client.image_to_video.create(model="gen4.5",prompt_image="https://example.com/image.jpg",prompt_text="Camera slowly pushes in, leaves rustling in the breeze",ratio="1584:672",duration=10,seed=12345).wait_for_task_output()
Image-to-Video Aspect Ratios:
- Widescreen:
1280:720(16:9),1584:672(21:9 ultra-wide) - Standard:
1104:832(4:3) - Portrait:
720:1280(9:16),832:1104(3:4)
Duration: 2-10 seconds | Pricing: 12 credits/second (60 credits minimum for 5 sec)
Gen-4 Turbo (Image-to-Video)
task = client.image_to_video.create(model="gen4_turbo",prompt_image="https://example.com/image.jpg",prompt_text="Camera slowly pushes in, leaves rustling",ratio="1280:720",duration=5,seed=12345 # Optional: reproducibility).wait_for_task_output()
Aspect Ratios: Landscape 1280:720, 1584:672, 1104:832 | Portrait 720:1280, 832:1104 | Square 960:960
Aleph (Video-to-Video)
Transform existing videos with text/image guidance:
task = client.video_to_video.create(model="gen4_aleph",video_uri="https://example.com/source.mp4",prompt_text="Transform to anime style",references=[{"uri": "https://example.com/style_ref.jpg"}] # Optional style reference).wait_for_task_output()
Aspect Ratios: Adds 848:480 (landscape) and 480:848 (portrait) to Gen-4 options.
Act-Two (Character Performance)
Drive character motion from reference performance. Objects require type discriminators:
task = client.character_performance.create(model="act_two",character={"type": "image", "uri": "https://example.com/character.jpg"}, # or type: "video"reference={"type": "video", "uri": "https://example.com/performance.mp4"}).wait_for_task_output()
Character types: image (character performs in static environment) or video (character performs with some of its own movement)
Veo (Google)
Google's Veo models for text-to-video and image-to-video. Veo models include audio generation - making them ideal when you need video with sound.
# Text-to-video with audio (no image required)task = client.image_to_video.create(model="veo3.1", # or "veo3", "veo3.1_fast"prompt_text="A cinematic shot of a rocket launching at sunset with roaring engines").wait_for_task_output()# Image-to-video with audiotask = client.image_to_video.create(model="veo3.1",prompt_image="https://example.com/starting_frame.jpg",prompt_text="Smooth camera movement through the scene with ambient nature sounds").wait_for_task_output()
Note: Gen-4.5 and Gen-4 models produce silent video. Use Veo if you need audio, or add audio separately with ElevenLabs models (see Audio Generation section).
Image Generation
Gen-4 Image with References
Use reference images with @mention syntax in prompts:
# gen4_image - reference_images is optionaltask = client.text_to_image.create(model="gen4_image",ratio="1920:1080",prompt_text="A beautiful mountain landscape at sunset").wait_for_task_output()# With referencestask = client.text_to_image.create(model="gen4_image",ratio="1920:1080",prompt_text="@EiffelTower painted in the style of @StarryNight",reference_images=[{"uri": "https://example.com/eiffel.jpg", "tag": "EiffelTower"},{"uri": "https://example.com/starry.jpg", "tag": "StarryNight"}]).wait_for_task_output()
Note: gen4_image_turbo requires reference_images (at least one). Use gen4_image for text-only generation.
Untagged references apply as general style:
reference_images=[{"uri": "https://example.com/subject.jpg", "tag": "subject"},{"uri": "https://example.com/style.jpg"} # No tag = style reference]
Audio Generation
ElevenLabs models for text-to-speech, sound effects, and voice processing.
Voice Presets: Maya, Arjun, Serene, Bernard, Billy, Mark, Clint, Mabel, Chad, Leslie, Eleanor, Elias, Elliot, Noah, Rachel, James, Katie, Tom, Wanda, Benjamin
Text-to-Speech
task = client.text_to_speech.create(model="eleven_multilingual_v2",prompt_text="Hello, welcome to RunwayML!",voice={"type": "runway-preset", "preset_id": "Maya"}).wait_for_task_output()
Sound Effects
task = client.sound_effect.create(model="eleven_text_to_sound_v2",prompt_text="Thunder rumbling in the distance, rain on a window").wait_for_task_output()
Voice Isolation
task = client.voice_isolation.create(model="eleven_voice_isolation",audio_uri="https://example.com/noisy_audio.mp3").wait_for_task_output()
Voice Dubbing
task = client.voice_dubbing.create(model="eleven_voice_dubbing",audio_uri="https://example.com/speech.mp3",target_lang="es" # Spanish).wait_for_task_output()
Supported languages: en, hi, pt, zh, es, fr, de, ja, ar, ru, ko, id, it, nl, tr, pl, sv, fil, ms, ro, uk, el, cs, da, fi, bg, hr, sk, ta
Speech-to-Speech
Convert speech to a different voice. Requires type discriminators:
task = client.speech_to_speech.create(model="eleven_multilingual_sts_v2",media={"type": "audio", "uri": "https://example.com/original.mp3"}, # or type: "video"voice={"type": "runway-preset", "preset_id": "Maya"}).wait_for_task_output()
Input Requirements
Size Limits
| Type | URL Limit | Data URI Limit | Ephemeral Upload | |
|---|---|---|---|---|
| Image | 16MB | 5MB (3.3MB pre-encoding) | 200MB | |
| Video | 32MB | 16MB | 200MB | |
| Audio | 32MB | 16MB | 200MB |
Supported Formats
Images: JPEG, PNG, WebP (no GIF)
Videos: MP4 (H.264/H.265/AV1), MOV (ProRes), MKV, WebM
Audio: MP3, WAV, FLAC, M4A, AAC
Base64 Data URIs
import base64with open("image.jpg", "rb") as f:data_uri = f"data:image/jpeg;base64,{base64.b64encode(f.read()).decode()}"task = client.image_to_video.create(model="gen4_turbo",prompt_image=data_uri,prompt_text="Gentle movement").wait_for_task_output()
Task Management
All operations are async. Use wait_for_task_output() (polls automatically, 10 min timeout).
Statuses: PENDING → RUNNING → SUCCEEDED / FAILED / CANCELED
THROTTLED = rate-limited, treat as PENDING
Canceling/Deleting Tasks
# Cancel running task or delete completed taskclient.tasks.delete(task.id)
See Task Management for manual polling and batch processing.
Prompting Tips
Gen-4 thrives on simplicity. Start simple, iterate.
- Describe single scenes (5-10 sec clips)
- Use clear physical descriptions, not conceptual language
- Reference subjects generically: "the subject", "she"
- Avoid negative phrasing - "no blur" produces unpredictable results
See Prompting Guide for camera movements and advanced techniques.
Error Handling
from runwayml import RunwayML, APIError, RateLimitErrorclient = RunwayML()try:task = client.image_to_video.create(...).wait_for_task_output()if task.status == "FAILED":print(f"Generation failed: {task.failure}")except RateLimitError:print("Rate limited - SDK retries automatically")except APIError as e:print(f"API error {e.status_code}: {e.message}")
| Code | Meaning | Action | |
|---|---|---|---|
| 400 | Invalid input | Fix request parameters | |
| 401 | Invalid API key | Check RUNWAYML_API_SECRET | |
| 429 | Rate limit | SDKs auto-retry with backoff | |
| 503 | Service unavailable | SDKs auto-retry |
Parameters Quick Reference
Video (gen4.5)
| Parameter | Type | Options | |
|---|---|---|---|
model | string | "gen4.5", "gen4_turbo", "veo3", "veo3.1" | |
duration | number | 2 to 10 (seconds) | |
ratio | string | Text-to-video: "1280:720" (16:9), "720:1280" (9:16)<br>Image-to-video adds: "1584:672" (21:9), "1104:832" (4:3), "832:1104" (3:4) | |
prompt_text | string | Motion/scene description (required) | |
prompt_image | string | URL or base64 (optional for text-to-video, required for image-to-video) | |
seed | number | Optional, for reproducibility |
Image (gen4_image)
| Parameter | Type | Options | |
|---|---|---|---|
model | string | "gen4_image", "gen4_image_turbo" | |
ratio | string | "1920:1080", "1280:720", etc. | |
prompt_text | string | Image description | |
reference_images | array | [{"uri": "...", "tag": "..."}] |
References
- Installation Guide - SDK setup
- Prompting Guide - Advanced prompting
- Task Management - Polling, batching
Official Documentation
- Runway API Documentation - Complete API reference
- Model Guide - All available models
- Developer Portal - API keys and account management