RunwayML

Discover how to use the RunwayML node in StackAI to generate AI-powered images and videos with customizable prompts, models, and output settings.

What is RunwayML?

RunwayML is an AI-powered node in StackAI that enables users to generate high-quality images and videos using advanced generative models. It supports both text-to-image and image-to-video workflows, making it ideal for creative projects, content generation, and multimedia automation.

How to use it?

The RunwayML node offers two main actions:

  1. Generate Video from Image

  2. Generate Image from Text

Each action has its own set of inputs, configurations, and outputs. Below are detailed explanations and examples for each.

1. Generate Video from Image

Create a video from an initial image, with optional text prompt guidance and customizable settings.

Inputs:

  • Image URL (string, required): The publicly accessible URL of the image to use as the first frame of the video. Example: "https://example.com/image.png"

  • Prompt (string, optional): A text prompt to guide the video generation. Example: "A futuristic cityscape at sunset"

Configurations:

  • Model (select, optional, default: Gen4 Turbo): The model variant for video generation. Options: "gen4_turbo"

  • Duration (select, optional, default: 5): The length of the generated video in seconds. Options: 5 or 10

  • Aspect Ratio (select, optional, default: 1280:720): The output video’s aspect ratio. Options:

    • 1280:720

    • 720:1280

    • 1104:832

    • 832:1104

    • 960:960

    • 1584:672

  • Seed (number, optional): Random seed for reproducible results (0-4294967295), using the same seed will produce the same image if all inputs and configurations are the same. Example: 123456

  • Watermark (boolean, optional, default: true): Whether to add a RunwayML watermark to the video. Options: true or false

Outputs:

  • Video URL (string): The URL to download or view the generated video.

  • Task ID (string): The unique identifier for the video generation task.

Example of Usage:

{
  "image_url": "https://example.com/image.png",
  "prompt": "A futuristic cityscape at sunset",
  "model": "gen4_turbo",
  "duration": 10,
  "aspect_ratio": "1280:720",
  "seed": 123456,
  "watermark": false
}

Output:

{
  "video_url": "https://runwayml.com/generated/video123.mp4",
  "task_id": "task_abc123"
}

2. Generate Image from Text

Create an image from a detailed text prompt, with options for model, resolution, and more.

Inputs:

  • Prompt (string, required): A detailed text description of the image to generate. Example: "A serene mountain landscape with a clear blue lake"

Configurations:

  • Model (select, optional, default: Gen4 Image): The model to use for image generation. Options: "gen4_image"

  • Resolution (select, optional, default: 1024:1024): The output image’s resolution/aspect ratio. Options include:

    • 1920:1080 (16:9)

    • 1080:1920 (9:16)

    • 1024:1024 (Square)

    • 1360:768 (Landscape)

    • 1080:1080 (Square)

    • 1168:880 (4:3)

    • 1440:1080 (4:3)

    • 1080:1440 (3:4)

    • 1808:768 (Wide)

    • 2112:912 (Ultra Wide)

    • 1280:720 (HD)

    • 720:1280 (Portrait HD)

    • 720:720 (Square HD)

    • 960:720 (4:3 HD)

    • 720:960 (3:4 HD)

    • 1680:720 (Cinematic)

  • Seed (number, optional): Random seed for reproducible results (0-4294967295), using the same seed will produce the same image if all inputs and configurations are the same. Example: 78910

  • Public Figure Threshold (select, optional, default: auto): Content moderation strictness for public figures. Options: "auto", "low"

  • Reference Images (string array, optional): Array of image URLs to use as references for generation. Can help augment the style of the image created. Example: ["https://example.com/ref1.jpg", "https://example.com/ref2.jpg"]

Outputs:

  • Image URL (string): The URL to download or view the generated image.

  • Task ID (string): The unique identifier for the image generation task.

Example of Usage:

{
  "prompt": "A serene mountain landscape with a clear blue lake",
  "model": "gen4_image",
  "resolution": "1920:1080",
  "seed": 78910,
  "public_figure_threshold": "auto",
  "reference_images": ["https://example.com/ref1.jpg"]
}

Output:

{
  "image_url": "https://runwayml.com/generated/image456.png",
  "task_id": "task_def456"
}

Summary Table

Action
Required Inputs
Optional Configurations
Outputs (Required)

Generate Video

image_url

prompt, model, duration, aspect_ratio, seed, watermark

video_url, task_id

Generate Image

prompt

model, resolution, seed, public_figure_threshold, reference_images

image_url, task_id

Advanced Settings

  • Retry on Failure: Enable retrying when the node execution fails

  • Fallback Branch: Create a separate branch that executes when this node fails, allowing you to handle errors gracefullyCreate a separate branch that executes when this node fails, allowing you to handle errors gracefully

PreviousRegexNextSalesforce

Last updated

Was this helpful?