# Video-First Generation Patterns — Experiments

Status: UNTESTED
Created: 2026-04-04
Source: X.com research (Dave Clark @Hrushworth, Framer @chaseleantj)

## Context

Our current pipeline generates still keyframes per shot, then feeds them to video models. These patterns skip the still-image compositing step entirely and let the video model handle spatial consistency internally.

Both patterns challenge our assumption that we need per-shot background plates or angle-specific compositing. They treat scene generation as a VIDEO problem from the start, not an IMAGE problem.

---

## Experiment 1: Blueprint Image Pattern

**Source:** Dave Clark (@Hrushworth), Feb 4 2026
**Post:** https://x.com/Hrushworth/status/2040406616806179001
**Model:** Kling 3.0, multi-shot feature

### The Pattern

1. Generate ONE hero image ("blueprint image") establishing character + world
2. Feed that SAME image as the start frame for EVERY multi-shot sequence in the scene
3. Each shot within the multi-shot gets its own prompt describing camera angle, action, wardrobe
4. Kling handles spatial consistency, camera changes, and character identity internally
5. Cut the resulting sequences together

### What Dave Did

- Single image of a female spy in a blue-lit nightclub environment
- Fed the same image into Kling 3.0 multi-shot
- Prompted different sequences with wardrobe changes and action:
  - "Shot1 1s: Cut to a close up of the same female spy now wearing a grey tanktop and underwear"
  - "Shot2 5s: Cut to close up shot of the same female spy now wearing a grey tanktop and underwear, profile angle of her jumping through the air firing her assault rifle..."
- Cut all the takes together into a short film
- Every sequence started from the SAME blueprint image

### Why This Matters for Us

- Eliminates the need for angle-specific background plates entirely
- No Seedream/NBP compositing step — video model does the spatial work
- Character identity anchored by the hero image, not by per-shot ref injection
- Wardrobe changes via prompt, not image refs (relevant for Dusty Phase 1→2, Sadie Phase 1→2)
- Maps directly to our `StepRunner.execute_multi_shot()` + coverage pass grouping

### Pipeline Mapping

```
CURRENT APPROACH:
  Hero image → Seedream grid (angle plates) → slice → composite character → feed to video

BLUEPRINT APPROACH:
  Hero image → Kling multi-shot (per-shot prompts) → done
```

### Test Plan

1. Use Dusty's hero image as the blueprint
2. Write a multi-shot sequence prompt covering the 6 shots from our test set:
   - Wide establishing (2s)
   - MCU polishing glass (3s)
   - OTS toward seat seven (2s)
   - Insert hands/glass (2s)
   - Profile neon backlight (3s)
   - Low angle bar surface (2s)
3. Run through Kling V3 multi-shot via StepRunner
4. Compare spatial consistency vs. our composited-plate approach
5. Compare cost: one multi-shot call vs. 6 Seedream edits + 6 video generations

### Open Questions

- Does wardrobe-via-prompt work reliably, or does identity drift?
- How much camera variety does Kling generate from a single still?
- Can we enforce specific focal lengths (24mm vs 85mm) in multi-shot prompts?
- Does the 15-second limit per sequence constrain coverage pass design?

---

## Experiment 2: SeedDance "In Between" Pattern

**Source:** Framer (@chaseleantj)
**Model:** SeedDance 2.0
**Status:** Cannot test yet — SeedDance not on fal.ai

### The Pattern

1. Upload FIRST FRAME and LAST FRAME of a scene
2. Prompt: "Show me what happens in between. USE MULTIPLE CAMERA ANGLES"
3. SeedDance generates the entire multi-shot video bridging both frames
4. Optional: add narrative direction ("MAKE SURE THE COUPLE IS FLIRTING")

### Why This Matters for Us

- Our shot plans already have first-shot and last-shot for each scene
- We could generate scene bookend frames (hero keyframes for first/last shot)
- SeedDance fills in all the coverage between them with camera variety
- Narrative direction in the prompt maps to our episode scripts' action descriptions
- Potentially replaces the entire per-shot generation pipeline for character scenes

### Pipeline Mapping

```
CURRENT:
  Script → Shot plan → Per-shot keyframes → Per-shot video → Assemble

IN-BETWEEN:
  Script → Scene bookend keyframes (first + last) → SeedDance fills middle → Done
```

### Afterimage Application

For a bar scene:
- First frame: Dusty behind bar, wide establishing (beginning of scene)
- Last frame: Dusty looking at empty seat seven, MCU (end of scene)
- Prompt: "Show me what happens in between. Use multiple camera angles. The bartender polishes a glass, glances at the empty stool, pauses, then walks toward it."
- SeedDance generates 10-15s of multi-angle coverage

### Test Plan (when SeedDance available on fal.ai)

1. Generate first/last keyframes for a bar scene using Seedream 5.0
2. Upload both + narrative prompt to SeedDance
3. Evaluate: camera variety, character consistency, spatial coherence
4. Compare against blueprint approach and per-shot approach

### Open Questions

- How much narrative control do we actually get?
- Does "USE MULTIPLE CAMERA ANGLES" reliably produce distinct cuts?
- Can we specify WHICH camera angles (MCU, OTS, insert)?
- Quality at 9:16 vertical?
- SeedDance fal.ai availability timeline?

---

## Relationship to Existing Pipeline

Both patterns complement, not replace, the existing architecture:

| Component | Still Useful? | Notes |
|-----------|--------------|-------|
| Shot plans | Yes | Drive prompt content for both patterns |
| Coverage passes | Yes | Group shots into sequences for blueprint pattern |
| Character refs | Maybe | Blueprint pattern may not need per-shot refs |
| Location refs | No (for character shots) | Both patterns let the model build the space |
| StepRunner | Yes | execute_multi_shot() handles both patterns |
| prompt_engine | Needs adaptation | Multi-shot prompts are different from per-shot prompts |

The NBP sequential-angles consultation already concluded: "Stop injecting location reference images for character shots. Let the model construct the 3D space around the character." These patterns are the video-generation equivalent of that insight.

---

## References

- Dave Clark post: https://x.com/Hrushworth/status/2040406616806179001
- Framer (Chase Lean) SeedDance hack: @chaseleantj on X
- NBP consultation: `consultations/starsend/nbp-sequential-angles-locations/SYNTHESIS.md`
- Coverage pass system: `recoil/pipeline/orchestrator/coverage_planner.py`
- Multi-shot execution: `recoil/execution/step_runner.py` → `execute_multi_shot()`