# BUILD SPEC — Workspace Pass Integration + Recent Feed + Segment Extraction **Date:** 2026-04-14 **Input:** consultations/recoil/workspace-pass-integration/SYNTHESIS.md + JT design decisions **Validation:** `cd ~/Dropbox/CLAUDE_PROJECTS/recoil && python3 -m pytest workspace/tests/ -v` **Working directory:** ~/Dropbox/CLAUDE_PROJECTS/recoil ## JT Design Decisions (locked) 1. **No intermediate pass-node step.** Pass videos get split into shot-level segments that file under their source shots as takes. The pass itself is metadata/provenance, not a primary navigation element. 2. **Visual treatment:** Pass-sourced takes should be visually distinguishable from direct-generation takes (e.g., a small badge or subtle indicator showing "from PASS_HATCH_001"). 3. **Recent feed:** Shows EVERYTHING — refs, casting explorations, video, frames. Not filtered to production only. 4. **Archive granularity:** Per-take. Must be able to archive individual takes, including individual segments extracted from passes. 5. **No dedicated A/B comparison.** Existing take cycling (arrow keys) is sufficient — select one, select another, arrow between them. 6. **Boundary frame thumbnails:** Defer to V2. Not in this build. ## Architecture Overview When a pass video lands in `output/video/ep_NNN/`, the workspace should: 1. Detect it via `_PASS_PATTERN` regex 2. Read PassStore for segment metadata (shot IDs, timestamps) 3. Use ffmpeg to extract segments into shot-level clips 4. Register extracted clips as takes under their source shots 5. Show pass provenance in the inspector when viewing a pass-sourced take 6. Provide a "Recent" tab that shows all generations reverse-chronologically ## Key Files | File | Role | |------|------| | `workspace/server.py` | Tree scanning, API endpoints, segment extraction | | `workspace/static/workspace.js` | Frontend tree, inspector, recent feed | | `workspace/static/workspace.css` | Styling for pass badges, recent feed | | `workspace/static/index.html` | Recent tab toggle | | `workspace/sidecar.py` | Sidecar enrichment for extracted segments | | `execution/pass_store.py` | Pass metadata (read-only from workspace) | --- ## Phase 1: Pass Detection + Segment Extraction (server-side) **Estimated time:** 10-15 min ### 1a. Add pass detection regex In `server.py`, add alongside `_SHOT_PATTERN`: ```python _PASS_PATTERN = re.compile( r'^(EP\d+)_PASS_(.+?)_take(\d+)' r'(?:_\d+)?' r'\.[a-zA-Z0-9]+$' ) ``` ### 1b. Add segment extraction function **IMPORTANT — Shot ID normalization (SPEC REVIEW BUG FIX):** PassStore shot IDs like `EP001_SH05A_HATCH_MS` have descriptive suffixes that break naive regex matching. Extract the base shot number and zero-pad to 3 digits to match existing `shot_NNN` convention: ```python def _normalize_shot_num(pass_shot_id: str) -> str: """Extract base shot number from PassStore shot ID and zero-pad. 'EP001_SH05A_HATCH_MS' -> '005a' 'EP001_SH12' -> '012' 'SH3' -> '003' """ m = re.match(r'(?:EP\d+_)?SH(\d+)([a-zA-Z]?)', pass_shot_id) if not m: return None num = m.group(1).zfill(3) suffix = m.group(2).lower() return num + suffix ``` Segment filenames use `shot_{padded}_FROM_{pass_id}_take{N}.mp4`: - Uppercase `_FROM_` distinguishes from model/hash tags in `_SHOT_PATTERN` - The `shot_` prefix + zero-padded number makes segments group with existing takes under the same shot node - Full source shot ID is preserved in the sidecar's `source_shot_id` field ```python def _extract_pass_segments( pass_video_path: Path, pass_id: str, segment_timestamps: dict[int, dict], segment_shot_ids: list[str], take_num: int, output_dir: Path, ) -> list[Path]: """Split a pass video into per-shot segment clips via ffmpeg. Each segment is saved as: shot_{padded}_FROM_{pass_id}_take{N}.mp4 Full source shot ID is written to the sidecar, not the filename. Returns list of extracted segment paths. Prerequisites: segment_timestamps must be non-empty with entries for each segment. If timestamps are missing, caller should skip. """ ``` For each segment: - Input: pass video, start time, duration - Normalize shot ID: `EP001_SH05A_HATCH_MS` → `005a` - Output: `output/video/ep_NNN/shot_005a_FROM_PASS_HATCH_001_take1.mp4` - ffmpeg command: `ffmpeg -ss {start} -t {duration} -i {input} -c:v libx264 -crf 18 -c:a aac {output}` - Always re-encode for frame-accurate cuts (stream-copy seeks to keyframes, producing inaccurate 2-5s segments). ### 1c. Write sidecar for each extracted segment Each extracted clip gets a sidecar JSON (note `source_shot_id` preserves the full PassStore ID): ```json { "source_type": "pass_segment", "source_pass_id": "EP001_PASS_HATCH_001", "source_shot_id": "EP001_SH05A_HATCH_MS", "source_take": 1, "segment_index": 0, "segment_timestamp_start": 0.0, "segment_timestamp_end": 5.0, "source_video": "EP001_PASS_HATCH_001_take1.mp4", "model": "seeddance-2.0", "cost_usd_segment": 0.5 } ``` ### 1d. Trigger extraction on tree scan In `get_tree()` after the tree scan completes (NOT inside `_scan_output_dir`): - For each file matching `_PASS_PATTERN` in the episode directories, check for a marker file `.{pass_filename}.extracted` in the same directory - If marker exists: extraction already done, skip - If no marker: read PassStore for this pass's record - Extraction requires BOTH conditions: 1. `segment_shot_ids` is a non-empty list 2. `segment_timestamps` is a non-empty dict with entries for each segment index - If BOTH exist: run `_extract_pass_segments()`, then write marker file - If shot IDs exist but timestamps are empty: pass video appears as ungrouped file with pass metadata. **This is the expected state before cut detection runs.** - If no PassStore record at all (manual drop): skip — file appears as ungrouped ### Validation - Drop a test pass video into `output/video/ep_001/` - Reload tree → segments extracted → appear as takes under source shots - Sidecar JSON exists for each segment --- ## Phase 2: Shot Node Integration for Pass-Sourced Takes **Estimated time:** 10-15 min ### 2a. Update shot grouping to include pass-sourced takes **No new regex needed.** The extraction naming from Phase 1b produces `shot_005a_FROM_PASS_HATCH_001_take1.mp4`, which already matches `_SHOT_PATTERN`: - `^shot_(005a)` — shot_num capture - `(?:_[A-Z0-9]+)*?` — matches `_FROM_PASS_HATCH_001` (all uppercase segments) - `(?:_take(\d+))?` — matches `_take1` These files automatically group under the same `SH005A` node as `shot_005a_take1.mp4`. To detect whether a take is pass-sourced (for inspector provenance), check for the sidecar's `source_type: "pass_segment"` field — NOT a filename regex. This is more reliable and decoupled from naming conventions. ### 2b. Take ordering Within a shot node's takes list, order by: 1. Direct takes first (shot_003_take1.mp4, shot_003_take2.mp4) 2. Pass-sourced takes after, grouped by pass (shot_003_FROM_PASS_HATCH_001_take1.mp4) Detection: a take is pass-sourced if its sidecar has `source_type: "pass_segment"`. The `_FROM_` substring in the filename is a secondary heuristic only. ### 2c. Take count badge Shot node badge (e.g., `SH03 ×5`) should count ALL takes including pass-sourced ones. ### Validation - Shot node shows combined take count - Arrow-key cycling goes through direct takes then pass-sourced takes - Inspector loads correctly for both types --- ## Phase 3: Pass Provenance in Inspector **Estimated time:** 10-15 min ### 3a. Detect pass-sourced takes in inspector When the selected file has a sidecar with `source_type: "pass_segment"`, show additional provenance info in the inspector: ``` FROM COVERAGE PASS Pass: PASS_HATCH_001 Segment: 1 of 3 (0.0s – 5.0s) Source: EP001_PASS_HATCH_001_take1.mp4 Total pass cost: $1.50 ``` ### 3b. Pass-sourced take badge in tree Pass-sourced takes should have a small visual indicator in the take cycling UI. Options: - A small "P" badge on the take counter - Dimmed border or accent color when viewing a pass-sourced take - Just the inspector info is enough (simpler) Recommend: inspector info only for V1. Badge in V2. ### 3c. API endpoint for pass detail ``` GET /api/pass/{project}/{pass_id} ``` Returns full PassStore record. Called by inspector when viewing a pass-sourced take. ### Validation - Select a pass-sourced take → inspector shows provenance section - Click through to source pass video (if still exists) works - API returns correct PassStore data --- ## Phase 4: Recent Feed **Estimated time:** 15-20 min ### 4a. Server endpoint ``` GET /api/recent/{project}?limit=50&offset=0 ``` - Walk ALL output directories: `output/video/`, `output/frames/`, `output/previs/`, `output/refs/` - Filter to media extensions: `.mp4`, `.jpg`, `.jpeg`, `.png`, `.webp` - Sort by `stat().st_mtime` descending - Skip: `_archive/`, `_meta/`, hidden files, `boundary_frames/`, `.json` sidecars - Overlay metadata from sidecars where available - Return flat list with: name, path, media_url, type (video/image), mtime, status, model, cost ### 4b. Frontend tab toggle Add to navigator panel header: ```html
``` When RECENT is active, replace the tree with a flat reverse-chronological list. ### 4c. Recent feed item rendering Each item shows: - Small thumbnail (lazy-loaded, 80px) - Filename (truncated to 30 chars) - Relative time ("2m ago", "1h ago", "yesterday") - Type badge: `VIDEO`, `FRAME`, `REF` - Status dot if available Click → load in viewer + inspector (same as tree selection). ### 4d. Polling Poll `/api/recent/{project}` every 5 seconds. Use mtime of newest item as change-detection hash — only re-render if it changes. ### Validation - RECENT tab shows all recent output files - New generation appears within 5 seconds of landing on disk - Click an item → loads in viewer - Refs from casting explorations appear alongside video takes --- ## Phase 5: Pass ↔ Shot Cross-Linking in Inspector **Estimated time:** 10-15 min ### 5a. Shot inspector: show coverage info When viewing a shot that has pass-sourced takes, add a "COVERAGE" section to the inspector: ``` COVERAGE From PASS_HATCH_001 (segment 1/3, 0-5s) From PASS_CORRIDOR_002 (segment 2/4, 4-7s) ``` Data source: scan pass-sourced takes for the current shot, extract pass IDs from filenames/sidecars. ### 5b. Clickable navigation Each coverage entry is clickable → navigates to the source pass video (original unsplit file) in the viewer if it still exists. ### Validation - Shot with pass-sourced takes shows COVERAGE section - Click navigates to pass video - Shot without coverage shows no section --- ## Phase 6: CSS + Filter Compatibility **Estimated time:** 5-10 min ### 6a. Pass-sourced take styling - In the take cycling indicator, pass-sourced takes get a subtle visual hint (e.g., a thin colored underline or a "P" prefix in the take counter: "P1 of PASS_HATCH") - Recent feed items get type-specific badge colors (video: blue, frame: green, ref: orange) ### 6b. Filter compatibility - Pass-sourced takes match the same filters as regular takes: - Media type: video - Model filter: match on sidecar model field - Text filter: match on pass_id, shot_id, filename - No pass-specific filter toggle needed for V1 ### 6c. Context menu Pass-sourced takes get the same context menu as regular takes: Pin, Archive, Copy Path. Archive moves the individual segment file only (not the source pass or other segments). ### Validation - Filter by model → pass-sourced takes from that model appear - Text search "PASS_HATCH" → shows relevant takes - Archive a single pass segment → only that file moves to _archive/ --- ## Pre-flight Checklist (for harness) Before Phase 1: - [ ] Confirm `ffmpeg` is available on Mac Studio: `which ffmpeg` - [ ] Confirm PassStore has test data: check `projects/tartarus/state/visual/passes/` - [ ] Confirm test pass video exists: `projects/tartarus/output/video/ep_001/EP001_PASS_HATCH_001_take1.mp4` - [ ] Run baseline tests: `python3 -m pytest workspace/tests/ -v` ## Post-build - Run `/bugfix` (mandatory) - Run `/simplify` (mandatory) - Hard-refresh browser to verify visual changes - Test with both tartarus and afterimage projects