Shade

WORKING SLICE
Index selected footage folders by extracting sparse keyframes and transcript segments, then search text or local embeddings with playable source timestamps. Keep an explicit per-file indexing report and removable-drive state.

Build this scoped Shade-inspired workflow with a documented data model and visible failure states.

Architecture
- Python CLI and FastAPI for a localhost media-search interface.
- SQLite with sqlite-vec for source manifests and model-versioned embeddings; FFmpeg keyframes, local CLIP-compatible embeddings and local speech transcription.

Prerequisites and limits
Python, FFmpeg, selected user-owned media folders and compatible downloaded embedding/speech models. Budget disk for previews and measure indexing time on a sample before scanning a whole drive.
Outside this release: Cloud streaming, face identity recognition and team asset synchronization are outside scope.

Data model and correctness
media paths, source hashes, sampled keyframes, timestamped transcripts, model-versioned vectors and index jobs
Invariant: Indexes never rewrite source footage; stale vectors are excluded after a file changes and similarity is not an identified person or verified event.
Save a job manifest with input hash, parameters and state. Write to temporary outputs, then atomically finalize only successful results; resume unfinished jobs without replacing originals.

Security and privacy
Bound file sizes and processing time, reject path traversal, and use argument arrays for subprocesses. Treat imported text as data and redact confidential source content from logs.

Recovery and export
Export sources, manifests and outputs with checksums. Keep failed-job diagnostics and allow retry into a new output path; restore the database and file directory together.

Implementation order
1. Phase 1 — Scope and fixtures. Implement this bounded workflow: Index selected footage folders by extracting sparse keyframes and transcript segments, then search text or local embeddings with playable source timestamps. Keep an explicit per-file indexing report and removable-drive state. Record prerequisites, select representative user-owned fixtures and document the unsupported features: Cloud streaming, face identity recognition and team asset synchronization are outside scope.
2. Phase 2 — Durable model. Model media paths, source hashes, sampled keyframes, timestamped transcripts, model-versioned vectors and index jobs Add migrations or a versioned document format, explicit validation, stable IDs and a visible import-error report. Preserve this rule: Indexes never rewrite source footage; stale vectors are excluded after a file changes and similarity is not an identified person or verified event.
3. Phase 3 — Complete the first useful path. Implement the workflow's input, review and output interface, with clear controls and explicit empty/error states. Save a job manifest with input hash, parameters and state. Write to temporary outputs, then atomically finalize only successful results; resume unfinished jobs without replacing originals.
4. Phase 4 — Permissions and integration failure. Bound file sizes and processing time, reject path traversal, and use argument arrays for subprocesses. Treat imported text as data and redact confidential source content from logs. Request integration credentials and permissions only for the enabled feature; show a disconnected state instead of mock results.
5. Phase 5 — Portable handoff. Export sources, manifests and outputs with checksums. Keep failed-job diagnostics and allow retry into a new output path; restore the database and file directory together. Include setup, operating limits, fixture walkthrough and shutdown/restart instructions in the README.
6. Phase 6 — Acceptance scenarios. Unplug a drive and retain its catalog with unavailable markers; reindex a changed clip without duplicating old segments or returning stale timestamps. Repeat the workflow after restart and with a denied permission or unavailable dependency; show recoverable failure rather than a success placeholder.

Acceptance
Unplug a drive and retain its catalog with unavailable markers; reindex a changed clip without duplicating old segments or returning stale timestamps.
Use real source data or clearly labeled fixtures. Explain unsupported input and provider failures; do not fabricate analytics, delivery receipts, accuracy claims or security guarantees.

Optional agent guidance
Optional external skill: [modern-python](https://github.com/trailofbits/skills/blob/main/plugins/modern-python/skills/modern-python/SKILL.md) — Set up Python projects with pyproject.toml, dependency management, linting, typing and automated checks. Review its instructions and compatibility before use; it does not grant deployment, data-access or publication permission.
Optional external skill: [sharp-edges](https://github.com/trailofbits/skills/blob/main/plugins/sharp-edges/skills/sharp-edges/SKILL.md) — Review security-sensitive APIs and configuration for dangerous defaults and easy-to-misuse interfaces. Review its instructions and compatibility before use; it does not grant deployment, data-access or publication permission.
Optional external skill: [web-design-guidelines](https://github.com/vercel-labs/agent-skills/blob/main/skills/web-design-guidelines/SKILL.md) — Review web interfaces for accessibility, keyboard focus, forms, navigation and interaction quality. Review its instructions and compatibility before use; it does not grant deployment, data-access or publication permission.
Project rule — data model: media paths, source hashes, sampled keyframes, timestamped transcripts, model-versioned vectors and index jobs
Project rule — preserve this invariant: Indexes never rewrite source footage; stale vectors are excluded after a file changes and similarity is not an identified person or verified event.
Project rule — acceptance evidence: Unplug a drive and retain its catalog with unavailable markers; reindex a changed clip without duplicating old segments or returning stale timestamps.

# AGENTS.md — Build guide for Shade

## Project scope
Index selected footage folders by extracting sparse keyframes and transcript segments, then search text or local embeddings with playable source timestamps. Keep an explicit per-file indexing report and removable-drive state.

Catalogue verdict: kinda. The search half is real and rebuildable. Extract keyframes with ffmpeg, embed them with CLIP, transcribe the audio with whisper, put the vectors in SQLite, and 'the drone shot over the bridge at golden hour' finds the clip on your own drives. The open-source stack for that is mature and the result is genuinely good. What does not survive the port is what Shade has grown into: cloud streaming so an editor opens full-res without waiting on a download, per-link permissions and guest access, review and approval, and a model pipeline that keeps improving without you retraining anything. Solo, on local storage, the DIY version wins outright. On a team, you are rebuilding a platform and calling it a script.
Use the implementation prompt below to define the deliverable. Complete each phase's acceptance checks before extending the scope.

## Working agreement
- Inspect the repository and its existing instructions before choosing paths, dependencies or commands. Keep one coherent stack and explain changes to the proposed architecture.
- Plan a vertical slice that accepts a real input and produces the useful output described below. Persist only the state the prompt calls for; respect memory-only and upstream-managed workflows. Use fixtures only when they are clearly labelled.
- After scaffolding, document the actual install, development, check and build commands in README and keep them synchronized with the package or project manifest. Do not report commands as successful unless they ran.
- Work in small steps. At handoff, list implemented flows, checks actually performed, remaining blockers, and any credentials or provider setup the owner must supply.
- Do not publish, spend money, contact customers, delete source data or run irreversible migrations without the project owner's authorization.

## Prerequisites
- Python, FFmpeg, selected user-owned media folders and compatible downloaded embedding/speech models. Budget disk for previews and measure indexing time on a sample before scanning a whole drive.
- Implementation components: Python CLI and FastAPI for a localhost media-search interface. SQLite with sqlite-vec for source manifests and model-versioned embeddings; FFmpeg keyframes, local CLIP-compatible embeddings and local speech transcription.
- Scope boundary: Cloud streaming, face identity recognition and team asset synchronization are outside scope.

## Stack and architecture
- Python CLI and FastAPI for a localhost media-search interface.
- SQLite with sqlite-vec for source manifests and model-versioned embeddings; FFmpeg keyframes, local CLIP-compatible embeddings and local speech transcription.
- Domain model: media paths, source hashes, sampled keyframes, timestamped transcripts, model-versioned vectors and index jobs

## Security and data integrity
- Bound file sizes and processing time, reject path traversal, and use argument arrays for subprocesses. Treat imported text as data and redact confidential source content from logs.
- Correctness boundary: Indexes never rewrite source footage; stale vectors are excluded after a file changes and similarity is not an identified person or verified event.
- Save a job manifest with input hash, parameters and state. Write to temporary outputs, then atomically finalize only successful results; resume unfinished jobs without replacing originals.
- Export sources, manifests and outputs with checksums. Keep failed-job diagnostics and allow retry into a new output path; restore the database and file directory together.

## Agent implementation rules
- Project rule — data model: media paths, source hashes, sampled keyframes, timestamped transcripts, model-versioned vectors and index jobs
- Project rule — preserve this invariant: Indexes never rewrite source footage; stale vectors are excluded after a file changes and similarity is not an identified person or verified event.
- Project rule — acceptance evidence: Unplug a drive and retain its catalog with unavailable markers; reindex a changed clip without duplicating old segments or returning stale timestamps.

## Optional agent skills and references
- Optional external skill: [modern-python](https://github.com/trailofbits/skills/blob/main/plugins/modern-python/skills/modern-python/SKILL.md) — Set up Python projects with pyproject.toml, dependency management, linting, typing and automated checks. Review its instructions and compatibility before use; it does not grant deployment, data-access or publication permission.
- Optional external skill: [sharp-edges](https://github.com/trailofbits/skills/blob/main/plugins/sharp-edges/skills/sharp-edges/SKILL.md) — Review security-sensitive APIs and configuration for dangerous defaults and easy-to-misuse interfaces. Review its instructions and compatibility before use; it does not grant deployment, data-access or publication permission.
- Optional external skill: [web-design-guidelines](https://github.com/vercel-labs/agent-skills/blob/main/skills/web-design-guidelines/SKILL.md) — Review web interfaces for accessibility, keyboard focus, forms, navigation and interaction quality. Review its instructions and compatibility before use; it does not grant deployment, data-access or publication permission.

Read the linked SKILL.md and its dependencies before adding a skill. Select only the skills matching this project's runtime and task; their documentation does not supply API access, credentials or approval to perform external actions. Pin the reviewed revision where the tool supports it. Follow the chosen agent's documented project-level installation mechanism.

## Distribution ideas
These are optional planning notes. Obtain the owner's approval before publishing or contacting anyone.
- Demonstrate the actual Shade-inspired workflow with owned or clearly labeled sample data: Index selected footage folders by extracting sparse keyframes and transcript segments, then search text or local embeddings with playable source timestamps. Keep an explicit per-file indexing report and removable-drive state.
- Publish a reproducible walkthrough with this observable result: Unplug a drive and retain its catalog with unavailable markers; reindex a changed clip without duplicating old segments or returning stale timestamps.
- Explain who can operate this scoped tool, its setup and ongoing costs, and these remaining product gaps: Cloud streaming, face identity recognition and team asset synchronization are outside scope. Avoid guaranteed savings, performance scores or implied endorsement.

## Engineering roadmap
1. Phase 1 — Scope and fixtures. Implement this bounded workflow: Index selected footage folders by extracting sparse keyframes and transcript segments, then search text or local embeddings with playable source timestamps. Keep an explicit per-file indexing report and removable-drive state. Record prerequisites, select representative user-owned fixtures and document the unsupported features: Cloud streaming, face identity recognition and team asset synchronization are outside scope.
2. Phase 2 — Durable model. Model media paths, source hashes, sampled keyframes, timestamped transcripts, model-versioned vectors and index jobs Add migrations or a versioned document format, explicit validation, stable IDs and a visible import-error report. Preserve this rule: Indexes never rewrite source footage; stale vectors are excluded after a file changes and similarity is not an identified person or verified event.
3. Phase 3 — Complete the first useful path. Implement the workflow's input, review and output interface, with clear controls and explicit empty/error states. Save a job manifest with input hash, parameters and state. Write to temporary outputs, then atomically finalize only successful results; resume unfinished jobs without replacing originals.
4. Phase 4 — Permissions and integration failure. Bound file sizes and processing time, reject path traversal, and use argument arrays for subprocesses. Treat imported text as data and redact confidential source content from logs. Request integration credentials and permissions only for the enabled feature; show a disconnected state instead of mock results.
5. Phase 5 — Portable handoff. Export sources, manifests and outputs with checksums. Keep failed-job diagnostics and allow retry into a new output path; restore the database and file directory together. Include setup, operating limits, fixture walkthrough and shutdown/restart instructions in the README.
6. Phase 6 — Acceptance scenarios. Unplug a drive and retain its catalog with unavailable markers; reindex a changed clip without duplicating old segments or returning stale timestamps. Repeat the workflow after restart and with a denied permission or unavailable dependency; show recoverable failure rather than a success placeholder.

## Paid-product capabilities outside this build
- cloud streaming of full-res files without downloading them first
- guest links with per-link permissions and roles
- built-in review, approval, and commenting
- face recognition and shot-type tagging that improves without your involvement
- team sync, so everyone searches the same index
- the NLE plugins and Slack integration

## Implementation prompt
WORKING SLICE
Index selected footage folders by extracting sparse keyframes and transcript segments, then search text or local embeddings with playable source timestamps. Keep an explicit per-file indexing report and removable-drive state.

Build this scoped Shade-inspired workflow with a documented data model and visible failure states.

Architecture
- Python CLI and FastAPI for a localhost media-search interface.
- SQLite with sqlite-vec for source manifests and model-versioned embeddings; FFmpeg keyframes, local CLIP-compatible embeddings and local speech transcription.

Prerequisites and limits
Python, FFmpeg, selected user-owned media folders and compatible downloaded embedding/speech models. Budget disk for previews and measure indexing time on a sample before scanning a whole drive.
Outside this release: Cloud streaming, face identity recognition and team asset synchronization are outside scope.

Data model and correctness
media paths, source hashes, sampled keyframes, timestamped transcripts, model-versioned vectors and index jobs
Invariant: Indexes never rewrite source footage; stale vectors are excluded after a file changes and similarity is not an identified person or verified event.
Save a job manifest with input hash, parameters and state. Write to temporary outputs, then atomically finalize only successful results; resume unfinished jobs without replacing originals.

Security and privacy
Bound file sizes and processing time, reject path traversal, and use argument arrays for subprocesses. Treat imported text as data and redact confidential source content from logs.

Recovery and export
Export sources, manifests and outputs with checksums. Keep failed-job diagnostics and allow retry into a new output path; restore the database and file directory together.

Implementation order
1. Phase 1 — Scope and fixtures. Implement this bounded workflow: Index selected footage folders by extracting sparse keyframes and transcript segments, then search text or local embeddings with playable source timestamps. Keep an explicit per-file indexing report and removable-drive state. Record prerequisites, select representative user-owned fixtures and document the unsupported features: Cloud streaming, face identity recognition and team asset synchronization are outside scope.
2. Phase 2 — Durable model. Model media paths, source hashes, sampled keyframes, timestamped transcripts, model-versioned vectors and index jobs Add migrations or a versioned document format, explicit validation, stable IDs and a visible import-error report. Preserve this rule: Indexes never rewrite source footage; stale vectors are excluded after a file changes and similarity is not an identified person or verified event.
3. Phase 3 — Complete the first useful path. Implement the workflow's input, review and output interface, with clear controls and explicit empty/error states. Save a job manifest with input hash, parameters and state. Write to temporary outputs, then atomically finalize only successful results; resume unfinished jobs without replacing originals.
4. Phase 4 — Permissions and integration failure. Bound file sizes and processing time, reject path traversal, and use argument arrays for subprocesses. Treat imported text as data and redact confidential source content from logs. Request integration credentials and permissions only for the enabled feature; show a disconnected state instead of mock results.
5. Phase 5 — Portable handoff. Export sources, manifests and outputs with checksums. Keep failed-job diagnostics and allow retry into a new output path; restore the database and file directory together. Include setup, operating limits, fixture walkthrough and shutdown/restart instructions in the README.
6. Phase 6 — Acceptance scenarios. Unplug a drive and retain its catalog with unavailable markers; reindex a changed clip without duplicating old segments or returning stale timestamps. Repeat the workflow after restart and with a denied permission or unavailable dependency; show recoverable failure rather than a success placeholder.

Acceptance
Unplug a drive and retain its catalog with unavailable markers; reindex a changed clip without duplicating old segments or returning stale timestamps.
Use real source data or clearly labeled fixtures. Explain unsupported input and provider failures; do not fabricate analytics, delivery receipts, accuracy claims or security guarantees.

Optional agent guidance
Optional external skill: [modern-python](https://github.com/trailofbits/skills/blob/main/plugins/modern-python/skills/modern-python/SKILL.md) — Set up Python projects with pyproject.toml, dependency management, linting, typing and automated checks. Review its instructions and compatibility before use; it does not grant deployment, data-access or publication permission.
Optional external skill: [sharp-edges](https://github.com/trailofbits/skills/blob/main/plugins/sharp-edges/skills/sharp-edges/SKILL.md) — Review security-sensitive APIs and configuration for dangerous defaults and easy-to-misuse interfaces. Review its instructions and compatibility before use; it does not grant deployment, data-access or publication permission.
Optional external skill: [web-design-guidelines](https://github.com/vercel-labs/agent-skills/blob/main/skills/web-design-guidelines/SKILL.md) — Review web interfaces for accessibility, keyboard focus, forms, navigation and interaction quality. Review its instructions and compatibility before use; it does not grant deployment, data-access or publication permission.
Project rule — data model: media paths, source hashes, sampled keyframes, timestamped transcripts, model-versioned vectors and index jobs
Project rule — preserve this invariant: Indexes never rewrite source footage; stale vectors are excluded after a file changes and similarity is not an identified person or verified event.
Project rule — acceptance evidence: Unplug a drive and retain its catalog with unavailable markers; reindex a changed clip without duplicating old segments or returning stale timestamps.

## Completion evidence
Demonstrate the prompt's acceptance scenarios against the scoped workflow. Include setup from a clean checkout and failure recovery. Check persistence across restart and export/restore only for the state the prompt says to store; for memory-only tools, confirm that temporary content is discarded as specified. Record actual results and remaining limitations. A detailed plan alone does not establish a working replacement.