Resemble AI

Build me a focused voice cloning + speech workflow for the personal core of Resemble AI. Requirements:

- Use exactly this stack: Python 3.12 + FastAPI + FFmpeg + React. Model source audio, processing jobs, transcript or edit segments, and exported media; store source IDs and timestamps for each.
- Paid product context: Voice cloning, speech generation, detection, and realtime voice APIs. Build only this DIY scope: Build the closest honest personal voice cloning + speech workflow using one user-selected local or API model, job history, preview, and export.
- For voice cloning + speech, require a user-owned text or audio input and record the selected voice/model with consent.
- Run synthesis or transformation as a resumable job and save the provider response or local model error.
- Preview the waveform and export WAV or MP3 with a manifest of inputs and settings.
- Use a local web page with input, progress, review, and export views. Required input or access: GPU-capable machine or model API key in .env; FFmpeg. Keep credentials in .env.
- Acceptance: with one labelled sample, show the input, saved intermediate state, and exported result; verify this invariant: A failed synthesis retains its source; output sample rate and duration must match the export manifest.
- Out of scope: low-latency inference infrastructure; licensed data, avatars, and production templates. Keep this a personal, inspectable workflow.
- Include a README with setup, a sample input, required keys or permissions, data location, and the supported scope.

# AGENTS.md — Build guide for Resemble AI

## Project scope
Build the closest honest personal voice cloning + speech workflow using one user-selected local or API model, job history, preview, and export.

Catalogue verdict: no. Do not mistake the interface for the product. Resemble AI's durable value is proprietary model, inference, safety, which a solo one-shot build cannot reproduce responsibly. The prompt therefore builds only the closest honest personal consolation tool.
Use the implementation prompt below to define the deliverable. Complete each phase's acceptance checks before extending the scope.

## Working agreement
- Inspect the repository and its existing instructions before choosing paths, dependencies or commands. Keep one coherent stack and explain changes to the proposed architecture.
- Plan a vertical slice that accepts a real input and produces the useful output described below. Persist only the state the prompt calls for; respect memory-only and upstream-managed workflows. Use fixtures only when they are clearly labelled.
- After scaffolding, document the actual install, development, check and build commands in README and keep them synchronized with the package or project manifest. Do not report commands as successful unless they ran.
- Work in small steps. At handoff, list implemented flows, checks actually performed, remaining blockers, and any credentials or provider setup the owner must supply.
- Do not publish, spend money, contact customers, delete source data or run irreversible migrations without the project owner's authorization.

## Prerequisites
- GPU-capable machine or model API key in .env
- Python 3.12
- FFmpeg
- Explicit README warning that this is a consolation build, not a production replacement

## Stack and architecture
- Use exactly this stack: Python 3.12 + FastAPI + FFmpeg + React.
- Model source audio, processing jobs, transcript or edit segments, and exported media; store source IDs and timestamps for each.
- Interface for this voice cloning + speech workflow: a local web page with input, progress, review, and export views.

## Security and data integrity
- A failed synthesis retains its source; output sample rate and duration must match the export manifest.
- Keep API keys and OAuth secrets in .env; redact them from logs, generated files, and error responses.
- State the practical limit: low-latency inference infrastructure.

## Agent implementation rules
- Project rule, data: Model source audio, processing jobs, transcript or edit segments, and exported media; store source IDs and timestamps for each.
- Project rule, behavior: For voice cloning + speech, require a user-owned text or audio input and record the selected voice/model with consent.
- Project rule, recovery: A failed synthesis retains its source; output sample rate and duration must match the export manifest.

## Optional agent skills and references
No installable skill is listed for this guide; use the project rules and the stack documentation.

Read the linked SKILL.md and its dependencies before adding a skill. Select only the skills matching this project's runtime and task; their documentation does not supply API access, credentials or approval to perform external actions. Pin the reviewed revision where the tool supports it. Follow the chosen agent's documented project-level installation mechanism.

## Distribution ideas
These are optional planning notes. Obtain the owner's approval before publishing or contacting anyone.
- Demonstrate this voice cloning + speech operation with sample or user-owned data: For voice cloning + speech, require a user-owned text or audio input and record the selected voice/model with consent.
- Show a concrete result or export: Preview the waveform and export WAV or MP3 with a manifest of inputs and settings.
- State the practical limit: low-latency inference infrastructure.

## Engineering roadmap
1. Phase 1, architecture and data: Use exactly this stack: Python 3.12 + FastAPI + FFmpeg + React. Model source audio, processing jobs, transcript or edit segments, and exported media; store source IDs and timestamps for each.
2. Phase 2, implement: For voice cloning + speech, require a user-owned text or audio input and record the selected voice/model with consent.
3. Phase 3, implement: Run synthesis or transformation as a resumable job and save the provider response or local model error.
4. Phase 4, review and output: Preview the waveform and export WAV or MP3 with a manifest of inputs and settings.
5. Phase 5, recovery and acceptance: Verify this invariant with a saved fixture: A failed synthesis retains its source; output sample rate and duration must match the export manifest. State the practical limit: low-latency inference infrastructure.

## Paid-product capabilities outside this build
- low-latency inference infrastructure
- licensed data, avatars, and production templates
- frontier generation quality
- voice or likeness safety systems

## Implementation prompt
Build me a focused voice cloning + speech workflow for the personal core of Resemble AI. Requirements:

- Use exactly this stack: Python 3.12 + FastAPI + FFmpeg + React. Model source audio, processing jobs, transcript or edit segments, and exported media; store source IDs and timestamps for each.
- Paid product context: Voice cloning, speech generation, detection, and realtime voice APIs. Build only this DIY scope: Build the closest honest personal voice cloning + speech workflow using one user-selected local or API model, job history, preview, and export.
- For voice cloning + speech, require a user-owned text or audio input and record the selected voice/model with consent.
- Run synthesis or transformation as a resumable job and save the provider response or local model error.
- Preview the waveform and export WAV or MP3 with a manifest of inputs and settings.
- Use a local web page with input, progress, review, and export views. Required input or access: GPU-capable machine or model API key in .env; FFmpeg. Keep credentials in .env.
- Acceptance: with one labelled sample, show the input, saved intermediate state, and exported result; verify this invariant: A failed synthesis retains its source; output sample rate and duration must match the export manifest.
- Out of scope: low-latency inference infrastructure; licensed data, avatars, and production templates. Keep this a personal, inspectable workflow.
- Include a README with setup, a sample input, required keys or permissions, data location, and the supported scope.

## Completion evidence
Demonstrate the prompt's acceptance scenarios against the scoped workflow. Include setup from a clean checkout and failure recovery. Check persistence across restart and export/restore only for the state the prompt says to store; for memory-only tools, confirm that temporary content is discarded as specified. Record actual results and remaining limitations. A detailed plan alone does not establish a working replacement.