Hyperframes Media
Published by waker240 in fullvideoproductionskill
What this skill does
Prepare Fish official-API narration, OpenAI word timestamps, and reviewed user-supplied local music/SFX for HyperFrames. Also covers captions, optional stock APIs and background removal. No private service, browser-session proxy, voice, or music library is bundled.
Add Hyperframes Media to your agent
Review the source and files first. When you are ready, copy the prompt instruction or use the CLI command supported by your environment.
Install with a prompt
Paste this into a compatible coding agent:
add this skill "hyperframes-media" from https://github.com/waker240/fullvideoproductionskillInstall with the CLI
Run this command in a controlled environment after reviewing the repository:
npx skills add https://github.com/waker240/fullvideoproductionskill --skill hyperframes-mediaSkill instructions
HyperFrames media
Use fish-audio-api for narration and media-use to ingest reviewed local assets. This package supplies workflows and code; users supply accounts, permitted voices, music, sound effects, images and video.
Requirements and scope
Node >=22.20 and FFmpeg/ffprobe must be on PATH. Read requirements before generation. The engine loads the video project's .env; existing environment values win. Fish uses its official API with FISH_API_KEY and FISH_REFERENCE_ID. Word timing uses OpenAI whisper-1 with OPENAI_API_KEY, or an explicitly configured OpenAI-compatible WHISPER_API.
The shared engine performs one POST per requested narration line and one transcription POST. Neither is automatically retried. It preflights both API configurations before creating voice audio. A whole --only tts rerun creates narration again; use the scaffolded paragraph pipeline for cached/surgical revisions. All generated audio needs listening review.
Shared engine
node <MEDIA_DIR>/scripts/audio.mjs --request audio_request.json --hyperframes . --out audio_meta.json --only tts
node <MEDIA_DIR>/scripts/audio.mjs --request audio_request.json --hyperframes . --out audio_meta.json --only bgm,sfx
The second command needs no API credentials. It preserves and validates any existing voice metadata. Example request (all files shown must be supplied by the user):
{
"lang": "zh",
"fish": {
"transport": "official-api",
"request": { "normalize": true, "prosody": { "speed": 1 } }
},
"lines": [
{ "id": "s0", "text": "这是一段待审听的旁白。", "sfx": [
{ "name": "confirmation", "path": "library/confirmation.wav", "offset_s": 0.2 }
] }
],
"bgm": { "path": "library/underscore.wav", "source": "reviewed-local", "volume": 0.18 }
}
fish.reference_id and fish.model are optional project overrides; no personal voice is supplied. fish.request uses official API fields, such as top-level temperature, top_p, normalize and prosody. The adapter owns text, reference ID and WAV format. It rejects credential-like fields and website backend configuration.
Output freezes assets/voice/<id>.wav and returns voices[].words as {id,text,start,end}. voice_id, fish_model and fish_transport describe the resolved non-secret profile. API keys and error bodies are never persisted. A transcription failure stops delivery with the voice WAV still on disk; do not invent word timing or claim captions are synchronized.
BGM/SFX come only from explicit local paths or the user's project manifest. Missing/non-audio files produce anomalies and are skipped; they never trigger search, generation or a fallback library. Review every anomaly before final delivery. Frozen files have content hashes so repeated filenames do not overwrite unrelated sources.
Discovery and approval
In a scaffolded project, use npm run audio:discover -- --type sfx --query "confirmation". No review catalogs are included. Install media-use in .agents/skills, or set MEDIA_USE_SKILL_DIR to that skill directory; set MEDIA_CATALOG or pass --catalog for your own workspace-local catalog. Empty search results mean the user must supply or select media. Catalog membership is not approval: audition, record source/license/hash and a review note, then ingest the exact bytes through media-use.
Reference guide
| Need | Read |
|---|---|
| Accounts, environment and optional tools | requirements |
| Fish request configuration | tts |
| Words and timing | transcription, TTS to captions |
| Caption authoring and cleanup | authoring, transcript handling |
| Local music and sound effects | BGM, SFX |
| Audition, gain matching, revisions | audio review |
| Optional stock APIs | stock media |
| Optional background-removal tooling | background removal |
API credentials do not replace the user's authorization for generation. Stay within the requested scope, preview a short sample, preserve original audio and keep listening/review decisions distinct from technical validation.
Files included
- assets/sfx/CREDITS.md
- assets/sfx/manifest.json
- references/audio-review-and-revision.md
- references/bgm.md
- references/captions/authoring.md
- references/captions/motion.md
- references/captions/transcript-handling.md
- references/remove-background.md
- references/requirements.md
- references/sfx-prompting.md
- references/sfx.md
- references/stock-media-apis.md
- references/transcribe.md
- references/tts-to-captions.md
- references/tts.md
- scripts/audio.mjs
- scripts/lib/sfx.mjs
- scripts/lib/tts.mjs
- scripts/tests/audio-engine.test.mjs
- SKILL.md

