VocAI
A semi-automated pipeline for anime-style Vocaloid music videos — beat-synced cuts, genuine Vocaloid vocals, karaoke subtitles. AI proposes, human disposes.
VocAI generates anime-style Vocaloid music videos in the vein of the classic Japanese producer scene: beat-synced cuts, 2.5D motion on illustrated stills, and karaoke lyric overlays. One project.json document drives a seven-stage Python pipeline — creative direction, audio and beat-grid analysis, genuine Yamaha Vocaloid vocal rendering (Hatsune Miku V6, driven programmatically), image generation for stills, motion, ASS karaoke subtitle generation, and final FFmpeg assembly — with a Gradio review UI as the human checkpoint between stages.
The design philosophy is “AI proposes, human disposes”: every stage’s output is reviewable and regenerating a single shot is cheap, so taste stays in the loop while the tedium is automated. Image generation runs locally on AMD hardware (ComfyUI via ZLUDA on an RX 7900 XTX), with cloud video models available for the heavy motion shots.
It’s equal parts media-pipeline engineering (timing math, subtitle formats, codec wrangling) and orchestration design — and a fun stress test of doing modern generative work on a non-NVIDIA stack.
Python, ComfyUI, FFmpeg, Gradio, Vocaloid 6. Private.