WindCone

WindCone is a full-stack content intelligence platform that automates the end-to-end pipeline from trend discovery to filmed video: scrape trending Reels, analyze them with Google Gemini Vision, synthesize patterns across multiple videos, generate scripts, clone a voice, and produce a lip-synced talking-head video — all in one workflow.

Platforms

• Web app — React 19 + Vite + Tailwind CSS, deployed on Vercel.
• iOS & Android — Flutter mobile app with Firebase auth.
• Mac & Windows desktop — Electron-packaged installers (DMG / EXE).
• Chrome Extension — Manifest V3, scrapes Instagram from within the browser.
• MCP server — Exposes trends, scripts, and analyses as Model Context Protocol tools for Claude Desktop and ChatGPT.

AI Pipeline

• Google Gemini 3 Flash (Vision + File API) — Analyzes video and audio in parallel for each scraped Reel. Aggregates findings across multiple videos to synthesize trend patterns. Generates ready-to-film scripts in the detected content style.
• Fish Audio API — Clones a voice from a reference audio file, then uses the clone for TTS of the generated script.
• FAL.ai (Sync Labs) — Lip-syncs the cloned-voice audio to a video frame, producing a talking-head video from a photo.
• HeyGen API — Alternative avatar pipeline for photo-based talking-head generation.

Engineering Highlights

• Instagram automation at scale — Playwright with persistent authenticated sessions bypasses Instagram's anti-bot defenses. Phase 0 (auto-login) maintains the session across searches in containerized Railway environments.
• Parallel video analysis — asyncio runs visual and audio Gemini analysis concurrently against the same uploaded file, halving analysis time vs. sequential calls.
• 4-phase async pipeline with real-time status — Search → Download → Analyze → Summarize runs as a background asyncio task. The frontend polls job status and renders per-Reel progress in real time.
• Voice cloning + lip-sync orchestration — Coordinates Fish Audio training (polling with exponential backoff), TTS generation, and FAL.ai lip-sync into a single pipeline. Falls back to HeyGen if the primary pipeline fails.
• MCP protocol integration — Backend exposes all data (trends, scripts, video analyses) as Claude/ChatGPT tools via the Model Context Protocol with API key authentication, enabling AI assistants to query WindCone directly.
• FFmpeg replaced with OpenCV — Thumbnail extraction moved from FFmpeg subprocess to OpenCV to avoid Railway's container restrictions on spawning heavy processes.

Deployment

Backend on Railway (FastAPI + Python + SQLite on mounted volume). Frontend on Vercel. Flutter on App Store and Google Play. Electron packaged as DMG and EXE. Extension published to Chrome Web Store.