Technical Guide: Real-Time Video Translation with Kami Subs

Practical Summary and Core Functionality

Kami Subs is a local, open-source tool designed to provide real-time, AI-generated subtitles for browser-based video content. By leveraging faster-whisper for transcription and various translation engines, it overlays translated text directly onto video players in Chrome or Edge. Unlike cloud-based SaaS solutions, Kami Subs operates entirely on the user's machine, ensuring privacy and eliminating per-minute fees. The system architecture consists of a browser extension that captures tab audio and a Python-based backend that processes the audio stream via WebSocket. This setup allows for seamless integration with platforms like YouTube, Twitch, and various lecture or news streams, provided the content is not protected by DRM.

System Prerequisites and Hardware Requirements

To utilize Kami Subs effectively, users must operate on a Windows 10 or 11 environment. The browser must be a Chromium-based version (116 or higher), such as Google Chrome or Microsoft Edge. A Python 3.10+ environment must be available on the system PATH. Hardware performance is a critical factor for real-time operation; while the tool can run on a CPU, it will introduce a 3-5 second latency. An NVIDIA GPU with CUDA 12 and cuDNN 9 support is strongly recommended to achieve sub-second latency. Users should also ensure approximately 1.6 GB of disk space is available for the default large-v3-turbo model, which is downloaded during the initial execution.

Installation and Backend Configuration

The installation process involves two primary components: the browser extension and the Python backend. After cloning the repository, users must set up the backend environment and install dependencies. The following commands are required for the initial backend setup:

cd kami-subs\backend
python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt

Once the backend is prepared, the extension must be loaded into the browser via chrome://extensions using the 'Load unpacked' feature. To enable the automatic backend startup, users must register the Native Messaging host by executing the provided PowerShell script. This ensures the backend launches automatically when the 'Start' button is clicked in the extension popup, removing the need for manual terminal management.

Operational Workflow and Usage

Once installed, the workflow is straightforward. Users navigate to a video tab, open the Kami Subs extension, and select their preferred model, device, and language settings. Clicking 'Start' initiates the backend, which connects to the browser via WebSocket at ws://127.0.0.1:8765/ws. The extension captures the audio stream, resamples it, and sends it to the backend for transcription and translation. The resulting JSON transcripts are then rendered as an overlay on the video. Users can adjust font size and position directly from the popup interface. When the session concludes, clicking 'Stop' terminates the backend process automatically.

Advanced Translation and Environment Variables

For users requiring high-quality, offline translation, the tool supports Meta's NLLB-200 model. This can be enabled by setting the KAMI_TRANSLATOR environment variable to nllb. Enabling this feature requires additional packages: transformers, torch, and sentencepiece. If these are not installed, the backend will silently fall back to the Google Translate endpoint. Advanced users can further tune performance by modifying environment variables such as KAMI_MODEL, KAMI_DEVICE, and KAMI_COMPUTE. These variables allow for fine-grained control over resource usage, such as switching to int8 quantization to reduce VRAM consumption on lower-end GPUs.

Limitations and Troubleshooting

Users should be aware of several documented limitations. The most significant is that DRM-protected content, such as Netflix or Disney+, will return silence because Widevine encryption prevents tabCapture from accessing the audio stream. Additionally, the current implementation is limited to a single active tab per extension instance. If subtitles appear delayed, it is likely that the selected Whisper model exceeds the hardware's real-time processing capacity; switching to a smaller model like tiny or small is recommended. If the Native Messaging host fails to register, users should verify that the registry key points to the correct path and that the extension ID matches the manifest.

Choosing When to Use Kami Subs

Kami Subs is best suited for users who require real-time, private, and free translation for non-DRM video content. It is an ideal solution for students watching lectures, viewers of live streams, or individuals consuming news content in foreign languages. Because it runs locally, it is particularly valuable for users who prioritize data privacy or those who wish to avoid the recurring costs associated with cloud-based transcription services. However, users must have the technical aptitude to manage a Python environment and the hardware resources to support local AI inference. For more information, visit the official repository at https://github.com/MohammdKopa/kami-subs.