AI Video Transcription
AI-powered subtitling and accessibility for e-commerce video content
March 2025
Project
AI Video Transcription
Technologies Used
Challenge
The product videos of an online shop needed better accessibility through transcription and subtitles, a requirement that has gained relevance with Germany's Accessibility Strengthening Act (BFSG).
The scope was substantial: thousands of product videos had to be identified, retrieved, and equipped with precise subtitles. Manual processing would have been far too time-consuming and expensive. On top of that, the video files were not listed in any central overview and had to be located first.
Additional challenges included:
- The lack of a central video overview or any clear structure of the storage locations
- Protection measures against automated access, which required careful, rate-limited retrieval
- The need for word-accurate subtitles for optimal accessibility
- Required adjustments for brand names and industry-specific terminology
Solution
I built an end-to-end pipeline that solved the problem in several steps:
1. Crawler for video identification
By analyzing the website structure, I discovered that the product videos followed a fixed naming scheme based on the respective product number. I built a Python crawler that systematically checked the catalog and identified over a thousand videos this way, without needing any central overview.
2. Server-friendly download management
For retrieval, I implemented a sequential, rate-limited process with delayed requests and clean session handling. This way the videos were downloaded reliably and without straining the servers.
3. Audio extraction and AI transcription
I used FFmpeg to extract the audio tracks from the videos. For transcription I used a specialized fork of OpenAI's Whisper (whisper-timestamped) that delivers word-accurate timestamps, a feature otherwise only available in the paid OpenAI API.
4. Subtitle file generation
I converted the word-accurate transcriptions into SRT and VTT formats. I implemented an intelligent subtitle segmentation algorithm that ensured they fit the screen well and appear in sync with the spoken words.
5. Automatic thumbnail generation
Using OpenCV, I analyzed the videos to extract high-quality frames as thumbnails. The algorithm took factors like color variety and level of detail into account to ensure the thumbnails are representative and appealing.
Code Examples
Results
The pipeline delivered impressive results:
- Successful identification and processing of over a thousand product videos
- Automatic creation of precise subtitles with an estimated error rate below 5% for regular text
- Significant accessibility improvement for hearing-impaired customers
- Enormous time and cost savings compared to manual transcription
- Reusable pipeline that can be applied to new product videos
Particularly noteworthy are the quality of the automatically generated subtitles and the precision of the word timings, which enable a synchronized reading experience. The only limitation was the recognition of specific brand names and technical terms, which could be improved in future versions with customized dictionaries.
Insights
The workflow shows the video transcription process.