Amazon Launches Scalable Multimodal Video Search Using Nova Models
Key Facts
- What: AWS demonstrates building a multimodal AI data lake for media and entertainment, enabling natural language, video-to-video, and hybrid search across 792,270 videos.
- Scale: Processed 8,480 hours (30.5 million seconds) of video from Multimedia Commons and MEVA datasets in 41 hours using Amazon Nova Multimodal Embeddings and Nova Pro.
- Cost: First-year total $27,328 (on-demand OpenSearch) or $23,632 (with Reserved Instances); one-time ingestion cost $18,088.
- Tech: Generates 1024-dimensional audio-visual embeddings in AUDIO_VIDEO_COMBINED mode; segments videos into 15-second chunks; stores in Amazon OpenSearch Service k-NN index.
- Performance: Processes 19,400 videos per hour with 600 parallel workers on four c7i.48xlarge EC2 instances.
Amazon has detailed a production-ready architecture that moves media and entertainment companies beyond manual tagging and keyword search to true semantic, multimodal video understanding.
The company published a technical blueprint showing how to build an AI data lake capable of natural language search across massive video libraries. The solution leverages Amazon’s recently released Nova multimodal models on Amazon Bedrock together with Amazon OpenSearch Service to index and retrieve video content based on its actual audio-visual meaning rather than human-added metadata.
According to the AWS blog post, the reference implementation processed 792,270 videos totaling 8,480 hours of content in just 41 hours. The datasets used were the Multimedia Commons collection (787,479 videos averaging 37 seconds) and the MEVA dataset (4,791 videos averaging 5 minutes). This scale demonstrates the system is ready for real-world media archives, news libraries, and entertainment catalogs.
How the Multimodal System Works
The architecture separates ingestion and search workflows. During ingestion, videos stored in Amazon S3 are sent to the Amazon Nova Multimodal Embeddings API in asynchronous mode. The API automatically segments each video into 15-second chunks — a length chosen to balance scene change detection with manageable embedding volume.
The system uses the AUDIO_VIDEO_COMBINED mode to create 1024-dimensional embeddings that capture both visual and audio information in a single vector. AWS chose the 1024-dimension variant over the higher 3072-dimension option, achieving approximately 3x storage cost savings with only minimal impact on accuracy. Embedding generation itself is dimension-agnostic in terms of pricing.
A companion Nova Pro model generates 10-15 descriptive tags per video drawn from a predefined taxonomy. These tags are stored in a separate text index to enable hybrid search. The post notes that Amazon Nova 2 Lite may offer better accuracy at lower cost for future tagging workloads.
Four Amazon EC2 c7i.48xlarge Spot instances running 600 parallel workers achieved a throughput of 19,400 videos per hour. Because the Bedrock asynchronous API limits accounts to 30 concurrent jobs, the pipeline uses a job queue and polling mechanism to stay within quota while maximizing utilization.
Search Capabilities and Architecture
The resulting system supports three distinct query types:
- Text-to-video search using natural language converted to embeddings
- Video-to-video search by directly comparing video embeddings
- Hybrid search combining 70% vector similarity with 30% keyword matching
Embeddings are indexed in an OpenSearch k-NN index optimized for semantic similarity, while tags use a standard text index with BM25 ranking. This dual-index approach allows the system to blend semantic understanding with traditional metadata search.
Cost Breakdown Shows Production Readiness
AWS provided detailed first-year economics for the 792,270-video corpus. Total cost comes to $27,328 using on-demand OpenSearch pricing or $23,632 with Reserved Instances.
The one-time ingestion cost of $18,088 breaks down as:
- Amazon Bedrock Nova Multimodal Embeddings: $17,096 (30.5M seconds at $0.00056 per second batch pricing)
- Nova Pro tagging: $571
- EC2 compute (4× c7i.48xlarge Spot instances for 41 hours): $421
Annual OpenSearch Service costs are estimated at $9,240 (on-demand) or $5,544 (Reserved Instances).
Impact
“This changes how media companies will search their archives — moving from manual tagging to semantic understanding of what is actually happening in the video and audio,” the architecture effectively demonstrates.
For media and entertainment organizations sitting on petabytes of video, the ability to search by natural language description or find similar footage by example video represents a significant productivity gain. The economics shown — roughly $0.023–$0.034 per video for the first year including storage and search capability — make the approach viable for large-scale deployments.
The solution also highlights the growing importance of multimodal embeddings in enterprise AI strategies. By creating a unified semantic representation of video content, organizations can build more intelligent applications for content discovery, rights management, editing assistance, and recommendation systems.
What's Next
The AWS post positions this architecture as a starting point for custom media AI data lakes. Organizations can extend the system with additional Nova models, incorporate more sophisticated metadata, or integrate with generative applications that require precise video retrieval.
As multimodal models continue to improve, the cost of embedding generation is expected to decrease while accuracy increases. The post specifically recommends evaluating Nova 2 Lite for tagging tasks in new deployments.
The reference implementation is available through the AWS blog, giving developers and media companies a practical blueprint for building their own scalable video understanding platforms on AWS infrastructure.
Sources
- Multimodal embeddings at scale: AI data lake for media and entertainment workloads
- AWS official documentation and Bedrock quotas (referenced in the announcement)
All technical specifications, pricing, and benchmark data in this article are sourced directly from official announcements. Competitor comparisons use publicly available data at time of publication. We update our coverage as new information becomes available.