Subscribe to our daily and weekly newsletters to receive the latest updates and exclusive content on industry-leading AI reporting. Learn more
Nvidia started Blueprint for AI agents That can today analyze videos as part of it CES 2025 Opening speech by CEO Jensen Huang.
With the new Nvidia AI Blueprint powered by Metropolis, companies and individuals can increase their productivity and security, and could even help Nvidia’s CEO improve his fastball pitch.
The next big moment in AI is literally in sight.
Today, more than 1.5 billion corporate cameras worldwide generate approximately 7 trillion hours of video per year. However, only a fraction of them are analyzed.
It is estimated that less than 1% of industrial camera video is viewed live by humans, meaning critical operational incidents can go largely unnoticed.
This is associated with high costs. For example, manufacturers lose trillions of dollars annually due to poor product quality or defects that they could have detected or even predicted earlier by using AI agents that can perceive, analyze and help humans take action.
Interactive AI agents with integrated visual perception capabilities can serve as such
Always-on video analysts help factories operate more efficiently, increase worker safety, keep track of everything running smoothly, and even improve an athlete’s game.
To accelerate the development of such agents, Nvidia today announced early access to them
a new version of the Nvidia AI Blueprint for video search and aggregation. Built on the Nvidia Metropolis platform – and now expanded with Nvidia Cosmos Nemotron Vision Language Models (VLMs), Nvidia Llama Nemotron Large Language Models (LLMs), and Nvidia NeMo Retriever – the blueprint provides developers with the tools to build and deploy AI agents that can analyze large amounts of video and image content.
The design integrates the Nvidia AI Enterprise software platform – which
includes Nvidia NIM microservices for VLMs, LLMs, and advanced AI frameworks for on-demand generation – to enable batch video processing that is 30x faster than real-time viewing.
The draft includes several agent AI features – such as thought chain reasoning, task scheduling and tool invocation – that can help developers streamline the creation of powerful and diverse visual agents to solve a range of problems.
AI agents with video analytics capabilities can be combined with other agents with different capabilities to enable even more sophisticated agent AI services.
Enterprises have the flexibility to build and deploy their AI agents from the edge to the cloud.
How AI agents from Video Analyst can help industrial companies
AI agents with visual perception and analysis capabilities can be fine-tuned to assist companies with industrial operations by:
● Increase productivity and reduce waste: Agents can help ensure this
Standard operating procedures are followed for complex industrial work
Processes such as product assembly. They can also be carefully tuned
Observe and understand nuanced actions and the order in which they occur
implemented.
● Increase asset management efficiency through better use of space:
Agents can help optimize inventory management in warehouses by performing 3D
Volume estimation and centralizing understanding across different cameras
streams.
● Improve security by automatically generating incident reports and
Summaries: Agents can process massive amounts of video and summarize them into contextual, informative incident reports. They can also help ensure
Compliance with personal protective equipment in factories, improving workers
Safety in industrial environments.
● Prevent accidents and production problems: AI agents can detect
atypical activities to quickly mitigate operational and security risks, be it in one
Warehouse, factory or airport, at an intersection or in another urban environment.
● Learn from the past: Agents can search operational videos
Archive and relevant information from the past and use it to solve problems or create new processes.
Video analysts for sports, entertainment and more
Another industry where video analytics AI agents will make a name for themselves is sports – a $500 billion market worldwide, with projected growth of hundreds of billions over the next few years.
Coaches, teams and leagues – professional and amateur – rely on video analytics to assess and improve player performance, prioritize safety and increase fan engagement through player analytics platforms and data visualization. With visual AI agents, athletes now have unprecedented access to deeper insights and opportunities for improvement.
During his CES opening keynote, Nvidia’s Huang introduced an AI video analysis agent that evaluated an amateur baseball player’s fastball pitching skills compared to those of a professional. With recorded video
from the ceremonial first pitch Huang threw for the San Francisco Giants
The video analysis AI agent was able to suggest areas to the baseball team
Improvement.
The $3 trillion media and entertainment industry is also expected to benefit from AI video analyst agents. Through the Nvidia Media2 initiative, these agents will help drive the creation of smarter, more tailored and more impactful content that can be adapted to individual viewer preferences.
Worldwide acceptance and availability
Partners from around the world are integrating the blueprint for building AI agents for video analytics into their own developer workflows, including Accenture, Infosys, Linker Vision, Pegatron, TATA Consultancy Services (TCS), Telit Cinterion and VAST.
Source link
