{"id":632136,"date":"2020-01-22T11:00:49","date_gmt":"2020-01-22T19:00:49","guid":{"rendered":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/?p=632136"},"modified":"2020-01-28T10:27:18","modified_gmt":"2020-01-28T18:27:18","slug":"project-rocket-platform-designed-for-easy-customizable-live-video-analytics-is-open-source","status":"publish","type":"post","link":"https:\/\/newed.any0.dpdns.org\/en-us\/research\/blog\/project-rocket-platform-designed-for-easy-customizable-live-video-analytics-is-open-source\/","title":{"rendered":"Project Rocket platform\u2014designed for easy, customizable live video analytics\u2014is open source"},"content":{"rendered":"

\"code<\/p>\n

Thanks to advances in computer vision and deep neural networks (DNNs) in what can arguably be described as the golden age of vision, AI, and machine learning, video analytics systems\u2014systems performing analytics on live camera streams\u2014are becoming more accurate. This accuracy offers opportunities to support individuals and society in exciting ways, like informing homeowners when a package has been delivered outside their door, allowing people to give their pets the attention they need when out for the day (opens in new tab)<\/span><\/a>, and detecting high-traffic areas so cities can consider adding a stop light.<\/p>\n

While DNN advancements and DNN inference are enablers, they alone are not enough when it comes to extracting valuable insights from live videos. Live video analytics requires keeping up with video frame rates, which can be as fast as 60 frames per second, making it crucial to effectively filter frames and avoid the costly processing of each frame. Project Rocket (opens in new tab)<\/span><\/a> provides a framework to do exactly that.<\/p>\n

Rocket-powered<\/h3>\n

Rocket\u2014which we\u2019re glad to announce is now open source on GitHub (opens in new tab)<\/span><\/a>\u2014enables the easy construction of video pipelines for efficiently processing live video streams. You can build, for example, a video pipeline that includes a cascade of DNNs in which a decoded frame is first passed through a relatively inexpensive \u201clight\u201d DNN like ResNet-18 (opens in new tab)<\/span><\/a> or Tiny YOLO (opens in new tab)<\/span><\/a> and a \u201cheavy\u201d DNN such as ResNet-152 (opens in new tab)<\/span><\/a> or YOLOv3 (opens in new tab)<\/span><\/a> is invoked only when required. With Rocket, you can plug in any TensorFlow (opens in new tab)<\/span><\/a> or Darknet (opens in new tab)<\/span><\/a> DNN model. You can also augment the above pipeline with, let\u2019s say, a simpler motion filter based on OpenCV background subtraction (opens in new tab)<\/span><\/a>, as shown in the figure below.<\/p>\n

\"diagram\"

The above figure represents one of several video pipelines that can be built for efficient, customizable live video analytics with the Project Rocket platform. In this pipeline, decoded video frames are filtered first using background subtraction detection and then low-resource DNN detection. Frames requiring further processing are passed through a heavy DNN detector.<\/p><\/div>\n

Cascaded pipelines, like the one above, allow for very efficient processing of live video streams by filtering out frames with limited relevant information and being judicious about invoking resource-intensive operations. Plus, Rocket also makes it easy to ship the outputs of the video analytics, such as the number of relevant objects in an object-counting application, to a database for after-the-fact review.<\/p>\n

\"diagram\"

The Project Rocket video analytics platform (above) is self-contained and allows people to plug in TensorFlow and Darknet DNN models to create pipelines for object detection, object counting, and the like to drive higher-level applications such as traffic prediction analysis and smart homes.<\/p><\/div>\n

Making streets safer<\/h3>\n

Project Rocket has been focusing on smart cities as its driving application. In partnership with the city of Bellevue, Washington (opens in new tab)<\/span><\/a>, we used the framework to help make the city\u2019s street system safer for drivers, riders, and pedestrians as part of its Vision Zero initiative to reduce traffic-related fatalities. With aggregate car and bicycle counts provided by a system built on the framework, for example, the city was able to assess the value of adding a bike lane (opens in new tab)<\/span><\/a> to its downtown area.<\/p>\n