AI and Video analytics

Video analytics is the study of video files by using machine learning algorithms to identify patterns in them. These patterns could be any feature of that video, for example, lighting properties, characters in the shot, scene sequence, music, camera features, etc.

In this project, the final scene of the movie The Good, the Bad and the Ugly” is chosen as the case study. Then a Python algorithm is written to read the video frame-by-frame and output the frame whenever the scene is changed. The following image shows a proportion of all the frames.

The ideal frames that the code should be able to detect

The algorithm compares every two frames and tries to identify how much different they are. There are techniques in image recognition used in the past which are designed for studying the similarities of two images. Such techniques are also tested but proven to be inefficient. After running a few tests, the code successfully printed out all the desired frames. As the following image indicates, the frames maintain the narrative of the screenplay and no part of the whole movie is being missed.

Smart video analytics
Image recognition is used to extract the significant frames of the scene.

Since the start of this project in April 2019, further studies are in progress to create effective timelines of various features in a movie. Below is a few samples. These diagrams help in studying successful movies and how to implement them in new ones.

Comparing three features of a sequence of frames in a movie

For further inquiries about this project please use the contact form.