Computer vision almost perfectly recognize objects in static images, and at the moment, the problems mostly arise only with the videos. Nevertheless, developers continue to create neural network for recognition of various actions on video, and one of the latest and most interesting projects is the work of an employee of Netflix Amir Zia. Using a database of 100 Hollywood films, he has trained a neural network to recognize a scene with a kiss, while avoiding overly explicit scenes.
The neural network recognizes the kisses, focusing on the movements of the lips of the actors and the sounds touching. To show the neural network what it needs to pay attention, Amir Ziai personally tagged kissing scenes with in the sample of the hundreds of Hollywood movies. Among the treated films were Anna Karenina (1935), Ghost (1990) and Casino Royale (2006). The developer did not expand the base of training films, because the neural network is stopped “wiser” after the eighties of the film. In the end, Amir was allocated 263 scene with a kiss and 363 of the stage without kissing.
The neural network learned to recognize kissing
A system of recognition of kisses consists of two parts. Analyzes first-second samples of music and video is called a binary classifier — it is a convolutional neural network ResNet for the analysis of visual data, and the neural network VGG to explore audio. The second part of the system, segmentator, selects fragments with kisses and gives them so that they do not recur.
After receiving the video, the system examines it for the presence of scenes with kissing, and displays them in the form of excerpts. Testing showed that the neural network copes with its task with 95% accuracy. However, it was observed that the system frequently makes an error in some scenes where kisses come amid expansive landscapes, or where kisses happen too fast.
Why such a neural network?
Though Amir Ziai and is a member of Netflix, created a neural network has no relation to the service. In recognition of kisses can be very useful in streaming services, as well as in Facebook, Instagram and TikTok. At least, slightly changing the algorithm, you can create a sorting system to explicit scenes. Also, this system can automatically sort videos by subject folders.
On the topic of neural networks recommend you to read about its ability to animate static portraits. Thanks to the work of the specialists from the Center for artificial intelligence Samsung and portraits of Dostoyevsky, Mona Lisa and albert Einstein literally alive — you look at them in our material.
To discuss this and other topics of science and technology, in our Telegram chat. In addition to interesting people, there you will find announcements of fresh materials of our website!