The concept of sports broadcast changes!?A new expression created by an unmanned AI camera seen in world gymnastics / rhythmic gymnastics -IT is even closer.Soft Bank News

The "2021 World Gymnastics / Rhythmic Gymnastics Championship Kitakyushu Tournament" was held in October 2021.Behind the brilliant acting of the players, experiments were actually being conducted to deliver a new form of sports broadcast.

We talked to the person in charge about the purpose of the experiment and the points of each technology.

Possibility of video content spread with AI camera

A group performance of rhythmic gymnastics taken with an unmanned AI camera.If you look at the video, you can see that the direction and zoom are adjusted in detail while recognizing the person.

Kanako Mori, who was in charge of the automatic tracking camera of the unmanned AI

Mori: Automatic tracking camera with unmanned AI has been installed at three points in the venue so that the group's performance can be seen from various angles.

In the synchronized movement of the players and the beautiful formations of the highlights, the players move in the same direction or fall apart.In this unmanned AI camera, AI recognizes the person, calculates instantly so that the left and right edges of the formation are settled, tracking in the direction of travel, or zooming out in the direction of the movement.AI is working automatically.

There are also spectators and referees in the venue, so the AI is specified in a detailed calculation of the camera orientation so that it can be pursued only by the movement of humans other than the players.It was hard to design an easy -to -recognize algorithm.I started preparing around summer, but I went to the venue for rhythmic gymnastics many times, learned various cases, such as the arrangement of the competition area and the movement of the referee, and derived the optimal setting value.

Mori: Recently, the number of multi -angle images such as VR and FR is gradually increasing, but in order to create a multilayal video content, there are many human resources such as camera and other equipment, such as photographers who operate the equipment.It will be.In addition, there is an aspect that photographers can shoot only at an angle that can secure space in which people are.

If you can automatically follow it with an AI camera like this time, you can take a new angle, such as reducing costs and labor costs, can be placed only for the camera, for example, from a place where people like the ceiling can not enter.I hope that a new multi -angle content will be born.

Mori: This seems to be watching live distribution videos at the seats of the venue.

Did you know that conventional live broadcasts are generally distributed several tens of seconds later than in the actual venue?

The video of the camera taken locally does not reach your terminal as it is, but is delivered through a lot of processes, such as compressing and converting data.

In this distribution, a low -delay distribution system that minimizes the time lag is adopted.In addition to reducing time by streamlining the huge processes generated during data conversion, it is not necessary to be buffering by performing a special protocol of data transmission and packet loss processing, almost real -time (0..1 to 0.I succeeded in regenerating smoothly even with a 3 second delay).

In the video, you can see that the movements of the players at the actual venue and the tablet images are almost the same, and are delivered in real time.

The customer at the venue saw this distribution video, but I was surprised that it seemed to be a real time that seemed to be shooting a video with a tablet.On TV, you can't change your perspective freely, but you can operate and switch the three viewpoints by yourself, so you said, "I want to see it at home."

It was my first time to conduct a distribution experiment at a venue where there were customers and the radio waves of the mobile phone was flying, but I found some issues to improve accuracy and have a very meaningful experiment.I think it was done.