Under development by Canon USA. The aim of the VR face-to-face application "Kokomo" that can be used with "digital camera + VR headset"

Interview with Canon USA Konno

——Please give us an overview of Kokomo announced this time.

"Kokomo" is a VR platform application that realizes face-to-face communication in virtual space using live 3D images with simple equipment such as "Canon digital camera" and "commercially available VR headset". Canon USA is developing it.

Using the image processing technology that Canon USA has cultivated over many years, the image of the user can be reproduced in a virtual space in 3D from the images taken with a compatible Canon camera, and in the virtual space using a commercially available VR headset. With the video call "Immersive Call" that you can make while watching the other party and the surroundings in 3D, you can enjoy the call as if you were talking face-to-face while checking each other's appearance with people in remote places. ..

By realizing face-to-face communication in virtual space with a simple configuration of Canon cameras, commercially available VR headsets, and smartphones, without preparing multiple cameras or complicated systems that require professional setup, far away We aim to realize an easy-to-use app for all users who want to "meet people in person" such as friends, family, and lovers who live apart.

——Please tell us the background behind the start of Kokomo's development.

Canon USA launched the "Post COVID Project" in-house in the summer of 2020 and has been engaged in various activities. Through the pandemic of the spread of the new coronavirus infection, many people have come to use video calls as a matter of course to connect with people in their personal lives and work. On the other hand, many people feel that existing video calls are not a substitute for face-to-face conversations with friends and family who are actually there, and that they are grateful for having face-to-face conversations. I think so.

Therefore, in October 2020, I decided on the initial concept, thinking that it would be great if I could realize "being there" from anywhere. Development began in November 2020 and was unveiled at CES 2022 in January 2022. We are proceeding with development at a speed of about one year from the start of development, and we are aiming for release within 2022.

——Mr. Konno was also involved in the “EOS Webcam Utility”.

Under the influence of the new coronavirus, "EOS Webcam Utility" can be installed even without knowledge of cameras and personal computers in order to meet the growing need for "easy use of higher quality images". We aimed to provide simple and easy software that anyone can use without even being aware of its existence. As of the end of December 2021, we were able to quickly develop solutions that meet needs, such as releasing the beta version at the fastest speed in the industry, about 3 weeks after planning, and as of the end of December 2021, it was accepted by many customers with about 3 million downloads worldwide. It led to the result.

Similarly, "Kokomo" aims to speedily develop solutions that meet needs by collecting actual voices from users who have the problem of not being able to meet people directly due to lockdown in a place close to the market. .. In addition to that need, I think there is also a need to meet directly with friends and family who have lived far away due to overseas assignments before the Korona-ka. We would like to solve such a customer's pain point of "not being able to meet people directly" with our technology and propose a new form of visual communication.

interview

Until Canon's Web camera software "EOS Webcam Utility Beta" is created

Over 300,000 downloads worldwide Listen to the development process and future plans

——Is the fact that the development of Kokomo started in November 2020 already looking ahead by the time the “EOS Webcam Utility” became the official version?

That's right. In the summer of 2020, when the "Post COVID Project" was launched in Canon USA, we started to create a concept that would lead to Kokomo, and in September 2020, when the official version of the "EOS Webcam Utility" was released, the concept was almost solidified.

news

Canon releases Web camera software "EOS Webcam Utility 1.0"

Official version that also supports Japanese

——You said that “EOS Webcam Utility Beta” was released about three weeks after the project was planned. How was the internal response and evaluation including the results?

In the in-house award for inventions of the entire Canon Group, it was commended for being the first to respond to the demand for Korona-ka. In addition to development, related departments such as top management, intellectual property, marketing, services, and public relations worked together to eliminate time loss and shorten the normal turn, which led to the fastest release in the industry. ..

The success of this Canon USA united activity was one of the triggers, and the aforementioned "Post COVID Project" was launched, leading to projects such as the hybrid work solution "AMLOS" in addition to the "Kokomo" developed and announced at CES 2022. I am.

The EOS Webcam Utility also won the Good Design Award in 2020, and it is also evaluated that it operates an online user forum in-house and realizes the ideal communication style that manufacturers should aim for in the social era. I did.

news

[CES2022] Canon, a web conference "AMLOS" that delivers multiple viewpoints with a single camera.Speakers and whiteboards are up

——Please tell us about the points you are focusing on in the development of Kokomo.

VR is widely used in a wide range of industries such as entertainment, tourism, and education. While communication in virtual space has become the mainstream of solutions via avatars, it is important for us to reflect ourselves, facial expressions, and clothes, and to look at the appearance of our important partner and make eye contact in communication. I thought that, and started planning. By using Canon's camera and image processing technology, we want people to forget that they are wearing a VR headset, and we want to make VR more realistic, so we are particular about live-action photography of people.

——You said, “Look at your important partner and match your eyes”. Is there a mechanism to match your eyes while wearing VR headsets? If there are any new technologies there, please let us know.

We cannot talk about the details because it is under development, but we are proceeding with the development based on the concept of "being able to see the other person's eyes and make face-to-face communication" in the virtual space.

In fact, at "CES 2022", visitors had a hands-on exhibition in which they had a conversation with Canon USA employees. The exhibition is a one-way way for customers to see Canon USA employees, but of course we are thinking of making it bidirectional when commercializing.

We will use the software development technology of Canon USA of the Irvine Office in California, which has been developing powerful software solutions utilizing Canon hardware for about 30 years, so please wait for a while until the release.

——Is Kokomo's face-to-face call limited to one-to-one?

I can't talk about the details because this is also under development, but I'm aiming for a one-to-one call first. In the future, I would like to be able to have a conversation with a few friends and the whole family.

——Please tell us more about the exhibition held at CES. How was the response?

At the booth, we asked visitors to wear VR headsets and made a video call while watching the other party (Canon USA employee) and the surrounding scenery in 3D in a virtual space that reproduced a trailer truck with a spectacular view of Malibu, California. We had you experience "Immersive Call".

Looking at the customers who actually experienced it, I feel a response to the Kokomo concept, such as conversations with gestures that are not done in ordinary video calls. In addition, some other companies who visited the venue said, "We were thinking about having such an experience," and I realize the importance of developing with a sense of speed.

——You can use “Meta Quest” (formerly Oculus series) for VR headsets, right?

The VR headset "Meta Quest" by Meta (formerly Facebook) in the United States is the most widely used VR platform / VR headset in the United States, and because it can support the technology of "Kokomo", it is the most suitable platform for customers. I decided that there was. First, we plan to distribute it on the "Meta Quest Store", and for other platforms, we will closely monitor market trends and consider them as appropriate.

——Does “easy equipment” mean that the corresponding cameras are a wide range of products?

At the time of the exhibition at CES 2022, we announced the compatibility of "EOS M200" and "EOS R5", but the specific compatible model is undecided. First of all, the intention of announcing these two models is to include the message that we are aiming to support a wide range of entry models such as the EOS M200 because one of the concepts is ease of use.

Of course, by using "EOS R5" and "RF 5.2mm F2.8 L DUAL FISHEYE", you can have a VR call experience with a high-quality 3D 180-degree VR image that has never been seen before. increase. Stay tuned for a complete list of compatible equipment when it's released in 2022.

news

Canon, stereo fisheye lens "RF 5.2mm F2.8 L DUAL FISHEYE" for 180 degree VR shooting. 275,000 yen

(14:00) Product images have been added.

——How do you use your smartphone with Kokomo? Can it be used without a personal computer?

We are developing it so that it can be used only with Canon cameras, commercially available headsets, and smartphones, and no personal computer is required. The smartphone that is familiar with the existing calling app is used as the trigger for the call, and the member registered with the "Kokomo" app downloaded to the smartphone is selected to start the video call "Immersive Call".

When the other party picks up the incoming call, a message such as "Please wear the VR headset" is displayed, so just specify the virtual space suitable for the content of the conversation, wear the VR headset to each other and start the conversation I am considering how to use it to start communication.

——Will Kokomo be sold as a solution? Or will it be distributed to general users like the EOS Webcam Utility?

First of all, we are considering offering through the VR platform "Meta Quest Store", but the price and sales method are undecided as to how to offer it. We are collecting customer feedback with the aim of launching in 2022.

——The name “Kokomo” is also characteristic.

The lyrics of an English song were close to the product concept and inspired me. It is a famous song in the United States, and one visitor sometimes introduced the original song of inspiration as "Do you know this song?" During the session. I hope you will get to know me.

——You said that you are developing for a release within the year. Please tell us if you have any future prospects.

We cannot talk about the details because it is under development, but we will proceed with the development using Canon cameras and Canon USA image processing technology. For example, in the VR space that is the background of a call, more than 2,000 images are stitched together using "EOS R5" that draws beautiful colors with high image quality and high precision in every detail, and Canon software "Digital Photo Professional". I am making it.

At the CES venue, we reproduced a trailer truck with a spectacular view of Malibu, California in a VR space, but in the future we are thinking of increasing the number of good places to meet and talk with people in a VR space. For example, there is a wide range of VR, such as the rooftop bar in New York, the beach in Hawaii, and the space that suits what you are talking about, or the space you want to visit. In addition to the feedback from this CES exhibition, we will continue to listen to customer feedback and proceed with development.