2021-09-07 Boeun Jang, Tokyo Trade Center, Japan
– Pay attention to technologies such as AI and AR using sensors and data –
– Various technologies are used in non-face-to-face situations, and there are also technologies that were difficult to use without spectators –
From July 2021 to August 7, 23, the '8 Tokyo Olympics (Tokyo 8)' was held in the metropolitan area centered on Tokyo, Japan. In past Olympics, the latest technology has been incorporated and used for relaying and measurement. The Olympics are also a venue to showcase new technologies, such as HD high-definition broadcasting at the 2020 London Olympics and 2020G self-driving buses at the 2012 PyeongChang Winter Olympics.
We would like to find out what kind of technology was used in this Olympic Games, which were held with unprecedented crowds in the midst of the COVID-19 situation.
Instantly reproduce the match with sensors and AI
At the Tokyo Olympics, the digital technology to understand the race has greatly evolved to the extent that the race process of track and field and swimming can be reproduced. The secret lies in image processing through new sensors and AI.
At the Tokyo Olympic Athletics Championships, for the first time in the Olympics, a 'motion sensor' was introduced that records the movements of athletes by attaching them to the body. This motion sensor was developed and operated by OMEGA Timing of Switzerland. The weight is 16g, and it is inserted into the athlete's wrist (number table) to measure 3-axis acceleration and angular acceleration. OMEGA Timing combined this sensor data with AI-processed data from camera images installed on audience seats and roofs to measure driving acceleration during races, trajectories of athletes running in lanes and corners, and weight load on athletes. There are 2000 types of data items, and the data is provided to players and coaches immediately after the game to be used for analysis. It is also used in replay videos that explain the game in relay broadcasts.
[OMEGA Timing's motion sensor (left) and competition camera (right)]
Source: Nikkei xTech
The data technology of the game has also changed the way of enjoying TV and online spectating. OBS (Olympic Broadcasting Service) introduced a technology that displays player information in the form of speech balloons by following the movement of players when broadcasting international broadcasts of ball games and team events such as basketball. In addition, important scenes such as scoring can be analyzed in detail by changing the cut or replay video.
The introduced technology is 'True View' developed by Intel, and it is a technology that makes 3DCG Volumetric Video. Dozens of high-resolution 5K cameras installed around the stadium capture the entire court and create XNUMXD image data (voxel) that records the width, length, and depth.
Camera shot image in True View
Material: Intel
Each camera is connected to a dedicated server in the stadium with an optical line cable, so video data is stored and processed on the server. By using this video data, when broadcasting replay video and highlights, viewers can see the game video reproduced from a 360-degree view, including the player's gaze and the goal line. It is said that it is also used to confirm play, which is difficult to judge with the naked eye.
Tokyo Olympics without spectators, cheering on social media
The Tokyo Olympics were held without spectators due to the spread of COVID-19 in Japan, but voices of support from each country were delivered at many stadiums. This is because messages of cheering videos gathered from all over the world flowed from the large video boards and information displays installed in the stadium.
The 'TOKYO 2020 Share The Passion Project', which delivers cheering videos and texts to players, is led by the Tokyo 2020 Organizing Committee, and has created a structure that allows easy participation from all over the world through the use of social media such as Twitter.
There are three types of messages: 'Videos within 3 seconds cheering players with dance and applause tailored to pre-prepared sound sources', 'Freely filmed videos within 15 seconds', and 'Text written within 30 characters'. When these cheers are posted on social media (Facebook, Instagram, TikTok, Twitter), the organizing committee extracts them into the system and edits them in a form that can be played back on the stadium video board.
The task is automatic, and the auto-fill feature was developed by Alibaba Cloud, a subsidiary of China's Alibaba Group, which sponsors the competition. AI technology that automatically detects and excludes videos that violate the terms of use, such as against public order and morals, is also Alibaba Cloud's technology.
Heat stroke and dense data warns
The Tokyo Olympics, held in midsummer amid the COVID-19 situation, required measures to avoid heat stroke and crowding. At the National Stadium, an outdoor stadium, a sensor-based heat stroke countermeasure system was introduced to about 1000 people who mainly move outdoors. This system is a combination of a biosensor that measures body temperature and pulse rate worn on the ear and a smartphone app, from 'safety' to 'attention' to urge hydration, 'warning' to urge rest, and 'to take immediate action'. It estimates the situation in four stages up to 'danger' and instructs an appropriate response.
[Alibaba Cloud's Heatstroke Countermeasure Biometric Sensor]
Source: Nikkei xTech
The smartphone app always transmits body temperature, pulse rate, and location information to the cloud service. The cloud service estimates the risk of heat stroke by combining body temperature and pulse rate by referring to the 'heat index' of the place where the person concerned is located in the location information.
The heat index was calculated based on temperature and humidity, radiated heat, and solar radiation by environmental sensors installed at 14 places in the National Stadium. This system was introduced and operated by Alibaba Cloud, a subsidiary of Alibaba in China.
Data was also used to visualize vehicles and people entering and exiting the stadium. It is sensor data that detects camera image data and traffic volume and crowds. The organizing committee collects anonymous video data, traffic data, and crowd data by placing IoT devices such as sensor-attached cameras in stadiums, parking lots, and buses. This data is processed with image recognition AI, etc., and the number of people in the area is measured to determine the share of the area and the traffic volume. When the target area reaches the threshold of capacity, it has a structure to notify the event management staff in real time.
The staff who received the notification were instructed to avoid crowding in the area or to take measures such as increasing staff in areas where there is a shortage of manpower. The Tokyo Olympics is the first time that data collected by sensors has been visualized and used for stadium operation.
Intel was responsible for the development and operation of the system. The IOC is promoting the 'IOC Data Utilization Project', and Intel is providing technical support for this project. The acquired data will be accumulated as performance and will be used in the operation of the next competition.
Face authentication gate considering diversity
The face authentication system, introduced to reduce the burden of expenses, has evolved to meet the 'Diversity & Inclusion' of the Tokyo Olympics. NEC, which developed the system, improved the design and function of the gate so that even a person in a wheelchair can perform face authentication smoothly.
[Face authentication gate used in Tokyo Olympics]
Source: NEC
At the Tokyo Olympics, about 300 face authentication gates were installed at the entrance of all stadiums, at the entrance to the athletes' village, and at the entrance to Tokyo Big Sight, where MPCs where news and broadcasters gather. The face authentication gate works with the system to verify the identity of the person with the face photo and the IC chip of the access card, and the target number is more than 10 people.
Face authentication gates using these two elements still exist, but the gate developed by NEC this time adopted a streamlined recessed design for the support that supports the camera. This is so that a person in a wheelchair can approach the gate without any hindrance and proceed with the admission procedure. The card holder has also been adjusted to a position that is easy to use for both standing and wheelchair users. Face authentication is also required when placing the card, but the camera at the top of the gate has a function to identify an approaching person and authenticate the face, so there is no need to bring your face. Based on this design, NEC is planning to adopt universal design and provide it to companies and government offices.
implication
Although the Tokyo Olympics, originally scheduled to be held in 2020, were postponed for a year due to the COVID-19 outbreak, the spread of infections continued and eventually was held on a reduced scale with no spectators. For this reason, the various ITs that were supposed to support pleasant spectators at the stadium were unfortunately not utilized much. For example, AR glasses, which were scheduled to be used for swimming watching, and NTT's relay image transmission technology, which was scheduled to be used for public viewing, lost an opportunity to show to the general public.
A manager of Company N, a company related to the Tokyo Olympics introduction system, said, "As the decision to hold no audience was delayed, there was very little time to respond. The opportunity to showcase digital technology in more games has disappeared.”
On the other hand, as a countermeasure against the COVID-19 infection, technology that had not had a chance to compete in the previous Olympics was used, such as managing the test results and health status of the event officials through a dedicated information system.
The Tokyo Olympics, held in such unprecedented circumstances, seem to have introduced cutting-edge technology without any attachment to their own country or other countries. In fact, many of the technologies of Intel in the United States and Alibaba Group in China were employed as core systems, and several Korean companies, including Samsung, were also used in various fields such as security and telecommunication.
The Olympics are not just about watching and enjoying the games, but also a place to introduce and showcase cutting-edge technology to the public. In this Olympic Games, the most popular technologies, such as AI, AR, and 5G, were fully utilized to support the competition. At the 2024 Paris Olympics and 2028 Los Angeles Olympics, what technologies will receive attention and become mainstream is already paying attention.
Source: Intel, NEC, etc. website, Tokyo 2020, Nikkei Electronics, Nikkei Shimbun, Nikkei Xtech, KOTRA Tokyo Trade Center