Machine Learning Event Update: NeurIPS 2020の画像

Recommended for

  • This article is recommended for people interested in the deep learning-based approaches and some use cases, to perform soccer analytics from gameplay videos.

Expected reading duration

10 minutes

Introduction: AI in Sports

Welcome to third part of our Object Detection blog series!
I’m Al of AI Research and Innovation Hub (ARIH). In this blog, we will go over two recent approaches and some use cases in the application of AI in soccer: automatic event annotation, and pass analysis and expected possession value (EPV).

Parts
1.  Introduction to Object Detection and ARIH’s Use Cases
2.  Deep Learning Approaches for Abandoned Object Detection
3.  AI in Soccer: Smart Analytics from Gameplay Videos
4.  Common challenges and possible solutions in Object Detection

Like in other domains, various tools and techniques in artificial intelligence are being used in sports analytics. In football, specifically, recent advances in deep learning-based object detection and tracking serve as baseline strategies for data extraction and extrapolation in gameplay videos. These techniques improve the assessment of game performances (practice and actual matches) by automating player performance tracking, and act as a helpful decision-making tool for managing teams and planning future matches.

出典:Automatic Pass Annotation from Soccer Video Streams Based on Object Detection and LSTM
キャプション:Fig. 3. Construction of the Object Position Vectors.
https://arxiv.org/pdf/2007.06475.pdf

Figure 1. Person detection in soccer gameplay video (Sorano et al, 2020)

AI Approaches and Sample Use Cases in Football

Automatic Event Annotation

One of the common AI approaches applied in soccer is automatic event annotation from gameplay videos, which are otherwise manually collected by human operators. These involve deep learning-based player and ball detection or tracking (such as the one shown in Figure 1), in order to annotate practice or official matches with game events i.e. ball possession, passes, shots, and fouls.

An example of this is in Sorano et al (2020), where a method to automatically recognize passes from video streams, called PassNet (shown in Figure 2) , was introduced. Using ResNet18 for feature extraction and YOLOv3 pre-trained on COCO dataset for object detection, they fed the extracted object position vectors to a sequence classification module composed of Bidirectional-LSTMs. The output is a pass vector that indicates, for each frame of the original sequence, whether or not it is part of a pass sequence.

出典:Automatic Pass Annotation from Soccer Video Streams Based on Object Detection and LSTM
キャプション:Fig. 1. Architecture of PassNet.
https://arxiv.org/pdf/2007.06475.pdf

Figure 2. Architecture of PassNet (Sorano et al, 2020)

More recently, with novel datasets for soccer event detection, other approaches are currently being explored such as in Karimi et al (2020), where they made use of the soccer event (SEV) dataset (see Figure 3), to perform event classification in gameplay videos. This dataset, composed of 6000 images, has ten event classes: ‘corner kick’, ‘free kick’, ‘to substitute’, ‘tackle’, ‘red card’, ‘yellow card’, ‘penalty kick’, ‘center circle’, ‘left penalty area’, and ‘right penalty area’, which is a step further from binary pass annotation such as in Sorano et al, and can pave the way for more advanced soccer analytics.

出典:Soccer Event Detection Using Deep Learning (Referential material)
キャプション:Fig. 5: Samples of SEV dataset
https://arxiv.org/pdf/2102.04331.pdf

Figure 3. Sample images from soccer event (SEV) dataset (Karimi et al, 2021)

Pass Analysis and Expected Possession Value (EPV)

Another main approach is pass analysis, which identifies what options are available to a player in possession of the ball at any given moment i.e. where on the field can a player pass the ball, and still retain team possession. An extension of this is a quantitative metric called the expected possession value (EPV), which is the team’s probability of scoring a goal at a given time in a field position.

In Arbues-Sanguesa et al (2020), for instance, a computational model to estimate the most feasible pass at any given a monocular video of a soccer math, was described. By taking into account the player orientation gathered from body pose estimations, and the spatial configuration of the offensive and defensive teams in the two-dimensional field, the feasibility of pass events within players of the same team was computed (as demonstrated in Figure 4 below).

出典:Using Player’s Body-Orientation to Model Pass Feasibility in Soccer
キャプション:Figure 1. In order not to take pairwise distances into account while computing orientation feasibility, all players are moved towards an equidistant distance (unit circle)., Figure 2. Individual scenarios of intersection given the relocated players of Figure 1.
https://arxiv.org/pdf/2004.07209.pdf

Figure 4. (left) ball and players’ position in the field, and (right) individual scenarios of intersection
(i.e., top right player is the best oriented candidate to receive the ball) (Arbues-Sanguesa et al, 2020)

Most recent attempt to capture the complex spatiotemporal relationships in soccer is the expected possession value (EPV), which is essentially an estimate of which team will score the next goal, given all the spatiotemporal information at any given time. Figure 1 provides an intuitive example of how the possession value changes in time, which gives information about the observed actions. Recent developments have extended this by providing practical insight into the factors that vary EPV, or the actions that could be taken to boost it.

 

 出典:A framework for the fine-grained evaluation of the instantaneous expected value of soccer possessions
キャプション:Fig. 1 Evolution of the expected possession value (EPV) of FC Barcelona during a match against Real Betis in La Liga season 2019/2020.
https://arxiv.org/pdf/2011.09426.pdf

Figure 5. EPV evolution of FC Barcelona during a match, with information on observed actions (Fernandez et al, 2020)

Fernandez et al (2020) formulated EPV as a composition of the expectations of three different on-ball actions: passes, ball drives, and shots. The resulting single EPV estimate is from the ensemble of models from the estimation of each component, thereby allowing the impact evaluation of both observed and potential actions, as demonstrated in Figure 6 below.

 

出典:A framework for the fine-grained evaluation of the instantaneous expected value of soccer possessions
キャプション:Fig. 6 A visual control room tool based on the EPV components.
https://arxiv.org/pdf/2011.09426.pdf

Figure 6. (left) Instantaneous value of selection probability of each on-ball action,
and the expected value of each action, as well as the overall EPV,
(right) evolution of EPV during the possession and EPV of the optimal passing option at every frame (Fernandez et al, 2020)

Summary

Professional sports teams have started to catch up on the adaptation of artificial intelligence to gain competitive advantage. In soccer, specifically, gameplay videos are being leveraged as a starting point for analytics. In this blog, two common AI approaches in soccer analytics were introduced, namely: (1) automatic event annotation, and (2) pass analysis and expected possession value (EPV).  In general, we were able to see that from leveraging the advancements in AI, it is possible to develop applications from simple pass annotations, to valuing player’s actions.

For more information about the use cases, please contact our team.

 

■ 本ページでご紹介した内容・論文の出典元/References

Danilo Sorano, Fabio Carrara, Paolo Cintia, Fabrizio Falchi, Luca Pappalardo, “Automatic Pass Annotation from Soccer Video Streams Based on Object Detection and LSTM”, Fig. 1. Architecture of PassNet., Fig. 3. Construction of the Object Position Vectors., arXiv:2007.06475v1, https://arxiv.org/pdf/2007.06475.pdf

Ali Karimi, Ramin Toosi, Mohammad Ali Akhaee, “Soccer Event Detection Using Deep Learning”, Fig. 5: Samples of SEV dataset, arXiv:2102.04331v1, https://arxiv.org/pdf/2102.04331.pdf

A.Arbues-Sanguesa, A.Martin, J.Fernandez, C.Ballester, G.Haro, “Using Player’s Body-Orientation to Model Pass Feasibility in Soccer”, Figure 1. In order not to take pairwise distances into account while computing orientation feasibility, all players are moved towards an equidistant distance (unit circle)., Figure 2. Individual scenarios of intersection given the relocated players of Figure 1., arXiv:2004.07209v1, https://arxiv.org/pdf/2004.07209.pdf

Javier Fernandez, Luke Bornn, Daniel Cervone, “A framework for the fine-grained evaluation of the instantaneous expected value of soccer possessions”, Fig. 1 Evolution of the expected possession value (EPV) of FC Barcelona during a match against Real Betis in La Liga season 2019/2020., Fig. 6 A visual control room tool based on the EPV components., arXiv:2011.09426v1, https://arxiv.org/pdf/2011.09426.pdf


マクニカのARIH(AI Research & InnovationHub)では、最先端のAI研究・調査・実装による評価をした上で最もふさわしいAI技術を組み合わせた知見を提供し、企業課題に対する最適解に導く活動をしています。
詳細は下記よりご覧ください。

関連記事

*テックブログ*
Introduction to Object Detection and ARIH's Use Cases | Object Detection Blog Series (Part 1)

*テックブログ*
Deep Learning Approaches for Abandoned Object Detection | Object Detective Blog Series (Part 2)

*テックブログ*
Common challenges and possible solutions in Object Detection | Object Detective Blog Series (Part 4)