Passing down traditional sports in a replay

The preservation of traditional sports and games could soon be guaranteed by new, cost-effective motion sensing technologies.

The World Sports Encyclopaedia may be taking an inventory of over 8 000 sports in the world, but globalisation and receding cultural differences have put many of them on the verge of disappearing. In Europe alone, a great deal of our 3 000 Traditional Sports and Games (TSGs) are either in decline or already lost in spite of their importance for cultural heritage. The question is, how do we keep them alive for the next generations to enjoy?

Launched in 2013, the REPLAY (Reusable low-cost platform for digitizing and preserving traditional participative sports) project answers this question through the tweaking and exploitation of existing motion sensing technologies. From a set of Wireless Inertial Measurement Units (WIMUs) and Kinect devices, the eight-strong consortium has created a unique solution to capture the unique 3D movement signatures of National and Local sporting Heroes (elite and well-known players of the TSG) and allows children to directly compare themselves to them in a funny and engaging manner. Furthermore, the consortium is now able to harness data from video footage and players, store it and use it to help the young master the techniques of elite players.

To test its technology, REPLAY focused on two very representative TSGs: Basque Pelota, which is one of the most popular Basque sports, and Gaelic sports (Gaelic Football and Hurling/Camogie) which are still very popular in Ireland. Maria Teresa Linaza, coordinator of REPLAY, agreed to discuss the outcomes of the project ahead of its end in February 2016.

Why did you take interest in the conservation of traditional sports? Why is it important?

Traditional Sports and Games (TSG) are part of Intangible Cultural Heritage, representing a symbol of the cultural diversity of a society and a backbone of the community. They are also an efficient means to convey values of solidarity, diversity, inclusiveness and cultural awareness. They ensure opportunities for social contact and for the exchange of experiences across generations.

It is therefore imperative that we understand, preserve, protect and promote TSG just like we protect works of art. New technologies offer great opportunities to do that.

How would you explain the fact that Gaelic and Basque traditional sports are doing so well when other TSGs have completely disappeared?

There are various reasons for that. First, both lie at the very foundation of their respective cultural traditions. For example, recordings of Basque Pelota can be found at the end of the Roman Empire. Secondly, the popularity of some games crosses state frontiers, with an international reach which preceded the idea of the nation itself. Basque Pelota is played in Europe (mainly Spain and France), in Latin America and in the United States.

Another key reason is that the equipment necessary for these games is usually cheap. For instance, two main modalities of Basque Pelota analysed within the REPLAY project enclose very simple equipment: Jai Alai is played in a three-wall court with a hard rubber ball caught and thrown with a curved wicker basket, while handball is played barehanded on a three-sided court with woollen balls covered with leather.

We believe the strength of these two traditions and their resistance to decline should serve to help other TSGs.

Can you tell us more about the technologies you developed? How can they help preserve TSGs?

The REPLAY platform revolves around three modules—motion capture, performance comparison, and 3D rendering and visualization of landscapes and characters—and is applied in three different scenarios.

First, the PLAY&LEARN scenario focuses on children and teenagers having access to a low cost motion capture set-up (Microsoft Kinect) to try at home, one player at a time, in order to learn and mimic the skills of a National/Local hero.

The second scenario, COACH&TRAIN, is oriented towards the capture of club player skills in order to allow coaches to better instruct their trainees on how to improve their game. The set-up for motion capture is more complex, including multiple Microsoft Kinects and WIMUs, and is located in a controlled environment, such as a sports club. Coaches have access to a database of skills from National/Local heroes and can gather information about how the elites perform a given technique. Apart from giving immediate feedback on the performed movement, the REPLAY platform also records and assesses the progress of a given player, allowing the coach to make better decisions when devising improvement plans for his athletes.

Finally, the INTERACT&PRESERVE scenario empowers visitors of a museum or exhibition centre through a visual interface enabling them to 'interact' with the rich content of TSGs and learn more about the related Cultural Heritage. First, the visitor can access 3D-captured content of National and Local Heroes in order to compete against them in a 3D environment. Then, this 3D interaction is enhanced by legacy videos and content about the TSGs, their history, instruments and modalities. Different stories are authored by museum curators, put in context and published in order to share and engage the visitors of the museum.

How do these technologies differ from similar market offerings?

As far as we know, until now there has been no market offering other than video that digitally captures skills in 3D or records the unique movement signatures of players. Mainstream sports like football, American football and basketball use very precise motion capture and analysis technologies owned by large companies like Vicon, but the cost of such an approach is very high. On the contrary, REPLAY is looking towards a low-cost capturing, processing and analysis platform based on off-the-shelf technologies combining Microsoft Kinect devices with WIMUs.

You can recover 3D motions even from historical films. How did you make that possible?

The main skills of 'National Heroes' can be recovered from TSG video legacy recorded with non-calibrated monocular cameras. This reconstruction generates an animated skeleton which represents the movements and poses of the player, along with the trajectory of the ball, all in a 3D environment. Such data can then be used in virtual museums, immersive virtual systems or video games.

To achieve this, a semi-automatic approach has been implemented in which a set of 2D representations of the scene's elements (camera, ball and player) are manually marked in key frames of the video. Then, an automatic process estimates the calibration parameters of the camera, the 3D positions, poses and body part dimensions of the players, as well as the trajectory of the ball. In-between frames are automatically calculated taking into account constraints related to human kinematics and collisions with the environment.

What types of markets and businesses are you targeting with this project?

Intangible Cultural Heritage is the natural market for the results of the project. One potential example of application is the preservation of handcrafters' skills by capturing and recognizing the gestures performed by some parts of the body (hands, arms and feet) using low-cost standard equipment.

Another potential scenario is rehabilitation, as determining whether the movements of the player are complete and correct can become a big problem. Multiple motion capture modalities should be evaluated and combined, including the already ubiquitous monocular Webcams, depth sensing devices such as Microsoft Kinect and emerging wireless low-cost sensors (mainly inertial sensors). The combination of one or more sensing devices will lead to different uses for the platform (i.e. the use of a low-cost Webcam to practice at home; a Microsoft Kinect device for motion capturing; and multiple Kinects and/or wearable sensors for large-scale installations at nursing homes or even hospitals).

The project ends in February. What are your plans until then and after it finishes?

In January 2016 the final REPLAY platform will be tested by real users both in the Basque Country and in Ireland. The PLAY&LEARN scenario will be evaluated with children (aged 8 to 16) from Pelota schools from the three Basque provinces and the south of France. For the Gaelic Sports, the three scenarios will be validated: the PLAY&LEARN scenario with children from primary schools in the Dublin area; the COACH&TRAIN scenario with coaches from several Irish clubs and their players; and the INTERACT&PRESERVE scenario with museum curators and visitors to the Croke Park Museum.

The third and final End-User Advisory Board Workshop will take place in Dublin on 28 January during the evaluation of the final REPLAY prototype. Then, a final handbook will analyse and specify the methodologies and most cost-effective hardware solutions to extend the results of the project to other TSGs.