Gymnastics’ latest twist? Robot judges that see everything

The rectangular grey boxes arrayed around the floor at the World Gymnastics Championships this week are easy enough to ignore. But these little boxes, 30 of them in all, hold outsize significance: according to Morinari Watanabe, the president of the International Gymnastics Federation, their presence at the championship signals “the beginning of the new history of gymnastics”.

The robot judges, in other words, have arrived.

Amid the displays of athleticism and artistry at this week’s championships, there also has emerged a development in technology that could hint at the trajectory of officiating across the sports landscape.

Each of the grey boxes keeping watch in Stuttgart, designed by the Japanese company Fujitsu and about the size and shape of a Wi-Fi router, contained a set of three-dimensional laser sensors that tracked the movements of each of the 547 gymnasts from 92 nations participating this week. That data was fed to an artificial intelligence system, accessible to the human judges, that measured and analysed skeletal positions, speeds and angles — some of them unavailable or simply missed by the judges — as the athletes went through their movements.

Thanks to all this, Watanabe explained, no longer would gymnasts — many of whom, he noted, had started gymnastics as young as age three and had trained competitively for more than a decade — risk having their efforts unceremoniously wasted by human error or interference. “This is a step toward the challenge of justice through technology,” said Watanabe.

The debut of such technology at the biggest gymnastics meet outside the Olympics represented a meaningful milestone in a sport periodically marred by judging controversies and often wracked with questions about political influence in scoring decisions.

For all the grand language, and for all the big-picture prophesying it has inspired about the future of sports — baseball is already experimenting with robot umpires and tennis is starting to expand electronic line-calling — the steps unveiled in Stuttgart were preliminary, and fairly subtle.

In gymnastics, at least, humans very much remain in control. For now.

At the world championships, the artificial intelligence system instead has served a supporting role, available to judges to confirm difficulty scores in two circumstances: in the event of inquiries (when gymnasts formally challenge the judges’ score) and blocked scores (when there is a large deviation between the sets of judges). And the technology, officially, remains limited to a few events: pommel horse, rings, and men’s and women’s vault.

The gymnastics federation did not release any detailed data related to the launch of the AI system this week. But Steve Butcher, the federation’s sports director, noted that reviews aided by the computer system had already resulted in “a few changes” at the championships, which continue through Sunday.

Butcher said judges were separately reviewing a large portion of the exercises through the Fujitsu system for the purposes of verification and education.

Despite the fanfare of the system’s introduction and its potential effect on scores, athletes and coaches said they were not paying too much attention to the AI judges. Laurent Landi, who coaches the American star Simone Biles, said that it was too early and the procedures not sufficiently integrated to fully ponder the implications of the technology. But he said once the computers proved their accuracy, they could represent a big, positive step for the sport.

“We all know how subjective scoring can be,” said Landi.

Others expressed wariness, arguing a computerised system contradicted the human heart of the sport.

“We shouldn’t turn it over to robots because, to be honest, I think that’s a little weird,” said Yul Moldauer, a member of the American men’s team. “I think we should always have human beings being the judges.”

Before this year’s competition, Moldauer and the hundreds of other competitors were asked to participate in what Butcher called a “body dimension measurement” procedure — effectively a precise scan of their bodies and movements — so that the analyses could proceed with maximum accuracy.

The pre-competition measurements were voluntary, but Butcher made sure to point out that several athletes who had agreed to be scanned this week later benefited during the championships by having one of their scores increased upon review.

Takehiko Ishii, an executive for Fujitsu, estimated that more than 90 per cent of the athletes had agreed to participate. Those who did not were evaluated using standardised body dimensions, but, he said, “some gymnasts have thick muscles and some gymnasts are thin, so it’s better to have the body dimension measurements”.

MyKayla Skinner, a member of the United States women’s team, said there was a lot of discussion, and some confusion, among her teammates when they were asked to review the paperwork to sign off on the scans.

“We were like: ‘I don’t know, do we? Do we not? This is so weird!’” said Skinner. “But I think in some ways it’s going to be very, very great, just because there’s so much politics that go into gymnastics.”

Photos: Courtesy Fujitsu

Asked about concerns about data privacy, Butcher said all of the athlete information collected at the competition would be discarded at a predetermined expiration date.

At the same time, Ishii said there had been preliminary conversations about whether such data could eventually be collected and sold as part of training materials for the public, with athletes receiving any royalties.

Amid all the other minor concerns, however, most discussions about the so-called robot judges have tended to lead to a larger, simpler question: Will there come a time when human judges are eliminated completely?

Ishii said the AI was not yet sophisticated enough to take over fully for humans. But one day it might, he said.

Coaches and gymnasts were more sceptical about an artificial intelligence system’s ability to ever fully evaluate a nuanced gymnastics routine.

“The computer cannot understand artistry, mastery, cannot understand the feeling in the movements you make,” said Enrico Casella, an Italian coach. “It can see all the angles better than judges, but the artistic part is impossible.”

The decision to unveil the technology this week came after a series of tests, beginning with one at last year’s world championships in Doha, Qatar. The International Olympic Committee is aware of the technology, but it has not publicly announced whether the AI system — or a different one — will be employed at the Olympics next summer in Tokyo.

Fujitsu and FIG said they envisioned a future in which the 3-D laser sensors were used not only in competition, but to enhance the viewing experience for fans. The federation and a Japanese broadcaster experimented this week with integrating the data into television broadcasts. Coaches, meanwhile, were considering their value as a training tool.

“Once we learn about — we’re calling them robots — once we determine what the robots can really see, then it could absolutely dictate the direction of coaching for us,” said Tom Forster, the high-performance team coordinator for the American women’s team. “The human eye can’t pick up everything.”