LOS ANGELES — Augmented reality on sports broadcasts isn’t new. In fact, there’s a gold standard: the first-down line for football games, invented two decades ago. Imagine the chaos and confusion in homes and sports bars across the country if they took that away. More recent additions to the genre include the virtual baseball strike zone, and the digital arc tracing the flight of a golf ball.
So what about basketball? Can augmented reality provide anything so indispensable that NBA fans would howl in protest if it were taken away?
Not yet. But the technology is impressive, and the potential is there.
That’s my takeaway after trying out Clippers CourtVision for two games. The experience was unveiled this week by former Microsoft CEO Steve Ballmer and his NBA team, the L.A. Clippers. Developed by technology company Second Spectrum with Ballmer’s backing and encouragement, CourtVision uses computer vision to see what’s happening on court, artificial intelligence to understand the game, and augmented reality to display animations and data on screen.
I've tried @LAClippers CourtVision from @SecondSpectrum and @Steven_Ballmer for two games. Very cool + promising example of AI + AR, if not yet an indispensable addition to the viewing experience like 1st down line in football. Here are the 3 main modes on one play. pic.twitter.com/lerf1Ljtii
— toddbishop (@toddbishop) October 21, 2018
The augmentation happens inside the app, using video from inside the arena. It’s not possible, for example, to hold up your phone and see the AR features via your own camera in real-time. However, viewers can customize the experience by selecting among different augmented reality modes.
In “Coach” mode, for example, when Clippers center Boban Marjanović set a screen and cut to the basket for a pass, the system put his name and number over his head to identify him as the player with the ball, diagrammed the play in real-time on the floor (below), and showed his point total above his head after he scored.
When Oklahoma City Thunder small forward Paul George grabbed a rebound from a missed shot, the system showed his rebound total. Turnovers, assists and other stats also appear above players in real time as they add to their totals, and facts such as a player’s free-throw percentage appear in the lower third of the screen as he’s shooting a foul shot.
In “Player” mode, shooting percentages hover over the head of each offensive player, changing dynamically as they move around the floor, indicating the quality of the shot and likelihood of scoring if they were to get the ball.
And a “Mascot” mode makes the experience more like watching a video game, with dynamic graphics and fun animations accompanying the action.
It all happens automatically, without human intervention, which is no small technological feat. The stream is currently delayed by two minutes from the live action, to allow time for the required processing, but that’s down significantly from earlier testing, and Ballmer and Second Spectrum say the time gap will get even shorter over time.
The technology is currently available in two forms: a standard experience for FOX Sports Prime Ticket subscribers in the Los Angeles area via the FOX Sports app; and a more advanced experience, with additional viewing angles and other options, available as a private beta in a standalone app for a limited number of users. The Clippers are the only NBA franchise providing the technology to fans for now, although the cameras exist in NBA arenas to roll out the experience across the league.
I used the standalone app on my phone in the stands over WiFi at Staples Center during the Clippers home opener against the Denver Nuggets on Wednesday night, and Friday night against the Thunder.
Admittedly, this isn’t the typical experience. In most cases, people will be using the app not at the arena but at home, and if they stick with just the CourtVision experience, they won’t notice the time delay. (The standard game broadcast is already delayed by about 30 seconds.) However, the situation underscored just how significant the 2-minute delay is, as I watched plays on my phone that had been rendered almost irrelevant by the scoring that had taken place in the meantime.
My favorite feature, by far, is the real-time updating of stats, floating above a player’s head on-screen in Coach and Player modes. For example, when a player scores, his points total updates and appears for a few seconds, and the assists total also updates for his teammate who passed the ball (below). While it’s not quite indispensable, I did miss these numbers when watching the normal view.
Almost as useful, for someone who doesn’t follow the team, was the system’s ability to identify the player with the ball in real-time. Hardcore fans won’t need this for the home team, but it’s important to note that all of these features are also available for the visiting team, so that’s useful, as well. Of course, this will be a familiar feature to anyone who has played a basketball video game.
Perhaps the most impressive technological accomplishment is the dynamic updating of effective field goal percentage, the probability that a player will make a shot from a particular spot on the floor, based on his past performance. The percentage changes smoothly and turns from red to green when a player is in a good position to score from a particular spot on the court.
But after watching this for a few plays, the shooting percentages started to feel more like a novelty, vs. the real utility of seeing in-game stats dynamically updated over each player’s head.
Mascot mode was a big “meh” for me. Maybe this will get kids more engaged with games, but personally I think the real appeal of CourtVision will be the ability for basketball fans to track stats in Coach mode and better understand the game by seeing field goal percentages in Player mode.
One feature of the CourtVision app is the ability to switch the audio away from the game commentators to “Sneaker Squeaks” mode. Unfortunately, this silences only the broadcasters, not the incessant music and sound effects that plague NBA games (and other professional sports). For this basketball purist, at least, the ability to take away that modern “advance” would make the technology truly indispensable.
In the meantime, CourtVision is an impressive technological accomplishment with useful features for basketball fans, and the potential to become even more valuable as it evolves over time.