Don’t adjust your settings. It’s not your fault. And you weren’t alone.
The NHL and MLB Advanced Media re-launched the League’s digital properties this week, with a new look for NHL.com, a refreshed NHL app and, perhaps most importantly, the transformation of GameCenter Live into NHL.TV as the official streaming service.
The hype for that last item was palpable: BAM is considered the best in the business when it comes to streaming. It has built its reputation on streaming baseball games with minimal glitches and impressive images.
The NHL returned to the ice after the All-Star Game with 12 games on Tuesday night. The glitches were many. The images were not always impressive. When, you know, you could actually see images.
“I don’t want to trivialize it: There’s nothing worse than a fan who wants to watch a live game and can’t,” said Bob Bowman, president of BAM. “You hate to have the fans be frustrated. But it’s from those kinds of events you get better.”
The fans were very frustrated on Tuesday night.
Some of it was the expected stubborn reaction to a new product, the “things aren’t they way I like them” lament. This is something that eventually sorts itself out, as users adapt and the product changes and everything meets in the middle.
More problematic was that for thousands of fans who stream games on laptops, gaming platforms and mobile devices, NHL.tv simple didn’t work, to an embarrassing degree.
— coach pete (@johnpeterblink) February 3, 2016
Here’s a sample of over 70 emails Puck Daddy received last night. (We’d also recommend reading Jason Gurwin’s “10 things NHL.tv got wrong” piece, which gets into some interesting details.)
First, I had issues logging in. I clicked on the game, found the little TV icon in the space-age floating weirdness that is the NHL's idea of a website, and clicked it. I was faced with the spinning wheel of doom for a minute. Two minutes. Three. I refreshed; nothing. Nothing. Nothing. (I checked to make sure: YouTube worked fine. Netflix? Fine. Flash? Still working. NHL.tv? WHO KNOWS.) Finally, after restarting my browser, switching browsers, restarting my computer, and going back to the original browser, I was able to get to the login screen.
Hallelujah, I thought! Finally. It's working.
The only game that would work for me was the Leafs-Bruins game. This was billed as the "Free Game of the Week" or some such, which is great, except that I'm a paying customer and should therefore be able to watch the not-free games, too. No dice. Spinning wheel of doom.
At this point, I give up for a bit. The Habs are going to lose; I don't need to have my hopes and dreams crushed in real time technicolor. I'm still hopeful, at this point, that things will magically work by the time the Oilers game starts. Surely NHL.tv wouldn't deprive me of McDavid's return, right?
Well, I got my wish. Sort of. The game loaded, but it was incredibly laggy; even with the stream quality switched all the way to low, it stopped and started and did a thing where it fast-forward played, like, ten seconds at a time. I made it to about five minutes left in the first period before it froze up completely. I tried waiting it out; I tried switching to the CBJ stream. I tried refreshing the page.
Spinning circle of doom.
- Ki Hunt
The stream started to skip. It replayed the same 15 second sequence, with Drew Remenda making some quip about Connor McDavid’s average shift length, about 6 times in a row. Naturally, I was getting pretty frustrated by this. It seemed to clear itself up, but from here on out, it became unwatchable. It would freeze every 20-30 seconds, the feed would become grainy and blurry upon unfreezing, and repeat. I closed the browser and re-opened the feed, only to get a frozen still of gameplay for 5 minutes.
At this point, I got up to make dinner since attempting to watch it was so frustrating.
Eventually, the stream came back to life, and I returned to attempt to watch the last 2 minutes of the period. Instead, I got to see the sequence from 1:50 to 1:15 replay, and this it not an exaggeration, for 10 minutes.
That’s right, the same 45 second clip looped for 10 minutes.
Here we are, in 2016, facing the same issues that I sent countless e-mails about in 2011. It’s the same old song. If the NHL cannot figure out a way to resolve this immediately, I am going to begin the process of using everything in my power to get a refund for this package. I am the core demographic for this product, and through its incompetency, the NHL has alienated me. I just wanted to watch my favourite terrible team in peace (and high definition).
- Kevin in Vancouver
I get home and drop my bags on the ground, my dog greets me, but he knows its hockey time, and he lives before he is tormented by the constant complaining throughout the night. I turn on the game, and watch the first 5 minutes in the first and then...it happens.
BLANK SCREEN. BAM.
I get the update on my phone, Landeskog scores. I missed it. I went through my shitty day to hear the horn, to see a goal and I missed it. I question everything. I question hockey. I question sports in general. "This must be fixed" "The NHL can't be this heartless can they?"
At this point my dog is probably in the basement, he knows. They say dogs and cats can sense your emotion, and he was about to call the police, if he had thumbs.
I check Twitter. The apocalypse in neigh, and everyone's app and video crashed. What do I do? Where do I go?
The first period ends. Ok, I have to find a way to watch this game's remaining two periods.
The NHL.tv app is a joke, and ruined me and my dog’s life. All I have to say is thank you to the pizza faced nerd in the basement of his mother's house on Reddit to put out a better stream than the National Hockey League's IT and engineering team. If I had the money for a hooker, you'd get the best mediocre hooker I could afford.
- Justin Harned
Every email was a tale of woe about the stream not working. The NHL.tv official Twitter support account was deluged with complaints – and a cheeky parody account called NHL GameCenter Dead expressed the fans’ outrage.
So what happened?
Primarily, said Bowman, it was a problem with one of BAM’s Content Delivery Networks (CDN).
When one game ends, it becomes archived. So the CDN needs to be configured so it knows that one game is no longer a live stream, and that another game is now being live streamed. According to Bowman, this “just wasn’t done right” on Tuesday night, leading to dozens of problems across the platforms: Streams slowing, freezing, lagging and crashing.
Much of this happened as the night went on and games reached their conclusions.
“We had a glitch last night. The game states changed, the config went down, and by the time you re-launch another CDN, it takes 20-30 minutes to re-launch that thing,” said Bowman.
This of course would lead to the question: Why not have a second CDN running in case the first one was glitchy on the night of the big launch?
“Should we have had CDMs spun up? The reason we didn’t spin up two is because if you have errors, it’s really hard to check on those errors when you have two of everything going, right? We tried to isolate it. It was a conscious decision,” said Bowman. “In hindsight, maybe we should have spun up two. But if we had to do it, we would have done it [that way] again.”
This might come as a surprise to those who know MLB.tv as a standard-bearer in streaming. But keep this in mind: It’s a partnership with the NHL. While BAM isn’t about to toss anyone under the bus, there are going to be obvious growing pains in a relationship that’s, for all intents, is in its infancy.
“We’re not ones to say it was this person or that person, but there are a lot of handshakes that go on, a lot of relying that goes on. In the end, it was up to us and it didn’t work,” said Bowman.
“It won’t happen tonight. It didn’t happen last night after we re-configged it. Some things you just can’t anticipate.”
Bowman believes the CDN problem accounted for the “lion’s share” of the streaming issues on Tuesday night. But they’re also aware that the problems at the launch got well beyond that, including some weirdness with the NHL app.
Like this for example, from Sean in Japan, as players who aren't playing in a game are listed as goal-scorers.
“It could be five separate issues. You go from a lab of 50 devices to the real world of, like, 100,000 different devices with different operating systems and connection speeds,” said Bowman.
He said there is a “punch list” of items collected last night on the website, the app and NHL.tv that BAM is working to remedy. He said, in the offseason, there will be a full reassessment of data delivery for the app.
Bowman said he’s confident the basic issues with streaming that sunk their launch this week will be fixed in short order, perhaps even for Wednesday's games.
The rest of it, he said, is being discovered and addressed as every corner of their products is being tested by NHL fans.
“You hate to have the fans be frustrated. But it’s from those kinds of events you get better,” he said. “If you’re honest with the fans, they’re honest with you, and that’s the best long-term policy.”