Then two veterans led them home.
''We don't worry about what the score is, we worry about playing better,'' Fever coach Lin Dunn said. ''And if you do things better, than slowly and surely now it's a 10-point game, now it's a five-point game. And that's all we focus on.''
Indiana won its fourth straight for the first time since 2012.
Hartley scored 16 and Ivory Latta added 12 for Washington, which had won three straight and was playing the second of back-to-back games.
''We just chipped away,'' Dunn said, ''and once we got close I felt like you could see Washington start to tense up a bit. And I think (in the) fourth quarter, the last five minutes, they were tired because they had a back to back.''
Stefanie Dolson added 10 points and nine rebounds for Washington.
Coleman scored 13 points in the first half to get Indiana close, and fellow fifth-year veteran Larkin scored all of her points after intermission.
The Fever shot just 35 percent (21 for 60), but the Mystics were worse, hitting 18 of 61 field goals (29.5 percent), including 2 of 15 from 3-point range.
''We survived for the last few games on great effort on rebounds and half-court defense,'' Mystics coach Mike Thibault said, ''and for the most part I think we actually did a pretty good job of that. . But we were (an) awful offensive basketball team tonight. Absolutely awful.''
The Mystics outrebounded Indiana 40-29, but turned over the ball 17 times, 12 in the second half.
Without January (sprained ankle), Indiana shot 2 of 18 in the first quarter and trailed 23-7 in the second.
''I think most nights a 16-point lead in the first half is fool's gold,'' Thibault said. ''I really do because it's almost too early to get it.''
Down nine at intermission, Indiana outscored Washington 24-9 in the third quarter, led by Larkins' 10.
''The first half, four fouls, held scoreless. I just knew it was going to be important for me to get going,'' she said.
The Fever took their first lead of the game, 43-41, late in the quarter on a 3-pointer by Maggie Lucas. The Fever led by 10 midway through the fourth quarter before Washington rallied.
Hartley's driving layup made it 62-61 with 10.9 seconds left, then Coleman was fouled and hit both free throws.
The Mystics had a chance, but when Dolson screened Hartley's defender, the 6-foot-3 Howard switched and deflected Hartley's attempt. The Mystics controlled the loose ball as time expired.