The 12th-seeded Paladins (7-23) have eliminated the fifth-seeded Bulldogs (11-21) in the first round for three straight years.
Two consecutive baskets by Austin put Furman ahead to stay, 44-42, with 8:05 remaining, and his two free throws with 11 seconds left provided the final margin.
Neither team led by more than six points and they were tied 25-25 at halftime.
Stephen Croone scored 11 points and his steal with 27 seconds left protected a one-point margin. Charlie Reddick added eight points and 10 rebounds for Furman, which will play fourth-seeded Appalachian State in the quarterfinals Saturday.