Kentavious Caldwell-Pope finished with 16 points for Georgia, but the fourth-leading scorer in the Southeastern Conference went 0 for 6 from the field in the second half.
Steele's consecutive 3-pointers gave Mississippi State (7-7, 2-0 SEC) an 11-point lead, its biggest of the game, with 5:02 remaining.
Charles Mann added 14 points for Georgia (6-9, 0-2), which was coming off a 33-point loss that was its worst Florida since 1967.
Georgia, which has dropped two straight games, got reserves Donte' Williams and Kenny Gaines from one-game suspensions levied by coach Mark Fox for breaking unspecified team rules.
But their return made little difference in helping the Bulldogs avoid their second straight loss. Williams scored five points in 24 minutes and went 3 for 9 from the free-throw line. Gaines had five points in 12 minutes.
Mississippi State took a six-point lead on Steele's 3 from the left wing with 11:33 remaining, but Georgia forced a 50-all tie on Tim Dixon's putback at the 9:01 mark.
Steele's previous career high was 19 points, scored last March 8 against Georgia.
Georgia began the game ranked No. 304 in scoring offense. Other than Caldwell-Pope's 4 for 9 performance on 3s, the rest of the team went 0 for 10.
Georgia, which honored former coach Hugh Durham and members of his 1983 Final Four team at halftime, took its biggest lead at seven on Tim Dixon's putback with 5:05 left in the first half.
The lead changed hands five times in the last 1:53 of the first half before Sword's putback with 1 second left putting Mississippi State up 36-35.
Mississippi State was coming off a 56-54 victory over South Carolina on Wednesday in Starkville.
The Bulldogs, who improved to 1-3 on the road in their first season under coach Rick Ray, are without Wendell Lewis indefinitely as the senior forward recovers from a right patella fracture. Freshmen DeAndre Applewhite and Jacoby Davis are out for the season with anterior cruciate ligament injuries.
But Mississippi State has gotten a boost the last two games from Sword, who had 18 points against South Carolina.