Arrieta finished a long pass from Tony Tchani in the 86th minute.
Lee Nguyen tied it at 3-3 for New England (6-14-5) in the 81st minute with a penalty kick against rookie keeper Matt Lampson.
''It was really a good day to celebrate my (29th) birthday,'' Arrieta said. ''I'm always under pressure to score. I'm a forward. That's what I like to do but the most important thing is the team needed to win.''
The Crew (10-8-6) are 2-0-2 in the past four games. They trailed 2-0 after goals by Ryan Guy in the 17th minute and Jerry Bengtson in the 23rd.
''It was some of our best stuff, some of our worst stuff all kind of wrapped into a first half and to a game,'' New England coach Jay Heaps said.
Federico Higuain made it 2-1 three minutes later with a 28-yard free kick.
Arrieta tied it in the 32nd and another Higuain free kick gave the Crew a 3-2 lead.
Each time Higuain powered the ball over the wall of defenders.
''Those free kicks were world class,'' Heaps said.
Higuain has three goals and one assist in three games since joining the Crew.
''The key is to go over the wall because once the kick goes over the wall, you have a better chance to score,'' he said. ''That's what I tried to do.''
- Jairo Arrieta
- Columbus Crew
- New England
- New England Revolution