Houston, which has finished with the worst record in the majors each of the past three seasons, won a series at Yankee Stadium for the first time in four tries.
The fourth-place Astros improved to 4-7 in the Bronx and will go for a three-game sweep Thursday afternoon.
''We haven't been giving up toward the end of the game. We just keep grinding out at-bats, and things are working out for us,'' Grossman said.
Stephen Drew hit his first homer for New York, and Ellsbury drove in a run with a bunt single. But the Yankees (63-61) lost for the seventh time in nine games and dropped five games out of a playoff spot.
They are 2 for 17 with runners in scoring position during the series and have totaled 20 runs in their last nine games, never scoring more than four in that stretch.
''We've got to score more than two runs. It's hard to win when you only score two runs,'' manager Joe Girardi said. ''Our pitching has been pretty good, but that's a pretty tall task.''
Ellsbury flied out against Jose Veras with two on to end it. New York stranded 10 in all.
''I knew it was in the park. It was a great sinker,'' Veras said.
Feldman (7-9) matched a career high by throwing 121 pitches and worked out of trouble to last 6 2-3 innings. He gave up two runs and eight hits, striking out a season-best seven.
The right-hander tossed 6 2-3 shutout innings to beat the Yankees on opening day. Houston had lost his previous six road starts, but he entered with a 2.30 ERA during four straight quality starts against New York.
''He's a veteran guy and he's been around the game for a long time. He knows how to pitch,'' Fowler said.
Making his second start since a long injury layoff, Michael Pineda took a 2-1 lead into the seventh. He threw 89 pitches and was pulled after a leadoff walk to Jason Castro.
Looking to rest his best relievers following a heavy workload the past few days, Girardi turned to David Huff (2-1) and Esmil Rogers. They failed miserably, ruining the manager's bobblehead night.
Houston smacked five straight one-out singles, four off Rogers, in a span of 10 pitches.
''We have relied on the home run quite a bit, and today was a prime example of keeping the line moving,'' Astros manager Bo Porter said. ''I felt like guys had great at-bats.''
No. 9 batter Jake Marisnick tied it with a single to left, Grossman put the Astros up 4-2, and Altuve added an RBI single as the boos increased for Rogers.
''When the other guys are used a lot, guys have to step up,'' Girardi said. ''It just didn't happen tonight.''
Kevin Chapman struck out Ellsbury with runners at second and third to end the bottom half. Veras worked the ninth for his first save since Sept. 2, 2013, with Detroit.
''It was nice to see Chapman come in, probably the biggest part of the game, and pick me up. I owe him a dinner or something like that,'' Feldman said.
Seemingly resigned to trying small ball, the Yankees went 4 for 5 on stolen base attempts in the first five innings. But one pitch after Chase Headley was caught at second in the fourth, Drew hit his fifth home run of the season. The other four came with Boston before a July 31 trade.
Astros: Regular closer Chad Qualls was unavailable because of a stiff back. He is day to day.
Yankees: Slumping DH-RF Carlos Beltran sat out because of pain in his troublesome right elbow and was examined by a doctor. He had his third cortisone shot and is day to day. ''Hopefully, this is the last one,'' said Beltran, who hopes to return to the lineup on Friday. ''At the end of the year I have to address it and get the surgery to get it over with, but right now I'm just trying to finish the season.'' ... RHP David Phelps (elbow inflammation) played catch and will try to get ready to come off the disabled list as a reliever, Girardi said.
LHP Dallas Keuchel (10-8, 3.11 ERA) makes his first career start against the Yankees in the series finale. RHP Brandon McCarthy is 4-2 with a 2.30 ERA in seven outings for New York since he was acquired from Arizona in early July.
HERE COMES TROUBLE
Fowler is 9 for 20 (.450) with six extra-base hits against the Yankees this year.
Yankees LF Brett Gardner tumbled head over heels over a low retaining wall to make a courageous catch in the ninth.