Costa struck an 89th-minute winner in Monday's game at Stamford Bridge to give Conte victory in his first Premier League match at the Chelsea helm.
But the Brazil-born Spain international was fortunate to be on the pitch, having earlier caught West Ham goalkeeper Adrian with a late challenge after he had already received a yellow card.
"Diego tried to put pressure on Adrian and before the contact, he stopped," Conte said, in comments that seemed to be contradicted by the evidence of television replays.
"For this reason I think the referee took the right decision. You know Diego is a passionate man and he puts passion in everything he does. In this situation, I think the referee took the right decision."
Chelsea were leading 1-0, courtesy of Eden Hazard's 47th-minute penalty, when Costa launched into a challenge on the dawdling Adrian, appearing to catch him high on the right calf.
Referee Anthony Taylor had booked Costa for dissent in the first half, but on this occasion he kept his cards in his pocket.
West Ham centre-back James Collins equalised with a neat finish 13 minutes from time, but Costa had the final say when he gathered a flick-on from debutant Michy Batshuayi and rifled home a low shot.
"From our point of view, we are not happy with that," West Ham manager Slaven Bilic said when asked about Costa's challenge.
"I don't think it was reckless, definitely not. It was not deliberate. But it was late, very late.
"Nine times out of 10, when you see that challenge on a goalkeeper, it ends up with a booking. Unfortunately this was the 10th time."
The Croatian was magnanimous enough to concede that Chelsea's victory was "deserved".
As well as a decisive late cameo by former Marseille striker Batshuayi, Conte also saw his summer signing from champions Leicester City, midfielder N'Golo Kante, turn in a typically industrious display on his competitive debut for the club.
- Ayew blow -
The France international was booked in the third minute for an untidy foul on Andy Carroll, but he grew into the game and Conte hailed his "fantastic stamina".
Kante's inclusion meant that there was no place in the starting eleven for Cesc Fabregas, but Conte insisted that he was "happy" with the former Arsenal midfielder.
Another positive for Conte was the performance of Hazard, whose form nosedived dramatically last season after he had been voted England's Player of the Year in Chelsea's 2014-15 title-winning campaign.
The Belgian winger looked alert and dangerous throughout, going close to breaking the deadlock with a curling effort and then putting Chelsea ahead with an emphatic penalty.
"Eden played a good game today with a great ability," Conte said.
"Hazard, Willian are players with great attitude with the other players. I am very pleased with Hazard's performance, but he knows he can improve a lot."
Conte also said he had been touched by the reception afforded him by the fans of his new club.
"I have to say thanks to the Chelsea fans because the welcome was fantastic," said the former Juventus and Italy coach.
"And to see the Italian flag in the stand, I had a bit of emotion. And with me (were) my wife and my daughter.
"I hope with work, with my players, we will pay (back) this welcome."
Compounding Bilic's disappointment was a thigh injury sustained by new recruit Andre Ayew that forced him off in the 34th minute.
The Ghana attacking midfielder, a club-record signing from Swansea City, will undergo a scan on Tuesday, but already looks a doubt for Thursday's Europa League play-off round first leg at Romanian side Astra Giurgiu.
"It's a thigh injury," Bilic said. "He told me that it's not a big one. But you never know. We have to wait for the scan tomorrow."