The Rossoneri conceded early in the first half and despite playing with a man advantage for the best part of the second, following Yuto Nagatomo's red card, they could not find an equalizer.
Allegri has put much of the blame for his side's fourth loss of the season on the shoulders of Valeri, who he believes made many mistakes on incidents that were "really obvious."
"I never talk about referees. But, I have to now. The referee got it wrong in this game," Allegri was quoted by Corriere della Sera."In the first half he made a technical error. He blew the whistle for halftime 12 seconds early. Then there was a last-man foul on [Urby] Emanuelson which was given against him.
"There was maybe a penalty on Robinho, the Nagatomo red card was obvious as he couldn’t not give him another yellow for that.
"This is not an excuse and perhaps today the referee was unlucky, but the incidents were really obvious."
The former Cagliari tactician expressed his disbelief as to how Riccardo Montolivo's goal was disallowed for an apparent foul on Inter goalkeeper Samir Handanovic.
He added: “I have no idea what the referee whistled for on the Montolivo goal. Let’s just say he was unlucky on the decisive incidents, as the goal would’ve been crucial. There was also a penalty and [Walter] Samuel was last man too.
“It’s not as if the referee did us a favor with Nagatomo’s red card, as he obviously stuck his arm out for an intentional handling offense.
“It’s a shame we are running into some decisions that go against us, but we can’t do anything about that. Maybe the referees are on as bad form as we are."
Despite his tirade at the match official, Allegri did admit that his side needs to improve in certain areas, especially in defending set-pieces.
“We have to look at the performance and improve some things. It’s a shame to lose this way, but we have to accept the result and look at where we got it wrong. At the moment we have conceded seven goals from set plays and that should be avoided. We’ll work on it over the break," he said.
The loss now sees Milan sit in 11th place with only seven points from seven games played.Follow GOAL.COM USA on Twitter
- Yuto Nagatomo