Guernsey FC manager Tony Vance says they will appeal Charlton Gauvain's red card in Monday's loss to South Park.
The midfielder was dismissed nine minutes into the second half after the assistant referee claimed he heard him use a racist term towards him.
Vance says the official misheard the comment, which was directed at one of Gauvain's team-mates, shortly after Sam King scored the only goal of the game.
"We're gathering all the information so we'll appeal," Vance told BBC Sport.
The loss means Guernsey's 3-1 win over Westfield in their opening game of the season remains their only points on the board after three matches, having also lost 4-0 at Ascot United on Saturday.
"All the opposition players, our own players, our bench, the supporters near the incident, all told the referee what he said, and not what the linesman thought," Vance added.
"The referee was in a really difficult situation, I thought he handled it really, really well, but unfortunately he sent him off because of the linesman's story.
"When the opposition are trying to tell the referee not to send a player off that tells a story."
The loss leaves Guernsey 15th in the 20-team Isthmian League South Central Division with three points from three matches ahead of a trip to 14th-placed Sutton Common Rovers on Saturday.
"Our response after Charlton's sending off was really good and we had a right good go," said Vance.
"But of course you're down to 10 men, it's not easy, you're making drastic substitutions rather than the ones you want to make given the situation.
"So it had a massive impact on the game, but the best thing about our play was that we were competitive and we gave it our best shot, but the game will unfortunately be talked about for things that we don't want to talk about."