A bronze memorial to the 17th century slave trader was pulled down during a demonstration on 7 June and dumped in Bristol Harbour.
It sparked a wave of pressure to remove statues of people involved in the slave trade across Britain and fears of further damage sparked violent protests including far-right groups.
The Colston statue was assessed to have £3,750 worth of damage after being recovered from the water by Bristol City Council.
Police have asked the Crown Prosecution Service for a charging decision against three men aged 21, 25 and 32 and a 29-year-old woman.
Another five people who were interviewed as part of the investigation have been offered a conditional caution for criminal damage.
The men, aged 18, 20, 29, 33 and 47, will not be prosecuted if they accept the terms within a week but if they refuse they could be charged.
They would have to tell the history commissioner set up by Bristol City Council the “reasons for their actions, as well as their concerns and thoughts going forward”.
Police said they would each pay a fine of £100, which would be sent to Bristol’s Nilaari charity, which supports black and ethnic minority communities.
They would also have to each do two hours’ of environment improvement works for the council, including removing graffiti and rubbish.
Police were nearby when the statue was toppled but did not intervene – a decision senior officers later said was to maintain public order as they were outnumbered.
No arrests were made at the time but officers reviewed CCTV footage and other pictures and video to identify suspects following a wave of criticism.
It came amid widespread Black Lives Matter protests across Britain sparked by the police killing of George Floyd in the US.
Colston had long been a divisive figure in Bristol, with people calling for his name to be removed from schools, roads and venues.
Priti Patel, the home secretary, called the toppling of the statue “utterly disgraceful” at the time, adding: “It is a completely unacceptable act. Sheer vandalism and disorder are completely unacceptable.”
But the mayor of Bristol, Marvin Rees, said the comments showed an “absolute lack of understanding”.
“I think circumstances came to a head and people felt the need to take the statue down,” he added.
“I can't and won't pretend the statue of a slave trader in a city I was born and grew up in wasn't an affront to me and people like me.”