Mr Floyd, an African-American, died on 25 May when a white police officer continued to kneel on his neck after he pleaded he could not breathe.
Thousands of people marched in London on Wednesday after the death sparked global protests against racism.
The Birmingham protest started outside the library but later moved through the city to the police's headquarters.
West Midlands Police said an estimated 4,000 people took part and there were no arrests.
"The protesters were loud and passionate, and made their voices clearly heard, but there were no arrests and no disorder," a force spokesman said.
Protest organisers UK Isn't Innocent said Britain had "a duty to stand in solidarity with the US while exposing the inner workings of racism and police brutality in the UK".
"We are tired and we have been tired for too long," lead organiser Hannah Ringane said.
"We have been taught that we won't be treated the same as everyone else, that we will be viewed as aggressors."
Carol Smith, who was among the demonstrators, said: "My grandchildren were born here, they have to have a different world to the world I have.
"They have to realise they have a right to be here, and they have a right to equality, just like everyone else who don't look like them.
"I can't give you the answer to racism. I didn't create it, people who look like me didn't create it."
Derrick Bassaragh added: "Young kids - black, white and Asian - demand parity, justice, love and equality.
"Not just for us, but changes for the world. These are the things that are demanded now."
Aston Villa and England defender Tyrone Mings - who was targeted with racist abuse while playing for his country against Bulgaria last year - indicated he would join demonstrators, urging his followers online to "stand for what's right".
People were originally due to gather in Victoria Square outside the council house, but when it became clear the numbers would be too large it was moved to Centenary Square.
Although it was billed as a stand-in demonstration, protesters moved on from the square and marched towards Lloyd House, the headquarters for West Midlands Police, shouting "justice now" and calling for an end to police brutality.
In the past month, the Independent Office for Police Conduct has begun nine investigations into West Midlands Police connected to alleged excessive use of force on black men and two officers have been suspended.
Ch Insp Sarah Tambling, from the force, said she was "really pleased with the atmosphere" at the protest in the city.