Flores, who was passed over for a job with the Giants last week, said the league and its owners ran their operation "like a plantation".
He has filed a class action lawsuit in the US courts.
The league and teams have denied the racism claims.
The suit, filed on the first day of Black History Month in the US, begins with an errant text message said to be from New England coach Bill Belichick congratulating who he thought was Brian Daboll on getting the Giants job.
However, it was Flores he was texting with, three days before Flores interviewed with the team.
In addition to the NFL and Giants, the Dolphins and Denver Broncos are named as defendants along with 'John Doe Teams 1-29', meaning every other NFL franchise.
The lawsuit says: "In certain critical ways, the NFL is racially segregated and is managed much like a plantation.
"Its 32 owners - none of whom are black - profit substantially from the labour of NFL players, 70% of whom are black."
Flores alleges he was fired after rejecting an offer from Dolphins owner Stephen Ross for $100,000 for every defeat late in the season which would have improved the club's position in the off-season entry draft.
He also says he refused to violate the league's tampering rules when owner Ross arranged a meeting between Flores and a "prominent quarterback".
"After the incident, Mr Flores was treated with disdain and held out as someone who was non-compliant and difficult to work with," the suit reads.
"From that point forward, Mr Flores was ostracised and ultimately he was fired."
Flores, 40, finished with a 24-25 record over three seasons at Miami.
The suit was filed in the United States District Court for the Southern District of New York and purportedly shows screen grabs from Flores' phone of text messages between he and Belichick.
"I hear from Buffalo & NYG that you are their guy," Belichick is alleged to have texted Flores, mistaking him for Daboll.
Belichick apologised when he realised his mistake and said: "I double checked and I misread the text. I think they are naming Daboll. I'm sorry about that."
The lawsuit says Flores subsequently gave an extensive interview for a job he already knew he would not get.
"An interview that was held for no reason other than for the Giants to demonstrate falsely to the League Commissioner Roger Goodell and the public at large that it was in compliance with the Rooney Rule," it says.
The Rooney Rule, introduced by the NFL in 2003, states clubs must interview at least one black, Asian or minority ethnic candidate for each head coach and general manager vacancy.
In a statement put out by his law firm, Flores said: "God has gifted me with a special talent to coach the game of football, but the need for change is bigger than my personal goals.
"My sincere hope is that by standing up against systemic racism in the NFL, others will join me to ensure that positive change is made for generations to come."
The Broncos were named in the suit for their "sham interview" with Flores in 2019.
"It was clear from the substance of the interview that Mr Flores was interviewed only because of the Rooney Rule, and that the Broncos never had any intention to consider him as a legitimate candidate for the job," the suit reads.
The NFL said Flores' allegations were baseless and it was committed to providing equal opportunities.
"Diversity is core to everything we do, and there are few issues on which our clubs and our internal leadership team spend more time. We will defend against these claims, which are without merit," it said.
The Dolphins released a statement saying they "vehemently deny any allegations of racial discrimination".
"The implication that we acted in a manner inconsistent with the integrity of the game is incorrect," the team said.
The Giants said: "We are pleased and confident with the process that resulted in the hiring of Brian Daboll.
"We interviewed an impressive and diverse group of candidates. The fact of the matter is, Brian Flores was in the conversation to be our head coach until the 11th hour.
"Ultimately, we hired the individual we felt was most qualified to be our next head coach."
In his complaint against the Broncos, Flores said then-general manager John Elway and president Joe Ellis showed up an hour late to the interview, adding "it was obvious that they had drinking heavily the night before".
The Broncos called those allegations "blatantly false" in their statement.
"There were five Broncos executives present for the interview, which lasted approximately three and a half hours - the fully allotted time and concluded shortly before 11am," it said.
"Pages of detailed notes, analysis and evaluations from our interview demonstrate the depth of our conversation and sincere interest in Mr Flores as a head coaching candidate.
"The Broncos will vigorously defend the integrity and values of our organization - and its employees - from such baseless and disparaging claims."