Williams told David Letterman during an appearance on the Late Show that he wasn't surprised Rio won the bid instead of Chicago.
"Chicago sent Oprah and Michelle Obama," Williams said. "Rio sent 50 Strippers and a pound of blow. It wasn't really fair, you know?"
Williams' comments have been replayed several times on news shows across Brazil on Tuesday, while city mayor Eduardo Paes says Williams is envious of Rio's achievement.
Rio's Olympic Committee said its lawyers will analyze whether legal action against Williams is possible.