Kentaro Kobayashi joked about the Holocaust in what is described as an inappropriate way in an old video.
In the clip from 1998, Kobayashi participates in a sketch, where it is joked about the Holocaust.
- It became known that during an earlier appearance, he used a language that mocked a tragic part of history, says Olympic director Seiko Hashimoto.
Kobayashi apologizes in a statement, calling some of the lines in the sketch extremely inappropriate.
"It was from a time when I did not manage to get laughs the way I wanted, and I think I tried to get people's attention in a prejudiced way."
He is the third prominent person to be forced to leave the lead of the ceremony, before it even took place.
In the past, the composer and the creative director have also been forced out.
The former has since become famous in an interview where he tells how he treated disabled classmates badly, and the creatively responsible Hiroshi Sasaki was allowed to leave in March after he suggested that a female comedian should act like a pig.
The Olympics will be inaugurated at Tokyo's Olympic Stadium on Friday night, local time.