Speaking at the Edinburgh Festival, Fincham compared the Simon Cowell talent show to Olympic long-distance runner Mo Farah, insisting that the missing viewers from last week's series nine launch were not worrying.
"It's not the 100m, it's the 10,000m. It's got years," said Fincham. "Think Mo Farah not Usain Bolt.
"It's an illusion to think that a show like The X Factor will go on a slide. People said that about Strictly after numbers fell for a couple of years and then it came back."
Last week's X Factor launch was the show's lowest since 2006.
When probed about the ratings drop, he added: "Oh gosh, X Factor is here for a long time.
"Last Saturday's X Factor was more than twice the size of the next biggest show on television.
"The figures were down, but it was the hottest weekend of the year and I think it's not unreasonable to say in the immediate aftermath of the Olympics viewing figures have gone down. People gorged themselves on TV in the Olympics.
"How long has The X Factor got implies that it does have a finite life. I don't agree with that proposition."
However, Fincham did express concern that the big entertainment shows "that dominated the last decade" such as I'm A Celebrity, The X Factor and Strictly were making it hard to create new series.
"It's difficult to grow similar brands on the same scale," he said.
