Host Dermot O'Leary was reportedly asked by producers to apologise to viewers over Tulisa Contostavlos's use of the word 'milf' (meaning 'mother I'd like to fuck').
After Contostavlos used the word to describe contestant Melanie Masson, O'Leary later apologised on her behalf, despite being used after the watershed at 10.30pm.
"I want to apologise for the language Tulisa used. "Naughty, naughty, but that's why we love her," said O'Leary.
Some viewers described the apology as "ridiculous", The Sun reports.
One said: "I cannot believe Tulisa isn't allowed to say MILF at 10.30pm. Has the world gone mad?"
"Tulisa said MILF at 10.30 and that's not allowed? What?" another added.
A senior X Factor source later admitted that the decision to apologise was perhaps not necessary, saying: "It was a bit over-cautious and over-sensitive."
The X Factor continues next Saturday (October 20) at 8.20pm on ITV1. Masson was ultimately eliminated from the show last night.
