The judge has said that he thinks Cowell believes the boyband pose a "threat" to the success of One Direction, according to the Daily Star.
"He knows Union J are a threat to One Direction, so he's not going to say much about them," Walsh is quoted as saying.
Cowell mentored One Direction to third place in the 2010 series, and the group have since gone on to achieve worldwide success. However, Walsh added that there is room for more than one successful boyband in the charts.
"There's always room for a new boyband if they are good enough and they work hard," he explained.
Walsh was also quick to add that his other boyband in this year's competition should also be taken seriously, saying: "I think we have got two great boybands. It's not just Union J, District 3 have come back into the race too."
District3 fell into the bottom two in the second week of the competition, but were praised by the judges this weekend for their performance of Madcon's 'Beggin'' and Chris Brown's 'Turn Up The Music'.
Union J meanwhile had to cancel a meet and greet last week, after it was feared that safety could be compromised due to the number of fans that turned up.
Walsh also added that Cowell is a big fan of three contestants so far.
"I know he watched the very first show and I know he loves Ella [Henderson]. He couldn't believe Jahmene [Douglas]. I told him he was amazing and he said, 'I agree with you, Jahmene's got great potential'. And there was also James Arthur. They were his three," he said.
A source close to Cowell said: "Simon will work extremely hard for any bands he signs to his label."
Cowell left the UK version of the show in 2010 to work solely on The X Factor USA, which airs on Fox in the US and on ITV2 in the UK.
