The singer had received the fewest amount of viewer votes along with Roger Wright following their live performances of 'Born This Way' and 'Handbags and Gladrags' respectively.
The pair then had to perform a sing-off version of Lord Andrew Lloyd Webber's 'Love Changes Everything'.
With the choice down to Lloyd Webber, he told them: "This is not easy now. Despite a fantastic performance, I've got to save Roger."
A relieved Wright told host Amanda Holden: "Any one of us five could have gone, Nathan is phenomenal.
"Twice in the bottom two, it's a humbling scenario, believe me."
A smiling James said of his elimination: "I would like to say thank you for all your comments. I've really tried to do my best, I'm sorry if it wasn't enough.
"Thank you to everyone for voting, this is first time I've been in the bottom two and that's a real achievement."
The final five contestants had earlier performed a cover of Take That's 'Greatest Day' with Gary Barlow on piano.
Ben Forster, David Hunter, Rory Taylor and Roger Wright will battle it out for a place in the grand final in tomorrow's show, which airs at 9pm on ITV1.
