Fans of the sci-fi drama have speculated that a 2013 episode could bring back former Doctors, but Baker told Cambridge News that such a reunion is "not going to happen".
"They don't need us," said the actor, who originally played the sixth Doctor between 1984 and 1986. "The programme is doing extremely well without us.
"Also, every time someone asks me that question I hold up a photograph of myself when I played the part, compared to what I look like now, and say, 'OK, how do you deal with that?' I've changed."
Baker insisted that he is "neither for nor against" reprising his Doctor Who role, as long as his incarnation of the Time Lord got "a fair crack of the whip".
"There would be two big questions I would ask: the first is, is my Doctor going to get a fair crack of the whip in the story, and not be eclipsed by anybody else?" he explained.
"And the other one is, what vast fortune are you offering me? I'd be quite brutal... and say, if they offered me a million quid, I'd go on and say one line for them. If they offered me a tenner, I wouldn't."
Paul McGann - who played the eighth Doctor in a 1996 TV movie - previously told Digital Spy that he would "love" to reappear on the show, while Peter Davison (the fifth Doctor) has admitted that he would be "very surprised" to be asked back.
In 2011, Christopher Eccleston - who starred as the ninth Doctor in 2005 - also appeared to rule out a return.
