The 29-year-old actor told reporters at a screening of his new BBC Olympic drama Bert and Dickie that he is "selective" about the parts he picks.
"Because we shoot [Doctor Who] for nine, ten months, all in all, you get about a month to do something else," he revealed.
Smith added that he only has time to shoot one additional project outside of Who per year.
"Of course, you have to be selective," he said. "But... I responded to the [Bert and Dickie] script. It's made by people I know at the BBC and people that are of great repute. And that's important.
"There was a good director attached and I always thought it would get a good, strong cast. But it just has to start with a script and I guess I just responded to that."
Bert and Dickie airs on Wednesday (July 25) at 8.30pm on BBC One.
