The 37-year-old singer - who has eight dogs with wife Ayda Field - said he was happy to leave the judging duties to his Take That bandmate, Gary Barlow.
'Gary has found his feet really quickly,' he told The Sun. 'It would take me a long time. I would like to be in his position for one night, but not four months. What's the point when I could be at home with the dogs? I didn't get in this game to work.
'He speaks with authority. I work with him and I know he's in a better position to be sat up there saying what he says than I would be.'
Robbie did join Gary as a mentor during the judges' houses stage of the competition and admitted that choosing the final four contestants was a tough task.
'You can't just choose people because they're a bit upset because last time they didn't get through. You have to desensitise yourself, but who wants to break a 16-year-old's heart?'