The New Zealand-born actor, who is starring in new flick The Next Three Days, says he and wife Danielle Spencer cosy up for 'date nights' in front of the talent show.
And he even confessed he'd be auditioning for the singing contest like a shot if he were a couple of decades younger.
The 46-year-old Oscar winner told Extra TV: 'I watch The X Factor'. We use that as our date night.
'I know that probably sounds pretty sad to some of you people out there.
'I know if I was 20 now, I'd be lining up with everybody else, trying to have a go.'
Crowe also gushed about fatherhood, telling host Mario Lopez - a new dad - that it 'just gets better and better'.
'I just came across some photographs of my little one, Tennyson, when he was just born,' he said.
'It's funny to look back on the relationship you have automatically as soon as they're there. It's very easy for me to get sentimental about it.'
Crowe and his wife have two sons, Charlie, 6, and Tennyson, 4.