The popular British show returns for its 11th series on October 8, but ITV bosses are in turmoil after Hill decided to leave the programme after nine years.
The comedian is said to have turned down a £1 million pay rise to continue, according to British newspaper The Sun.
Last year Hill, 46, admitted that he 'dreaded' making the show because watching TV is 'all-consuming'.
He will now allegedly work on Simon Cowell's rumoured X Factor musical.
A source told the publication: 'Harry kept telling people how much he hated doing it but they thought he was joking. He wasn't.'
An ITV spokesperson said that another series would air in 2012, while production company Avalon added: 'Harry has a lot of commitments in the next year, but a final decision has not yet been reached.'