The 38-year-old former judge announced that she would not be returning to the talent show last weekend, and according to The Mirror, a third change of the dates by show bosses was the final straw.
'She'd had enough of being messed around and treated like she was insignificant. The third change of dates was the final straw,' said the insider.
And Dannii wrote on her blog this week, 'I'm not furious, upset or livid. We simply couldn't make the dates work.'
Her departure has reportedly left the situation behind the scenes in chaos, with producers desperately looking for a female judge to replace her and Cheryl Cole, who is now working on the US programme.
'ITV wants things signed and sealed but Simon is still changing his mind and thinking of new names. The feeling is that without Cheryl and Dannii a big sexy female star is needed.
'If they end up with Tulisa and Kelly Rowland, who are the favourites for the jobs, fans will be very disappointed.'