Reports had claimed that the 28-year-old's northern accent was to blame for her sacking from the talent competition.
But Steve has since denied the reports about 'angel' Cole.
Jones, 34, told British newspaper The Sun: 'I spoke to her and said I was disappointed I wasn't going to spend a lot more time with her because she's a cool girl. The powers that be know exactly what happened. I'm my own little island backstage.'
He added: 'But the only thing I will say is it was nothing to do with the accent. I watched her judge about 140 contestants and only twice did someone say, 'Excuse me?'.
'If anything the people backstage were just looking in the monitors and saying, 'Oh my god it's an angel. She's beautiful'. Because she is, of course.'