The crooner took to the stage on American Idol in 2007 and said that everything went to pieces when he caught the music mogul rolling his eyes.
He told the Radio Times, 'I had to sing Call Me Irresponsible in front of 20 million people. And the first thing I saw when I got on stage was Simon Cowell.
'I sang about four lines and he looked at me and did this (rolls his eyes dismissively) and that was it, I was done. It killed me. My confidence dropped.'
And he dismissed reports that his 'aloof' reaction had anything to do with being under the influence of alcohol.
'I get the attitude of, 'Oh really? Well, f*** you.' And if you watch the clip on YouTube, I'm so aloof I look bored.
'And it became quite a thing for me in America because all the news agencies started to report that I was drunk. But I'm such a control freak I would never drink before going on stage.'
But when news of the episode eventually got back to Cowell via their mutual friend Dannii Minogue years later, Buble said he had no recollection of it.
'She goes and tells him, and comes back saying this: the man didn't even know he did it. He was just probably sitting on his fifth hour of the show going, 'Kill me now... and I've got to go to the bathroom'.
'But I'm so sensitive I took it as a rejection. Anyway, every time I've ever talked to him since then he's been incredibly nice.'