The star, who forged a successful solo career for himself after leaving the group in 1995, said even when Gary's own solo career flopped he couldn't stop trying to get one over on him.
The Sun reports that in the documentary about the band's recent reunion, Take That: Look Back, Don't Stare, Robbie confesses, 'My problem has always been with Gary. It always was with Gary. I wanted to crush him, I wanted to crush the memory of the band, and I didn't let go. 'You know, even when he was down, I didn't let go. And for that I sincerely apologise.
'You only hurt so much because you love them so much.'
But he said that being back in a band is much more interesting than being a solo artist.
'All of a sudden I'm inspired, there's something to work on and some people to work with,' he said. 'There's nothing for me to do other then be in Take That.
'I'm f****** bored. I'm scared, I'm lonely, and I've said everything I want to say in a record. I'm enjoying this sharing too much to want to go back to the Robbie Williams band.
'I'm breaking me up. Yeah, I'm breaking me up because of music similarities.'