Matt, who won last year's series of the show, also said he can't wait for the 'X Factor winner's label' to go away so he just be an individual recording artist.
Cardle told British Look magazine: 'I massively compromised myself for X Factor. I wore yellow trousers and sang Katy Perry's Firework for f***s sake!
'Singing pop songs without my guitar and covered in make-up didn't sit right.
'I hear people in the street saying, 'There's the guy that won 'X Factor'. I can't wait for that to just disappear.'
Despite the 28-year-old's criticisms of the singing competition, which saw him perform his new song Run For Your Life on Sunday night's show, he says the programme taught him a lot about himself.
He added: 'I think I'm a better person than I was back then. The X Factor taught me a lot about myself.''