The 30-year-old former Destiny's Child star is set to join singer Chris Brown for three weeks in September, which will mean she gets back to the UK just days before the live shows start.
A source said to UK newspaper Daily Mirror: 'Kelly is taking a real risk jetting off in the middle of her first series of the X Factor. Some of the cast and crew are worried she is not taking it seriously.'
The tour will mean she has no one-to-one time with her finalists in person before the shows start.
However, a spokesperson for the show confirmed that she will call her acts from abroad before the live shows begin.