The BBC1 show has dropped almost 2million viewers since last year and at its peak 3.7million people were tuned in.
The latest show was knocked by the sunny weather meaning more people were outdoors and not watching their TV sets.
A source at the BBC said it hopes more viewers will be tuning in later in the series.
The source told UK newspaper The Sun: 'The last series went out in January so it was cold dark nights versus lovely sunshine now.
'You can't compare like for like. This is a bigger, longer-running series.'
The show sees Deeley as the host and judges include Arlene Phillips, Louise Redknapp and Nigel Lythgoe.