The 42-year-old, who was previously married to Sandra Bullock, lives in Austin, Texas, with his kids Chandler, 16, Jesse Jr, 14, and Sunny, seven. Meanwhile, Von D lives in Los Angeles, where she shoots her reality show LA Ink.
Talking to People magazine, James said that the commuting back and forth became too much.
'I'm so sad because I really love her,' he said. 'The distance between us was just too much.'
The couple got engaged in January and had been planning to tie the knot this summer on their one-year dating anniversary.
Von D has also confirmed the split on Twitter.
'I am no longer w Jesse,' she wrote, 'and out of respect for him, his family and myself, thats all the info I'd like to share. Thanks for respecting that.'