The rocker - who has sons Kingston, two, and five-month-old Zuma with wife Gwen Stefani, as well as daughter Daisy Lowe, 19, from a previous relationship - admits he is too tired to consider expanding his brood in the near future.
When asked if he wants more children, he said: "That would seem logical, but so would sleep. Sleep would be nice, and a bit of selfish time, which isn't around the corner just yet. But, the men aren't in control of that stuff, as you well know."
Gavin admits raising two young children has not been easy, and he and Gwen were unable to leave the boys together at first because of Kingston's jealousy.
He added: "Kingston has become super sweet and very protective over Zuma. At first it was best not to leave them alone, but now Kingston's just turned a corner and just a real sweetheart with him which is so sweet and just a relief because it can go the other way."
The singer insists he and Gwen won't push their children into pursuing a music career, but Kingston will experience life on the road when he tours with his dad next month.
He told America's OK! magazine: "You'd have to ask them if they want to be musicians, they both certainly have enough opinions to figure it out for themselves.
"Kingston is coming to the first couple of shows with me and I'm really looking forward to that. They keep trying to throw things into my schedule to do during the day and work but I'm like, 'Nooo, I'm busy.' I think it's going to be a lot of fun."