I completely understand where you're coming from. When I had my first child, I noticed that for the entire first year I had to scale way back on the business I was running. It was frustrating as I watched others growing their businesses while mine stayed pretty static. When she was older I got a high school girl to come over and play with her for just 3 hours a day. I wrote my first book in those three hours a day. She was happy, I was happy, all was well.
When the second child came, I knew in advance what was going to happen to my time so I arranged for people to manage my business for me so I could give him the attention he needed.
When the second child turned three and the first was 6 I got a lot of freedom back. They are now both in school full time so I am finally working a full 8 hours a day again. It's glorious to be working on my own goals while also giving as much quality time to my kids as I can. Plus now they like playing with each other.
So, to get back to you...
It's a balance. If you need/want more time to work on the things that are important to you, consider bringing in some help. You can let your kids play with someone while you're in the house working or learning.