In an effort to get back into boxing again before it's waaaaaaaaay too late, I'm currently training twice a day six days a week. I love the weight training and most of the boxing specific training (especially the sparring, which is the fun part - usually). Running, ab work and some of the tedious drills that we do over and over and over and over again: not so much....
Any type of full contact or non-contact martial arts type training can be REALLY fun. You might want to try that out if you haven't. And no, you don't have to be a violent person.