I did some canoeing and kayaking with scouts while in high school and it was lots of fun. I only paddled the kayaks in flat water and I'll have to admit I liked the rapids better, which I only did in canoes. There were long stretches of flat, then a bit of excitement going through the rapids.
Basically, to get started you'd need to take a whitewater kayak clinic. My scout troop would not let anyone take a kayak into whitewater unless they could roll back to upright from a capsized position. Then you can find some guided trips on whitewater rivers. For example, here's a web page I found on guided kayak trips in Tennessee and Georgia:
Guided Trips
The rivers in that area are gorgeous in the summer. If there are rivers near you, you can probably find similar offerings on them. After a few guided trips, you would need to invest in your own equipment and find some kind of local kayaking club on the internet. I'm sure they exist.
Good luck.