I use
NearlyFreeSpeech and
Amazon S3 for hosting.
NFS will host many types of dynamic content and they have SSH support, ect. It's a high-quality host too IMO, and they won't take your site down unless it violates a law. Domain names there are $7.50 and you can do whatever you want with them.
S3 is cheaper than NFS but can't host dynamic content or anything. So you should probably host all of your images, ect, with them and your scripts with NFS.
Both charge based on your usage. To use MySQL in NFS, you need to buy a MySQL process, which will cost one cent a day.