Why Amazon S3?
The main reason for switching to S3 as a host for this site was cost. This blog gets almost no traffic so I anticipate my monthly S3 bill to be less than $1 USD. That is pretty cheap. I was previously using a VPS from Linode. It was $19.99 a month. I love Linode, and next time I need a VPS I will go with them. But it was overkill for this blog. I was simply serving static HTML from Apache.
Using Amazon's Simple Monthly Calculator I estimate that even if this site gets a couple orders of magnitude more popular, I'll still be paying less than $4.00 a month.
I also no longer have to worry about server security and server uptime. Those are two pretty big pluses.