You can find all the links on Werner Vogel's blog. I share his enthusiasm for the new host-a-website feature. I was asking for it for a long time. I guess, from reading the tea leaves, he was too.
Anyway, I had to do a bunch of trial and error to get my first S3-hosted static site configured. I wanted to leave a howto behind me so next time I won't have to do the head-scratching again. Maybe it will help you too.
1. Go to the S3 panel on the AWS website and create a new bucket called r2.reallysimple.org in the US-Standard region.
2. Upload a file called index.html to the new bucket. Say something simple like Hello World. You can copy the HTML from my test file if you want.
4. Click Permissions in the lower-right panel. Then click on Edit bucket policy. Paste in the template text you grab from this file. Edit it to replace YOUR-BUCKET-NAME-GOES-HERE with the name of your bucket (in the example r2.reallysimple.org). Click Save.
Now you should be able to go r2.reallysimple.org and see the text of your index file.