I haven't looked at Azure itself to say how they expect it to be used. A lot of these "cloud" offerings are really meant to be consumed internally though, not by the public directly.

How they handle DNS would be a good question. I assume Microsoft is using some sort of dynamic routing behind a "front door" router. This would let your application appear to be in the same place (in terms of DNS names or even IP addresses) even though it may "float" among different physical servers.

Another quick look suggests that yes, you could just host a web site in Azure. That really isn't what Azure is meant to be used for though. Simple web hosting is probably cheaper to obtain through conventional web hosting arrangements. When they talk about "web applications" I think they mean the term more generally, as in "crap glued together using Web Services as an IPC technology" and not a conventional ASP.Net, PHP, etc. "web application."