That is pretty much how it works. In more detail:
You load an image for the first time:
  1. Internet Exlplorer sends a request for image. It opens a connection and downloads the image to your computer. The image is saved as a temporary file.
  2. When close your browser it will save these files to the cache (temporary internet files)
  3. Two days later you request the same image again. Internet explorer sees that it has previously cached the image and sends a request to the web server to the effect: "If this resource has been updated since xx/xx/xx xx:xx then send me the new copy".
  4. If the image has not been updated since the last request, Internet Explorer uses the cached copy from the temporary internet files.

Its worth noting that it is not only web browsers that cahce resources. Your request may pass through several proxy servers - each of which may have a cached copy of the resource. This makes the Internet seem a lot faster than it actaully is and without this caching the whole lot would probably grind to a halt.