Update for Joomla 2.5.x
Note that Joomla versions 1.6 upwards include this as an option in the global configuration settings which you can access using your Joomla admin panel:

1. Template Manager
2. Options
3. Preview Module Positions -> Disabled or Enabled

You will still need to suffix the frontend URL with ?tp=1


Joomla 1.5.x

 This is another feature which took me a few years to notice in Joomla.  If you've ever installed a template for Joomla and spent hours working out where or how you are going to put what where, try adding ?tp=1 after your index.php url (ie. http://www.yourdomain.com/?tp=1)

If this doesn't work for you try this:

1. Log in to the backend of your website (into the Joomla Admin Panel)
2. Go to Extensions > Template Manager > select your template
3. Click on the big Preview button in the top right
4. Click on the link 'Show in New Window'
5. Bookmark the web address of the new window

The bookmark/favorite that you have just made can be refreshed when you change things on your site and should reflect exactly how your website is overlayed with the module positions.  You will need to repeat the process when you select a different template but otherwise the bookmark will last as long as you keep that template.