For Windows XP:

Open My Computer
Click on Tools
Click Folder Options
Click the View tab
Put a check in the box next to "Do not cache thumbnails"
Click on OK

For Windows Vista and Windows 7:

Click the Start button.
Click on Control Panel.
Click on Folder Options.
Click on the View tab.
Under the Files and Folders section check the button next to "Always show icons, never thumbnails"
Click on Apply.
Click on OK.