A note on calibration apps. If your monitor is a reasonably new one, you will be in the small minority of those that calibrate, and so you might make what you see less-accurate as compared to what most people see. I know that what I see after using Adobe Gamma is very different from what I see when I go to the library or a friends house to double check the same peice. When I leave my monitor stock, the color-shifting is so small and uniform (on reasonably new monitors) that everything still looks good, even though it is a bit different. Colors can even vary on a new monitor for the first few minutes after you boot up, so I let mine get good and warm for about 15 minutes before I start.
The only place I have really had trouble with color is using drop-shadowed JPEGs combined with numeric background colors for the web. The colors shift a teeeny bit when you compress the images. You'll tweek the UI to perfection on your own monitor and then you'll go to a friend's house and it won't even be close to blending with the background. Anytime I need to do this I use a small, equally-compressed (same app) JPEG swatch as the background.
I have had this trouble in skins a bit as well. If you use different programs to make the JPEGS, the compression will be different, and it will look like patchwork to some people. Sometimes the font color won't match the skin, either. A good test is to drop your own screen color-depth to 256 and see if your colors are still all seemless and uniform. If not, the varied compression will show on some older monitors. I dunno if PNG has the same problem.
if you are really, really paranoid that what you see might not be seeing the real thing, there are fancy thingamajiggers for sale that attach to the screen to self-calibrate the monitor optically. (I played with one, and found it to be economically equivalent to throwing money in the twa-let.) The people I talk to that do design for magazines pretty much say they have to take what they see on the screen with a grain of salt, and go by industry-standard numeric color values.
Anyway, i'm rambling... If your monitor is old, calibrate it, but if it is new, I'd leave it alone.