Is this the terrain rendered in 3D from the galaxy map, or the planet surface tiles?
Both. All planets have a heightmap, which is basically a greyscale version of the map on the colony screen (the game uses it to determine terrain types on tiles, and colorizes it based on the class/environment/etc). If the planet doesn't have a custom texture defined, the colorized heightmap is used as the texture on the globes as well.
When you turn the variety slider all the way down, you're basically going from using all of those heightmaps (or if you don't have the package installed, the game generates them at game creation), to just a couple or even one per class, AFAIK. So all planets of each class having the same terrain is the price for the decreased memory use. If you don't care about that, then by all means turn it down to 0.