Actually... WAY back in the day, tower cases were rare & motherboards were mounted horizontally so it'd be parallel to your desk. This would leave the add-on cards such as Modem/Sound Card/Video Cards to be standing straight up & down.
Cooling was really neglected in design work up until probably the late 90s "Yes early/mid 90s CPUs would have passive cooling"
There are some cases out there that actually flip the board upside down, the only issue is the heat from the CPU rises into add-on cards with that design, as well as the BTX motherboard standard which fizzled out due to lack of proper scaling when it came to cooling. -- BTX died about 10 or so years ago.