It is widely repeated in the popular literature and elsewhere that the singularity at the center of a black hole is a point. It is not true. Two observers who free-fall into a spherical black hole along two different angular trajectories at the same time $t$ do not encounter each other at the central singularity; rather, they lose causal contact with each other already well away from the singularity. Counterintuitively, in general relativity two points can be spatially close yet causally distant. The singularity is a surface, not a point. The story for rotating black holes is more complicated, but the same conclusion holds. For a rotating black hole, the singular surface almost certainly resides at its inner horizon, where even the tiniest classical or quantum perturbations ignite the exponential mass inflation instability, precipitating collapse to a spacelike singular surface. There are implications for quantum gravity. We argue that, whatever the ultimate theory of quantum gravity may be, the quantum states of a black hole probably reside at its effectively 2-dimensional singular surface, which coevolves unitarily with, and in thermodynamic equilibrium with, the hot atmosphere of trapped Hawking radiation that the black hole generates within its event horizon.