Mantle convection shapes Earth's surface by generating dynamic topography. Observational constraints and regional convection models suggest that surface topography could be sensitive to mantle flow for wavelengths as short as 1,000 and 250 km, respectively. At these spatial scales, surface processes including sedimentation and relative sea‐level change occur on million‐year timescales. However, time‐dependent global mantle flow models do not predict small‐scale dynamic topography yet. Here we present 2‐D spherical annulus numerical models of mantle convection with large radial and lateral viscosity contrasts. We first identify the range of Rayleigh number, internal heat production rate and yield stress for which models generate plate‐like behavior, surface heat flow, surface velocities, and topography distribution comparable to Earth's. These models produce both whole‐mantle convection and small‐scale convection in the upper mantle, which results in small‐scale (<500 km) to large‐scale (>104 km) dynamic topography, with a spectral power for intermediate scales (500 to 104 km) comparable to estimates of present‐day residual topography. Timescales of convection and the associated dynamic topography vary from five to several hundreds of millions of years. For a Rayleigh number of 107, we investigate how lithosphere yield stress variations (10–50 MPa) and the presence of deep thermochemical heterogeneities favor small‐scale (200–500 km) and intermediate‐scale (500–104 km) dynamic topography by controlling the formation of small‐scale convection and the number and distribution of subduction zones, respectively. The interplay between mantle convection and lithosphere dynamics generates a complex spatial and temporal pattern of dynamic topography consistent with constraints for Earth.