Recent advances in synthetic methods enable designing subunits that self-assemble into structures with precise, finite sizes and well-defined architectures, but yields are frequently suppressed by the formation of off-target metastable structures. Increasing the complexity (the number of distinct subunit types) can inhibit off-target structures, but leads to slower kinetics and higher synthesis costs. Here, we study icosahedral shells formed of programmable triangular subunits as a model system, and identify design principles that produce the highest target yield at the lowest complexity. We use a symmetry-based construction to create a range of design complexities, starting from the maximal symmetry Caspar-Klug assembly up to the fully addressable, zero-symmetry assembly. Kinetic Monte Carlo simulations reveal that the most prominent defects leading to off-target assemblies are disclinations at sites of rotational symmetry. We derive symmetry-based rules for identifying the optimal (lowest-complexity, highest-symmetry) design that inhibits these disclinations, leading to robust, high-fidelity assembly of targets with arbitrarily large, yet precise, finite sizes. The optimal complexity varies non-monotonically with target size, with `magic' sizes appearing for high-symmetry designs in which symmetry axes do not intersect vertices of the triangular net. The optimal designs at magic sizes require 12 times fewer inequivalent interaction-types than the (minimal symmetry) fully addressable construction, which greatly reduces the timescale and experimental cost required to achieve high fidelity assembly of large targets. This symmetry-based principle for pruning off-target assembly generalizes to diverse architectures with different topologies.