A high energy lepton collider is the perfect discovery machine for particle physics, allowing to directly produce particles at the highest possible energies. However, it also has a huge potential for precision physics. I will discuss two different, and complementary ways in which such a machine could allow to indirectly probe new physics at an unprecedented level: high-energy scattering processes that have maximal sensitivity to new physics effects, and high-rate processes such as those induced by vector boson radiation. I will discuss a few most significant examples related to the muon g-2, electroweak boson production, and Higgs physics measurements. In particular, a muon collider could provide the first high-energy test of the muon g-2 anomaly, and could improve the bound on the muon electric dipole moment by three orders of magnitude.