Symmetry is one of the most general and useful concepts in physics. A system that has a symmetry is fundamentally constrained by it. The same constraints do not apply when the symmetry is broken. The quantitative determination of how much a system breaks a symmetry allows to reach beyond this binary situation and is a necessary step towards the quantitative connection between symmetry breaking and its effects. We introduce measures of symmetry breaking for a system interacting with external fields (particles). They can be computed from measurements of the system-mediated coupling strengths between subspaces of incoming and outgoing fields (particles) that transform in a definite way under the symmetry. The generality of these symmetry breaking measures and their tight connection to experimental measurements make them applicable to a very wide range of physics, like quantification of phase transitions, constraints in dynamical evolution, and the search for hidden symmetries.