By connecting light to magnetism, cavity magnon-polaritons (CMPs) can link quantum computation to spintronics. Consequently, CMP-based information processing devices have emerged over the last years, but have almost exclusively been investigated with single-tone spectroscopy. However, universal computing applications will require a dynamic and on-demand control of the CMP within nanoseconds. Here, we perform fast manipulations of the different CMP modes with independent but coherent pulses to the cavity and magnon system. We change the state of the CMP from the energy exchanging beat mode to its normal modes and further demonstrate two fundamental examples of coherent manipulation. We first evidence dynamic control over the appearance of magnon-Rabi oscillations, i.e., energy exchange, and second, energy extraction by applying an anti-phase drive to the magnon. Our results show a promising approach to control building blocks valuable for a quantum internet and pave the way for future magnon-based quantum computing research.