The linear accelerator FLUTE (Far Infrared Linac and Test Experiment) at KIT serves as a test facility for accelerator research and for the generation of ultra-intense coherent THz radiation. To achieve stable THz photon energy and optimal beam trajectory, the energy of the electrons emitted from the RF photo-injector must be stable. The accelerating voltage of the RF cavity has been shown to be a significant influencing factor. Here, we report on the development of a slow closed-loop feedback system to stabilize the RF power and thus the accelerating voltage in the RF photo-injector cavity. With this closed-loop feedback system the relative standard deviation of the RF power in the cavity can be improved by 8.5 %.