In high energy particle physics, machine learning has already proven to be an indispensable technique to push data analysis to the limits. So far widely accepted and successfully applied in the event reconstruction at the LHC experiments, machine learning is today also increasingly often part of the final steps of an analysis and, for example, used to construct observables for the statistical inference of the physical parameters of interest. This thesis presents such a machine learning based analysis measuring the production of Standard Model Higgs bosons in the decay to two tau leptons at the CMS experiment and discusses the possibilities and challenges of machine learning at this stage of an analysis. To allow for a precise and reliable physics measurement, the application of the chosen machine learning model has to be well under control. Therefore, novel techniques are introduced to identify and control the dependence of the neural network function on features in the multidimensional input space. Further, possible improvements of machine learning based analysis strategies are studied. A novel solution is presented to maximize the expected sensitivity of the measurement to the physics of interest by incorporating information about known uncertainties in the optimization of the machine learning model, yielding an optimal statistical inference in the presence of systematic uncertainties.