The important role of automated optical inspection in the manufacturing process necessitates the design of optimized and precise inspection setups which can fulfill the application demands. Due to the high dimensionality of the design space, a manual choice of the geometrical and optical parameters is tedious and often not optimal. In this paper, we propose the calculation of two comprehensive evaluation metrics, namely the lateral resolution and the measurement uncertainty, based on using a CAD model of the workpiece and real-time graphical simulations. Automatic setup evaluation enables us to determine the setup adequacy with respect to the part tolerances and inspection quality requirements, which opens up the path to automatic optimization and virtual rapid prototyping of machine vision systems. We present the optical inspection of a cylinder head using laser triangulation as an application. Based on graphical simulation results, the proposed metrics are evaluated for different configurations of the setups and the achieved performance characteristics are compared against the part inspection tolerances on different areas of the CAD model.