Large-scale smart meter roll-outs all over the world are one effect of the ongoing energy transition. This poses a significant risk to consumer's privacy. Battery based load hiding (BBLH)---where an energy storage system is employed to obscure actual demand patterns---is one possibility to still retain privacy. In recent years many different BBLH algorithms have been proposed. But although most of them were assessed with some formally defined privacy measure, the current state of the art sorely lacks any comparability.
We give an overview of privacy measures proposed for this scenario, available storage technologies, and datasets used for the assessment of BBLH. Furthermore, we conduct a study of how these factors influence the ratings of several state-of-the-art BBLH algorithms. Our results illustrate the need for standardization as well as further research into meaningful privacy measures. Achieving this is necessary for private households to make an informed decision on which BBLH algorithm is best for their specific situation.