Copy That? The Influence of Process Model Representation Format on LLM Comprehension

LLMs are increasingly applied across the business process management (BPM) lifecycle, e.g., for analyzing or documenting processes. How reliably they support these applications depends on how well they comprehend the underlying process model. In practice, the same process model may be provided to an LLM in different ways, e.g., as a structured serialization or a rendered diagram. However, it is unclear whether and to what extent the choice of representation format affects comprehension. We address this gap with a systematic study that assesses comprehension through interpretation quality, i.e., the graded correctness of task responses. We compare the representation formats of the same process model in three categories: natural-language text, rendered diagrams, and structured serializations. This yields four isolated formats (textual narrative, PNG, BPMN-XML, Mermaid) and selected hybrid combinations, all derived from a common set of BPMN models. We evaluate these format configurations across six LLMs, eight processes, and eleven task types. An LLM-as-a-judge scores responses against reference answers using a graded metric and an error taxonomy. ... mehrThe results show that the representation format substantially affects interpretation quality, with no single format being best across all LLMs. The isolated PNG is consistently the weakest, while hybrid combinations of natural-language text with BPMN-XML or Mermaid emerge as the strongest formats overall. These findings establish the representation format as a design variable for LLM-supported BPM tasks and yield implications for its selection.