A LLM-based voice user interface for voice dialogues between user and industrial machines

Recent advancements in Large Language Models (LLMs) have significantly expanded the role of Artificial Intelligence (AI) in manufacturing. One promising area for LLM integration is machine control, particularly for industrial equipment like milling, drilling, and turning machines. Despite their critical role, these machines often exhibit limitations in user-friendliness, operational flexibility, accessibility, and operator safety which are partially based on console-based interfaces. Voice user interfaces (VUIs) mitigate those limitations. The current literature on VUIs highlights challenges like constrained instruction sets, voice recognition, and reliance on physical machines for testing. The proposed approach addresses these challenges by using LLMs, and virtual prototypes of physical machines, for intensive training and testing of the VUI. The paper describes this approach, explores application challenges, identifies key implementation limitations. This article also provides a comparison of accuracies of 5 pre-trained transformer based language models for understanding a set of sample commands that can be issued to the milling machine providing idea on the usability of pre-trained transformer models as the core conversational-component of the conceptualised VUI.