M4E Research #3: November & December 2023
LLM-MARS: Large Language Model for Behavior Tree Generation and NLP-enhanced Dialogue in Multi-Agent Robot Systems
Abstract: This paper introduces LLM-MARS, first technology that utilizes a Large Language Model based Artificial Intelligence for Multi-Agent Robot Systems. LLM-MARS enables dynamic dialogues between humans and robots, allowing the latter to generate behavior based on operator commands and provide informative answers to questions about their actions. LLM-MARS is built on a transformer-based Large Language Model, fine-tuned from the Falcon 7B model. We employ a multimodal approach using LoRa adapters for different tasks. The first LoRa adapter was developed by fine-tuning the base model on examples of Behavior Trees and their corresponding commands. The second LoRa adapter was developed by fine-tuning on question-answering examples. Practical trials on a multi-agent system of two robots within the Eurobot 2023 game rules demonstrate promising results. The robots achieve an average task execution accuracy of 79.28% in compound commands. With commands containing up to two tasks accuracy exceeded 90%. Evaluation confirms the system's answers on operators questions exhibit high accuracy, relevance, and informativeness. LLM-MARS and similar multi-agent robotic systems hold significant potential to revolutionize logistics, enabling autonomous exploration missions and advancing Industry 5.0.
Authors: Artem Lykov, Maria Dronova, Nikolay Naglov, Mikhail Litvinov, Sergei Satsevich, Artem Bazhenov, Vladimir Berman, Aleksei Shcherbak, Dzmitry Tsetserukou
Link: https://arxiv.org/abs/2312.09348
Real-Time Performance of Industrial IoT Communication Technologies: A Review
Abstract: With the growing need for automation and the ongoing merge of OT and IT, industrial networks have to transport a high amount of heterogeneous data with mixed criticality such as control traffic, sensor data, and configuration messages. Current advances in IT technologies furthermore enable a new set of automation scenarios under the roof of Industry 4.0 and IIoT where industrial networks now have to meet new requirements in flexibility and reliability. The necessary real-time guarantees will place significant demands on the networks. In this paper, we identify IIoT use cases and infer real-time requirements along several axes before bridging the gap between real-time network technologies and the identified scenarios. We review real-time networking technologies and present peer-reviewed works from the past 5 years for industrial environments. We investigate how these can be applied to controllers, systems, and embedded devices. Finally, we discuss open challenges for real-time communication technologies to enable the identified scenarios. The review shows academic interest in the field of real-time communication technologies but also highlights a lack of a fixed set of standards important for trust in safety and reliability, especially where wireless technologies are concerned.
Authors: Ilja Behnke, Henrik Austad
Autoencoder-Based Visual Anomaly Localization for Manufacturing Quality Control
Abstract: Manufacturing industries require efficient and voluminous production of high-quality finished goods. In the context of Industry 4.0, visual anomaly detection poses an optimistic solution for automatically controlled product quality with high precision. In general, automation based on computer vision is a promising solution to prevent bottlenecks at the product quality checkpoint. We considered recent advancements in machine learning to improve visual defect localization, but challenges persist in obtaining a balanced feature set and database of the wide variety of defects occurring in the production line. Hence, this paper proposes a defect localizing autoencoder with unsupervised class selection by clustering with k-means the features extracted from a pre-trained VGG16 network. Moreover, the selected classes of defects are augmented with natural wild textures to simulate artificial defects. The study demonstrates the effectiveness of the defect localizing autoencoder with unsupervised class selection for improving defect detection in manufacturing industries. The proposed methodology shows promising results with precise and accurate localization of quality defects on melamine-faced boards for the furniture industry. Incorporating artificial defects into the training data shows significant potential for practical implementation in real-world quality control scenarios.
Authors: Devang Mehta, Noah Klarmann