The integration of artificial intelligence (AI) into software architecture has the potential to greatly support practitioners in architecting machine learning (ML) and reinforcement learning (RL) systems. In the context of Industry 4.0 cyber-physical production systems (CPPSs), this work presents our approach, research problems, and open challenges when designing ML and RL systems and applying AI-powered architecting to enhance automation and reduce manual effort. We describe a novel approach with six advanced contributions: organising architectural design decisions (ADDs) linked to the ML workflow and deployment, extending machine learning operations (MLOps) into reinforcement learning operations (RLOps), introducing formal architectural model development, adding automatic conformance checks, using large language models (LLMs) in RL conformance assessment and creating MLOps and RLOps pipelines using LLMs and a low-code approach. We also describe how earlier contributions in supporting the architecting process could be enhanced using AI. We discuss four further key open research challenges: RL model versioning, the development of RL communication protocols, working with agentic AI, and supporting RL within continuous integration and continuous delivery pipelines.

Top