Reinforcement Learning (RL) is increasingly used in modern Industry~4.0 systems, enabling intelligent control and adaptation in dynamic production environments. However, the management and operationalization of RL models is less mature than established machine learning operations (MLOps) practices, which are primarily designed for traditional machine learning and deep learning workflows. Unlike these conventional approaches, which rely on static datasets and predefined training pipelines, RL agents learn continuously and interact with evolving environments, adding to the complexity of model management. This work investigates how RL models are managed in practice and how their training pipelines are architected and maintained. We study an industrial RL pipeline through a real production-automation case study, analyzing system artifacts and configurations using a catalog of RL-specific Architectural Design Decisions (ADDs) as the analytical lens. The work contributes a structured catalog of 15 ADDs for RL model management and pipeline implementation and demonstrates how static analysis can assess conformance to these practices. To further support our findings, we present a tool that detects architectural patterns and identifies missing practices, enabling engineers to build more reliable, traceable, and reproducible RL systems.

Top