Agentic AI Architecture for Evaluating and Improving Reinforcement Learning Pipelines

Agentic AI Architecture for Evaluating and Improving Reinforcement Learning Pipelines

Abstract

Reinforcement Learning (RL) systems are gaining traction in application areas such as robotic systems, self-driving vehicles, and automated industrial operations.In these safety-critical domains, lifecycle management practices -- including model versioning, registry integration, automated evaluation, retraining triggers, and staged deployments -- are just as important as algorithmic performance in ensuring reliability and reproducibility. However, these practices are often scattered across source code, configuration files, CI/CD workflows, and deployment manifests, making it difficult to automate detection and correction. This paper introduces an Agentic AI architecture that utilizes multiple specialized LLM-based agents to evaluate RL pipelines against a curated catalog of 15 architectural best practices. The agents are specialized (e.g., file selection, detection, fix suggestion, code generation), and an orchestrated pipeline with automated validation gates combines practice detection and LLM reasoning to identify both explicit and implicit practice gaps and suggest, or where appropriate, even apply fixes. We validate the approach in a large-scale industrial cyber-physical production case study and in five open-source RL repositories with diverse architectures, demonstrating the scalable detection of missing practices and the generation of actionable fixes. The system achieved an overall detection F$_1$-score of 0.80 and a mean generation-quality score of 2.40, indicating that it reliably identifies missing practices and produces functionally valid code artifacts.

Grafik Top
Authors
  • Ntentos, Evangelos
  • Zdun, Uwe
Grafik Top
Shortfacts
Category
Paper in Conference Proceedings or in Workshop Proceedings (Paper)
Event Title
EEE/ACM 5th International Conference on AI Engineering - Software Engineering for AI
Divisions
Software Architecture
Event Location
Rio de Janeiro, Brazil
Event Type
Conference
Event Dates
12-13 April 2026
Date
12 April 2026
Export
Grafik Top