Reinforcement learning (RL) systems deployed in production face monitoring challenges absent in traditional machine learning: policies degrade silently under environment drift, reward signals can be gamed, and agents may develop hacking behaviors that evade standard evaluation. Despite growing practitioner interest, no systematic study has mapped the architectural design decisions (ADDs) practitioners face when building monitoring infrastructure for deployed RL agents. We address this gap through a Straussian Grounded Theory analysis of 29 gray literature sources, performing open, axial, and selective coding to extract ADDs, decision options, and decision drivers related to post-deployment RL monitoring. Our analysis identifies seven ADDs organized along a layered monitoring pipeline, spanning signal selection, statistical degradation testing, false alarm calibration, automated response mechanisms, and reward hacking detection and mitigation, together with 57 decision options and 72 decision drivers grounded in verbatim evidence. The emerging theory reveals that monitoring must be multi-signal and adversary-aware, as aggregate metrics can remain stable while dangerous behaviors grow in specific interaction contexts. The results are formalized as a decision model with UML visualizations to support practitioners in navigating the RL monitoring design space, with the full catalog available in our replication package.

Top