RoboCup 2D Soccer Simulation (RCSS2D) has long served as a standard robot soccer simulator; however, its workflow is implemented entirely in C++, which complicates integration with modern reinforcement learning pipelines in Python. Prior work such as Half Field Offense in RCSS2D is limited to half-field scenarios. We propose R2D-RL (RCSS2D-RL), an interface built on top of helios-base, a sample team codebase for RCSS2D, and it exposes full match gameplay in RCSS2D as a Python environment for multi-agent reinforcement learning.