coadapt-marl

A small, readable, decentralized multi-agent RL harness under partial observability. Each environment is a world of N actors; the agent setup is N independent local policies, bound one-to-one to the actors. Every agent sees only its own measurement plus delayed messages relayed from its graph neighbours — so information about a moving target arrives later the farther an agent is from the source, and coordination collapses (the information-arrival wall).

The pipeline

spaces  -> action types (Discrete / MultiDiscrete / Box)
env     -> the Env contract + EnvSpec
envs/   -> the N-actor worlds (task state, per-actor measurement, team reward)
network -> the communication layer: who hears whom, and how stale (partial observability)
agent   -> N independent local policies (no parameter sharing)
rollout -> the observe -> act -> step -> communicate loop (scan over time, vmap over seeds)
solvers -> from-scratch learners (reinforce, ppo, grpo, trpo, recurrent)
storage -> save a run as a structured dict file
plotting-> read those files and draw the standard figures

Quick start

pip install -r requirements.txt
python3 -m pytest tests/ -q                         # fast, pure JAX/CPU

bash examples/01_train_single.sh   relay_discrete ppo 300
bash examples/04_compare_solvers.sh 300 16          # all solvers -> comparison table

Where to go next

• Using the code — the environments, solvers, entry points.
• Scripts reference — every script and all its flags.
• Results — output layout, file schema, reading figures.
• Create a new solver — the plug-in interface.
• Create a new environment — the Env contract.
• API reference — every function, from the source.