[docs] Documentation for POCA and cooperative behaviors#5056

Merged

ervteng merged 300 commits intomainfrom

develop-coma2-docs

Mar 12, 2021

This pull request is big! We're only showing the most recent 250 commits

Commits on Dec 21, 2020

Make comms one-hot
Ervin Teng
committed

Commits on Dec 23, 2020

Fix S tag
Ervin Teng
committed

Commits on Jan 4, 2021

Merge branch 'master' into develop-centralizedcritic-mm
Ervin Teng
committed
Additional changes
Ervin Teng
committed
Some more fixes
Ervin Teng
committed

Commits on Jan 6, 2021

Self-attention Centralized Critic
Ervin Teng
committed

Commits on Jan 11, 2021

separate entity encoder and RSA
andrewcoh
committed
clean up args in mha
andrewcoh
committed
more cleanups
andrewcoh
committed
fixed tests
andrewcoh
committed
Merge branch 'develop-attention-refactor' into develop-centralizedcritic-mm
Ervin Teng
committed
Merge branch 'develop-attention-refactor' into develop-centralizedcritic-mm
Ervin Teng
committed
entity embeddings work with no max
Ervin Teng
committed
remove group id
Ruo-Ping Dong
committed
very rough sketch for TeamManager interface

Chris Elion
authored and
Ruo-Ping Dong
committed

Commits on Jan 12, 2021

One layer for entity embed
Ervin Teng
committed
Use 4 heads
Ervin Teng
committed
add defaults to linear encoder, initialize ent encoders
andrewcoh
committed
Merge branch 'master' into develop-centralizedcritic-mm
Ervin Teng
committed
Merge branch 'develop-lin-enc-def' into develop-centralizedcritic-mm
Ervin Teng
committed
add team manager id to proto
Ruo-Ping Dong
committed
team manager for hallway
Ruo-Ping Dong
committed
add manager to hallway
Ruo-Ping Dong
committed
send and process team manager id
Ruo-Ping Dong
committed
remove print
Ruo-Ping Dong
committed
Merge branch 'develop-centralizedcritic-mm' into develop-cc-teammanager
Ruo-Ping Dong
committed

Commits on Jan 13, 2021

small cleanup
Ruo-Ping Dong
committed
default behavior for baseTeamManager
Ruo-Ping Dong
committed
add back statsrecorder
Ruo-Ping Dong
committed
update
Ruo-Ping Dong
committed
Team manager prototype (#4850)

Ruo-Ping Dong
and
Chris Elion
authored
Remove statsrecorder
Ervin Teng
committed
Fix AgentProcessor for TeamManager
Ervin Teng
committed
Merge branch 'develop-centralizedcritic-mm' into develop-cc-teammanager
Ruo-Ping Dong
committed
team manager
Ruo-Ping Dong
committed

Commits on Jan 14, 2021

New buffer layout, TeamObsUtil, pad dead agents
Ervin Teng
committed
Use NaNs to get masks for attention
Ervin Teng
committed

Commits on Jan 15, 2021

Add team reward to buffer
Ervin Teng
committed
Try subtract marginalized value
Ervin Teng
committed

Commits on Jan 20, 2021

Add Q function with attention
Ervin Teng
committed
Some more progress - still broken
Ervin Teng
committed
use singular entity embedding (#4873)
andrewcoh
authored
I think it's running
Ervin Teng
committed

Commits on Jan 21, 2021

Actions added but untested
Ervin Teng
committed

Commits on Jan 22, 2021

Commits on Jan 23, 2021

Commits on Jan 26, 2021

Commits on Jan 27, 2021

Commits on Jan 29, 2021

Add trust region to COMA updates
Ervin Teng
committed
Remove Q-net for perf
Ervin Teng
committed
Weight decay, regularizaton loss
Ervin Teng
committed
Use same network
Ervin Teng
committed

Commits on Feb 1, 2021

add base team manager
Ruo-Ping Dong
committed

Commits on Feb 4, 2021

Remove reg loss, still stable
Ervin Teng
committed
Black format
Ervin Teng
committed

Commits on Feb 5, 2021

add team reward field to agent and proto
Ruo-Ping Dong
committed
set team reward
Ruo-Ping Dong
committed
add maxstep to teammanager and hook to academy
Ruo-Ping Dong
committed

Commits on Feb 8, 2021

check agent by agent.enabled
Ruo-Ping Dong
committed

Commits on Feb 9, 2021

remove manager from academy when dispose
Ruo-Ping Dong
committed
move manager
Ruo-Ping Dong
committed
put team reward in decision steps
Ruo-Ping Dong
committed
use 0 as default manager id
Ruo-Ping Dong
committed
fix setTeamReward

Ruo-Ping Dong
and
vincentpierre
authored
change method name to GetRegisteredAgents
Ruo-Ping Dong
committed
address comments
Ruo-Ping Dong
committed
Merge branch 'develop-base-teammanager' into develop-agentprocessor-teammanager
Ervin Teng
committed
Merge branch 'develop-base-teammanager' into develop-agentprocessor-teammanager
Ervin Teng
committed
Revert C# env changes
Ervin Teng
committed
Remove a bunch of stuff from envs
Ervin Teng
committed
Remove a bunch of extra files
Ervin Teng
committed
Remove changes from base-teammanager
Ervin Teng
committed
Remove remaining files
Ervin Teng
committed
Remove some unneeded changes
Ervin Teng
committed
Make buffer typing neater
Ervin Teng
committed
AgentProcessor fixes
Ervin Teng
committed
Back out trainer changes
Ervin Teng
committed
Separate Actor/Critic, remove ActorCritics
andrewcoh
committed
update policy to not use critic
andrewcoh
committed
add critic to optimizer, ppo runs
andrewcoh
committed
fix precommit errors
andrewcoh
committed
use delegate to avoid agent-manager cyclic reference
Ruo-Ping Dong
committed
fix test_networks
andrewcoh
committed
put team reward in decision steps
Ruo-Ping Dong
committed

Commits on Feb 10, 2021

fix unregister agents
Ruo-Ping Dong
committed
Update SAC to use separate policy
Ervin Teng
committed
add teamreward to decision step
Ruo-Ping Dong
committed
typo
Ruo-Ping Dong
committed
unregister on disabled
Ruo-Ping Dong
committed
remove OnTeamEpisodeBegin
Ruo-Ping Dong
committed
make critic a property
andrewcoh
committed

Commits on Feb 11, 2021

change name TeamManager to MultiAgentGroup
Ruo-Ping Dong
committed
more team -> group
Ruo-Ping Dong
committed
fix tests
Ruo-Ping Dong
committed
fix tests
Ruo-Ping Dong
committed
remove commented code
andrewcoh
committed
Merge remote-tracking branch 'origin/develop-base-teammanager' into develop-agentprocessor-teammanager
Ervin Teng
committed
Use attention tests from master
Ervin Teng
committed
Revert "Use attention tests from master"
Ervin Teng
committed
Use attention from master
Ervin Teng
committed
Renaming fest
Ervin Teng
committed
Use NamedTuples instead of attrs classes
Ervin Teng
committed
fix saver test
andrewcoh
committed
Move value network for SAC to device
Ervin Teng
committed
Merge remote-tracking branch 'origin/develop-critic-optimizer' into develop-critic-optimizer
Ervin Teng
committed
add SharedActorCritic
andrewcoh
committed
test for SharedActorCritic
andrewcoh
committed
fix agent processor test
andrewcoh
committed
Bug fixes
Ervin Teng
committed

Commits on Feb 12, 2021

remove GroupMaxStep
Ruo-Ping Dong
committed
add some doc
Ruo-Ping Dong
committed
fix sac shared
andrewcoh
committed
Fix mock brain
Ervin Teng
committed
np float32 fixes
Ervin Teng
committed
more renaming
Ervin Teng
committed
fix test policy
andrewcoh
committed
Test for team obs in agentprocessor
Ervin Teng
committed
Test for group and add team reward
Ervin Teng
committed
doc improve

Ruo-Ping Dong
and
Ervin T.
authored

Commits on Feb 13, 2021

Merge branch 'master' into develop-base-teammanager
Ruo-Ping Dong
committed
Merge branch 'master' into develop-base-teammanager
Ruo-Ping Dong
committed
Merge branch 'develop-base-teammanager' of https://github.com/Unity-Technologies/ml-agents into develop-base-teammanager
Ruo-Ping Dong
committed

Commits on Feb 16, 2021

store registered agents in set
Ruo-Ping Dong
committed

Commits on Feb 17, 2021

remove unused step counts
Ruo-Ping Dong
committed
Merge branch 'develop-base-teammanager' into develop-agentprocessor-teammanager
Ervin Teng
committed
Global group ids
Ervin Teng
committed
coma trainer and optimizer
andrewcoh
committed
MultiInputNetBody
andrewcoh
committed

Commits on Feb 19, 2021

Fix Trajectory test
Ervin Teng
committed

Commits on Feb 23, 2021

Merge branch 'master' into develop-agentprocessor-teammanager
Ervin Teng
committed
Remove duplicated files
Ervin Teng
committed
Merge branch 'develop-agentprocessor-teammanager' into develop-coma2-trainer-mm
Ervin Teng
committed
Running COMA (not sure if learning)
Ervin Teng
committed
Add team methods to AgentAction
Ervin Teng
committed
Right loss function for stability, fix some pypi
Ervin Teng
committed
Buffer fixes
Ervin Teng
committed
Group reward function
Ervin Teng
committed

Commits on Feb 24, 2021

Add PushBlockCollab config and fix some stuff
Ervin Teng
committed
Fix Team Cumulative Reward
Ervin Teng
committed
Buffer fixes
Ervin Teng
committed
clean ups (#5003)
andrewcoh
authored
Merge branch 'master' into develop-coma2-trainer
andrewcoh
committed
Add test for GroupObs
Ervin Teng
committed
Change AgentAction back to 0 pad and add tests
Ervin Teng
committed
Addressed some comments
Ervin Teng
committed

Commits on Feb 25, 2021

Address some comments
Ervin Teng
committed
Add more comments
Ervin Teng
committed
Rename internal function
Ervin Teng
committed
Move padding method to AgentBufferField
Ervin Teng
committed
Merge branch 'main' into develop-agentprocessor-teammanager
Ervin Teng
committed
checkout ppo/optimizer from main
andrewcoh
committed
Merge branch 'develop-agentprocessor-teammanager' into develop-coma2-trainer
andrewcoh
committed
Fix slicing typing and string printing in AgentBufferField
Ervin Teng
committed
Fix slicing typing and string printing in AgentBufferField
Ervin Teng
committed
Fix to-flat and add tests
Ervin Teng
committed

Commits on Feb 26, 2021

Commits on Mar 3, 2021

get value estimate test
andrewcoh
committed

Commits on Mar 5, 2021

[coma2] Make group extrinsic reward part of extrinsic (#5033)
Ervin T
authored
[coma2] Add support for variable length obs in COMA2 (#5038)
Ervin T
authored
Fix pypi issues
Ervin Teng
committed
Action slice (#5047)
andrewcoh
authored
Merge branch 'main' into develop-coma2-trainer
Ervin Teng
committed

Commits on Mar 8, 2021

add torch no_grad to coma LSTM value computation
andrewcoh
committed
torch coma tests: lstm, cur, gail
andrewcoh
committed
Fix warning message format
Ervin Teng
committed
Fix warning message formatting again
Ervin Teng
committed
Documentation for COMA2 and cooperative behaviors
Ervin Teng
committed
Add link to Environment Design
Ervin Teng
committed
Remove typo
Ervin Teng
committed

Commits on Mar 9, 2021

C# multiagent doc (#5058)
Ruo-Ping Dong
authored
Remove redundant heading
Ervin Teng
committed
Add changelog entries
Ervin Teng
committed
Address comments
Ervin Teng
committed
[coma2] Group reward reporting fix (#5065)
Ervin T
authored

Commits on Mar 10, 2021

Copy obs before removing NaNs (#5069)
Ervin T
authored
Change comments on COMA2 trainer
Ervin Teng
committed
Merge branch 'main' into develop-coma2-trainer
Ervin Teng
committed
Properly restore and test COMA2 optimizer
Ervin Teng
committed
multiinput netbody tests
andrewcoh
committed
Merge branch 'develop-coma2-trainer' of https://github.com/Unity-Technologies/ml-agents into develop-coma2-trainer
andrewcoh
committed
cleanup some mypy types (#5072)
Chris Elion
authored
coma -> poca
Ervin Teng
committed
Rename folders and files
Ervin Teng
committed
Fix two more coma references
Ervin Teng
committed
Update docs
Ervin Teng
committed
Multiagent simplerl (#5066)

andrewcoh
and
Ervin Teng
authored
add docstrings to network body
andrewcoh
committed
ource /Users/ervin/.virtualenvs/mlagents-38/bin/activate
Ervin Teng
committed
Update tests
Ervin Teng
committed
rename to MultiAgentNetwork, docstring
andrewcoh
committed
fix references to ppo
andrewcoh
committed
docstrings to poca optimizer
andrewcoh
committed

Commits on Mar 11, 2021

Update pushblock screenshot
Ervin Teng
committed
Add description of Dungeon Escape
Ervin Teng
committed
Move common loss functions for PPO and POCA (#5079)
Ervin T
authored
Turn on the SimpleMultiAgentGroup
Ervin Teng
committed
Add dungeon escape screenshot
Ervin Teng
committed
[poca] Remove add_groupmate_rewards from settings (#5082)
Ervin T
authored
Merge branch 'main' into develop-coma2-trainer
Ervin Teng
committed
Untrack PB Collab Config
Ervin Teng
committed
Update comment and fix reporting of group dones
Ervin Teng
committed
Merge branch 'develop-coma2-trainer' into develop-coma2-docs
Ervin Teng
committed
Address comments
Ervin Teng
committed
Fix coma reference
Ervin Teng
committed
Remove mention of envs
Ervin Teng
committed
Add diagram, correct capitalizations
Ervin Teng
committed
correct some more capitalizations
Ervin Teng
committed
Address some comments
Ervin Teng
committed
Remove dungeon escape
Ervin Teng
committed

Commits on Mar 12, 2021

Clean up docs a bit
Ervin Teng
committed
Fix references to MultiAgentGroup
Ervin Teng
committed
Merge branch 'main' into develop-coma2-docs
Ervin Teng
committed