Deep Reinforcement Learning Algorithms with MindSpore

this is fork by Deep Reinforcement Learning Algorithms with PyTorch

\

This repository contains MindSpore implementations of deep reinforcement learning algorithms and environments.

Algorithms Implemented

1. Deep Q Learning (DQN) _{^{(Mnih et al. 2013)}}
2. DQN with Fixed Q Targets _{^{(Mnih et al. 2013)}}
3. Double DQN (DDQN) _{^{(Hado van Hasselt et al. 2015)}}
4. DDQN with Prioritised Experience Replay _{^{(Schaul et al. 2016)}}
5. Dueling DDQN _{^{(Wang et al. 2016)}}
6. REINFORCE _{^{(Williams et al. 1992)}}
7. Deep Deterministic Policy Gradients (DDPG) _{^{(Lillicrap et al. 2016 )}}
8. Twin Delayed Deep Deterministic Policy Gradients (TD3) _{^{(Fujimoto et al. 2018)}}
9. Soft Actor-Critic (SAC) _{^{(Haarnoja et al. 2018)}}
10. Soft Actor-Critic for Discrete Actions (SAC-Discrete) _{^{(Christodoulou 2019)}}
11. Proximal Policy Optimisation (PPO) _{^{(Schulman et al. 2017)}}
12. DQN with Hindsight Experience Replay (DQN-HER) _{^{(Andrychowicz et al. 2018)}}
13. DDPG with Hindsight Experience Replay (DDPG-HER) _{^{(Andrychowicz et al. 2018 )}}
14. Hierarchical-DQN (h-DQN) _{^{(Kulkarni et al. 2016)}}
15. Stochastic NNs for Hierarchical Reinforcement Learning (SNN-HRL) _{^{(Florensa et al. 2017)}}
16. Diversity Is All You Need (DIAYN) _{^{(Eyensbach et al. 2018)}}

Environments Implemented

1. Bit Flipping Game _{^{(as described in Andrychowicz et al. 2018)}}
2. Four Rooms Game _{^{(as described in Sutton et al. 1998)}}
3. Long Corridor Game _{^{(as described in Kulkarni et al. 2016)}}
4. Ant-{Maze, Push, Fall} _{^{(as desribed in Nachum et al. 2018 and their accompanying code)}}

Usage

The repository's high-level structure is:

├── agents                    
    ├── actor_critic_agents   
    ├── DQN_agents         
    ├── policy_gradient_agents
    └── stochastic_policy_search_agents 
├── environments   
├── results             
    └── data_and_graphs        
├── tests
├── utilities             
    └── data structures            

i) To watch the agents learn the above games

To watch all the different agents learn Cart Pole follow these steps:

git clone https://github.com/p-christ/Deep_RL_Implementations.git
cd Deep_RL_Implementations

conda create --name myenvname
y
conda activate myenvname

pip3 install -r requirements.txt

python results/Cart_Pole.py

For other games change the last line to one of the other files in the Results folder.

ii) To train the agents on another game

Most Open AI gym environments should work. All you would need to do is change the config.environment field (look at Results/Cart_Pole.py for an example of this).

You can also play with your own custom game if you create a separate class that inherits from gym.Env. See Environments/Four_Rooms_Environment.py for an example of a custom environment and then see the script Results/Four_Rooms.py to see how to have agents play the environment.