#! https://zhuanlan.zhihu.com/p/143392167

如需转发，烦请邮件告知 junjialiu@sjtu.edu.cn

Reinforcement-Learning-in-Robotics Content 专栏目录

This is a private learning repository for Reinforcement learning techniques, Reasoning, and Representation learning used in Robotics, founded for Real intelligence.

Reinforcement Learning Foundation

1. 神经网络基础：反向传播推导与卷积公式 [Zhihu]
2. 强化学习基础 Ⅰ：马尔可夫与值函数 [Zhihu]
3. 强化学习基础 Ⅱ：动态规划，蒙特卡洛，时序差分 [Zhihu]
4. 强化学习基础 Ⅲ：on-policy, off-policy & Model-based, Model-free & Rollout [Zhihu]
5. 强化学习基础 Ⅳ：State-of-the-art 强化学习经典算法汇总 [Zhihu]
6. 强化学习基础 Ⅴ：Q learning 原理与实战 [Zhihu]
7. 强化学习基础 Ⅵ：DQN 原理与实战 [Zhihu]
8. 强化学习基础 Ⅶ：Double DQN & Dueling DQN 原理与实战 [Zhihu]
9. 强化学习基础 Ⅷ：Vanilla Policy Gradient 策略梯度原理与实现 [Zhihu]
10. 强化学习基础 Ⅸ：一文读懂 TRPO 原理与实现 [Zhihu]
11. 强化学习基础 Ⅹ：一文读懂两种 PPO 原理与实现 [Zhihu]
12. 强化学习基础 Ⅺ： Actor-Critic & A2C 原理与实现 [Zhihu]
13. 强化学习基础 Ⅻ：DDPG 原理与实现 [Zhihu]
14. 强化学习基础 XIII：Twin Delayed DDPG TD3原理与实现 [Zhihu]

Model-based RL

1. Model-Based RL Ⅰ：Dyna, MVE & STEVE [Zhihu]
2. Model-Based RL Ⅱ：MBPO原理解读 [Zhihu]
3. Model-Based RL Ⅲ：从源码读懂PILCO [Zhihu]

Probabilistic in Robotics

1. PR 序：机器人学的概率方法学习路径 [Zhihu]
2. PR Ⅰ：最大似然估计MLE与最大后验概率估计MAP [Zhihu]
3. PR Ⅱ：贝叶斯估计/推断及其与MAP的区别 [Zhihu]
4. PR Ⅲ：从高斯分布到高斯过程、高斯过程回归、贝叶斯优化 [Zhihu]
5. PR Ⅳ：贝叶斯神经网络 Bayesian Neural Network [Zhihu]
6. PR Ⅴ：熵、KL散度、交叉熵、JS散度及python实现 [Zhihu]
7. PR Ⅵ：多元连续高斯分布的KL散度及python实现 [Zhihu]
8. PR Sampling Ⅰ：蒙特卡洛采样、重要性采样及python实现 [Zhihu]
9. PR Sampling Ⅱ：马尔可夫链蒙特卡洛 MCMC及python实现 [Zhihu]
10. PR Sampling Ⅲ：M-H and Gibbs 采样 [Zhihu]
11. PR Structured Ⅰ：Graph Neural Network: An Introduction Ⅰ [Zhihu]
12. PR Structured Ⅱ：Structured Probabilistic Model 结构化概率模型 [Zhihu]
13. PR Structured Ⅲ：马尔可夫、隐马尔可夫 HMM 、条件随机场 CRF 全解析及其python实现 [Zhihu]
14. PR Structured Ⅳ：General / Graph Conditional Random Field (CRF) 及其 python 实现 [Zhihu]
15. PR Structured Ⅴ：GraphRNN——将图生成问题转化为序列生成 [Zhihu]
16. PR Reasoning 序：Reasoning Robotics 推理机器人学习路线与资源汇总 [Zhihu]
17. PR Reasoning Ⅰ：Bandit问题与 UCB / UCT / AlphaGo [Zhihu]
18. PR Reasoning Ⅱ：Relational Inductive bias 关系归纳偏置及其在深度学习中的应用 [Zhihu]
19. PR Reasoning Ⅲ：基于图表征的关系推理框架 —— Graph Network [Zhihu]
20. PR Reasoning Ⅳ：数理逻辑（命题逻辑、谓词逻辑）知识整理 [Zhihu]
21. PR Memory Ⅰ：Memory systems 2018 – towards a new paradigm 【重磅综述】记忆系统——神经科学的启示 [Zhihu]
22. PR Perspective Ⅰ：Embodied AI 的新浪潮 —— new generation of AI [Zhihu]
23. PR Efficient Ⅰ：机器人中的数据高效强化学习 [Zhihu]
24. PR Efficient Ⅱ：Bayesian Transfer RL with prior knowledge [Zhihu]
25. PR Efficient Ⅲ：像训练狗狗一样高效地训练机器人 [Zhihu]
26. PR Efficient Ⅳ：五分钟内让四足机器人自主学会行走 [Zhihu]
27. PR Efficient Ⅴ：自预测表征，让RL agent高效地理解世界 [Zhihu]

Meta-Learning

1. Meta-Learning: An Introduction Ⅰ [Zhihu]
2. Meta-Learning: An Introduction Ⅱ [Zhihu]
3. Meta-Learning: An Introduction Ⅲ [Zhihu]

Imitation Learning

1. Imitation Learning Ⅰ：模仿学习 (Imitation Learning) 入门指南 [Zhihu]
2. Imitation Learning Ⅱ：DAgger透彻理论分析 [Zhihu]
3. Imitation Learning Ⅲ：EnsembleDAgger 一种贝叶斯DAgger [Zhihu]

RL from Demonstrations

1. RLfD Ⅰ：Deep Q-learning from Demonstrations 解读 [Zhihu]
2. RLfD Ⅱ：Reinforcement Learning from Imperfect Demonstrations under Soft Expert Guidance [Zhihu]

Multi-agent Reinforcement Learning

1. MARL Ⅰ：A Selective Overview of Theories and Algorithms 【重磅综述】 多智能体强化学习算法理论研究 [Zhihu]

Paper Reading

Active Visual Navigation

1. Reading：利用物体关系的目标驱动视觉导航 [Zhihu]
2. Reading：Learning to learn how to learn-Meta自适应视觉导航 [Zhihu]
3. Reading：Bayesian Relational Memory 在视觉导航中的应用 [Zhihu]
4. Reading：Attention+3D空间关系图在视觉导航中的应用 [Zhihu]
5. Reading：机器人导航的半参数化拓扑记忆结构 [Zhihu]
6. Reading：将Transformer应用到机器人视觉导航中 [Zhihu]

RL robotics in the physical world with micro-data / data-efficiency

1. 【重磅综述】如何在少量尝试下学习机器人强化学习控制 [Zhihu]

Others

1. End-to-End Robotic Reinforcement Learning without Reward Engineering: [Medium] [Zhihu]
2. Overcoming Exploration in RL with Demonstrations: [Medium] [Zhihu]
3. The Predictron: End-To-End Learning and Planning: [Zhihu]
4. IROS2019 Paper速读(一) [Zhihu]
5. IROS2019 Paper速读(二) [Zhihu]
6. IROS2019 Paper速读(三) [Zhihu]
7. IROS2019 Paper速读(四) [Zhihu]

Simulator

刘浚嘉：MuJoCo自定义机器人建模指南

Tools

1. Tools 1：如何用 PyQt5 和 Qt Designer 在 Pycharm 中愉快地开发软件 [Zhihu]
2. Tools 2：Arxiv 论文提交流程——看这篇就够了 [Zhihu]
3. Tools 3：Python socket 服务器与客户端双向通信（服务器NAT，文件传输） [Zhihu]
4. Tools 4：Python三行转并行——真香！[Zhihu]
5. Tools 5：Python三行转并行后续——多进程全局变量 [Zhihu]

专栏关联Github代码库

Reinforcement-Learning-in-Robotics
Machine-Learning-is-ALL-You-Need