Merge pull request #94 from automl/main

Dev update

CITATION.bib

@@ -1,7 +1,14 @@
-@inproceedings { BenEim2021a,
-  author = {Carolin Benjamins and Theresa Eimer and Frederik Schubert and André Biedenkapp and Bodo Rosenhahn and Frank Hutter and Marius Lindauer},
-  title = {CARL: A Benchmark for Contextual and Adaptive Reinforcement Learning},
-  booktitle = {NeurIPS 2021 Workshop on Ecological Theory of Reinforcement Learning},
-  year = {2021},
-  month = dec
+@inproceedings { BenEim2023a,
+  author       = {Carolin Benjamins and
+                  Theresa Eimer and
+                  Frederik Schubert and
+                  Aditya Mohan and
+                  Sebastian Döhler and
+                  André Biedenkapp and
+                  Bodo Rosenhahn and
+                  Frank Hutter and
+                  Marius Lindauer},
+  title        = {Contextualize Me - The Case for Context in Reinforcement Learning},
+  journal      = {Transactions on Machine Learning Research},
+  year         = {2023},
 }

docs/index.rst

@@ -16,12 +16,40 @@ Welcome to the documentation of CARL, a benchmark library for Contextually Adapt
 Reinforcement Learning. CARL extends well-known RL environments with context, making
 them easily configurable to test robustness and generalization.
 
-CARL is being developed in Python 3.9.
-
 
 Feel free to check out our `paper <https://arxiv.org/abs/2110.02102>`_ and our
 `blog post <https://www.automl.org/carl-a-benchmark-to-study-generalization-in-reinforcement-learning/>`_ on CARL!
 
+What is Context?
+----------------
+
+.. image:: ../figures/concept.png
+  :width: 75%
+  :align: center
+  :alt: CARL contextually extends Brax' Fetch.
+
+Context can change the goals and dynamics of an environment. 
+The interaction interval in Pendulum, for example, can make that environment muhc easier or harder.
+The same is true for the composition of a Mario level. 
+So context is a tool for creating variations in reinforcement learning environments.
+In contrast to other approaches like procedural generation, however, context can easily be defined and controlled by the user.
+That means you have full control over the difficulty and degree of variations in your environments.
+This way, you can gain detailed insights into the generalization capabilities of your agents - where do they excel and where do they fail?
+CARL can help you find the answer!
+
+If you're interested in learning more about context, check out our `paper <https://arxiv.org/abs/2202.04500>`_ or context in RL or the corrsponding `blog post <https://www.automl.org/contextualize-me-the-case-for-context-in-reinforcement-learning/>`_.
+
+What can you do with CARL?
+--------------------------
+
+With CARL, you can easily define train and test distributions across different features of your favorite environments.
+Examples include:
+- training on short CartPole poles and testing if the policy can transfer to longer ones
+- training LunarLander on moon gravity and seeing if it can also land on mars
+- training and testing on a uniform distribution of floor friction values on Halfcheetah
+... and many more!
+
+Simply decide on a generalization task you want your agent to solve, choose the context feature(s) to vary and train your agent just like on any other gymnasium environment
 
 Contact
 -------

docs/source/api/autoapi_link.rst

@@ -0,0 +1 @@
+.. include:: ../../autoapi/src/envs/carl_env/index

docs/source/api/index.rst

@@ -6,6 +6,7 @@ This page gives an overview of all CARL environments.
 
 
 .. toctree::
+    autoapi_link
     ../../autoapi/src/envs/carl_env/index
     ../../autoapi/src/envs/classic_control/index
     ../../autoapi/src/envs/box2d/index

docs/source/cite.rst

@@ -10,15 +10,16 @@ If you use CARL in your research, please cite us with the following Bibtex entry
 .. code:: text
 
     @inproceedings { BenEim2021a,
-      author = {Carolin Benjamins and
-                Theresa Eimer and
-                Frederik Schubert and
-                André Biedenkapp and
-                Bodo Rosenhahn and
-                Frank Hutter and
-                Marius Lindauer},
-      title = {CARL: A Benchmark for Contextual and Adaptive Reinforcement Learning},
-      booktitle = {NeurIPS 2021 Workshop on Ecological Theory of Reinforcement Learning},
-      year = {2021},
-      month = dec
+      author       = {Carolin Benjamins and
+                  Theresa Eimer and
+                  Frederik Schubert and
+                  Aditya Mohan and
+                  Sebastian Döhler and
+                  André Biedenkapp and
+                  Bodo Rosenhahn and
+                  Frank Hutter and
+                  Marius Lindauer},
+      title        = {Contextualize Me - The Case for Context in Reinforcement Learning},
+      journal      = {Transactions on Machine Learning Research},
+      year         = {2023},
     }

docs/source/environments/carl_env.rst

@@ -1,18 +1,13 @@
 The CARL Environment
 ====================
 
-CARL extends the standard `gym interface <https://gymnasium.farama.org/content/basic_usage/>`_  with context.
+CARL extends the standard `gymnasium interface <https://gymnasium.farama.org/content/basic_usage/>`_  with context.
 This context changes the environment's transition dynamics and reward
 function, creating a greater challenge for the agent.
 During training we therefore can encounter different contexts and train for generalization.
 We exemplarily show how Brax' Fetch is extended and embedded by CARL. Different
 instantiations can be achieved by setting the context features to different values.
 
-.. image:: ../figures/concept.png
-  :width: 75%
-  :align: center
-  :alt: CARL contextually extends Brax' Fetch.
-
 
 Here we give a brief overview of the available options on how to create
 and work with contexts.