Blog

7 Model-based RL Papers I liked from NeurIPS 2020

30 Dec 2020

Here is a bunch of brief informal impressions on the ideas behind some of the reinforcement learning papers I enjoyed at NeurIPS 2020. For the full story, including the experimental or theoretical results, I encourage you to check out the full works, that are linked. The common thread in all of these works? Some form of, explicit or implicit, model-based reinforcement learning.


8 Albums I liked from 2020

30 Dec 2020

2020 was a difficult year for many. In difficult times, music, and art in general, has a very special role for society and individuals. To thank the artists that helped me and thousands of other music lovers go through this year, I want to give my impressions on my favorite eight albums from the last twelve months, restricting myself to three macro-genres that I enjoy, neoclassical, jazz and rock.


How to learn a useful Critic? By model-based RL!

30 Apr 2020

There is a number of potential gains in using an approximate model in reinforcement learning, in terms, for instance, of safety and, as most commonly affirmed, sample-efficiency. However, there is an advantage that should not be forgotten and that is, perhaps, the most interesting: approximating the environment dynamics can unlock peculiar learning modalities that would be impossible in a model-free setting. We will see in this blog post that model-based techniques can be leveraged to obtain a critic that is tailor-made for policy optimization in an actor-critic setting. How? By allowing the critic to explicitly learn to produce accurate policy gradients.


Model-based Policy Gradients

26 Jan 2020

pirandello You recognized it. This is the super-handy expression for the policy gradient derived more than 20 years ago by Richard Sutton and colleagues. In this blog post, I will show how the Policy Gradient Theorem can offer a lens to interpret modern model-based policy search methods. Yes, even the ones that do not directly consider it.


Albums For A Year: towards introspective data science

31 Dec 2017

pirandello Luigi Pirandello was an outstanding author, whose writings shine in the history of European literature. Apart from a considerable number of plays and novels, he wrote also many novelle, (not too) short stories gathered in the collection Novelle per un anno (“Novelle for a year”).