Cicero denouncing Catiline at a meeting of the Roman Senate. A famous instance of the successful use of reason.

I recently read . As the title suggests, the book attempts to make sense of what seems to be a paradox of human’s ability to reason. The book seeks to explain how a mechanism which has been assumed to have evolved in humans in order to improve their problem solving ability seems to be so poor at that job. While reading the book, it struck me that some of the ideas presented within were relevant for not just cognitive science, but also for the field of artificial intelligence (AI). …


A Proposal

The field of Deep Reinforcement Learning (DeepRL) has made significant gains in the past half-decade. To go from sometimes solving simple Atari games to achieving superhuman performance in complex modern games like Starcraft 2 and DOTA in such a short time is quite an accomplishment. Indeed, given such successes one might think that RL has solved most interesting problems in games. There are however domains where there is still much progress to be made. One such area is in tasks which change over time, and require online adaptation on the part of the agent. The field of Meta-RL has sought…


When I shared my thoughts on the book Sapiens by Yuval Noah Harari, I expressed the hope that the book would present a sweeping and all-encompassing theory of human culture. Despite being compelling in its own right, it largely failed to live up to such an expectation. On the other hand, written in 1978 by Rene Girard (hereafter referred to as Things Hidden), more than delivers on the same promise. In this work, Girard, a philosopher and historian, presents a remarkably simple hypothesis which he claims can account for large swaths of…


I recently read a new pre-print titled , by Adam Santoro, Andrew Lampinen, and collaborators at DeepMind, and was so taken by it that I thought I would write up some brief thoughts. Unlike much of the research published by DeepMind, which is largely empirical in nature, this paper presents a philosophical perspective on the classic symbolic vs connectionist debate. Rather than making the now-typical proposal to combine symbolic reasoning and neural network approaches into a hybrid system where deep learning is used for “low level” processing, and a classical AI system is used at the…


Sapiens by Yuval Noah Harari was released in English in 2014, but it was not until last month that I finally read it. As such, the thoughts presented here are likely not new. Though, I hope they are at least of interest.

Before reading Sapiens, my only experience with Harari was watching about fifteen minutes of an interview he gave at Google for his more recent book Homo Deus, which was published in English in 2016. I don’t perfectly remember the conversation but it was far enough into gene editing and cognitive enhancement that I felt I had heard enough…


A few years ago wrote a on the basics of Reinforcement Learning (RL). In those, I walked through a number of the fundamental algorithms and ideas of RL, providing code along the way. This is a kind of continuation of that series, with the difference being that this article (and possibly future ones) will dive into some of the more eclectic and experimental elements of the RL world. I want to begin with a topic close to my heart and recent research interests, something called the successor representation (SR). …


As those who follow this blog are probably aware, I spend thinking about Reinforcement Learning (RL). These thoughts naturally extend into my everyday life, and the ways in which the formalisms provided by RL can be applied to the world beyond artificial agents. For those unfamiliar, Reinforcement Learning is an area of Machine Learning dedicated to optimizing behaviors in the context of external rewards. If you’d like an introduction, I wrote a on the topic a couple years ago. …


Looking beyond the hype of recent Deep RL successes

In recent weeks DeepMind and OpenAI have each shared that they developed agents which can learn to complete the first level of the Atari 2600 game Montezuma’s Revenge. These claims are important because Montezuma’s Revenge is important. Unlike the vast majority of the games in the (ALE), which are now easily solved at superhuman level by learned agents, Montezuma’s Revenge has been hitherto unsolved by Deep Reinforcement Learning methods and was thought by some to be unsolvable for years to come.


What goes into a stable, accurate reinforcement signal?

(This post assumes some familiarity with machine learning, and reinforcement learning in particular. If you are new to RL, I’d recommend checking out a I wrote in 2016 on the topic as a primer)

Introduction

Since the of the ML-Agents platform a few months ago, I have been surprised and delighted to find that thanks to it and other tools like OpenAI Gym, a new, wider audience of individuals are building Reinforcement Learning (RL) environments, and using them to train state-of-the-art models. The ability to work with these algorithms, previously something reserved for ML PhDs, is…


A bear learning to hunt for fish with it’s parent.

A few weeks ago OpenAI made a splash in the Deep Learning community with the release of their paper “.” The work contains impressive results suggesting that looking elsewhere than Reinforcement Learning (RL) methods may be worthwhile when training complex neural networks. It sparked a debate around the importance of Reinforcement Learning, and perhaps it’s less than necessary status as the go-to technique for learning to solve tasks. What I want to argue here is that instead of being seen as two competing strategies, one of which being necessarily better than the…

Arthur Juliani

Research Scientist. Interested in Artificial Intelligence, Neuroscience, Philosophy, and Literature.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store