Deep learning has become the dominant paradigm for solving a myriad of tasks ranging from natural language processing to computer vision. However, the adoption of deep neural networks as agents in the digital and physical worlds has been hampered by their lack of data efficiency. In this thesis, we propose methods to build and leverage world models for sample-efficient decision making.

Firstly, we propose to initialize agents for text-based environments with pretrained language models. We introduce a fine-tuning procedure that blends imitation learning and reinforce- ment learning into a common objective. The resulting models drastically outperform previous approaches while being significantly more sample efficient.

Secondly, we develop a novel world model architecture composed of a discrete autoencoder that builds a vocabulary of image tokens and an autoregressive transformer that composes the vocabulary over time. We train agents to solve environments by learning from synthetic trajectories imagined in the world model. Our agents achieve superhuman performance with only a few hours of real data.

Finally, we improve the world model architecture to render it scalable to more complex domains. We encode differentials between frames instead of the frames themselves, thus reducing the sequence length of the model, and inject absolute state representations in its con- ditioning to simplify the world modeling task. We demonstrate that our approach surpasses existing methods when scaling the number of interactions in a challenging environment.