Curiosity Driven Deep Reinforcement Learning

Deep Reinforcement Learning: Mastering Sparse Rewards & Curiosity-Driven Exploration

This advanced deep reinforcement learning course dives into cutting-edge AI research, empowering experienced coders to implement state-of-the-art algorithms from scratch using PyTorch. Ideal for those already familiar with actor-critic agents, this fast-paced course focuses on tackling the challenge of learning in environments with sparse rewards – a critical hurdle for achieving artificial general intelligence (AGI).

Key Topics & Skills:

• Curiosity-Driven Reinforcement Learning: Discover how to imbue agents with an intrinsic sense of curiosity, enabling them to explore effectively and learn valuable policies even when external rewards are scarce or absent. Learn to implement the Intrinsic Curiosity Module (ICM), a versatile component applicable to various RL algorithms.
• Asynchronous Advantage Actor-Critic (A3C): Implement the A3C algorithm from the ground up, exploring asynchronous methods for deep reinforcement learning. Learn to leverage multithreading in Python and parallel processing to significantly speed up training. We'll also delve into generalized advantage estimation for improved A3C performance. A unique aspect of this course is learning to train A3C agents without a GPU.
• Sparse Reward Environments: Master techniques for navigating environments where rewards are infrequent, such as maze-like scenarios where rewards are only received upon reaching the goal. See a clear performance boost using curiosity-driven methods compared to standard A3C.
• PyTorch Implementation: Gain hands-on experience coding these advanced algorithms using the popular PyTorch framework. A TensorFlow 2 version is also in development.
• Modular & Extensible Code: Develop skills in writing clean, extensible, and modular code, making it easier to adapt these techniques to other reinforcement learning algorithms like Proximal Policy Optimization (PPO), Soft Actor-Critic (SAC), and Deep Q-Networks (DQN).

Hands-on Projects:

• A3C Implementation: Code the A3C algorithm from scratch, including multithreading and generalized advantage estimation.
• ICM Implementation: Build and integrate the Intrinsic Curiosity Module into a deep reinforcement learning framework.
• OpenAI Gym Atari: Train and test your agents in the Pong environment from the OpenAI Gym Atari library, achieving near world-class performance.
• Sparse Reward Maze: Train an agent to navigate a maze with sparse rewards, demonstrating the power of curiosity-driven learning.

Prerequisites:

This is a fast-paced course designed for motivated and advanced students with prior experience coding actor-critic agents. A brief review of fundamental reinforcement learning concepts will be provided.

You will learn to:

• Implement complex deep reinforcement learning papers.
• Utilize multi-core CPUs for efficient parallel processing.
• Write extensible modular code.
• Modify the OpenAI Gym Atari Library.