Learning Progress

This is an implementation of the curriculum introduced in the paper Multi-task curriculum learning in a complex, visual, hard-exploration domain: Minecraft (Kanitscheider et al 2021). It has been used to achieve strong performance in minecraft without offline data. The method tracks the completion rate of individual binary tasks over time. It maintains two exponential moving averages, one fast and one slow, of this task progress over the course of training. By measuring the difference between these two moving averages, we can determine whether the agent is making recent progress on a task. If the difference is positive, the agent is learning to solve the task. If the difference is negative, the agent is forgetting how to solve a task. To improve performance in both cases, the curriculum samples tasks according to the magnitude of the learning progress. You can reference the paper for more details.

class syllabus.curricula.learning_progress.LearningProgressCurriculum(task_space: TaskSpace, **kwargs)

Bases: Curriculum

Provides an interface for tracking success rates of discrete tasks and sampling tasks based on their success rate using the method from https://arxiv.org/abs/2106.14876. TODO: Support task spaces aside from Discrete

REQUIRES_CENTRAL_UPDATES = False

REQUIRES_STEP_UPDATES = False

on_step(obs, rew, term, trunc, info) → None

Update the curriculum with the current step results from the environment.

update_task_progress(task: int, progress: float)

Update the success rate for the given task using a fast and slow exponential moving average.