Curriculum#

Syllabus’s Curriculum API is a unified interface for curriculum learning methods. Curricula following this API can be used with all of Syllabus’s infrastructure. We hope that future curriculum learning research will provide implementations following this API to encourage reproducibility and ease of use.

The full documentation for the curriculum class can be found Global Synchronization

The Curriculum class has three main jobs:

  • Maintain a sampling distribution over the task space.

  • Incorporate feedback from the environments or training process to update the sampling distribution.

  • Provide a sampling interface for the environment to draw tasks from.

In reality, the sampling distribution can be whatever you want, such as a uniform distribution, a deterministic sequence of tasks, or a single constant task depending on the curriculum learning method.

To incorporate feedback from the environment, the API provides multiple methods:

syllabus.core.curriculum_base module#

class syllabus.core.curriculum_base.Curriculum(task_space: TaskSpace, random_start_tasks: int = 0, task_names: Callable | None = None)#

Bases: object

Base class and API for defining curricula to interface with Gym environments.

add_task(task: Any) → None#
log_metrics(writer, step=None, log_full_dist=False)#

Log the task distribution to the provided tensorboard writer.

Parameters:

writer – Tensorboard summary writer.

property num_tasks: int#

Counts the number of tasks in the task space.

Returns:

Returns the number of tasks in the task space if it is countable, TODO: -1 otherwise

sample(k: int = 1) → List | Any#

Sample k tasks from the curriculum.

Parameters:

k – Number of tasks to sample, defaults to 1

Returns:

Either returns a single task if k=1, or a list of k tasks

property tasks: List[tuple]#

List all of the tasks in the task space.

Returns:

List of tasks if task space is enumerable, TODO: empty list otherwise?

update(update_data: Dict[str, tuple])#

Update the curriculum with the specified update type. TODO: Change method header to not use dictionary, use enums?

Parameters:

update_data (Dictionary with "update_type" key which maps to one of ["step", "step_batch", "episode", "on_demand", "task_progress", "add_task", "noop"] and "args" with a tuple of the appropriate arguments for the given "update_type".) – Dictionary

Raises:

NotImplementedError –

update_batch(update_data: List[Dict])#

Update the curriculum with batch of updates.

Parameters:

update_data – List of updates or potentially varying types

update_on_demand(metrics: Dict)#

Update the curriculum with arbitrary inputs.

Parameters:

metrics – Arbitrary dictionary of information. Can be used to provide gradient/error based updates from the training process.

Raises:

NotImplementedError –

update_on_episode(episode_return: float, trajectory: List | None = None) → None#

Update the curriculum with episode results from the environment.

Parameters:

  • episode_return – Episodic return

  • trajectory – trajectory of (s, a, r, s, …), defaults to None

Raises:

NotImplementedError –

update_on_step(obs: Any, rew: float, term: bool, trunc: bool, info: dict) → None#

Update the curriculum with the current step results from the environment.

Parameters:

  • obs – Observation from teh environment

  • rew – Reward from the environment

  • term – True if the episode ended on this step, False otherwise

  • trunc – True if the episode was truncated on this step, False otherwise

  • info – Extra information from the environment

Raises:

NotImplementedError –

update_on_step_batch(step_results: List[Tuple[int, int, int, int]]) → None#

Update the curriculum with a batch of step results from the environment.

This method can be overridden to provide a more efficient implementation. It is used as a convenience function and to optimize the multiprocessing message passing throughput.

Parameters:

step_results – List of step results

update_task_progress(task: Any, progress: Tuple[float, bool]) → None#

Update the curriculum with a task and its progress.

Parameters:

  • task – Task for which progress is being updated.

  • progress – Progress toward completion or success rate of the given task. 1.0 or True typically indicates a complete task.