Controlling the physical world is an integral part and arguably a prerequisite of general intelligence. And, the only known example of general-purpose intelligence had emerged in primates which had been successful in manipulating the world for millions of years in history. Physical control tasks as we know share many common properties and it is sensible enough to consider them as a distinct class of behavioural problems.

Unlike board games, language and other symbolic domains, physical tasks are fundamentally continuous in a state, time as well as action. Their dynamics are subject to second-order equations of motion, that implies that the underlying state is composed of position-like and velocity-like variables, while state derivatives are acceleration-like. Sensory signals i.e. observations usually carry meaningful physical units and vary over corresponding timescales.

This decade has seen rapid progress when it comes to the application of Reinforcement Learning (RL) techniques to difficult problem domains such as video games. The Arcade Learning Environment resulted to be a vital facilitator of these developments, therefore, providing a set of standard benchmarks for the evaluation and comparison of learning algorithms.

And The DeepMind Control Suite results in the provision of a similar set of standard benchmarks for continuous control problems.

The DeepMind Control Suite is a set of continuous control tasks with a standardized structure and interpretable rewards, that is intended to serve as performance benchmarks for reinforcement learning agents.

Architecture:

As we already know that The Control Suite is a set of stable, well-tested continuous control tasks that are easy to use and modify, therefore, tasks are written in Python and physical models are defined using MJCF. Standardized action, observation and reward structures make benchmarking simple and learning curves easy to interpret.

Since the tasks are powered by the MuJoCo physics engine, that in return makes them not only easy to use but also to modify. They also include benchmarks for several learning algorithms.

Future Work:

The DeepMind Control Suite proves to be a starting place when it comes to the design and performance comparison of reinforcement learning algorithms for physics-based control. It offers a wide range of tasks, that range from near-trivial to quite difficult. The uniform reward structure allows for robust suite-wide performance measures.

The Control Suite is now being shared with the wider community and it is hoped that it will be found useful. It is also looked forward to the diverse research that the Suite may enable, and to integrating community contributions in all its future releases.

Since, in this release, Several elements are missing from the Control Suite.

There are several features that were hoped to be included but unfortunately did not make it into this release; it is, therefore, intend to add these in the near future.

A few of them include:

• A quadrupedal locomotion task,

• An interactive visualiser with which to view and perturb the simulation,

• Support for C callbacks and multi-threaded dynamics,

• A MuJoCo TensorFlow op wrapper and Windows™ support.

For further insight into the same links are provided below:

For More Information: GitHub

Source and Link to the PDF: Click Here

DeepMind Control Suite Illustration Video:

Video Source: Yuval Tassa