Baselines requires python3 (>=3.5) with the development headers. You’ll also need system packages CMake, OpenMPI and zlib. Those can be installed as follows


sudo apt-get update && sudo apt-get install cmake libopenmpi-dev python3-dev zlib1g-dev

Mac OS X

Installation of system packages on Mac requires Homebrew. With Homebrew installed, run the follwing:

brew install cmake openmpi

Windows 10

We recommend using Anaconda for windows users.

  1. Create a new environment in the Anaconda Navigator (at least python 3.5) and install zlib in this environment.
  2. Install MPI for Windows (you need to download and install msmpisetup.exe)
  3. Clone Stable-Baselines Github repo and replace the line gym[atari,classic_control]>=0.10.9 in by this one: gym[classic_control]>=0.10.9
  4. Install Stable-Baselines from source, inside the folder, run pip install -e .

4. [Optional] If you want to use atari environments, you need to install this package: (using again pip install -e .)

Stable Release

pip install stable-baselines

Bleeding-edge version

With support for running tests and building the documentation.

git clone && cd stable-baselines
pip install -e .[docs,tests]

Using Docker Images

If you are looking for docker images with stable-baselines already installed in it, we recommend using images from RL Baselines Zoo.

Otherwise, the following images contained all the dependencies for stable-baselines but not the stable-baselines package itself. They are made for development.

Use Built Images

GPU image (requires nvidia-docker):

docker pull araffin/stable-baselines

CPU only:

docker pull araffin/stable-baselines-cpu

Build the Docker Images

Build GPU image (with nvidia-docker):

docker build . -f docker/Dockerfile.gpu -t stable-baselines

Build CPU image:

docker build . -f docker/Dockerfile.cpu -t stable-baselines-cpu

Note: if you are using a proxy, you need to pass extra params during build and do some tweaks:

--network=host --build-arg HTTP_PROXY= --build-arg http_proxy= --build-arg HTTPS_PROXY= --build-arg https_proxy=

Run the images (CPU/GPU)

Run the nvidia-docker GPU image

docker run -it --runtime=nvidia --rm --network host --ipc=host --name test --mount src="$(pwd)",target=/root/code/stable-baselines,type=bind araffin/stable-baselines bash -c 'cd /root/code/stable-baselines/ && pytest tests/'

Or, with the shell file:

./ pytest tests/

Run the docker CPU image

docker run -it --rm --network host --ipc=host --name test --mount src="$(pwd)",target=/root/code/stable-baselines,type=bind araffin/stable-baselines-cpu bash -c 'cd /root/code/stable-baselines/ && pytest tests/'

Or, with the shell file:

./ pytest tests/

Explanation of the docker command:

  • docker run -it create an instance of an image (=container), and run it interactively (so ctrl+c will work)
  • --rm option means to remove the container once it exits/stops (otherwise, you will have to use docker rm)
  • --network host don’t use network isolation, this allow to use tensorboard/visdom on host machine
  • --ipc=host Use the host system’s IPC namespace. IPC (POSIX/SysV IPC) namespace provides separation of named shared memory segments, semaphores and message queues.
  • --name test give explicitely the name test to the container, otherwise it will be assigned a random name
  • --mount src=... give access of the local directory (pwd command) to the container (it will be map to /root/code/stable-baselines), so all the logs created in the container in this folder will be kept
  • bash -c '...' Run command inside the docker image, here run the tests (pytest tests/)