Installation Tutorial for DigitalOcean

Introduction

QuantRocket can easily be deployed to the cloud. Cloud deployments have several advantages over local deployments, especially for live trading:

  • your cloud provider's internet connectivity is typically much more reliable than your home internet
  • run Docker on Linux, its native platform
  • access your cloud deployment securely from any computer
  • a dedicated environment for your algos, no CPU/memory resource contention between your algos and your personal computer use

DigitalOcean is an excellent choice for running QuantRocket in the cloud. Compared to Amazon (AWS), DigitalOcean offers better pricing, faster hardware, and simpler server configuration and management.

This tutorial will show you how to use Docker to provision a DigitalOcean Droplet (i.e. cloud server) and deploy QuantRocket to it.

If you're new to Docker, it may be helpful to read the overview of QuantRocket's architecture before starting.

Install Docker

To run QuantRocket in the cloud, first you need to install Docker on your local computer. You'll use your local installation of Docker to create and control your cloud server.

Follow the instructions below to download and install Docker (Docker Engine, Docker Compose, and Docker Machine) for your operating system.

Install Docker on Windows 10 Professional

Visit Docker's website and follow the instructions to download and install Docker Desktop , which includes Docker Engine, Docker Compose, and Docker Machine.

If prompted during installation, do not check the box to use Windows containers. Docker Desktop for Windows can run Windows containers or Linux containers but not both at the same time. QuantRocket uses Linux containers.

Docker container mode

After the installation, start Docker Desktop if it isn't already started (Start > Docker).

Install Docker on Mac

Visit Docker's website and follow the instructions to download and install Docker Desktop , which includes Docker Engine, Docker Compose, and Docker Machine.

Install Docker on Linux

Visit Docker's website and follow the installation instructions for your distribution:

When finished, run Docker's hello-world container to verify the installation:

$ sudo docker run hello-world

By default, you have to use sudo to run Docker. Forgetting to do so can result in confusing error messages, so follow the steps to allow yourself to manage Docker as a non-root user. Verify that it worked by running (without sudo):

$ docker run hello-world

You may also want to configure Docker to start on boot.

Next, follow the instructions for installing Docker Compose on Linux.

When finished, verify your installation:

$ docker-compose --version

Then, install Docker Machine:

$ base=https://github.com/docker/machine/releases/download/v0.14.0 &&
  curl -L $base/docker-machine-$(uname -s)-$(uname -m) >/tmp/docker-machine &&
  sudo install /tmp/docker-machine /usr/local/bin/docker-machine

Verify the installation:

$ docker-machine version

Create DigitalOcean Droplet

Now that Docker is installed on your local computer, you can use it to create a DigitalOcean Droplet (cloud server). In a later step you'll deploy QuantRocket to this Droplet.

Follow the tutorial on Docker's website to sign up for a DigitalOcean account and generate an access token.

Use Docker Machine to create a DigitalOcean Droplet:

$ docker-machine create --driver digitalocean --digitalocean-access-token=<your-access-token> --digitalocean-region nyc3 --digitalocean-size s-6vcpu-16gb --digitalocean-backups --digitalocean-monitoring quantrocket

Note the following options in the above example:

  • --digitalocean-access-token: replace with your access token
  • --digitalocean-region: indicate the region where the server should be created. Choose a region near where you live and/or near an exchange you care about. (Don't worry too much about the region location since ultra-low latency isn't essential to the kinds of trading strategies you can run with QuantRocket.)
  • --digitalocean-size: indicate the size of the Droplet. The minimum size for running QuantRocket is 4gb, but 8gb or 16gb is better. The larger the Droplet, the larger the data analyses you can run. Check out the DigitalOcean API docs for available sizes.
  • --digitalocean-backups: optional but recommended (extra cost)
  • --digitalocean-monitoring: optional but recommended
  • Name the Droplet "quantrocket". The name will be used with Docker Machine and will also appear in the DigitalOcean user interface.

See Docker's website for the full list of available DigitalOcean options.

After you create the Droplet, you'll see it in the DigitalOcean user interface when you log in to your DigitalOcean account.

Setup Custom Domain

During the deployment process, QuantRocket will obtain an SSL certificate from Let's Encrypt so that you can securely access your cloud deployment over HTTPS. You must provide your own custom domain to use for the SSL certificate.

If you're on the Supersonic plan or higher and don't have your own domain, we can provide one for you. Open a support ticket and provide the IP address obtained below.

Use Docker Machine (or the cloud provider interface) to check the IP of your cloud server:

$ docker-machine ip quantrocket

Log in to your DNS provider and create an A record that points your custom domain to the IP of your cloud server. For example, if the IP address is 1.2.3.4 and your domain is abccapital.com, you could create an A record that points quantrocket.abccapital.com to IP address 1.2.3.4.

In a subsequent step, you will provide your custom domain (quantrocket.abccapital.com) as the environment variable HOUSTON_DOMAIN. QuantRocket will use this to obtain a SSL certificate from Let's Encrypt for your custom domain.

Download Docker Compose file

To install QuantRocket, download a Docker Compose file which tells Docker how to create the QuantRocket stack. A Compose file is a YAML file that defines a multi-container Docker application.

Open a command line prompt (PowerShell on Windows, Terminal on Mac/Linux).

Create a folder for QuantRocket under your home directory:

$ cd ~
$ mkdir quantrocket
$ cd quantrocket

Copy and paste the following command to download the latest Compose file and save to your computer:

$ curl 'https://www.quantrocket.com/composefiles/latest/cloud/docker-compose.yml' -o docker-compose.yml

(You can also download the Compose file from the downloads page.)

Set cloud environment variables

To deploy QuantRocket to the cloud, you must set several environment variables which define the domain and authentication credentials to use for your cloud deployment.

In your Compose file, you'll see a section like the following:

houston:
  image: 'quantrocket/houston:1.4.0'
  environment:
    BASIC_AUTH_USER: '${HOUSTON_USERNAME}'
    BASIC_AUTH_PASSWD: '${HOUSTON_PASSWORD}'
    LETSENCRYPT_DOMAIN: '${HOUSTON_DOMAIN}'

This tells Docker to look for environment variables called HOUSTON_USERNAME, HOUSTON_PASSWORD and HOUSTON_DOMAIN on your computer and pass them to the houston container, QuantRocket's API gateway.

These environment variables serve the following purpose:

  • ${HOUSTON_USERNAME}/${HOUSTON_PASSWORD}: username and password which you'll use to connect remotely to your deployment (choose a strong password)
  • ${HOUSTON_DOMAIN}: the custom domain from an earlier step (for example quantrocket.abccapital.com)
Make sure you choose a username and password for ${HOUSTON_USERNAME} and ${HOUSTON_PASSWORD} when running QuantRocket in the cloud. Otherwise, your QuantRocket services will be publicly accessible to anyone with the URL!

To set the environment variables, follow the instructions appropriate to the operating system of your local computer (not the OS of the cloud server).

Windows 10 Professional

In PowerShell, set the environment variables using the following syntax:

$ [Environment]::SetEnvironmentVariable("HOUSTON_USERNAME", "whateveryouwant", "User")
$ [Environment]::SetEnvironmentVariable("HOUSTON_PASSWORD", "pickastrongpassword", "User")
$ [Environment]::SetEnvironmentVariable("HOUSTON_DOMAIN", "quantrocket.abccapital.com", "User")
The environment variables won't take effect until you restart PowerShell, so the close the window now and open a new one.

Mac

On Mac, set your environment variables in .profile, which gets executed each time you open Terminal.

$ touch ~/.profile
$ echo 'export HOUSTON_USERNAME=whateveryouwant' >> ~/.profile
$ echo 'export HOUSTON_PASSWORD=pickastrongpassword' >> ~/.profile
$ echo 'export HOUSTON_DOMAIN=quantrocket.abccapital.com' >> ~/.profile

Source the file to make the environment variables available in the current terminal session (not required for future terminal sessions):

$ source ~/.profile

Linux

On Linux, set your environment variables in .bashrc, which gets executed each time you open a terminal.

$ touch ~/.bashrc
$ echo 'export HOUSTON_USERNAME=whateveryouwant' >> ~/.profile
$ echo 'export HOUSTON_PASSWORD=pickastrongpassword' >> ~/.profile
$ echo 'export HOUSTON_DOMAIN=quantrocket.abccapital.com' >> ~/.profile

Source the file to make the environment variables available in the current terminal session (not required for future terminal sessions):

$ source ~/.bashrc

Deploy QuantRocket

Use Docker Machine to activate the environment you created earlier. Running docker-machine env quantrocket will cause Docker to print the platform-specific command you should run to activate your environment:

$ docker-machine env quantrocket
$ # then type the command docker-machine displays, for example on Linux/Mac:
$ eval $(docker-machine env quantrocket)

After activation, any Docker commands you run will run on your Droplet rather than on your local computer. Now you can use docker-compose to deploy QuantRocket to the Droplet:

$ cd ~/quantrocket
$ docker-compose -p quantrocket up -d

Docker Compose will read your docker-compose.yml, pull the images down from Docker Hub onto your Droplet, and create and run containers from the images. This process takes several minutes. You will see output like this:

Output
Creating network "quantrocket_default" with the default driver
Pulling houston (quantrocket/houston:0.8.0)...
ad74af05f5a2: Pull complete
c1da3f388f16: Extracting [============================================>      ]  110.3MB/125.3MB
3f388f16: Downloading [===========================================>       ]    108MB/125.3MBte
4c2b7a4aefc0: Download complete
e61a44100a50: Download complete
8e960303c169: Waiting
f2a1d6599d9d: Waiting

You can list all the containers that are running on your Droplet:

$ docker ps
Output
CONTAINER ID        IMAGE                               COMMAND                  CREATED             STATUS              PORTS                           NAMES
476bd8607707        quantrocket/history:0.2.6           "/usr/bin/tini -- ..."   37 seconds ago      Up 33 seconds       80/tcp                          quantrocket_history_1
96e93387015c        quantrocket/master:0.2.11           "/usr/bin/tini -- ..."   37 seconds ago      Up 32 seconds       80/tcp                          quantrocket_master_1
fc4b87285158        quantrocket/jupyter:4.4.0.7         "/usr/bin/tini -- ..."   38 seconds ago      Up 33 seconds       80/tcp                          quantrocket_jupyter_1
...

You should now be able to access the Jupyter environment in your browser at your custom domain from earlier. For example, if your custom domain is quantrocket.abccapital.com, your deployment URL is https://quantrocket.abccapital.com. You may want to bookmark this URL.

Your browser will prompt you to enter the username and password you set earlier (and will store the credentials so you don't have to re-enter them each time).

Let's Encrypt imposes rate limits of 5 duplicate certificates per week, so don't redeploy QuantRocket more frequently than that to the same cloud server (i.e. same IP) or the certificate installation will fail, in which case a certificate error will prevent you from connecting to your deployment and you'll have to wait for the rate limit to reset or redeploy to a different cloud server.
In your Docker Compose file, you might notice that port 80 (HTTP) is also exposed in addition to port 443 (HTTPS). Port 80 is only used by Let's Encrypt for SSL certificate validation and can't be used to connect to your QuantRocket deployment.

In JupyterLab you'll see the JupyterLab dashboard. Open the Quickstart notebook (QuickStart.ipynb) to find an interactive tour and other resources to help you get started with QuantRocket:

JupyterLab home