Installation Tutorial for Cloud

QuantRocket can easily be deployed to the cloud. Cloud deployments have several advantages over local deployments, especially for live trading:

  • your cloud provider's internet connectivity is typically much more reliable than your home internet
  • run Docker on Linux, its native platform
  • access your cloud deployment securely from any computer
  • a dedicated environment for your algos, no CPU/memory resource contention between your algos and your personal computer use

This tutorial will show you how to deploy QuantRocket to your preferred cloud provider.

Note for users running private clouds. This tutorial installs QuantRocket in a public cloud and secures the deployment with SSL and HTTP Basic Auth. Alternatively, to install QuantRocket in a private cloud without using SSL and Basic Auth, (1) launch your cloud instance and SSH onto it; (2) follow the Linux tutorial to install QuantRocket on the cloud instance; and (3) instead of going to http://localhost:1969 to access JupyterLab, go to http://<private_ip_of_instance>:1969.

Create Cloud Server

To deploy QuantRocket to the cloud, you need a cloud server running Docker and accessible by SSH. The exact steps vary by cloud provider, and tutorials are readily available on the web, so a step-by-step tutorial is not provided here. However, the basic steps are as follows:

  1. Check the system requirements and determine the appropriate memory, CPU, and storage to give to your cloud server.
  2. Create a cloud server using your preferred cloud provider (Amazon Web Services, DigitalOcean, Google Cloud, Microsoft Azure, etc.)
  3. Set up SSH access to the cloud server using public key authentication (not password authentication).
  4. Install Docker on the cloud server. To do so, check out the Linux tutorial or find a tutorial specific to your cloud provider.

To verify that you have successfully completed these steps, login to the server using SSH. If public key authentication is set up correctly, you will not need to enter a password:

$ ssh your_username@your_cloud_server_ip

Once you are on the cloud server, verify that Docker is installed and running by entering a docker command, such as:

$ docker info
Important: Be sure to open ports 80 and 443 on your cloud server. Depending on your cloud provider, these ports may or may not be open by default.

Setup Domain

Cloud deployments must be accessed via a domain you provide (for example quantrocket.abccapital.com); they cannot be accessed via the IP address of the cloud server. During the deployment process, QuantRocket will obtain an SSL certificate from Let's Encrypt so that you can securely access your cloud deployment over HTTPS. You must provide your own custom domain to use for the SSL certificate.

Log in to your DNS provider and create an A record that points your custom domain to the IP of your cloud server. For example, if the IP address is 1.2.3.4 and your domain is abccapital.com, you could create an A record that points quantrocket.abccapital.com to IP address 1.2.3.4.

In a subsequent step, you will provide your custom domain (quantrocket.abccapital.com) as the environment variable HOUSTON_DOMAIN. QuantRocket will use this to obtain a SSL certificate from Let's Encrypt for your custom domain.

Install Docker Locally

Next, you must install Docker on your local computer. This will allow you to type Docker commands on your local computer but have those commands run on your cloud server.

Follow the instructions below to download and install Docker for your operating system.

Install Docker Desktop

Visit Docker's website and follow the instructions to download and install Docker Desktop on your computer.

During the installation, follow the Docker prompts to enable the WSL 2 backend. You may also be prompted to install the WSL 2 Linux kernel from Microsoft. See the Docker instructions for more detail.

After the installation, start Docker Desktop if it isn't already started (Start > Docker).

To verify the installation, open PowerShell (click Start and type "powershell") and run:

$ docker run hello-world

If you run into any problems during installation, please refer to Docker's installation guide for Windows. QuantRocket simply requires a standard installation of Docker Desktop for Windows.

Do not select any options in Docker Desktop to use Windows containers. Docker Desktop for Windows can run Windows containers or Linux containers but not both at the same time. QuantRocket uses Linux containers.
QuantRocket only supports Macs with Intel chips, not Macs with the newer M1 chip. We will support Macs with M1 chips as soon as possible but are currently waiting for necessary updates to occur in Docker Hub and in certain underlying libraries that QuantRocket depends on.

Install Docker Desktop

Visit Docker's website and follow the instructions to download and install Docker Desktop on your computer.

After the installation, start Docker Desktop if it isn't already started.

Install Docker Engine

Visit Docker's website and follow the installation instructions for your distribution:

When finished, run Docker's hello-world container to verify the installation:

$ sudo docker run hello-world

By default, you have to use sudo to run Docker. Forgetting to do so can result in confusing error messages, so follow the steps to allow yourself to manage Docker as a non-root user. Verify that it worked by running (without sudo):

$ docker run hello-world

You may also want to configure Docker to start on boot.

Install Docker Compose

Next, follow the instructions for installing Docker Compose on Linux .

When finished, verify your installation:

$ docker-compose --version

Download Docker Compose file

To install QuantRocket, download a Docker Compose file which tells Docker how to create the QuantRocket stack. A Compose file is a YAML file that defines a multi-container Docker application.

Open a command line prompt (PowerShell on Windows, Terminal on Mac/Linux).

Create a folder for QuantRocket under your home directory:

$ cd ~
$ mkdir quantrocket
$ cd quantrocket

Copy and paste the following command to download the latest Compose file and save to your computer:

$ curl 'https://www.quantrocket.com/composefiles/latest/cloud/docker-compose.yml' -o docker-compose.yml

(You can also download the Compose file from the downloads page.)

Set cloud environment variables

To deploy QuantRocket to the cloud, you must set several environment variables which define the domain and authentication credentials to use for your cloud deployment.

In your Compose file, you'll see a section like the following:

houston:
  image: 'quantrocket/houston:1.4.0'
  environment:
    BASIC_AUTH_USER: '${HOUSTON_USERNAME}'
    BASIC_AUTH_PASSWD: '${HOUSTON_PASSWORD}'
    LETSENCRYPT_DOMAIN: '${HOUSTON_DOMAIN}'

This tells Docker to look for environment variables called HOUSTON_USERNAME, HOUSTON_PASSWORD and HOUSTON_DOMAIN on your computer and pass them to the houston container, QuantRocket's API gateway.

These environment variables serve the following purpose:

  • ${HOUSTON_USERNAME}/${HOUSTON_PASSWORD}: username and password which you'll use to connect remotely to your deployment (choose a strong password)
  • ${HOUSTON_DOMAIN}: the custom domain from an earlier step (for example quantrocket.abccapital.com)
Make sure you choose a username and password for ${HOUSTON_USERNAME} and ${HOUSTON_PASSWORD} when running QuantRocket in the cloud. Otherwise, your QuantRocket services will be publicly accessible to anyone with the URL!

To set the environment variables, follow the instructions appropriate to the operating system of your local computer (not the OS of the cloud server).

In PowerShell, set the environment variables using the following syntax:

$ [Environment]::SetEnvironmentVariable("HOUSTON_USERNAME", "whateveryouwant", "User")
$ [Environment]::SetEnvironmentVariable("HOUSTON_PASSWORD", "pickastrongpassword", "User")
$ [Environment]::SetEnvironmentVariable("HOUSTON_DOMAIN", "quantrocket.abccapital.com", "User")
The environment variables won't take effect until you restart PowerShell, so close the window now and open a new one.

On Mac, set your environment variables in .profile, which gets executed each time you open Terminal.

$ touch ~/.profile
$ echo 'export HOUSTON_USERNAME=whateveryouwant' >> ~/.profile
$ echo 'export HOUSTON_PASSWORD=pickastrongpassword' >> ~/.profile
$ echo 'export HOUSTON_DOMAIN=quantrocket.abccapital.com' >> ~/.profile

Source the file to make the environment variables available in the current terminal session (not required for future terminal sessions):

$ source ~/.profile

On Linux, set your environment variables in .bashrc, which gets executed each time you open a terminal.

$ touch ~/.bashrc
$ echo 'export HOUSTON_USERNAME=whateveryouwant' >> ~/.profile
$ echo 'export HOUSTON_PASSWORD=pickastrongpassword' >> ~/.profile
$ echo 'export HOUSTON_DOMAIN=quantrocket.abccapital.com' >> ~/.profile

Source the file to make the environment variables available in the current terminal session (not required for future terminal sessions):

$ source ~/.bashrc

Setup Docker Context

By default, any docker or docker-compose commands you type on your local computer will run locally. However, you can send those commands to your cloud server instead by defining a Docker Context that points to your cloud server. Supply a name for the context (cloud in this example) and specify the SSH connection parameters using the following syntax:

$ docker context create cloud --docker "host=ssh://your_username@your_domain_or_ip"

There are two ways to activate the cloud context you created. The first option is to explicitly specify the --context parameter each time you run a docker or docker-compose command. For example:

$ docker-compose --context cloud up -d

If you are running multiple QuantRocket deployments (such as one locally and one in the cloud), this explicit approach may be best to avoid confusion about which deployment you are targeting.

Alternatively, you set the context persistently using docker context use:

$ docker context use cloud

Any subsequent docker or docker-compose commands will run against your cloud server, without the need for the --context parameter. If needed, you can switch back to the local deployment by activating the specially named 'default' (that is, local) context:

$ docker context use default

Deploy QuantRocket

Newer versions of Docker Desktop ship with Docker Compose V2, but QuantRocket utilizes Docker Compose V1, so you should disable V2:

$ docker-compose disable-v2

Now you can use docker-compose to deploy QuantRocket to the cloud server:

$ cd ~/quantrocket
$ docker-compose --context cloud -p quantrocket up -d

Docker Compose will read your docker-compose.yml, pull the images down from Docker Hub onto your cloud server, and create and run containers from the images. This process takes several minutes. You will see output like this:

Output
Creating network "quantrocket_default" with the default driver
Creating volume "quantrocket_codeload" with default driver
Creating volume "quantrocket_db" with default driver
Creating volume "quantrocket_flightlog" with default driver
Creating volume "quantrocket_settings" with default driver
Pulling codeload (quantrocket/codeload:2.5.0)...
2.5.0: Pulling from quantrocket/codeload
852e50cd189d: Pull complete
a6236801494d: Downloading [======================================>            ]  62.63MB/80.38MB
679c171d6942: Download complete
8fe74afaa3e3: Download complete
d9ada034b8a6: Downloading [=============>                                     ]   4.94MB/18.95MB

You can list all the containers that are running on your deployment:

$ docker --context cloud ps
Output
CONTAINER ID   IMAGE                         COMMAND                  CREATED             STATUS             PORTS                                            NAMES
d26da467ac03   bc77b14f9c3d                  "uwsgi-quantrocket -…"   About an hour ago   Up About an hour   80/tcp                                           quantrocket_master_1
b1569b403b38   f2226cb5f6c3                  "uwsgi-quantrocket -…"   About an hour ago   Up About an hour   80/tcp                                           quantrocket_fundamental_1
fcf1c7b0fb36   1ca2dbd9bfd2                  "uwsgi-quantrocket -…"   About an hour ago   Up About an hour   80/tcp                                           quantrocket_realtime_1
952089561d20   fcade8902208                  "uwsgi-quantrocket -…"   About an hour ago   Up About an hour   80/tcp                                           quantrocket_history_1
...

You should now be able to access the Jupyter environment in your browser at your custom domain from earlier. For example, if your custom domain is quantrocket.abccapital.com, your deployment URL is https://quantrocket.abccapital.com. You may want to bookmark this URL.

Your browser will prompt you to enter the username and password you set earlier (and will store the credentials so you don't have to re-enter them each time).

If your browser times out trying to connect, check the houston logs for error messages by running: docker-compose --context cloud logs houston
You must type 'https' in the URL. Typing 'http' will not automatically redirect to 'https'.
Let's Encrypt imposes rate limits of 5 duplicate certificates per week, so don't redeploy QuantRocket more frequently than that to the same cloud server (i.e. same IP) or the certificate installation will fail, in which case a certificate error will prevent you from connecting to your deployment and you'll have to wait for the rate limit to reset or redeploy using a different domain name.
In your Docker Compose file, you might notice that port 80 (HTTP) is also exposed in addition to port 443 (HTTPS). Port 80 is only used by Let's Encrypt for SSL certificate validation and can't be used to connect to your QuantRocket deployment.

In JupyterLab you'll see the JupyterLab dashboard. Click the Quickstart button to find an interactive tour and other resources to help you get started with QuantRocket:

JupyterLab home