QuantRocket logo


Historical Data Collection

For backtesting we will collect 1-minute bid-ask bars for all CL futures.

First, start IB Gateway:

Collect CL futures chain

Collect contract details for all available CL futures:

Monitor flightlog for a completion message:

quantrocket.master: INFO Collecting NYMEX FUT listings from IBKR website (CL only)
quantrocket.master: INFO Requesting details for 1 NYMEX listings found on IBKR website
quantrocket.master: INFO Saved 140 NYMEX listings to securities master database

Define universe of CL futures

Next we define a universe of CL futures for easy reference. To do so, download a CSV of CL futures from the securities master database:

Then upload the CSV to create the "cl-fut" universe:

Define rollover rules

For the purpose of defining calendar spreads, we must define rollover rules to specify which contract should be considered the front month and the various back months. Example rules are defined in quantrocket.master.rollover.yml, where we specify to rollover 10 business days before expiration. See the usage guide for more rollover rule options.

The master service looks for this file in the codeload directory, so move it there to install it:

Collect historical data

Next we collect 1-min historical data with the following parameters:

Then we collect the data. Be prepared for intraday data collection to take some time (perhaps a day or so depending on several variables).

Monitor flightlog for completion:

quantrocket.history: INFO [cl-1min-bbo] Collecting history from IBKR for 144 securities in cl-1min-bbo
quantrocket.history: INFO [cl-1min-bbo] Saved 22664 total records for 50 total securities to quantrocket.v2.history.cl-1min-bbo.sqlite

Next Up

Part 2: Calendar Spread Research