# DATASET guide

> Every finished SPX session as files on your own machine, with every field's book, unit and start date named.

Source: https://firmtape.com/docs/dataset

The DATASET licence, monthly or yearly. Reading single sessions on the site stays free.

## Why it matters

The archive is free to read one session at a time, on the site and in the terminal. Research over the whole of it needs the sessions as files you can compute on, and a clear rule for which book every number comes from.

DATASET answers one question: **how do I run my own study over every session FirmTape has measured?** It delivers the same day files the terminal reads, one a day, with the per-visitor limit on distinct days removed for your account.

Every session carries three books. The **volume** book is the conventional public object, built here from the tape; the **open-interest** book uses the morning's positions; the **measured** book is signed print by print from the tape and its quotes. Their magnitudes are never comparable, so every field names its book.

## Where to find it

Start the licence on [pricing](https://firmtape.com/pricing). Once it is active, Account, Plans and billing shows how to download; every field is on the [data dictionary](https://firmtape.com/dataset/dictionary).

| You want | Go to |
| --- | --- |
| The licence | [Pricing](https://firmtape.com/pricing), DATASET |
| What a field means | The [data dictionary](https://firmtape.com/dataset/dictionary) |
| Which days exist | The day list the account page names |
| To show the numbers to other people | [Licensing](https://firmtape.com/licensing) |

## Read it in 30 seconds

1. A field's suffix names its book: `_volcp` or `_vol` the volume book, `_oi` the open-interest book; a neutral name is the measured book.
2. The dictionary's **since** column says from which session a field exists; earlier sessions do not carry it.
3. Levels are SPX index points and times are New York time.
4. A day file covers one finished session; a session is never rewritten after its evening.
5. The calendar file carries each scheduled release and what usually followed its kind.

## How to use it

**The files**

### The day files and the dictionary

**Why:** the same session files the terminal reads, as data for your own code. **Where:** with the licence active, Account, Plans and billing, **How to download**; every field on the [data dictionary](https://firmtape.com/dataset/dictionary). **Plan:** The DATASET licence.

1. Fetch the day list to see which finished sessions exist.
2. Fetch each day's file with your account's token as a bearer header, from a script.
3. Look up each field on the [data dictionary](https://firmtape.com/dataset/dictionary): its meaning, its unit, its book and the session it starts from.

![The data dictionary: every field of the day files and every LAB column, grouped by book, with meaning, unit and start date](https://archive.firmtape.com/press/features/data/dictionary.eb9d819f.webp)

> **What it measures:** A day file is one finished SPX session as the terminal reads it: minute series, per-strike ladders and the session's levels. The dictionary lists every field of the files and every LAB column, generated from the same source the LAB compiler reads, so the two never disagree.

How to read it:

- A field's **since** column is the first session that carries it; a study over all sessions should start each field there.
- Times are New York time; levels are SPX index points.

**Limits:** Finished sessions only. Your own research only: the files are not for showing to other people.

**Developer reference:** [/docs/developers/dataset](https://firmtape.com/docs/developers/dataset)

**Related:** [DATASET: Three books in every file](https://firmtape.com/docs/dataset#three-books) · [DATASET: A new session every evening](https://firmtape.com/docs/dataset#evening-delivery) · [LAB: The LAB vocabulary](https://firmtape.com/docs/lab#vocabulary)

### Three books in every file

**Why:** three readings of the same dealer positioning, side by side, and a rule that keeps them from being mixed. **Where:** the book tabs on the [data dictionary](https://firmtape.com/dataset/dictionary). **Plan:** The DATASET licence; the dictionary is free to read.

1. Open the dictionary and pick **VOLUME**, **MEASURED**, **OPEN INTEREST** or **NO BOOK**.
2. Read the line under the book's name: how to spot its fields by suffix.
3. Compare magnitudes only inside one book.

![The data dictionary's book tabs with the volume book open: cp times cumulative session volume, spotted by the suffix _volcp or _vol](https://archive.firmtape.com/press/features/data/dictionary.eb9d819f.webp)

> **What it measures:** The **volume** book signs cumulative session volume by call or put (the conventional public object, built here from the tape); the **open-interest** book uses the morning's open interest; the **measured** book is signed print by print from the tape and its quotes. A field with no book (price, time, implied vol) is marked so.

How to read it:

- `_volcp` and `_vol` are the volume book, `_oi` the open-interest book; a neutral payload name is the measured book.
- The volume book is the default: it is what the chart draws when no book is named.
- Two books disagreeing on the regime is expected; they measure different things.

**Limits:** Magnitudes are never comparable across books.

**Related:** [DATASET: The day files and the dictionary](https://firmtape.com/docs/dataset#archive-files) · [LIVE: The three books](https://firmtape.com/docs/live#the-book)

### A new session every evening

**Why:** the archive grows by one session a day, and your copy can grow with it. **Where:** the day list, the evening after each session. **Plan:** The DATASET licence.

1. After the close, fetch the day list again.
2. Fetch the new day's file; earlier days never change.
3. Fetch the calendar file too for the scheduled releases and what usually followed each kind.

> **What it measures:** A finished session is written the evening it closes and is not rewritten afterwards, so a day you already hold stays valid.

**Limits:** Today's session as it trades is LIVE, not part of the files.

**Related:** [DATASET: The day files and the dictionary](https://firmtape.com/docs/dataset#archive-files) · [REPLAY: The session archive and session pages](https://firmtape.com/docs/replay#session-archive)

### Older days in the levels API

**Why:** one account for recent levels in a chart and the long history in a study. **Where:** the levels API's history call, with your account's key. **Plan:** The DATASET licence together with LIVE's levels API.

1. Ask the levels API for a day's history as usual.
2. Days older than the API's usual window answer when the account holds the licence.

> **What it measures:** The levels API answers the last 90 calendar days of history for an account key; a day further back needs the DATASET licence.

**Limits:** The free archive on the site is untouched by this rule: every finished session stays free to read there.

**Developer reference:** [/docs/developers/live](https://firmtape.com/docs/developers/live)

**Related:** [LIVE: Levels API](https://firmtape.com/docs/live#levels-api) · [DATASET: The day files and the dictionary](https://firmtape.com/docs/dataset#archive-files)

**The licence**

### Dataset licence

**Why:** the whole archive, all three gamma books, on your own machine, with no credits and no rate limit. **Where:** the licence on [pricing](https://firmtape.com/pricing); every field in the [data dictionary](https://firmtape.com/dataset/dictionary). **Plan:** The dataset licence.

1. Start the licence from [pricing](https://firmtape.com/pricing); delivery starts that evening.
2. Download with the token you already hold for LAB, under your licence.
3. Read each field's meaning, unit, book and start date in the [data dictionary](https://firmtape.com/dataset/dictionary).
4. Pick the book by its suffix: the volume, measured and open-interest books are never mixed.

![The data dictionary: every field of the archive files and every LAB column, grouped by the volume, measured and open-interest books](https://archive.firmtape.com/press/features/data/dictionary.eb9d819f.webp)

> **What it measures:** Every finished SPX session minute by minute, full per-strike detail in all three books, plus the market calendar with what usually followed each kind of event. A session lands minutes after the evening build.

How to read it:

- Magnitudes are never comparable across books: a field's suffix names its book.
- A field's start date is in the dictionary; some exist only from the day they were added.

**Limits:** For your own research. Showing the numbers to other people is [licensing](https://firmtape.com/licensing). Licensed raw vendor data is never part of it.

**Developer reference:** [/docs/developers/dataset](https://firmtape.com/docs/developers/dataset)

**Related:** [DATASET: Licensing for your product](https://firmtape.com/docs/dataset#licensing) · [LAB: The LAB vocabulary](https://firmtape.com/docs/lab#vocabulary) · [REPLAY: The session archive and session pages](https://firmtape.com/docs/replay#session-archive)

### Licensing for your product

**Why:** the numbers inside your app, site, bot or paid channel, on published terms with no sales call. **Where:** [Licensing](https://firmtape.com/licensing). **Plan:** A redistribution licence, priced on the page.

1. Read the tiers on [Licensing](https://firmtape.com/licensing): levels only, the stream fifteen minutes late, or the stream in real time to an audience ceiling.
2. Press **Request this licence**; a countersigned email and a key follow.

![The four licence tiers on /licensing: LEVELS, STREAM 15 MIN, STREAM and STREAM 250, each with its price and its ceiling of people](https://archive.firmtape.com/press/features/data/licensing.6d90a781.webp)

| Tier | What it carries |
| --- | --- |
| LEVELS | Live levels each minute, ES and NQ, the archive as files; any audience |
| STREAM, 15 MIN | The book every second and the strike ladder with vanna and charm, fifteen minutes late; any audience |
| STREAM | The same in real time, up to 50 people |
| STREAM 250 | The same in real time, up to 250 people |
| Larger audiences | Printed on the same page, up to 1,000 people and beyond |

How to read it:

- Attribution goes wherever the numbers are shown, with a link to firmtape.com; only the tier above 1,000 people drops it.

**Limits:** No tier covers a dealer-positioning terminal built on the stream, and none carries the raw vendor tape.

**Related:** [DATASET: Dataset licence](https://firmtape.com/docs/dataset#dataset-licence) · [LIVE: Levels API](https://firmtape.com/docs/live#levels-api)

## Add to DATASET

| Add-on | What it adds |
| --- | --- |
| [Redistribution licences](https://firmtape.com/licensing) | Show FirmTape's levels or its stream to other people in your app, site, bot or channel, on printed terms |

## Use it with other tools

| Tool | How it pairs |
| --- | --- |
| [REPLAY: The session archive and session pages](https://firmtape.com/docs/replay#session-archive) | Read a session on the site before you study it in bulk. |
| [LAB: The LAB vocabulary](https://firmtape.com/docs/lab#vocabulary) | Rebuild a LAB column from the fields it is built from, listed on the dictionary. |
| [LIVE: Levels API](https://firmtape.com/docs/live#levels-api) | Recent history over the levels API; older days with the licence. |

## Good to know

- The licence is for your own research: internal use, no redistribution, no competing service.
- Delivery uses your account's token as a bearer header, from a script, with no browser.
- Fields added later carry their start date; a study over all sessions should check it.

### If something looks wrong

| You see | Do this |
| --- | --- |
| A download is refused | The licence must be active on the account the token belongs to; check Account, Plans and billing. |
| A field is missing on an older day | It starts later: read its since column on the dictionary. |
| Two books disagree on the regime | They measure different things: the measured book disagrees with the convention book in a large share of minutes by design. |

## Glossary

| Term | Meaning |
| --- | --- |
| [volume book](https://firmtape.com/learn/what-is-gex) | cp-signed cumulative session volume per strike: the conventional public gamma object, built from the tape. |
| [open-interest book](https://firmtape.com/learn/open-interest-vs-volume) | The same construction on the morning's open interest. |
| measured book | The book signed print by print from the tape and its quotes. |
| day file | One finished session as the terminal reads it, every minute and every strike. |
| [data dictionary](https://firmtape.com/dataset/dictionary) | Every field of the files and every LAB column, by book, with unit and start date. |