Docs & data dictionary

Terminology

The same words appear across the site — here is how they relate.

Data line

A collection pipeline and its QA-verified output — the unit of inventory on the catalog. Most data lines are published as a dataset; a line marked “coming soon” is verified but not yet purchasable.

Dataset

A published product with its own page under /datasets — sample, data dictionary, datasheet and tiered pricing.

Package

A downloadable file within a dataset: an S/M/L tier, a table or a free sample, each with a manifest-verified row count, size and sha256.

Slice

A custom cut of a dataset (by country, industry, time range…) prepared on request — priced from $49 so you only pay for the part you need.

Protocol access — Hugging Face Hub

Every DataForge package is hosted on the Hugging Face Hub under huggingface.co/zalizedata. Free/open datasets live in public repos — load them with load_dataset, the hf:// filesystem protocol (pandas, DuckDB, Polars) or plain HTTP resolve links, no signup. Paid and restricted editions are delivered from private (or gated) repos: request access on the dataset page and, once approved, use the exact same protocols with your access token.

# pip install -U datasets huggingface_hub
from datasets import load_dataset

ds = load_dataset("zalizedata/app-category-opportunity-scan",
                  "category_scores", split="train")
print(ds)

# Query straight over the hf:// protocol — no download step
import pandas as pd
df = pd.read_parquet(
    "hf://datasets/zalizedata/app-category-opportunity-scan/"
    "data/category_scores/train-00000-of-00001.parquet")