Skip to content
DataForge

Job Postings Dataset with Salaries: Free Sources, ATS Scraping, and What Clean Data Looks Like

Published 2026-08-04 · DataForge team

“Job postings dataset” is one of those searches where the top results quietly disappoint. Kaggle serves stale one-off dumps; LinkedIn data is locked behind anti-scraping walls and its ToS; aggregator APIs price at enterprise budgets. Meanwhile the best source is hiding in plain sight: thousands of companies publish their live jobs on public ATS (applicant tracking system) endpoints that return clean JSON. This guide maps the realistic options, shows how ATS collection works, and is honest about what still requires real parsing effort — salaries most of all.

The realistic sources

Kaggle and academic dumps. Free, instantly downloadable, fine for coursework. Almost always a single historical snapshot (the popular “LinkedIn Job Postings 2023-24” set, US-only scrapes, etc.), with no refresh, inconsistent fields, and murky provenance. If your analysis needs current postings or a time series, these can’t help.

LinkedIn / Indeed scraping. The data everyone wants and the legal/technical mess nobody should want. Aggressive anti-bot systems, account bans, and explicit ToS prohibitions. Third-party scraped LinkedIn datasets exist; buying them means inheriting that provenance risk.

Government feeds. The US National Labor Exchange, state job banks, and Europe’s EURES publish postings openly — decent for labor-market research, weak on tech coverage and structure.

Public ATS endpoints — the good stuff. Companies on Greenhouse, Lever, Ashby, Workable and SmartRecruiters expose their job boards as public JSON APIs (it’s how their career pages render). The data is first-party, live, and legally collected from public endpoints:

import requests

# Greenhouse: any company's public board, as JSON
boards = ["stripe", "figma", "duolingo"]
for b in boards:
    js = requests.get(f"https://boards-api.greenhouse.io/v1/boards/{b}/jobs?content=true").json()
    for j in js["jobs"]:
        print(b, "|", j["title"], "|", j["location"]["name"], "|", j["absolute_url"])

Lever (https://api.lever.co/v0/postings/{company}?mode=json) and Ashby work the same way. The catch: you need to discover thousands of board slugs, poll them on a schedule, deduplicate reposts, and — the hard part — extract structure from HTML description blobs.

Why salary is the hard field

US pay-transparency laws (Colorado, California, New York, Washington…) pushed salary ranges into postings — but as free text inside the description: “the annual base range for this role is $148,000–$212,000”, “£65-80k DOE”, “$70/hr on W2”. A usable salary field requires parsing that handles currencies, hourly/annual units, ranges, and equity mentions, then validates against sanity bounds. This single field is most of the difference between a postings list and a postings dataset. In our 390,000-posting corpus every row carries parsed salary_min/salary_max/salary_currency, plus derived seniority and tech_stack tags extracted from descriptions.

With that structure, questions become one-liners:

-- Median advertised base for senior backend roles, US remote vs on-site
SELECT remote_policy,
       COUNT(*) AS postings,
       PERCENTILE_CONT(0.5) WITHIN GROUP (
         ORDER BY (salary_min + salary_max) / 2.0) AS median_base
FROM postings
WHERE seniority = 'senior'
  AND 'python' = ANY(tech_stack)
  AND salary_currency = 'USD'
GROUP BY remote_policy;

Building your own ATS pipeline: an honest effort estimate

If you’re weighing DIY seriously, here is what the pipeline actually involves, from experience running one at 6,500-board scale. Board discovery is the underrated first problem: there’s no directory of Greenhouse slugs. You discover them from careers-page URLs (crawling company sites for boards.greenhouse.io links), from job aggregators’ outbound links, and from guessing normalized company names — expect weeks to reach a few thousand boards, with an ongoing trickle of new ones. Polling is easy (the endpoints are fast JSON) but needs politeness: daily cadence, conditional requests, and back-off for boards that disappear (companies switch ATS vendors constantly — ~2–3% churn per month in our experience). Deduplication matters because companies repost the same role across locations or refresh postings to bump them; a stable content hash on title+description+location catches most of it. Parsing is the long tail: salary extraction alone covers dozens of phrasing patterns, and seniority inference from titles (“Staff”, “Principal”, “III”, “Sr.”) needs a curated mapping that you will still be editing a year in. None of this is hard computer science; all of it is sustained maintenance. Budget a solid month to build and a few hours a week forever — or buy the output as a snapshot and spend your time on the analysis instead.

What people build with postings data

  • Compensation benchmarks. Advertised ranges are forward-looking (what companies pay now to hire), complementing backward-looking survey data. Pairs well with our H-1B salary data — committed wages vs advertised ranges.
  • Sales and recruiting intelligence. A company posting five Kubernetes roles is buying DevOps tooling and losing engineers to somebody. Postings are the rawest public hiring signal — see our hiring signals guide for the derived version.
  • Tech-adoption tracking. Stack mentions across postings measure real-world technology demand better than survey hype cycles.
  • Labor-market research. Posting velocity, remote-share trends, and pay-transparency compliance studies.

A final note on scope honesty, because postings data gets oversold: a corpus built on public ATS endpoints covers the modern-ATS tech-economy slice — thousands of companies, disproportionately software, biotech and venture-backed — not the whole labor market. Retail shift work, government jobs and most enterprise postings live elsewhere. For the analyses this guide describes (tech compensation, stack demand, growth signals), that slice is precisely the population of interest, which is why the trade is worth making; just don’t present tech-slice numbers as economy-wide statistics.

Free sources vs. a cleaned dataset

Kaggle dumps / DIY ATS scraping DataForge dataset
Cost Free / your time From $99
Freshness Stale / you maintain the poller Monthly snapshot, 390,000+ live postings
Coverage One dump / boards you found 6,500+ company boards across 5 ATS platforms, US + EU
Salary Raw text, mostly unusable Parsed min/max/currency on every posting with detected pay
Structure Title + description blob Seniority, remote policy, tech-stack tags, normalized company
Provenance Often unclear Public ATS endpoints only, documented per-row source_url

Honest guidance: for a class project, grab a Kaggle dump. To monitor a handful of specific companies, hit their Greenhouse/Lever endpoints directly with the snippet above — it’s easy and free. The dataset earns its price at breadth (thousands of boards), history (monthly snapshots), and the parsed fields you’d otherwise spend weeks regex-wrangling. Inspect the free 500-row sample to judge the parsing quality yourself.

FAQ

Is scraping ATS endpoints legal? These are public, unauthenticated JSON APIs that companies deliberately expose to publish their jobs. We collect only from such public endpoints and record the source URL on every row. (LinkedIn/Indeed scraping is a different story — that’s why we don’t do it.)

Why not just use the LinkedIn dataset from Kaggle? Provenance (scraped against ToS), staleness (fixed snapshot), and no parsed salary structure. Fine for a portfolio notebook; risky as a business input.

How many postings have salary data? It varies by geography — US postings under pay-transparency laws disclose most often. The datasheet reports the exact fill rate per field; the sample lets you verify.

How do postings compare to BLS JOLTS data? They measure different things and disagree instructively. JOLTS is a survey-based estimate of all US job openings — statistically representative, two months lagged, no company or salary detail. ATS postings are a census of published openings at covered companies — immediate, granular, but skewed toward tech and companies that use modern ATS platforms, and inflated by evergreen requisitions (roles companies always accept applicants for) and ghost postings (roles kept up for pipeline or appearances). Labor economists use postings for timing and composition signals, JOLTS for levels. If your analysis needs “how many openings exist”, use JOLTS; if it needs “which companies are hiring for what, this month, at what pay”, postings are the only game in town.

What share of postings are ghost jobs? Nobody has a clean measure, and be suspicious of anyone quoting one confidently. Detectable proxies: postings that stay open far beyond the category-median fill time, companies whose posting count exceeds plausible team growth, and reposted-identical requisitions. In our corpus, collected_at history plus stable job_id deduplication lets you compute posting age and repost patterns yourself — which is the honest way to handle ghosts: measure and filter by your own threshold rather than trusting a vendor’s opaque “verified” flag.

Which ATS platforms matter most? Greenhouse and Lever dominate venture-backed tech; Ashby is growing fast in startups; Workable and SmartRecruiters skew European and mid-market. Enterprise ATSs (Workday, SuccessFactors, Taleo) host far more total jobs but don’t expose comparably clean public JSON — their career sites are semi-structured HTML, which is why most postings datasets (ours included) cover the modern-ATS slice explicitly rather than claiming the whole labor market.

Can I get history? Each monthly snapshot is a separate edition; subscription tiers deliver them as they’re produced. Postings carry collected_at, and reposts are deduplicated by stable job_id.