The dataset is rebuilt weekly from live job boards. When a number moves because the pipeline was wrong rather than because the market moved, it belongs here.
Every revision that changed a published number. Newest first. Dates are the day the corrected data went live. Current build: 266 companies, 18,670 de-duplicated postings, 8,041 with a published salary, median $200,000.
classification
Role families: a single-tier classifier was sorting 21.5% of rows into "other"
What was wrong
The role-family bucket came from one rule list that tested `engineer` first. Sales Engineer, Solutions Engineer and Support Engineer were therefore counted as engineering, and the bare word `data` pulled data-center technicians into data/ML.
What changed
Four tiers: disambiguation, then specific titles, then generic words, then broad engineering words. 21 unit tests cover the traps (Technical Account Manager must not become sales; Senior Accountant must not become sales).
measure
before
after
engineering share
39.7%
27.4%
other share
21.5%
8.5%
remote vs on-site gap, engineering
-11.5%
-6.2%
statistics
Per-company publication rate divided two different things
What was wrong
The rate was computed as paid rows in the CSV divided by open postings on the board. Those are different populations, and the CSV is sampled, so one company reported a 105.9% publication rate.
What changed
Numerator and denominator now come from the same de-duplicated population. Companies with fewer than 25 open postings are left out of the table instead of producing a meaningless percentage.
measure
before
after
companies in the table
n/a
146
companies with 40+ postings publishing no salary
n/a
11
data
Currency conversion used a hardcoded table of ten currencies
What was wrong
Anything outside USD/EUR/GBP/CNY and six more came back as "no salary". Filtering by salary then deleted those postings, so the filter reported "this job has no pay" when the truth was "we could not convert it".
What changed
Live rates for 166 currencies from open.er-api.com, cached 24 hours, with the snapshot timestamp stored in every row and in the run summary. Unconvertible currencies stay null instead of being guessed.
measure
before
after
postings with a salary
7,984
8,004
rows hidden by the salary filter
~1,170
0
dates
Greenhouse publish date was actually the last-edit date
What was wrong
The pipeline read updated_at because created_at does not exist in Greenhouse responses. A 2024 evergreen role edited for a typo looked like it was posted today, which also broke the "only new jobs" mode.
What changed
Read first_published, fall back to created_at, then updated_at. Documented per board on the freshness page, because each board exposes a different date field.
measure
before
after
postings older than 180 days, Greenhouse
0.0%
14.5%
median posting age, all boards
22 days
51 days
dedupe
De-duplication ignored location and deleted 17% of real postings
What was wrong
Rows were merged on company + title. The same title in Dublin, Singapore and Chicago is three different jobs, so about 17% of the dataset disappeared. A normal de-duplication rate on a multi-board scrape is 1-3%.
What changed
The key is company + normalized title + normalized location. Hyphens are normalized too, otherwise US-Chicago and us chicago stay apart.
measure
before
after
share removed by de-duplication
17.4%
1.4%
postings with a salary
6,833
7,984
median salary
$204,750
$200,000
infrastructure
Site moved off a host that disallowed crawling
What was wrong
The static host served a platform-injected robots.txt with Disallow: / on the bare path while our own file appeared under any query string. Every URL answered 200, so nothing looked wrong, but crawlers that respect robots.txt never entered.
What changed
Moved to GitHub Pages under a path that serves our own robots.txt, and added an automated crawlability check that compares the bare path against a cache-busted one and fails if a sitemap entry points at another host.