Six APIs, zero keys: what 18,825 real job postings contain

I wanted a job feed I could filter myself, so I tried to fetch one from the source instead of the aggregators. It turns out that most company career pages are thin wrappers around a handful of applicant tracking systems (ATS), and those systems expose JSON endpoints without keys or login.

Six of them answered. I guessed 527 plausible company slugs, 259 responded, and that produced 18,825 real job postings. Here is what I found in them — including two bugs that silently published numbers that were off by two orders of magnitude.

The six endpoints

All public, all read-only, none of them need an account. Some have changed shape or disappeared: SmartRecruiters returned 404 on both endpoints I tried, and Teamtailor customers usually sit behind their own domain so the documented JSON path just 404s.

Workday is the awkward one. It needs a tenant, a cluster number and a site name before you can address a board at all, and guessing is close to hopeless: 24 well-known companies × 2 clusters × 4 site names = 192 probes, exactly one hit. Everything else had to be supplied by hand.

What a sweep returns

• 18,825 postings from 259 companies (49% of guessed slugs)
• 8,071 with a published salary (43%) — but that number only exists because of the mistake below, which is the most interesting thing I learned

Finding 1: I was wrong about Greenhouse

My first sweep looked like this:

93% of every salary-bearing row came from Ashby. Greenhouse had the volume and essentially no pay, so the obvious conclusion was "Greenhouse doesn't publish salaries".

That conclusion was wrong, and the reason is that I was asking for the wrong payload. ?content=false returns postings with the metadata array set to null for all of them. Ask for ?content=true and the pay range is sitting right there in the description text — 44% of Greenhouse postings have one.

The trap is that Greenhouse's content field is double-escaped HTML: the description literally contains <div class="content-intro">. One html.unescape is not enough. If you unescape once and search for $, you find nothing and conclude there is no salary. You need to unescape until the string stops changing (I cap it at three rounds).

Finding 2: Ashby hides its compensation behind one flag

Ashby returns beautiful structured compensation — compensationTiers, summaryComponents with minValue, maxValue, currencyCode, interval — but only when you pass ?includeCompensation=true. Without the flag you get a gorgeous list of jobs and zero numbers. Same failure mode as Greenhouse, different cause: the API is right, the request is wrong.

Finding 3: the $1,536,000 ghost

Lever returns structured ranges:

{"min": 128000, "max": 176000, "currency": "USD", "interval": "per-month-salary"}

That is a monthly salary — in a Latin American band — labeled USD. Annualise it and you get $1,536,000 for a DevOps engineer. It briefly topped the entire scan and looked completely legitimate, because it was a well-formed number in a well-formed field.

Then this one, from a long Greenhouse description:

The base salary range is 210,000 USD - 310,000 USD. Some hourly employees exist.

The word "hourly" appears somewhere in the text, so my annualiser multiplied an annual range by 2,080 and produced $436,800,000.

Both bugs share a shape: the output was plausible. A range that is off by 12× or 2,080× does not look broken; it looks like a senior role at a rich company.

I stopped trying to repair these and started deleting them. Anything that annualises below $5,000, above $5,000,000, or above $1,500,000 in USD loses its salary fields entirely while the posting stays. And the unit words are only read from the ~40 characters after the last number in the sentence, not from the whole description:

def _unit_tail(window, limit=40):
    # "…210,000 USD - 310,000 USD. Some hourly employees exist."
    # -> only look at " USD." , so "hourly" later in the text cannot fire

Since then the maximum salary in a full sweep is $850,000, which belongs to the posting it is attached to.

Lesson: in a data product, one wrong order of magnitude is worse than one missing field.

Finding 4: the pay range is not always a number

American postings often write the currency after each number:

The base salary range is 196,000 USD - 310,500 USD.

My regex was built for $196,000 - $310,500, so it matched the first number plus "USD", stopped, and produced 196000 - 196000. The output looked perfect — a single plausible number — which is exactly the kind of bug that ships. You need a separate pattern for number currency - number currency, and you need to decide what to do about postings that list several bands (… for Level 4, and … for Level 5); I take the first band and say so in the docs.

Finding 5: dedupe and "only new" are the real features

Large companies post the same role on several boards, so a raw sweep duplicates heavily: 18,825 rows collapsed to 15,528. And if you (or your users) pay per row, the feature everyone actually wants is "only the postings I have not seen yet" — which needs a stable fingerprint per posting plus somewhere to persist the seen set. Mine is company + normalised title + url, hashed, stored in a key-value store, capped at 50k entries, FIFO.

The normalisation detail that matters: strip punctuation and case, but do not strip location qualifiers. Engineer (Remote) and Engineer (Berlin) are different jobs.

If you want to try this yourself

1. Do not trust labels, tags or category counts — measure the payload.
2. Ask for compensation explicitly (Ashby), and fall back to reading the description (Greenhouse, Workday).
3. Guard every annualisation with a plausibility bound, and drop rather than repair.
4. Deduplicate on company + normalised title + url, never on title alone.
5. Store a fingerprint per posting if you want incremental results.

Honest limits

Slug guessing worked 49% of the time. Workable boards often answer with an empty shell. Some postings are evergreen and never close, so "new" is fuzzy at the edges. Salary only exists when a company chooses to publish it — 43% of rows here, not 100%. And a wrong number is worse than a missing one, so I drop aggressively and tell users when a company returned nothing.

I packaged this as a small tool that scans the six APIs, normalises salaries to annual USD, dedupes across boards and can return only new postings: ATS Job Feed. It ships the list of 259 companies that were verified live, so you do not have to guess slugs, and it can return only postings that actually state a salary.

If you pull job data for a living: which of these six APIs has bitten you the hardest? I would like to know which shape I should handle next.

Inspect the actual rows before trusting any of these numbers. Both sample outputs are mirrored as static tables — job feed: https://sjdfi0wjvosdju9.github.io/samples/ats-jobs.html, lead radar: https://sjdfi0wjvosdju9.github.io/samples/freelance-leads.html. No account needed. The selection notes (including the markets I rejected) are at https://sjdfi0wjvosdju9.github.io/how-we-choose.html.