How I Added dbt Cloud to Coral — My Open Source Hackathon Journey

What is Coral?

Coral is an open-source tool that lets you query any API, database,
or file as SQL — no ETL, no warehouse, no glue code.

The Problem I Solved

dbt Cloud is used by thousands of data engineers. But there was no
way to query dbt Cloud jobs, runs, and environments via SQL. I built that.

What I Built

PR #627 — dbt Cloud Community Source

Added 3 tables to Coral:

• dbt_cloud.jobs — all jobs with schedules and state
• dbt_cloud.runs — run history with status codes
• dbt_cloud.environments — deployment environments

The Code

# manifest.yaml (simplified)
name: dbt_cloud
backend: http
tables:
  - name: jobs
  - name: runs  
  - name: environments

Live Demo

-- Query 890 open PRs from Coral repo
SELECT number, title, state 
FROM github.pulls 
WHERE owner='withcoral' AND repo='coral' 
AND state='open' 
LIMIT 1000

Result: 890 rows in ~10 seconds. No ETL. Just SQL.

Status Codes I Learned

dbt Cloud run status codes:

• 1 = Queued
• 2 = Starting
• 3 = Running
• 10 = Success
• 20 = Error
• 30 = Cancelled

Challenges

1. Pagination — dbt Cloud uses offset pagination, had to implement correctly
2. Timestamps — ISO8601 format needed special handling in Coral DSL
3. Virtual columns — filter echo columns needed for debugging

What I Learned

• How to write a Coral source manifest (DSL v3)
• HTTP backend configuration
• Open source contribution workflow — fork, branch, PR, review
• Real API pagination patterns

The Result

PR #627 merged into withcoral/coral ✅

Now any Coral user can:

SELECT j.name, r.status, r.started_at
FROM dbt_cloud.runs r
JOIN dbt_cloud.jobs j ON j.id = r.job_id
WHERE r.status = 20  -- Error
ORDER BY r.started_at DESC
LIMIT 20;

Try It Yourself

coral source add --file sources/community/dbt_cloud/manifest.yaml
coral sql "SELECT * FROM dbt_cloud.jobs LIMIT 10"

Links

• Merged PR: https://github.com/withcoral/coral/pull/627
• Coral repo: https://github.com/withcoral/coral
• Pirates of the Coral-bean hackathon: https://wemakedevs.org/hackathons/coral

Built during Pirates of the Coral-bean hackathon by @WeMakeDevs 🏴‍☠️