How to Build a Data Catalog Without Enterprise Tools

A data catalog is just an inventory of what data you have, where it lives, and what it means. Enterprise tools like Alation, Collibra, and Atlan charge $50-200K/year to answer those questions. You don't need them. If you're a startup or mid-market company with under 500 tables, you can build a catalog that's genuinely useful with open-source tools or even a well-maintained spreadsheet. I've done it both ways.

What a data catalog actually connects

Before picking a tool, it helps to see what you're building. A catalog sits between your data sources and the people who consume data. It doesn't move data -- it indexes metadata.

  Sources                     Catalog                    Consumers
  --------                    -------                    ---------
  PostgreSQL   ---\                                /---> Analysts
  MySQL        ---|  schema, descriptions,        |---> Data engineers
  Snowflake    ---|  lineage, freshness,    ------|---> Product managers
  BigQuery     ---|  ownership, glossary          |---> Executives
  S3 / GCS     ---|                                \---> New hires ("what
  APIs         ---/                                      data do we have?")

The catalog answers five questions: What tables exist? What do the columns mean? Who owns each dataset? Where does the data come from? Is it fresh? If your catalog answers those, it's working. Everything else is polish.

What a useful data catalog contains

Not every team needs every feature. But here's the full list, roughly in order of how much each one reduces "hey, do we have data on X?" Slack messages:

• Table and view inventory -- every table, with a human-readable description of what it contains
• Column-level documentation -- not just types, but what status_code actually means (is 3 "cancelled" or "refunded"?)
• Ownership -- who to ask when something breaks or when you need context
• Data freshness -- when was this table last updated, and is that normal?
• Lineage -- where does this data come from, and what downstream tables or dashboards depend on it?
• Data quality status -- are tests passing? Are there nulls where there shouldn't be?
• Business glossary -- what does "MRR" mean in this company? Is "customer" the same as "account"?

Now let's look at the tools that can get you there.

dbt docs -- the easiest starting point

If you're already using dbt, you have the foundation of a data catalog and might not know it. Every model can have a description. Every column can have a description. Tests document expectations. And the DAG is your lineage.

dbt docs generate
dbt docs serve

That gives you a searchable web interface with:

• Model and column descriptions (from your schema.yml files)
• A visual DAG showing how models depend on each other
• Test results and source freshness
• Column types and stats

The catch: dbt docs only covers what dbt manages. If you have raw tables that aren't dbt sources, they won't appear. And the UI is functional but not pretty -- it hasn't changed much in years.

Best for: Teams already on dbt who want a catalog with zero additional tooling. Write better schema.yml files and you're 70% there.

For a deeper comparison of Core vs Cloud (which adds a hosted catalog), see our dbt Core vs dbt Cloud breakdown.

DataHub -- the full-featured open-source option

DataHub started at LinkedIn and is now the most mature open-source metadata platform. It does everything: schema discovery, lineage, ownership, tags, search, a business glossary, data quality integration, and access policies.

The feature list is impressive. The setup is not trivial. A production DataHub deployment involves:

• Docker Compose or Kubernetes
• Kafka (for metadata events)
• MySQL or PostgreSQL (for metadata storage)
• Elasticsearch (for search)
• A frontend React app

That's a lot of moving parts. If you have a platform team or a data engineer who enjoys infrastructure, DataHub rewards the investment. The ingestion framework supports dozens of sources -- Snowflake, BigQuery, Postgres, dbt, Airflow, Looker, Tableau, and more. Lineage is automatic for most of them.

Best for: Teams with 10+ data practitioners who need a real metadata platform and have the engineering capacity to operate it. If you'd run Airflow, you can run DataHub.

OpenMetadata -- the newer, simpler alternative

OpenMetadata is the most promising newer entrant. It's UI-first, which means non-engineers can actually use it without training. It covers discovery, lineage, data quality (built-in, not a separate tool), collaboration (comments, tasks, announcements), and a glossary.

Setup is simpler than DataHub -- it uses a single backend service with a MySQL or Postgres database and Elasticsearch. No Kafka dependency. You can have it running locally in 15 minutes with Docker Compose.

The ingestion framework is solid. It connects to most major databases, warehouses, dashboards, and pipelines. The lineage view is clean. And the data quality features mean you don't need a separate tool like Great Expectations for basic checks.

Best for: Teams that want DataHub-level features with less operational overhead. If you're choosing between DataHub and OpenMetadata today, start with OpenMetadata unless you need a specific DataHub integration.

Amundsen -- search-first discovery

Amundsen came out of Lyft and takes a different angle: it's a search engine for data. The core experience is typing "customers" and seeing every table, dashboard, and column related to customers, ranked by usage and popularity.

It's built on a graph database (Neo4j or Atlas) with Elasticsearch for search. The architecture is clean but maintenance has slowed compared to DataHub and OpenMetadata. The community is smaller, updates are less frequent, and some connectors lag behind.

That said, if your primary problem is discovery -- "I know we have revenue data somewhere, but I can't find it" -- Amundsen's search experience is genuinely good. It was built for that single use case and it shows.

Best for: Large organizations where the main problem is discovery across hundreds of tables. Less maintained than DataHub or OpenMetadata, so factor in the risk of a slower-moving project.

Google Sheets or Notion -- the pragmatic option

I know. It sounds wrong. But for a team with fewer than 50 tables, a shared spreadsheet is often the right answer.

Create a sheet with these columns:

| Table Name | Schema | Description | Owner | Refresh | Source | Key Columns | Notes |
|------------|--------|-------------|-------|---------|--------|-------------|-------|
| orders     | public | All orders  | @anna | Hourly  | Stripe | id, user_id | ...   |
| users      | public | User accts  | @mike | Real-time| App DB | id, email   | ...   |

This works until it doesn't. The failure mode is predictable: someone adds a table and forgets to update the sheet. Columns get renamed and the sheet goes stale. Around 50-100 tables, the maintenance overhead starts to outweigh the simplicity benefit.

Best for: Teams with under 50 tables, no dbt, and no appetite for running another service. Better than nothing, and nothing is what most small teams have.

dbt + Great Expectations + a wiki -- the DIY combination

This is the stack I've seen work best for teams between "spreadsheet" and "DataHub." You stitch three tools together, each handling what it does best:

  dbt                     Great Expectations        Notion / Confluence
  ---                     --------------------      -------------------
  Model descriptions      Data quality checks       Business glossary
  Column docs             Freshness monitoring      Tribal knowledge
  Lineage (DAG)           Anomaly detection         Process docs
  schema.yml = catalog    Test suites               "What does MRR mean?"

dbt handles the technical metadata -- what tables exist, what columns they have, how they connect. Great Expectations handles quality -- are values in range, are there unexpected nulls, is the data fresh. The wiki handles business context -- what "active customer" means, why Q3 2024 numbers look weird, who approved the metric definition change.

The tradeoff: no single search interface. You'll look in dbt docs for schema questions, Great Expectations for quality questions, and the wiki for business questions. For most teams under 100 tables, that's fine. You know where to look.

Best for: dbt shops that want quality monitoring and business documentation without adopting a full metadata platform. For teams already evaluating open-source analytics tools, this combination slots in cleanly.

Comparison table

Start here: which approach fits your team

Don't overthink this. The decision tree is short:

1. Already using dbt? Start with dbt docs generate. Write better descriptions in your schema.yml files. You'll get 70% of a catalog for zero effort.
2. Under 50 tables, no dbt? Make a Google Sheet. Seriously. Update it in your PR review process.
3. 50-200 tables, want something real? OpenMetadata. Simpler to set up than DataHub, and the built-in quality features save you from running Great Expectations separately.
4. 200+ tables, dedicated platform team? DataHub. It's the most mature, the integration library is the deepest, and LinkedIn runs it at scale.
5. Just need discovery, not governance? Try Amundsen -- or honestly, just connect your database to a tool that indexes everything automatically and let people search.

That last point matters. A lot of catalog projects stall because the setup is the easy part -- the hard part is getting people to write descriptions and keep them current. Any tool that auto-indexes schemas and types gets you halfway there without asking anyone to fill out a form.

Where Fastero fits

Fastero isn't a data catalog. But it solves the discovery half of the problem. When you connect a database, Fastero indexes every table, column, and type automatically. You can ask "what tables do we have about customers?" in natural language and get an answer in seconds, without anyone having written a single description.

It won't replace DataHub for a 50-person data team that needs governance, lineage, and a glossary. But for teams where the real problem is "I don't know what data we have" -- which is most teams -- it covers that in about 30 seconds.

For teams looking at the broader tooling picture, our data engineering tools roundup covers how catalogs fit alongside orchestration, transformation, and visualization.

FAQ

How much does a data catalog cost? Enterprise tools (Alation, Collibra, Atlan) run $50-200K/year depending on data volume and seats. Every open-source option in this post is free to use -- you pay only for the infrastructure to host it. dbt docs and a spreadsheet cost literally nothing.

Do I need a data catalog if I have fewer than 20 tables? Probably not as a dedicated tool. A README in your repo or a pinned Slack message listing your key tables is enough. The need for a catalog grows with the number of people who ask "do we have data on X?" -- not just the number of tables.

Can dbt replace a data catalog? Partially. dbt docs cover schema, descriptions, lineage, and test results. What's missing is ownership workflows, a business glossary, search across non-dbt sources, and data quality monitoring beyond dbt tests. For many teams, "dbt docs plus a wiki" is genuinely enough.

How do I get my team to actually use the catalog? Make it the answer to every "where is this data?" question. When someone asks in Slack, reply with a catalog link instead of a direct answer. Put catalog links in dashboard descriptions. Add a "check the catalog first" step to your onboarding doc. Adoption is a habit problem, not a tooling problem.

What's the difference between a data catalog and a data dictionary? A data dictionary is a subset -- it lists tables and columns with definitions. A catalog adds lineage, ownership, quality, freshness, search, and collaboration features. Most teams start with a dictionary (which a spreadsheet handles fine) and graduate to a catalog when they need lineage or cross-team discovery. Our open-source dashboard tools guide covers how visualization tools connect to this metadata layer.

Try Fastero free -- connect a database, Fastero indexes every table and column automatically. Not a full catalog, but the discovery problem is solved in 30 seconds. No credit card required.