FFastero
Back to blog

Blog article

How to Choose a Data Catalog Without Overengineering

Most data catalog content is written by vendors selling $50k/year enterprise tools. If you're a 5-15 person data team asking 'do I need a data catalog,' the honest answer is usually not yet — here's the decision framework, the poor-man's-catalog SQL, and the signals that tell you when you've actually outgrown it.

Fastero Dev TeamFastero Dev Team
2026-07-18
data catalogdata governanceOpenMetadataDataHubmetadata managementsmall data teams
How to Choose a Data Catalog Without Overengineering

I evaluated three data catalogs last year — Atlan, a self-hosted OpenMetadata instance, and a Notion wiki someone on the team had already started. We picked the Notion wiki. Not because Atlan wasn't impressive (it was), but because we had six data sources, four people who touched them regularly, and no compliance requirement forcing our hand. Buying a catalog would have meant paying five figures a year to solve a problem we didn't have yet, and creating a new problem — an unmaintained tool — that we would have.

Most of what you'll read about data catalogs is written by the companies selling them, which means it starts from "yes, you need one" and works backward to justify the price tag. This isn't that. This is the framework I actually used, including the SQL query you can run right now to get 80% of the value for $0.

What a data catalog actually does

Strip away the marketing and a data catalog is doing some combination of four things:

Discovery. Answering "does a table for X exist, and where?" without asking in Slack. This is the thing everyone actually wants.

Documentation. Column-level descriptions, table owners, freshness expectations — the tribal knowledge that otherwise lives in one senior analyst's head.

Lineage. Tracing a column in a dashboard back through every transformation to its source table. Useful for debugging and for "if I change this, what breaks downstream."

Governance. Access controls, PII tagging, data classification, audit trails for who queried what. This is the part regulators and enterprise security reviews actually care about.

Here's the thing the vendors don't lead with: these four capabilities have wildly different costs and wildly different payoffs depending on your team size, and almost nobody needs all four on day one.

  • Discovery is nearly free — most of it can come from your warehouse's own metadata.
  • Documentation is not a tooling problem, it's a discipline problem. A catalog doesn't write descriptions; a person does, and the same person who wouldn't write it in a wiki won't write it in Atlan either.
  • Lineage gets genuinely hard to do well once your transformation logic spans dbt, Python scripts, and ad-hoc SQL — and genuinely valuable once you have enough tables that "what breaks if I touch this" is a real question. Below a certain size, you already know the answer.
  • Governance is the one capability that's actually driven by an external requirement (SOC 2, HIPAA, a customer security questionnaire), not by team size or table count. If nothing is forcing you to prove access control and PII handling, you're paying for governance theater.

The mistake I see most often is teams buying a tool for all four capabilities when they only have a real need for one — usually discovery.

The "do I need one?" decision framework

Team size here means people who regularly write queries, build dashboards, or touch the warehouse — not your whole company.

Under 5 data people: No

At this size, the person asking "where's the customer table" can just ask the other three people, and probably already knows the answer. A catalog at this stage is a solution actively looking for a problem. Use information_schema (SQL below) plus a single shared doc for the handful of tables that genuinely need a paragraph of context — the ones where the table name lies about what's in it, or the metric has a non-obvious definition.

The failure mode to watch for isn't "we don't have a catalog," it's spending a week evaluating catalog vendors instead of writing the queries you actually need.

5-15 data people: Maybe

This is the zone where discovery starts to genuinely hurt — enough tables that nobody has the full picture, enough people that "just ask" stops scaling, but usually not enough regulatory pressure to need real governance yet.

If you're here, start with OpenMetadata or DataHub — both are free, open-source, and self-hostable. Don't start with a paid tool at this stage. The open-source options give you discovery and basic documentation without a contract, and critically, without a per-seat price that punishes you for onboarding more people to look things up.

The catch, covered below, is that "free" only means free of license cost. It is not free of maintenance.

15+ data people with compliance needs: Yes

Once you're large enough that lineage and access governance are real operational needs — and especially once a customer security questionnaire, SOC 2 audit, or HIPAA requirement is asking you to prove who can see what data and where a given field originated — it's time to evaluate Atlan, Alation, or Collibra. These tools earn their price at this scale: dedicated stewardship workflows, enterprise SSO and access policies, audit logging, and lineage that spans BI tools, warehouses, and orchestration in one graph.

Notice what's doing the work in this tier: it's the compliance need, not the headcount. A 12-person team at a healthcare company with HIPAA obligations should skip straight to this tier. A 40-person team with no regulatory exposure and no customer security reviews might reasonably stay in the tier below.

The hidden cost nobody puts in the pitch deck: maintenance

Here's what actually kills catalog adoption at small teams, and it isn't the licensing fee.

A catalog is a mirror of your data estate. Mirrors don't stay accurate on their own — someone has to keep pointing them at reality. Every new table needs a description. Every deprecated table needs to be marked deprecated, not left to rot with a green "active" badge. Every renamed column needs its old documentation updated. That's a standing job, not a one-time setup task, and at a 6-person data team there is no headcount for "catalog maintainer" — that responsibility either gets bolted onto someone's existing job (where it loses every priority fight) or it doesn't happen.

The result is shelfware with extra steps: a catalog that's 60% accurate is worse than no catalog, because now people are actively misled instead of just uninformed. I've seen a table marked "source of truth for revenue" that had been superseded eight months earlier — the catalog entry outlived the table's actual relevance by most of a year, and someone built a board deck off it before the mismatch got caught.

This is also why OpenMetadata and DataHub being "free" is a half-truth. The license is free. Someone still has to run the ingestion connectors, review what gets auto-cataloged, and — this is the part that actually takes ongoing time — write and update the documentation that turns a table listing into something useful. Auto-discovery gets you table and column names for free. It does not get you "this column is net of refunds, not gross" for free. A human has to know that and type it in.

If you can't name the specific person who owns catalog upkeep, you're not ready for a catalog. You're ready for the lightweight alternative below, which fails more gracefully because there's less of it to go stale.

Lightweight alternatives to a full catalog

Your warehouse's own metadata. Every major warehouse — Postgres, Snowflake, BigQuery, Redshift — exposes information_schema (or an equivalent) that already knows every table, column, and data type in your database. No ingestion job, no sync lag, always accurate because it's generated from the actual schema at query time. This is a poor man's catalog you already have:

-- List every table with column count and last-modified info (Postgres)
SELECT
    t.table_schema,
    t.table_name,
    COUNT(c.column_name) AS column_count,
    obj_description(
        (t.table_schema || '.' || t.table_name)::regclass, 'pg_class'
    ) AS table_comment
FROM information_schema.tables t
JOIN information_schema.columns c
    ON t.table_schema = c.table_schema
   AND t.table_name = c.table_name
WHERE t.table_schema NOT IN ('pg_catalog', 'information_schema')
GROUP BY t.table_schema, t.table_name
ORDER BY t.table_schema, t.table_name;
 
-- Column-level detail for a specific table, including any comments you've set
SELECT
    c.column_name,
    c.data_type,
    c.is_nullable,
    col_description(
        (c.table_schema || '.' || c.table_name)::regclass::oid,
        c.ordinal_position
    ) AS column_comment
FROM information_schema.columns c
WHERE c.table_schema = 'public'
  AND c.table_name = 'orders'
ORDER BY c.ordinal_position;

COMMENT ON TABLE orders IS 'One row per completed order, net of refunds'; and COMMENT ON COLUMN orders.total_amount IS 'USD, tax-exclusive'; cost you thirty seconds each and live inside the database itself — no separate tool to keep in sync, no risk of the documentation drifting from the schema because they're stored in the same place. Snowflake and BigQuery have the equivalent (information_schema.tables/columns, and COMMENT ON in Snowflake, --description in BigQuery's bq CLI or DDL).

dbt docs. If your transformations run through dbt, you already have dbt docs generate — a browsable, searchable site with column descriptions, tests, and a dependency graph, built from schema.yml files you're plausibly maintaining anyway for testing. This gets you real (if partial) lineage for free, scoped to whatever runs through dbt.

A Notion or Confluence wiki. Unglamorous, but it works because the barrier to updating it is nearly zero and everyone already knows how to use it. One page per major domain (orders, users, marketing), a table of key tables with a one-line description and an owner, updated when someone notices it's wrong. This was our actual solution, and eighteen months later it's still the thing people check first.

Schema auto-discovery in your existing tools. Some tools you already use will surface table and column structure automatically, without becoming a second system you have to maintain. Fastero's schema explorer, for one, auto-discovers tables and columns the moment you connect a database — it's not governance, it's not lineage, it's not going to help with a SOC 2 audit. It's just enough context, visible right where you're already writing queries, that you're not tab-switching to a doc (or worse, to Slack) to remember whether the column is created_at or createdAt.

None of these four options require evaluating vendors, signing a contract, or assigning someone the job title of catalog maintainer. That's the point.

When to upgrade

The lightweight approach isn't a permanent stance, it's a default until you have evidence you've outgrown it. Watch for these signals:

  • You're fielding the same "where's the data for X" question more than once a week, and it's coming from different people each time — the wiki isn't scaling with headcount anymore.
  • A customer security questionnaire or compliance audit asks for a data inventory, access log, or PII classification you can't produce quickly. This is the clearest signal of all, because it's an external deadline, not an internal judgment call.
  • You've had a real incident — a metric was wrong in a board deck, a PII column got exposed in a shared dashboard — that traces back to nobody knowing a table's actual definition or classification.
  • Your transformation logic has outgrown dbt docs' reach — lineage now spans multiple orchestration tools, ad-hoc Python, and reverse-ETL jobs, and no single existing tool shows the full path.
  • You've hired a dedicated data platform or governance role. If there's finally a person whose job includes catalog upkeep, the maintenance objection above stops applying, and the calculus changes.

When two or more of these are true, go back to the 5-15 or 15+ tier above and actually run the evaluation. Until then, the SQL query above and a wiki page someone actually updates will outperform an enterprise catalog that nobody has time to keep honest.


Try Fastero free — connect your data sources, ask questions in plain English, and automate the reports nobody has time to build. No credit card required.