Data Catalog Entry

When to use

• A new table, view, or dataset has been created and needs to be discoverable
• Analysts keep asking the same questions about a table's meaning or ownership
• A compliance or audit requirement mandates documentation of sensitive data
• Onboarding new team members who need to understand available data assets
• Auditing catalog completeness to find undocumented tables

Process

1. Extract technical metadata, pull schema, column names, types, primary keys, foreign keys, and row count from INFORMATION_SCHEMA or the source system. Use scripts/catalog_extractor.py to automate this for database tables.
2. Collect business context, interview the data owner to capture the business purpose, owning team, criticality (critical / high / medium / low), and known use cases. Record the business-friendly display name.
3. Write column descriptions, for each column, write a one-sentence plain-language description, note example values, and document any business rules (valid values, constraints, format requirements).
4. Assess data quality, calculate or estimate completeness, freshness (hours since last update), and duplicate rate. Document known issues and how they affect downstream use.
5. Document lineage, record upstream sources (where the data comes from) and downstream consumers (dashboards, models, reports that depend on it).
6. Add governance details and publish, specify access level (public/restricted/confidential), sensitivity (PII, financial, health), compliance tags, retention policy, and access instructions. Complete assets/catalog_entry_template.md and submit to the catalog.