What is a Data Catalog?
A Data Catalog is a centralized metadata management system that enables organizations to discover, understand, and manage all their data assets. Like a library catalog system, it organizes information about where data lives, what it contains, and how it relates to other data.
Key Components
- Data Assets: Tables, files, streams, APIs
- Metadata: Schema, owners, descriptions, quality scores
- Data Lineage: Data flow and transformation tracking
- Business Glossary: Common terminology definitions
- Classification & Tags: Data categorization and labeling
Why Organizations Need It
- Eliminate data silos and redundant work
- Reduce data discovery time from days to minutes
- Ensure regulatory compliance (GDPR, CCPA)
- Improve data quality and trust
- Enable self-service analytics
- Support data democratization
Core Features
1. Data Discovery: Google-like search for data assets
2. Metadata Management: Automated collection and enrichment
3. Data Lineage: End-to-end data flow visualization
4. Business Glossary: Standardized terminology
5. Quality Monitoring: Automated quality scoring
6. Access Control: Fine-grained permissions
Reference Standards
The data catalog discipline is rooted in established international standards that define how metadata is registered, exchanged, and governed.
- ISO/IEC 11179 — Information technology — Metadata registries (MDR). The foundational multi-part standard for describing data elements, value domains, and classification schemes.
- W3C Data Catalog Vocabulary (DCAT) version 3 — RDF vocabulary for publishing metadata about datasets, data services, and catalogs on the web.
- schema.org Dataset — Lightweight web-discoverable dataset description used by major search engines.
- Dublin Core Metadata Element Set, ISO 15836-1:2017 — Fifteen-element vocabulary providing minimum descriptive metadata.
- ISO/IEC 25012:2008 — Data quality model with fifteen characteristics (accuracy, completeness, consistency, credibility, currentness, accessibility, compliance, confidentiality, efficiency, precision, traceability, understandability, availability, portability, recoverability).
FAIR Principles
Modern catalogs implement the FAIR data principles now codified across the European Open Science Cloud and the Research Data Alliance: Findable, Accessible, Interoperable, Reusable. The WIA-DATA-007 catalog services map directly onto these four pillars: persistent identifiers (F), open and standardised retrieval (A), use of shared vocabularies (I), and clear licensing (R).
Why This Matters Now
The volume of organisational data routinely exceeds the comprehension of any individual team. A formal catalog is no longer optional — it is the operating system for data work. Without it, the same dataset is rebuilt three times a year, consent provenance is impossible to demonstrate to a regulator, and machine-learning teams train on the same biased subset because it is the only one anyone remembers.
The chapters that follow show how to build the catalog so that it remains useful at organisational scale, conformant with the international standards above, and trusted by the people who own each dataset.
Practical Application
An organisation adopting WIA-DATA-007 as its first formal catalog typically begins with three workshops:
- Inventory workshop — Stewards from each domain list the top twenty datasets that already serve regular consumption. The output is a CSV with name, owner, classification, and rough sensitivity tier.
- Connector workshop — A platform engineer wires up the catalog crawler against each datastore, using the conformance-tested connectors aligned with W3C DCAT, ISO/IEC 11179-3 metamodel, and the ResourceSync sitemap conventions.
- Tier-up workshop — Each owner spends thirty minutes raising their flagship dataset from bronze to silver: filling description, owner, column docs, tags, and freshness.
By the end of the third workshop, between forty and eighty assets are silver-tier and discoverable. This is the smallest deployment that materially changes consumer behaviour and validates the architecture chosen in the next chapter.