Skip to main content
Version: v0.20.0

DCAT Metadata Structure

Since the Dataspace Protocol (DSP) uses DCAT 3 (Data Catalog Vocabulary) for describing catalog entries, we have setup a DCAT structure for the TSG. This document describes the DCAT structure used in the TNO Security Gateway for representing data catalogs, datasets, and their distributions.

Overview​

The implementation of DCAT enables standardized discovery and exchange of dataset information between participants. The implementation extends DCAT with domain-specific vocabularies like HealthDCAT-AP for health data scenarios.

Core DCAT Structure​

The TSG uses a hierarchical DCAT structure to organize data resources:

Catalog
├── DataService (DSP API endpoint)
└── Dataset
    ├── dct:conformsTo → Ontology, legislation, standards
    │   └── (e.g., HealthDCAT-AP, CSVW, domain ontologies)
    └── Distribution
        ├── dct:conformsTo → Distribution schema (JSON Schema, XSD, etc.)
        ├── dcat:mediaType → Media type of source data
        └── DataService
            ├── endpointURL → DSP API endpoint
            ├── endpointDescription → dspace:connector
            ├── dspace:dataPlaneType → IRI indicating data plane type

Component Details​

Catalog​

The Catalog represents a collection of datasets and data services offered by a participant in the data space. It serves as the top-level container for all discoverable resources.

Key Properties:

  • dcat:dataset - References to datasets in the catalog
  • dcat:service - References to data services (typically the DSP API)
  • dspace:participantId - Identifier of the participant offering this catalog
  • dct:publisher - Publisher of the catalog

DataService (Catalog Level)​

The catalog-level DataService represents the DSP API endpoint that provides access to the catalog and facilitates data space protocol interactions.

Key Properties:

  • dcat:endpointURL - URL of the DSP API
  • dcat:endpointDescription - Description of the service (typically references dspace:connector). This reference is used to automatically link the DSP Data Service to new datasets.

Dataset​

A Dataset represents a logical collection of data that can be accessed through one or more distributions. Datasets can conform to various standards, ontologies, or legislation.

HTTP dataset publication​

Applications publish DCAT metadata and private backend configuration through the control plane's management API. The HTTP data plane receives the private configuration when a dataflow is prepared. Applications provide dataset authoring and editing.

A published HTTP dataset has:

  • Dataset-level conformsTo references to the shared semantic model and its version.
  • A distribution with format: "tsg:HTTP" and the response media type.
  • Distribution-level conformsTo references to schemas and OpenAPI documents.
  • Private assetConfig containing the backend URL and authorization header.

The control plane attaches the DSP access service when presenting the catalog. See dataset publication and vocabulary hubs for complete SDK submissions, including Semantic Treehouse references.

Analytics dataset publication​

Analytics-orchestration authors and publishes metadata for analytics datasets. The analytics data plane executes jobs and serves data; metadata analysis and publication belong to the application.

Automated Metadata Generation:

The analytics-orchestration metadata workflow supports:

  1. Deterministic Analysis:

    • Column data types detection (string, integer, float, date, boolean)
    • Statistical profiling (min/max values, unique counts, null percentages)
    • Pattern detection (email addresses, phone numbers, medical codes)
    • Temporal coverage extraction from date columns
    • Data quality metrics (completeness, consistency)

  2. LLM-Enhanced Metadata (optional):

    • Semantic title and description generation
    • Keyword extraction for discovery
    • Theme classification (using DCAT and HealthDCAT-AP themes)
    • Column-level semantic annotations

  3. DCAT Dataset Creation:

    • Generates comprehensive dataset metadata including CSVW table schema
    • Creates distribution with dcat:format set to tsg:analytics
    • Includes HealthDCAT-AP extensions for health data (age ranges, coding systems)
    • Embeds data quality measurements using DQV (Data Quality Vocabulary)
    • References CSVW for column-level metadata (variable dictionary)

The resulting dataset includes rich semantic metadata aligned with DCAT 3, HealthDCAT-AP, and CSVW standards, enabling fine-grained discovery and understanding of the data without exposing the actual content.

Example Generated Properties:

  • healthdcatap:numberOfRecords - Row count
  • healthdcatap:hasCodingSystem - Detected medical coding systems (ICD-10, LOINC, etc.)
  • dqv:hasQualityMeasurement - Completeness and validity metrics
  • csvw:tableSchema - Full variable dictionary with semantic annotations

Domain Extensions​

For health data, datasets may include HealthDCAT-AP properties:

  • healthdcatap:numberOfRecords - Number of records in the dataset
  • healthdcatap:minTypicalAge - Minimum typical age of subjects
  • healthdcatap:maxTypicalAge - Maximum typical age of subjects
  • healthdcatap:hasCodingSystem - Medical coding systems used

Distribution​

A Distribution represents a specific available format or access mechanism for a dataset. Each distribution can have its own schema, media type, and access service.

Key Properties:

  • dct:title - Title of this specific distribution
  • dct:conformsTo - Schema or standard for this distribution
    • Examples: JSON Schema, XML Schema (XSD), Avro Schema
  • dcat:mediaType - IANA media type of the data
    • Examples: text/csv, application/json, application/parquet
  • dcat:format - Format identifier (may be different from mediaType)
  • dcat:byteSize - Size of the distribution in bytes
  • dcat:accessService - Reference to the DataService providing access

DataService (Distribution Level)​

The distribution-level DataService describes how to access a specific distribution, including the data plane endpoint and any technical descriptions.

Key Properties:

  • dcat:endpointURL - URL of the DSP API (for transfer negotiation)
  • dcat:endpointDescription - Type of connector (typically dspace:connector)

Dataset Versioning​

DCAT provides multiple properties to manage dataset versions, allowing participants to track evolution of datasets over time and maintain relationships between different versions.

Version Properties​

DCAT defines several properties for version management:

  • dcat:version - A version number or identifier (e.g., "1.0", "2.3.1", "2024-01-15")
  • dcat:hasVersion - Links to other versions of this dataset (can be multiple)
  • dcat:isVersionOf - Points to the parent/base dataset that this is a version of
  • dcat:hasCurrentVersion - Points to the current/latest version
  • dcat:previousVersion - Points to the immediately preceding version

Versioning in TSG​

Publish the base dataset and its versions as separate metadata submissions. The base has hasVersion and hasCurrentVersion links; each version has version, isVersionOf, and optionally previousVersion. Only transferable versions need private backend configuration. References and shared domain properties must be supplied explicitly on each version.

See the version-specific model example for a base dataset and a version linked to a vocabulary hub.

Versioning Best Practices​

  1. Semantic Versioning: Use semantic versioning (MAJOR.MINOR.PATCH) for APIs and data schemas
  2. Date-based Versioning: Use ISO 8601 dates (YYYY-MM-DD) for time-series data or periodic releases
  3. Breaking Changes: Increment major version when making breaking changes to schema or semantics

Schema Versioning​

When versioning datasets, also version the conformance schemas:

{
  "@type": "Dataset",
  "@id": "https://example.org/datasets/research-data/v2.0.0",
  "dcat:version": "2.0.0",
  "distribution": [
    {
      "@type": "Distribution",
      "conformsTo": "https://example.org/schemas/research-v2.schema.json",
      "mediaType": "application/json"
    }
  ]
}

This ensures consumers can validate data against the correct schema version and understand structural changes between versions.

Implementation Examples​

HTTP Data Plane Dataset​

{
  "@type": "Dataset",
  "@id": "https://example.org/datasets/patient-data",
  "title": "Patient Demographics",
  "description": "Anonymized patient demographic data",
  "conformsTo": [
    "https://healthdataeu.pages.code.europa.eu/healthdcat-ap/",
    "https://example.org/ontology/demographics-v1"
  ],
  "distribution": [
    {
      "@type": "Distribution",
      "title": "JSON API Distribution",
      "conformsTo": "https://example.org/schemas/patient-schema.json",
      "mediaType": "application/json",
      "accessService": {
        "@type": "DataService",
        "endpointURL": "https://connector.example.org/api/dsp",
        "endpointDescription": "dspace:connector",
        "dataPlaneType": "tsg:HTTP"
      }
    }
  ]
}

Analytics Data Plane Dataset​

{
  "@type": "Dataset",
  "@id": "https://example.org/datasets/medical-records",
  "title": "Medical Records for Analysis",
  "description": "Encrypted medical records for federated analytics",
  "conformsTo": [
    "https://healthdataeu.pages.code.europa.eu/healthdcat-ap/"
  ],
  "distribution": [
    {
      "@type": "Distribution",
      "title": "CSV Distribution",
      "conformsTo": "https://www.w3.org/TR/tabular-metadata/",
      "mediaType": "text/csv",
      "format": "tsg:analytics",
      "accessService": {
        "@type": "DataService",
        "endpointURL": "https://connector.example.org/api/dsp",
        "endpointDescription": "dspace:connector",
        "dataPlaneType": "tsg:analytics"
      }
    }
  ]
}

Custom DCAT Properties (Application Profiles)​

Applications include custom vocabulary properties directly in the public JSON-LD dataset sent to the management API. When using the SDK's Dataset class, its extraProps constructor option serializes to those same top-level properties.

Use the bundled contexts through defaultContext() for DSP, TSG and health terms. Use full IRIs or an explicit JSON-LD context for other namespaces. Linking a profile in conformsTo does not import its context or validate conformance.

The publication guide includes complete request examples for:

  • GeoDCAT-AP: a location reference, spatial resolution and temporal resolution.
  • HealthDCAT-AP: record count, typical age range, population coverage and a vocabulary-hub model reference.
  • Shared properties: explicit merging for each version or collection item.

Conformance References​

Dataset Level (dct:conformsTo)​

At the dataset level, dct:conformsTo typically references:

  • Application Profiles: HealthDCAT-AP, DCAT-AP
  • Domain Ontologies: Medical terminologies, industry standards
  • Legislation: GDPR, HIPAA, domain-specific regulations
  • Metadata Standards: CSVW for tabular data

Distribution Level (dct:conformsTo)​

At the distribution level, dct:conformsTo typically references:

  • Data Schemas: JSON Schema, XML Schema (XSD), Avro Schema
  • Table Schemas: CSVW Table Schema for CSV files
  • API Specifications: OpenAPI specification

Best Practices​

Choosing Conformance References​

  1. Dataset conformsTo: Use for high-level semantic models, application profiles, and regulatory frameworks
  2. Distribution conformsTo: Use for technical schemas that validate the data format

Media Type Selection​

  • Use standard IANA media types whenever possible
  • For file-based resources, dcat:mediaType indicates the file format
  • For API-based resources, consider whether mediaType represents:
    • The format of API responses
    • The format of the underlying data source
    • Both (if they align)

Multiple Distributions​

Provide multiple distributions when:

  • Data is available in multiple formats (CSV, JSON, Parquet)
  • Different access patterns are supported (API vs. file download)
  • Different data plane types can access the same dataset
  • Different conformance levels or schemas apply to different views

Next: Learn about Design Decisions or return to Standards and Protocols.