Configuration
The Analytics Orchestration application is configured through a config.yaml
placed next to the application, or at the path named by CONFIG_PATH. Values
are type checked at startup, and every field can be overridden by an
environment variable of the form TSG__<PATH> (for example
TSG__DATASPACE__WALLET_BASE_URL).
Wallet
The application needs a reachable wallet. It is not optional: the wallet holds the identity every governance message is signed with, verifies the messages peers send, and publishes the DID service entry through which those peers find this application in the first place.
Governance channel
channel covers how the application talks to its counterparties. The defaults
are safe for a dataspace whose participants all run this application; the two
settings worth revisiting are tokenTtlSeconds (shorter is stricter, at the
cost of tolerating less clock skew) and allowedParticipants, which turns the
inbox from "any participant the wallet can verify" into a closed list.
Discovery
directory controls how counterparty addresses are resolved. In a normal
deployment nothing needs to be configured here: endpoints come from the
registry's DID documents. staticEndpoints exists for local development and
for peers whose DID document does not yet carry an orchestration service.
Participants
participants predates agreement-driven discovery and remains as a fallback,
so that a deployment without an agreement layer keeps working. When a run is
scoped to a project agreement, the agreement decides who takes part.
Inspection sources
The orchestration app declares its read-only inspection sources directly in its
normal configuration. The analytics data plane declares execution access in
its own configuration. A shared sourceId connects them; their physical paths
and least-privilege credentials may differ. The control plane receives neither
configuration.
# orchestration config.yaml
datasources:
sources:
- sourceId: research-lake
type: s3
endpoint: https://s3.internal.example
bucket: research
region: eu-west-1
local sources require root; http sources require baseUrl; s3 sources
require bucket. Local roots resolve relative to config.yaml. With Helm, put
the list under config.datasources.sources and credentials under the matching
list index in configFromSecrets.datasources.sources. This uses the chart's
existing ConfigMap and Secret environment projection without another mounted
file.
The private execution asset contains only { sourceId, path }. The media type
belongs to the catalog distribution and the orchestration database. In a
DMZ/LAN deployment, put the orchestration app and execution runner in the LAN;
the DMZ data-plane server needs neither source configuration nor credentials.
Component ownership
| Component | Source information |
|---|---|
| Control plane | Dataset/distribution metadata and opaque { sourceId, path } private asset config; no source locations or credentials |
| Orchestration app | Role-local source locations and inspection credentials, catalog media type and inferred metadata |
| Analytics data plane (LAN runner) | Role-local execution locations and credentials for the same logical source IDs |
Configuration parameters
| Key | Required | Type | Description | Default |
|---|---|---|---|---|
DatabaseConfig | ||||
db | Yes | DatabaseConfig | Database configuration | |
db.type | Yes | "sqlite" | "postgres" | Type of database | |
db.database | Yes | String | Name of the database | |
db.synchronize | Boolean | Synchronize database schema | ||
db{type=sqlite} | Yes | SQLiteConfig | Database configuration | |
db{type=sqlite}.type | "sqlite" | "postgres" | Type of database | "sqlite" | |
db{type=sqlite}.database | Yes | String | Name of the database | |
db{type=sqlite}.synchronize | Boolean | Synchronize database schema | ||
db{type=postgres} | Yes | PostgresConfig | Database configuration | |
db{type=postgres}.host | Yes | String | Host of the database | |
db{type=postgres}.port | Yes | Number | Port of the database | |
db{type=postgres}.username | Yes | String | Username of the database | |
db{type=postgres}.password | Yes | String | Password of the database | |
db{type=postgres}.ssl | Unknown | SSL configuration of the database | ||
db{type=postgres}.type | "sqlite" | "postgres" | Type of database | "postgres" | |
db{type=postgres}.database | Yes | String | Name of the database | |
db{type=postgres}.synchronize | Boolean | Synchronize database schema | ||
ServerConfig | ||||
server | ServerConfig | Server configuration | ||
server.listen | String | IP address the server listens on | "0.0.0.0" | |
server.port | Number | Port the server listens on | 3000 | |
server.publicDomain | String | Public domain of the server | "localhost" | |
server.publicAddress | String | Public address of the server | "http://localhost:3000" | |
server.subPath | String | Sub path of the server | ||
DataspaceConfig | ||||
dataspace | Yes | DataspaceConfig | Dataspace connectivity | |
dataspace.controlPlaneBaseUrl | Yes | String | Base URL of the control plane this application drives | |
dataspace.tenantSlug | String | Tenant slug to address on the control plane and wallet (sent as X-Tenant-Slug on every request) | ||
dataspace.ssoBridgeBaseUrl | String | Base URL of the SSO bridge issuing management tokens | ||
dataspace.walletBaseUrl | Yes | String | Base URL of this participant's wallet. Required: the governance channel signs and verifies every message with it, and the application advertises its own inbox as a DID service through it. | |
DataspaceAuthConfig | ||||
dataspace.auth | DataspaceAuthConfig | Management API authentication | ||
dataspace.auth.method | String | OAuth client authentication method | "client_secret_post" | |
dataspace.auth.clientId | String | OAuth client identifier | ||
dataspace.auth.clientSecret | String | Client secret, for the client_secret_post method | ||
dataspace.auth.privateKeyJwk | Object | Private key in JWK format, for the private_key_jwt method | ||
dataspace.auth.accessToken | String | Pre-obtained access token, for the access_token method | ||
dataspace.pollingIntervalMs | Unknown | Delay between DSP negotiation/transfer status checks, milliseconds | 1000 | |
dataspace.pollingMaxRetries | Unknown | Maximum DSP status checks before a negotiation or transfer times out | 20 | |
ExecutionPlaneConfig | ||||
executionPlane | ExecutionPlaneConfig | This participant's own analytics data plane, used to observe local execution | ||
executionPlane.baseUrl | Yes | String | Base URL of this participant's analytics data plane management API for local execution observation | |
DataspaceAuthConfig | ||||
executionPlane.auth | DataspaceAuthConfig | Management API authentication. Defaults to dataspace.auth when omitted | ||
executionPlane.auth.method | String | OAuth client authentication method | "client_secret_post" | |
executionPlane.auth.clientId | String | OAuth client identifier | ||
executionPlane.auth.clientSecret | String | Client secret, for the client_secret_post method | ||
executionPlane.auth.privateKeyJwk | Object | Private key in JWK format, for the private_key_jwt method | ||
executionPlane.auth.accessToken | String | Pre-obtained access token, for the access_token method | ||
DataSourcesConfig | ||||
datasources | Yes | DataSourcesConfig | Participant-local read-only sources used to browse and describe datasets | |
SourceConfig | ||||
datasources.sources | Yes | SourceConfig[] | Role-local sources available for dataset inspection; local paths resolve from this configuration file | |
datasources.sources[].sourceId | Yes | String | Stable logical source identifier shared across components | |
datasources.sources[].type | Yes | "local" | "http" | "s3" | Source type: local, http or s3 | |
datasources.sources[].root | String | local: root directory visible to this component | ||
datasources.sources[].baseUrl | String | http: base URL under which dataset paths are resolved | ||
datasources.sources[].bucket | String | s3: bucket containing the datasets | ||
datasources.sources[].endpoint | String | s3: endpoint URL for an S3-compatible service | ||
datasources.sources[].region | String | s3: region | ||
datasources.sources[].forcePathStyle | Boolean | s3: use path-style addressing | ||
datasources.sources[].authorization | String | http: upstream Authorization header value | ||
datasources.sources[].accessKeyId | String | s3: access key ID | ||
datasources.sources[].secretAccessKey | String | s3: secret access key | ||
datasources.sources[].pvcName | String | local: PVC mounted into Kubernetes jobs | ||
datasources.sources[].presignTtlSeconds | Number | s3: lifetime in seconds of generated presigned URLs | ||
AuthConfig | ||||
auth | Yes | AuthConfig | Management API authentication configuration | |
auth.enabled | Boolean | Enable authentication | true | |
auth.openIdConfigurationURL | String | OpenID configuration URL | ||
auth.callbackURL | URL | Callback URL the auth service will redirect users | ||
auth.redirectURL | URL | Redirect URL to UI after login/logout | ||
auth.clientId | String | Client ID | ||
auth.clientSecret | String | Client secret (required for client_secret_post authentication) | ||
auth.tokenEndpointAuthMethod | "client_secret_post" | "private_key_jwt" | Token endpoint authentication method: client_secret_post (default) or private_key_jwt | "client_secret_post" | |
auth.privateKeyJwk | Object | Private key in JWK format for private_key_jwt authentication. Alternative to privateKeyJwkFile. | ||
auth.privateKeyJwkFile | String | Path to file containing private key JWK for private_key_jwt authentication. Alternative to privateKeyJwk. | ||
auth.permissionPath | String | JSON path to extract permissions from the token | "$.permissions[*]" | |
ChannelConfig | ||||
channel | ChannelConfig | Governance channel between orchestration applications | ||
channel.serviceType | String | DID service type under which orchestration inboxes are advertised and discovered. Every participant of a dataspace must agree on this value. | "AnalyticsOrchestration" | |
channel.serviceId | String | Identifier of the DID service entry this application registers for itself | "#analytics-orchestration" | |
channel.registerDidService | Boolean | Register the inbox as a DID service on the wallet at startup. Disable when the DID document is managed out of band. | true | |
channel.tokenTtlSeconds | Unknown | Lifetime of an outgoing request envelope, seconds. Inbound envelopes living longer than this are rejected. | 300 | |
channel.maxBodyBytes | Unknown | Maximum accepted size of an inbound governance message, bytes | 262144 | |
channel.requestTimeoutMs | Unknown | Timeout of a single outgoing governance request, milliseconds | 10000 | |
channel.maxRetries | Unknown | Number of times an outgoing governance request is retried before it fails | 3 | |
channel.retryDelayMs | Unknown | Delay before the first retry of an outgoing request, milliseconds. Doubles with every further attempt. | 500 | |
channel.allowedParticipants | String | Participants whose messages are accepted. Leave empty to accept any participant the wallet can verify; set it to close the deployment. | ||
DirectoryConfig | ||||
directory | DirectoryConfig | Counterparty endpoint discovery | ||
directory.cacheTtlMs | Unknown | How long a resolved counterparty endpoint is reused before the registry is consulted again, milliseconds | 300000 | |
DirectoryEntryConfig | ||||
directory.staticEndpoints | DirectoryEntryConfig[] | Static counterparty endpoints, taking precedence over discovery | ||
directory.staticEndpoints[].id | Yes | String | Participant identifier (DID) this entry applies to | |
directory.staticEndpoints[].orchestrationEndpoint | String | Base URL of the counterparty's orchestration inbox | ||
directory.staticEndpoints[].dspAddress | String | Base address of the counterparty's DSP endpoints | ||
ParticipantConfig | ||||
participants | ParticipantConfig[] | Counterparties this application can run algorithms with. Discovery through project agreements and the registry replaces this list; it remains as a fallback for deployments without an agreement layer. | ||
participants[].id | Yes | String | Participant identifier (DID) of the counterparty | |
participants[].dspAddress | Yes | String | Base address of the counterparty's DSP endpoints, without a trailing path | |
participants[].orchestrationDatasetId | Yes | String | Dataset of type tsg:analytics that grants the right to submit algorithms | |
participants[].datasetIds | String | Datasets of type tsg:analytics-data the submitted algorithm may read | [] | |
participants[].role | Yes | String | Algorithm role assigned to this participant | |
OrchestrationDatasetConfig | ||||
orchestrationDataset | OrchestrationDatasetConfig | The orchestration dataset this application owns and publishes | ||
orchestrationDataset.datasetId | String | Catalog identifier of this participant's orchestration dataset | "analytics-orchestration" | |
orchestrationDataset.title | String | Catalog title of that dataset | "Analytics orchestration" | |
orchestrationDataset.description | String | Catalog description of that dataset | "Submit federated analytics algorithms to this participant" | |
PollingConfig | ||||
polling | PollingConfig | Event polling configuration | ||
polling.intervalMs | Unknown | Delay between two poll cycles of a participant, milliseconds | 1000 | |
polling.pageSize | Unknown | Maximum number of events fetched per poll | 100 | |
polling.waitMs | Unknown | Long-poll window offered to the data plane, milliseconds | 10000 | |
polling.runTimeoutMs | Unknown | Maximum run duration from start through completion, milliseconds | 21600000 | |
ProjectsPollerConfig | ||||
projectsPoller | ProjectsPollerConfig | Project-agreement protocol recovery sweep | ||
projectsPoller.intervalMs | Unknown | Delay between sweeps that retry non-terminal project agreements, milliseconds | 45000 | |
SeedConfig | ||||
seed | SeedConfig | Runnable-sample dataset and project-agreement seeding | ||
seed.enabled | Boolean | Seed the datasets and projects below at bootstrap | ||
PublishDataDatasetDto | ||||
seed.datasets | PublishDataDatasetDto[] | Data datasets to publish idempotently at bootstrap | ||
seed.datasets[].sourceId | Yes | String | Source ID from this orchestration app's configured sources | |
seed.datasets[].path | Yes | String | Path within the source; no bytes are uploaded | |
seed.datasets[].mediaType | Yes | String | Media type of the object the dataset points at | |
seed.datasets[].metadata | Unknown | Operator-reviewed metadata, optionally suggested from the data | ||
seed.datasets[].datasetId | Yes | String | Catalog identifier of the dataset | |
seed.datasets[].title | Yes | String | Catalog title of the dataset | |
seed.datasets[].description | String | Catalog description of the dataset | ||
SeedProjectConfig | ||||
seed.projects | SeedProjectConfig[] | Project agreements to create (or receive), sign and link the datasets above to | ||
seed.projects[].id | Yes | String | Business identifier of the seeded project agreement | |
seed.projects[].title | Yes | String | Title of the seeded project agreement | |
ProjectAgreementParticipant | ||||
seed.projects[].participants | Yes | ProjectAgreementParticipant[] | Participants of the agreement, in order. The first entry's participant creates it; every deployment seeding this project id must configure the same participants in the same order. | |
seed.projects[].participants[].title | Yes | String | undefined | |
seed.projects[].participants[].didId | Yes | String | undefined | |
seed.projects[].autoSign | Boolean | Sign the agreement automatically on receipt instead of waiting for an operator. Only applies to a participant seeded with this project id. | ||
seed.projects[].linkDatasets | String | Ids of this participant's own seeded datasets to link to the project once it reaches finalized | [] | |
seed.intervalMs | Unknown | Delay between sweeps that resume seeding while a peer or a project is not yet ready, milliseconds | 5000 | |
AuditModuleConfig | ||||
audit | AuditModuleConfig | Audit logging configuration | ||
audit.enabled | Boolean | Enable audit logging | true | |
audit.minSeverity | "debug" | "info" | "warning" | "error" | "critical" | Minimum severity level to log | "info" | |
audit.logDenied | Boolean | Always log denied access attempts regardless of action type | true | |
audit.logDelegated | Boolean | Log delegated access attempts when enabled, while still respecting action-specific success filters | true | |
audit.logMutations | Boolean | Log successful mutation actions (create, update, delete, manage) | true | |
audit.logExecute | Yes | Boolean | Log successful execute actions | |
audit.logReads | Yes | Boolean | Log successful read actions — disabled by default as reads are frequent and rarely relevant for auditing | |
audit.sensitiveResources | Unknown | Resources that are always logged regardless of action-specific success filters | ["w.key","w.credential","sso.user"] | |
AuditHandlersConfig | ||||
audit.handlers | AuditHandlersConfig | Handler configuration | ||
audit.handlers.console | Yes | Boolean | Enable console audit log handler | |
audit.handlers.database | Boolean | Enable database audit log handler | true | |
audit.handlers.otlp | Yes | Boolean | Enable OTLP audit log handler | |
OtlpAuditHandlerConfig | ||||
audit.otlp | OtlpAuditHandlerConfig | OTLP audit log handler configuration | ||
audit.otlp.url | String | OTLP HTTP logs endpoint URL | "http://localhost:4318/v1/logs" | |
audit.otlp.headers | Object | Additional HTTP headers to include in OTLP requests | ||
audit.otlp.timeoutMillis | Unknown | OTLP export timeout in milliseconds | 5000 | |
audit.otlp.concurrencyLimit | Unknown | Maximum number of concurrent OTLP export requests | 1 | |
audit.otlp.serviceName | String | Service name included on OTLP log attributes | "tsg-dsp-api" | |
audit.otlp.serviceVersion | String | Optional service version included on OTLP log attributes |