Deployment, migration and recovery

Bootstrap

Deploy SSO Bridge, Wallet, Control Plane and tenant-management-api with their additive database migrations. The Dockerfile-tenant-management image includes the UI; helm-charts/tenant-management follows the shared TSG chart conventions. Native production APIs use their root URL, while embedded Docker APIs use /api. Register the actual receiving API base, including any deployed prefix.

Provision two separate SSO clients before tenant creation:

• A shared public browser client for the manager, with authorization_code, PKCE, exact callback URL and tokenEndpointAuthMethod: none. The manager sets tenancy.mode: authority; its global identity flow requests only openid profile email. Wallet/Control Plane shared browser clients also need their service audiences, bounded permissions and refresh support.
• An operator-controlled platform client with client_credentials and receiving audiences for SSO, Wallet, Control Plane and any additional service. This is a service actor. Configure it under manager provisioning, independently of the public browser client. Use a secret reference or private-key client authentication rather than committing credentials in YAML.

On each consumer register the exact manager issuer/subject, receiving instance, audience and operations. For example, a Wallet registration is:

auth:
  enabled: true
  expectedAudience: wallet-api
  clientId: wallet-browser
  openIdConfigurationURL: https://sso.example/.well-known/openid-configuration
  callbackURL: https://wallet.example/api/auth/callback
  redirectURL: https://wallet.example/
tenancy:
  mode: managed
  serviceInstanceId: wallet
  managementUrl: https://tenants.example/
  controlPlaneUrls: [https://cp.example/api]
  provisioningIdentities:
    - principal: { issuer: https://sso.example, subject: tenant-management }
      serviceInstanceId: wallet
      audience: wallet-api
      operations: [configuration, status, authorization, lifecycle]

SSO uses its issuer as receiving audience and its own instance ID. Control Plane uses its API audience and additionally registers tenancy.walletUrls, tenancy.tokenEndpoints and tenancy.profiles. Its named profile contains the operator-owned IAM/data-plane/catalog options. Runtime DID, Wallet endpoints and tenant workload credentials come from manager provisioning, not initTenants. Wallet creation supports the default profile with generated keys; shared imported key/certificate or credential/holder seeds are rejected for new tenants.

Catalog datasets in a Control Plane profile are templates. Provisioning assigns stable tenant-specific IDs to their datasets, distributions and access services, including references between them. Read the published catalog for the resulting IDs. Imports preserve the existing catalog and its identifiers.

Manager sso, services and profiles register destinations and grant ceilings; tenants cannot submit arbitrary URLs or secrets. Set an optional uiUrl on each Wallet or Control Plane entry to enable the manager's Open Wallet and Open Control Plane actions. Use the browser root (including its deployed subpath), for example https://gateway.example/wallet/; the UI passes the canonical tenant ID through the normal tenant selection and authorization flow. Services without uiUrl remain visible without a launch action. Register platformPrincipals for operator provisioning and administration; use creators only for explicitly allowed self-service creation. Keep selfServiceCreation: false for hosted deployments. Exact human tenancy.platformPrincipals separately authorize shared consumer settings. Ordinary services and organization administrators cannot use provisioning or platform routes.

Allow the manager's SSO provisioning identity to perform the account operation when it should create or resolve login accounts and issue first-login setup links. Configure SSO tenancy.trustedBrowserClients with the exact first-party manager, Wallet and Control Plane browser client IDs to enable automatic SSO continuation. Account creation, shared browser authorization, and service grants remain separate capabilities. An organization role does not grant global SSO account authority.

Provide stable SESSION_SECRET and a TENANCY_SECRET of at least 32 characters through environment/secret references to manager, SSO, Wallet, Control Plane and browser-facing data-plane workers. Each service's replicas must share its secrets and database. TENANCY_SECRET encrypts server-held OAuth/workload credentials; preserve it with database backups. Replacing it without re-encrypting stored ciphertext prevents recovery. Use HTTPS public addresses; session cookies are HttpOnly, SameSite=Lax and secure for HTTPS. Configure the trusted ingress to set the forwarded protocol correctly. The manager itself uses global identity authorization (tenancy.mode: authority); the managed consumer switch belongs to SSO, Wallet and Control Plane.

Matching configuration across services

For one Wallet and Control Plane pair, configure these references together. API bases include the deployed prefix, such as /api, and omit a trailing slash. The allowlists remain explicit in the local sample as well as deployed systems.

Manager value	Must match
sso.audience	SSO server.publicAddress, the token issuer.
sso.apiUrl	SSO's actual API base; Control Plane tenancy.tokenEndpoints contains this base plus /oauth/token.
services[].serviceInstanceId	The corresponding consumer's tenancy.serviceInstanceId and provisioning identity serviceInstanceId.
services[].audience	That consumer's auth.expectedAudience, provisioning identity audience, and SSO browser client's allowed audience.
Wallet services[].apiUrl	Wallet's published management API base and an entry in Control Plane tenancy.walletUrls.
Control Plane services[].apiUrl	An entry in Wallet tenancy.controlPlaneUrls.
profiles.<name>.services	Registered manager service IDs, exactly one Wallet and one Control Plane.
profiles.<name>.controlPlaneProfile	A key in Control Plane tenancy.profiles.
provisioning.clientId	Each consumer's tenancy.provisioningIdentities[].principal.subject; its issuer matches sso.audience.
auth.clientId for manager, Wallet and Control Plane	A shared SSO browser registration and an entry in SSO tenancy.trustedBrowserClients.
platformPrincipals	Exact human issuer/subject pairs for manager authority. Consumer shared settings use the same field name under tenancy; register each operator explicitly where needed.

provisioningIdentities remains shared configuration because both Wallet and Control Plane provisioning guards use it, as does SSO. Wallet connector URLs and Control Plane IAM allowlists/profiles are defined by their respective apps.

Standard-stack integration

The CLI generates the coordinated four-service stack with apiVersion: tsg.dataspac.es/v1alpha2 and stacks[].managedTenancy.adminUserId. Use tools/cli/examples/07-tenant-management.yaml as the starting configuration. It registers the browser and provisioning clients, consumer allowlists, profiles, service URLs and session/encryption secret references. The administrator UUID must remain stable after the initial deployment. Organizations are created or imported through Tenant Management after the stack starts.

Managed CLI stacks support path/subdomain routing, Ingress/Gateway API, both database modes and both confidential client authentication methods. Participant seeds are rejected. HTTP and analytics workers are supported for organizations already created or imported through Tenant Management. They require an explicit Control Plane registration, canonical tenant execution scopes and scoped OAuth credentials in the referenced Secret. Analytics also requires a dedicated job namespace, service account and allowed images. Dataspace Starter is supported as a shared browser client; optional organization choices need canonical tenant UUIDs and the Control Plane audience. See the CLI configuration prerequisites.

The CLI does not dynamically provision workers, registrations or workload credentials when organizations are created. It rejects the Gaia-X Credential Workbench and Analytics Orchestration CLI profiles. Deploy those apps separately with managed service targets, service audiences and a registered shared browser client. Workbench also needs its membership issuer; Analytics Orchestration needs its canonical tenant ID and managed workload grants.

The standalone helm-charts/tenant-management chart remains available for operator-wired deployments. helm-charts/core is the shared template library.

Configuration cutover

The current Wallet, Control Plane and SSO reject the old seeded configuration. Remove initTenants, local member/invitation seeds, authentication compatibility flags and legacy client kinds before starting the new binaries. SSO clients must explicitly declare shared, workload or platform, their permitted audiences and exact redirects. Existing client rows also require a reviewed explicit kind. There is no automatic authority inference or legacy mode.

Removing configuration does not import an existing participant. Keep its state and complete the explicit import below to preserve its DID, keys, datasets and published URLs. New tenant creation generates a new participant identity.

Import existing organizations

1. Back up all databases, configuration and encryption secrets. Inventory each relationship using evidence from existing local rows and published endpoints. Freeze legacy tenant/membership writers during cutover. Decide a canonical UUID and exactly one verified human owner; resolve ambiguous/system owners explicitly. Matching slugs alone is insufficient.
2. Record the Wallet and Control Plane local primary IDs and their separate local URL slugs, existing DIDs/keys, resource IDs, issuer/subject mappings, ownership aliases, revoked memberships and explicit grants per service. Do not combine permissions from similarly named accounts or infer email matches.
3. Prepare ImportManagedTenant JSON with tenantId, name, canonical slug, profile, memberships, grants, mappings, localTenantIds and localTenantSlugs. Mapping entries contain principal, localSubject and ownershipAliases. Imports preserve consumer local IDs, key material, DIDs, catalogs and published URLs. An existing Wallet connector service URL must match the explicitly registered imported Control Plane URL. Existing data-plane registrations absent from the managed profile become API-managed on import. Their dataset links survive credential rotation and later profile updates. Registrations listed in the profile remain config-managed.
4. Preserve pending invitations in optional invitations entries with tokenHash (SHA-256 hex of the possession proof), role, optional exact principal and future expiresAt in epoch milliseconds. Existing proofs can be retained; email-only legacy invitations require an operator-generated proof and explicit acceptance in the manager. Share those proofs separately. Invitation acceptance does not union legacy service grants. Include reviewed desired grants/mappings explicitly, or grant access after acceptance.
5. Upgrade the receiving services in a maintenance window, with unsupported startup seed fields removed and all provisioning identities registered. Existing rows remain stored, but protected access is denied until their import is ready. Submit POST /imports as an explicitly registered human platform administrator with a stable idempotency key. Poll its operation and verify local identities, public URLs, signature verification and service grants before reopening access.
6. Remove the retired independent writers and keep the reviewed import manifest with the deployment record. Activate managed browser flows only with the full manager/SSO/consumer lifecycle available.

Before cutover, rollback means restoring the backed-up databases/configuration and legacy deployment together. After managed mutations or credential rotation, do not switch individual services back to stale legacy snapshots. Freeze writes and perform a coordinated restore from a mutually consistent backup, or repair the managed deployment forward. Canonical mappings/tombstones must remain consistent. No automatic rollback or global-token isolation fallback is provided.

Wallet-only identities

A profile may bind one Wallet without a Control Plane. Omit controlPlaneProfile, leave walletPermissions empty and configure no data planes for that profile. The manager provisions the Wallet identity and SSO authorization without creating connector endpoints or Control Plane workload credentials. The Gaia-X sample uses this profile for its Notary and membership authority.

Recovery and validation

Persistent operations, receipts and outbox leases survive restart. Reuse the original idempotency key; do not reserve a replacement tenant to recover a failed operation. Check non-secret operation codes/readiness and the registered service availability/configuration. The worker retries with backoff capped at 20 seconds. Workload rotation is an owner operation and keeps Wallet keys/DIDs unchanged.

Service-grant recovery requires an exact human principal registered in platformPrincipals; an organization owner or administrator has no recovery authority solely because of that role. GET /capabilities reports isPlatformAdministrator for this operator authority. If an owner clears their own grants, or a successor receives ownership without service grants, a platform administrator can submit POST /tenants/:id/grants/recover-owner with {"revision": <current revision>}. The caller need not be a tenant member. The audited operation adds only the current profile's reviewed ownerGrants for the active owner and bound services. It preserves existing imported permissions, local principal mappings, other principals' grants and workload credentials. Deleted/deleting tenants and stale revisions cannot be recovered through this operation. Permissions beyond that profile require a separate operator-reviewed policy/configuration change; the recovery request accepts no permission list.

The CLI exposes tsg tenant --url <api-base> list|get|create|import|operation|lifecycle. Supply a human token via TSG_TENANT_TOKEN; create/import require a stable --request-id. Platform creation also needs --owner <file>, containing a new or existing account selector. TenantManagementSdk also exposes account onboarding, setup-link delivery, operation history/retry and owner-grant recovery. The CLI covers the listed automation commands; account onboarding uses the SDK or UI. A bearer credential for a consumer must be bound to that consumer's audience and canonical tenant, while its request selector remains the local tenant ID/slug. For an imported participant, SDK controlPlaneTenantSlug and walletTenantSlug preserve different service URL names; controlPlaneAudience, walletAudience and walletAuth keep each receiving service's authorization separate. The tenantSlug option remains the shared addressing default. Set auth and walletAuth explicitly; a Control Plane token is never reused for Wallet or SSO. Configure ssoAuth and ssoAudience separately when the SDK needs SSO API access.

Run the manager's focused test suite with:

pnpm --filter @apps/tenant-management-api test --maxWorkers=2

The tests cover partial-failure restart, paired snapshots, revocation deadlines, invitation proofs, ownership/grant bounds and concurrent refresh leases. Future services implement the same configuration/status/authorization/lifecycle endpoints with strict provisioning policy, ConsumerProvisioning, their own projection and local resource checks. A generic adapter is covered by the worker suite.

Internals

Each target's local subjects and ownership aliases share one unique namespace. The manager validates the complete namespace before creating or updating a tenant, including membership additions and invitation acceptance. A new canonical subject that collides with an imported local subject or alias is rejected atomically; the tenant revision and pending invitation remain unchanged. Resolve that local identity relationship explicitly before retrying rather than assigning two principals the same legacy identity.