Skip to main content
Version: Next

Data plane signaling upgrade

Upgrade the control plane, HTTP data plane and analytics data plane together. Mixed versions are unsupported.

If not already applied, the signaling migration deletes existing dataflows on all three services and marks existing REQUESTED, STARTED and SUSPENDED DSP transfers as TERMINATED. Existing flow credentials stop working. Completed transfer history and agreements remain. Start new transfers after the upgrade. Rolling back the database schema cannot recover deleted flow state.

Providers now send one DPS start request with the transfer process ID as dataFlowId. Private backend configuration travels in metadata, and data addresses use the DSP DataAddress representation. A 202 response leaves the transfer requested until a started callback arrives. Duplicate readiness notifications do not send another DSP start. The provider persists the request and finishes its HTTP response before dispatching DPS start, including for planes that send an early callback. A failed, malformed or lost start response terminates the transfer and triggers best-effort cleanup. Data-plane startup failures are reported through DSP termination after request acceptance. Failed completion notifications are retried when the plane repeats its completed callback.

Only the party that suspended a transfer may initiate its resume. For consumer-initiated pull resume, the consumer waits for the provider's second start message with a fresh address. Data planes keep the backend configuration captured at start through suspension and resume. Consumer resume requests are recorded before dispatch so an early provider response can be distinguished from an unsolicited resume. A failed resume terminates the transfer with best-effort data-plane cleanup; start a new transfer to try again. This includes a failed DSP start message during consumer resume, even when the cause is a transient network error.

DPS start dispatch after an accepted request runs in the control-plane process once the response is sent. If that process stops in between, the transfer stays REQUESTED; terminate it and request a new transfer.

DPS TCK fixture details and exclusions are in tools/e2e-testing/lib/dps-tck/README.md. Its metadata fallback is test-only. Passing those checks establishes signaling behavior under that fixture, not token renewal or unrestricted third-party interoperability.

Worker identity and registration​

Current TSG HTTP and analytics workers require authenticated signaling and explicit canonical execution owners. A worker serving one tenant and a worker serving several tenants use the same configuration model. Neither self-registers on startup. Remove the old mode, endpoint, tenant-slug, registration-delay and registration-title fields from the worker's controlPlane configuration. Keep its stable dataplaneId and configure accepted owners explicitly:

controlPlane:
  controlplaneId: control-plane-api
  dataplaneId: shared-http-dp
  executionScopes:
    - issuer: https://sso.example.org
      controlPlaneId: control-plane-api
      tenantId: 6cc414b6-402d-4816-893e-1646f5b17c6e

Replace the example UUID with the canonical tenant ID returned by Tenant Management after provisioning. Preserve the existing worker identity when retaining its endpoint and history. Tenant slugs and participant DIDs are separate values. For analytics split clients, configure their runner assignments as described in split mode.

Configure the worker registration under the existing managed control-plane profile:

tenancy:
  profiles:
    default:
      # Retain the profile's existing IAM and catalog settings.
      dataPlanes:
        - dataplaneId: shared-http-dp
          endpoint: https://plane.example/api
          healthEndpoint: https://plane.example/api/health
          profiles: [https://w3id.org/dspace-sig/profile/http-pull]
          executionLease: true
          title: HTTP worker

Use the actual worker API root and health URL for your deployment. Provisioning creates a separate registration in each tenant, even when all point to one worker. An analytics registration advertises its supported analytics profiles. Configure Tenant Management's worker target and the matching SSO managedSignalingTargets so the tenant's existing control-plane workload credential can signal that worker. See signaling authentication.

The registration migration removes old registrations and dataset links. After that migration, use managed provisioning or its configuration reconciliation operation to apply the profile, then relink or republish datasets. Restarting a worker does not create a registration. Catalog records remain but cannot start transfers until linked again.

Drain legacy unowned transfers before cutover. Current workers cannot serve flows without verified immutable execution ownership and an active execution lease. Keep existing databases and signing keys; do not merge histories or assign unowned rows to a guessed tenant. Start new transfers through the tenant-local registrations.

Third-party registrations​

The control plane also supports explicitly configured third-party registrations through operator-owned tenancy.profiles.<profile>.dataPlanes, applied during managed provisioning. These entries do not independently provision the canonical authorization required by a TSG worker. Historical startup tenant seeds and CLI participant data-plane fields are no longer supported.

The registration API remains available at PUT /tenants/:tenantSlug/dataplanes and DELETE /tenants/:tenantSlug/dataplanes/:dataplaneId, relative to the control plane's API root. A PUT supplies the full registration. It contains dataplaneId, endpoint, profiles, and optional labels, title, healthEndpoint and private authorization. Authenticated callers may register or remove only their own plane identity. API registrations have source api; configured registrations have source config. Health stays unknown when no health endpoint is configured.

A private authorization object can select a separate signaling OAuth client for a third-party integration. Managed TSG registrations use the tenant-bound credential supplied through provisioning. Registration does not establish receiver issuer trust. See signaling authentication for claims, credentials and callback ownership.

Profiles and HTTP credentials​

The supported formats are https://w3id.org/dspace-sig/profile/http-pull, https://tsg.dataspac.es/profile/analytics-execution, and https://tsg.dataspac.es/profile/analytics-data. Unknown profiles and the former tsg:HTTP, tsg:analytics and tsg:analytics-data aliases are rejected. Catalogs publish canonical values. Analytics execution supplies a provider execution endpoint for control requests and application-provisioned bidirectional events. Its neutral name intentionally deviates from the DPS push/pull suffix rule. Analytics data grants access without an endpoint.

HTTP addresses carry a JWT in authorization and authType: bearer, with expiry and refresh properties. Applications should call sdk.transfers.dataAccess(transferId) before sending data requests to obtain current credentials. Management transfer responses omit endpoint properties; dataAccess is the only way to obtain the credentials. See HTTP dataflow execution for renewal and lifecycle behavior.

HTTP JWT startup requires a persistent signing key. The CLI preserves its signing Secret; Helm mounts the Secret named by dataAccessKeySecret. Keep the key and database across restarts, and run one HTTP data-plane replica. Legacy unowned or opaque-token flows do not establish current execution authority. Start new tenant-owned transfers after the upgrade.