The backend of the new portal, which replaces the Tractus-X portal: companies join a data space and
manage their membership. This is the hackathon scaffold: a running installation as a modular
monolith (Python 3.12, FastAPI), built
from the module specs in portal-backend-spec.
Every module of the spec is already mounted, as a stand-in. A team replaces one folder under
modules/ with its implementation; the rest of the installation keeps working, and demo.py
shows at once whether the flows still run end to end.
Prerequisites: uv, git and Docker with Compose.
git clone https://github.com/tx-arena-events/portal-backend-spec.git spec # read-only, not committed
uv sync # Python 3.12 and the dependencies (behind a TLS proxy: UV_NATIVE_TLS=1 uv sync)
uv run pytest -q # all tests, about 5 seconds
docker compose up -d # Keycloak, the test Portal IAM of C2-M5; ready after about a minute
uv run python demo.py # starts the CSP stand-in, OSP A and OSP B, runs scenarios 1 to 4
demo.py prints one line per check and a summary; exit code 0 means everything is green.
Before the day, uv run python check.py checks the whole setup (tools, repositories, network,
Docker images, Keycloak, ports, demo.py) and lists what is still to do.
Only some scenarios: uv run python demo.py --scenarios 1,2,3.
After every git pull, recreate Keycloak so that it imports the current realm:
docker compose up -d --force-recreate. An older realm stops the installations at start
(“Keycloak runs an older realm than …”).
Teams of C2-M2 and C2-M3 need Docker with Keycloak: both modules call its Admin API, which the
in-memory IAM of the tests does not have. With one of them in the repository, the in-process
scenario tests are skipped; demo.py against Keycloak is the check.
Keep the three processes running for your own calls (UI, curl, Swagger at /docs):
uv run python demo.py --serve # CSP stand-in :8100, OSP A :8000, OSP B :8001; Ctrl-C stops them
uv run python iam/test/get-token.py erika # token of a test person
uv run python demo.py --no-start --scenarios 1,2,3 # in a second terminal
start_processes() in demo.py shows the environment of each process (keys, trust list, CSP
addresses). One installation alone, with the built-in CSP stand-in of C3-M2 (no CSP_URL):
IAM_URL=http://localhost:8080/auth SETTINGS_FILE=config/osp-a/installation.json uv run uvicorn --factory scaffold.app:create_app --port 8000.
Without Docker, uv run python demo.py --small-iam runs on the in-memory IAM of the scaffold’s own tests;
there is no sign-in through the browser and no Admin API, so C2-M2 and C2-M3 cannot be built against it.
Every commit is a contribution to Eclipse Tractus-X. Before the day:
Author field of your commits (git config user.email).team/<n> in this fork. The UI team does the same with
dataspace-mgm-ui.gh repo fork eclipse-tractusx/dataspace-mgm-services --default-branch-only --clone # one team member; the others: git clone <fork URL>
git checkout -b team/<n>
git push -u origin team/<n>
On the day: commit and push to team/<n> of the team fork. In the merge window at 14:45 (the only one; code freeze
after it) the team opens one pull request from <fork-owner>:team/<n> to main of
eclipse-tractusx/dataspace-mgm-services;
the ECA check runs on every commit author and a Tractus-X committer merges. No ECA, no merge. main of the fork stays
equal to upstream. From the command line:
gh pr create --repo eclipse-tractusx/dataspace-mgm-services --base main --head <fork-owner>:team/<n>.
The user interface lives in dataspace-mgm-ui.
| Folder | Content |
|---|---|
portal_common/ |
shared building blocks: token validation (tokens), service_token(module), Problem Details (problems), installation settings (settings), queries to neighbours with the error rule (neighbours), event relay (events), access to the CSP with private_key_jwt (csp), spec access (spec), spec-driven stand-ins (standin) |
modules/<module>/ |
one folder per module of the spec (interface.py, router.py, see Module structure); c12_m1 is Process Orchestration (provided), all others are stand-ins |
controllers/ |
APIs of the installation outside the modules, laid out as in the Industry Core Hub backend: one folder per API category and version (controllers/fastapi/routers/<category>/v1/<name>_api.py), mounted under /api/v1. First: the event inbox POST /api/v1/events/inbox (FRAMEWORK 6) |
scaffold/ |
the installation app (create_app) and the module registry |
iam/ |
the test Portal IAM: Keycloak with the realm of C2-M5, test persons and a stand-in for the EBW-Connector (see iam/README.md) |
ebw/ |
for tests with a real wallet app on a phone: EBW-Connector (C5-M1), Business Wallet, test issuer and OSP 2.0 demo portal; runs on its own (see ebw/README.md) |
tools/ |
test tools: access to the Keycloak test installation (keycloak_login, keycloak_admin, iam_backend), in-memory IAM of the scaffold’s own tests (test_iam), shared CSP stand-in (csp_standin, own process), EBW-Connector double for attestations (ebw_signer), provider system double (provider_double), keys |
config/osp-a, config/osp-b |
per installation: settings (installation.json, FRAMEWORK 4.8), configuration variables of the modules (module-config.json, e.g. ATTESTATION_CLIENT_IDS, ISSUER_MODE; the process environment wins), clients at the EBW-Connector (ebw-clients.json, C5-M1 6.8) |
testdata/ |
one test data package for all modules, IDs as in FRAMEWORK 7a (see Test data) |
demo.py |
acceptance demo, HTTP only, with content checks |
tests/ |
tests of every part, including scenarios 1 to 3 in one process |
iss of the own
installation, aud portal-core. A token of another installation gives 401.tokens.require_spec(module, method, path) takes the permission
from x-permission in openapi.yaml. When the spec renames a permission, nothing changes in code.tokens.service_token("c3-m1") returns the token of the service account
svc-c3-m1 with every step and query permission of the spec. Process Orchestration and
neighbours.query() use it for calls over HTTP; calls in the same process carry no token.problems.route_class(module): validation 400 (not 422),
unhandled errors 500 as Problem Details, steps always with retryable.neighbour-rejected, retryable: false; 5xx, timeout,
no connection -> 502 upstream-error, retryable: true. In the same process, the Problem
raised by the neighbour’s interface function counts with its status, as over HTTP.id, source
urn:<module>, type (the name of the event), time, datacontenttype and the content in
data. events.publish(MODULE, "<Type>", {data}) puts it into the outbox and returns it at
once; a relay thread delivers it to every on_event(event), which reads event["type"] and
event["data"]. A failing subscriber does not stop the others. Events with secret fields are
refused. GET /scaffold/events?type=<Type> lists what was published (eventType= is still
accepted).
A split-out module delivers its events over HTTP to the event inbox POST /api/v1/events/inbox
(application/cloudevents+json, or binary mode with ce- headers; its own service token).sse.Broker per
module with a replay buffer in memory (SSE_REPLAY_SIZE), sse.response(...) for the
text/event-stream answer: Last-Event-ID, heartbeat (SSE_HEARTBEAT_SECONDS), end at token
expiry. Token in the Authorization header only; the server part of the UI opens the stream.<MODULE>_URL (e.g. C3_M1_URL), else
MODULE_URLS (JSON), else MODULE_URLS_DEFAULT (the scaffold sets in-process). in-process
calls the interface function of the module directly, an HTTP URL calls over HTTP with the
service token, empty means the built-in stand-in of the caller.Binding convention for every module folder modules/<id>/ (id like c1_m2):
interface.py holds the public functions of the module. Other modules import only from there,
e.g. from modules.c1_m2.interface import get_application_internal; never a neighbour’s router,
service or data.create-company -> create_company), a query by
its operationId in snake_case (getApplicationInternal -> get_application_internal).def create_company(input: dict) -> dict; input is the JSON body of
POST /steps/<step> (with processId, stepRunId), the return value is the step result
{stepRunId, state, result} exactly as the HTTP response. A query is
def get_application_internal(**params) -> dict | list; the keyword arguments are the path and
query parameters by their names in openapi.yaml, the return value is the JSON body. No token
in the same process (FRAMEWORK 4.2).portal_common.problems.Problem (type, status, retryable). The HTTP layer
turns it into Problem Details; a caller in the same process receives the exception.router.py is a thin HTTP layer: it checks token and permission and calls the module’s own
interface functions.neighbours.query(caller, target, "query_company",
companyId=...) calls modules.c3_m1.interface.query_company for in-process, otherwise HTTP
with the service token, or raises 502 neighbour-not-configured when the address is empty.c3-m1.create-company of flows.yaml as
modules.c3_m1.interface.create_company(input) and a lookup c8-m1.getOfferInternal as
get_offer_internal(**params) when the module is in-process, otherwise over HTTP.__getattr__ = standin.missing_interface(MODULE) at
the end of interface.py adds every step and query the module does not define yet (steps answer
DONE with the stand-in results of C12-M1 11.3, queries 501), so a team replaces a stand-in
folder and nothing else changes.tests/test_interfaces.py checks that every interface has a function for every step and query
(/steps, /internal) of its openapi.yaml and that modules import each other only through
modules.<id>.interface.
modules/<module>/interface.py and router.py (or start from them) and
write the module there; further files of the module (service, data) stay inside the folder.interface.py provides the steps and queries (see above). router.py provides router (an
APIRouter under /api/<module>), reset(), optionally on_event(event) and on_startup().
Keep __init__.py empty.reset() or at call time, never at import.uv run pytest -q && uv run python demo.py before every merge. GET /scaffold/modules shows
which modules are stand-ins.Team forks from before the CloudEvents change: producers keep working: the flat form
events.publish({"eventType": "...", ...}) still publishes, with a DeprecationWarning and one
warning per module and type in the log; change it
to events.publish(MODULE, "<Type>", {...}). Consumers must change: on_event(event) reads the
name from event["type"] and the fields from event["data"], no longer event["eventType"] and
top-level fields.
The test IDs, test persons and the start state of a hackathon installation come from
FRAMEWORK 7a;
the files under testdata/ follow it, and entries that 7a does not define say
“scaffold-only, not in FRAMEWORK 7a” in _comment.
OSP Admin), Rolf and Renate (OSP Reviewer) are OSP staff without a company
(testdata/persons-osp-a.json). Scenario 1 binds Erika as Company Admin.c0000001-0000-4000-8000-000000000001
and the BPN BPNLIUK2REMLK3F4 in scenario 1; Beispiel Logistik GmbH (c0000002-…, without Business
Wallet, COMMERCIAL_REG_NUMBER HRB 200002) is in testdata/applicant-companies.json for the path with
supporting documents, without a demo scenario.osp-a-ui and osp-b-ui, each registered in
config/osp-*/ebw-clients.json and allowed in ATTESTATION_CLIENT_IDS. The verification attestation
carries it as client_id; C1-M2 refuses an attestation for another client (demo-osp).Company Admin of the OSP company, app provider in scenario 3), and for
OSP B, for which 7a defines nothing, Gerd Beispiel, Rita Zweit, Beispiel Werkzeugbau GmbH and the OSP B
company 33333333-….The Portal IAM is the Keycloak in iam/, started with docker compose up -d: one Keycloak with the realm
CX-Central for OSP A and CX-Central-B for OSP B. With IAM_URL set, an installation validates tokens
against it, and the stand-ins of C2-M2, C2-M3 and C11-M1 use it through tools/iam_backend. Teams of C2-M2
and C2-M3 call its Admin API as their specs describe. Every start of an installation and /scaffold/reset
restore the start state of its realm. TEST_TOOLS=false leaves all test tools out and never touches the realm.
Without IAM_URL, an installation uses the in-memory IAM of tools/test_iam.py; pytest runs on it.
ebw/ holds the EBW-Connector (C5-M1), a Business Wallet in the browser, a test issuer and a small OSP 2.0
demo portal, for trying sign-in and onboarding with a real wallet app on a phone. It runs on its own, with its
own virtual environment, and needs nothing else from this repository: see ebw/README.md.
The scaffold does not use it: the modules and demo.py work with the stand-in TEST-EBW of iam/ and
with tools/ebw_signer.py.
At the hackathon no real credentials are issued: OSP A and OSP B both run on the route STUB of C4-M1
(C4-M1 11.6, 14). OSP A has ISSUER_MODE OWN with empty ISSUER_ROUTES, OSP B ISSUER_MODE CSP
with empty CSP_ISSUER_URL (config/osp-*/module-config.json). OSP B keeps the dataspace wallet of the CSP.
The path through the issuer of the CSP (route CSP_ISSUER, C4-M1 11.5) stays available for the demo at
the Community Days and is not used at the hackathon (OP-53): uv run python demo.py --csp-issuer
points CSP_ISSUER_URL of OSP B to the issuer API of the CSP stand-in, and scenario 4 then checks the
credentials in the CSP wallet and the callbacks of the CSP issuer.
Runs every flow of flows.yaml of the spec (FLOWS_FILE replaces it). At start-up it checks the
flows against all openapi.yaml files and event tables of the spec and refuses to start on an
unknown step, lookup or event, a source that is not earlier, or a required input without a
source (README 6.2; FLOWS_CHECK=warn turns the last two into warnings). Lookups follow README
6.8 (operationId, field paths, object mappings, join); a failed lookup is an error of the step.
With an empty module address, the built-in stand-in answers with the results of README 11.3.
Processes are kept in memory.
Live state for the UI (README 7.6 to 7.8): GET /api/c12-m1/processes/{processId}/events for OSP
staff and GET /api/c12-m1/applications/{applicationId}/events for the applicant, event
process-changed with the full state of the process. The stand-in of C11-M1 streams its inbox at
GET /api/c11-m1/notifications/events. Try it:
TOKEN=$(uv run python iam/test/get-token.py erika)
curl -N -H "Authorization: Bearer $TOKEN" localhost:8000/api/c11-m1/notifications/events
These points are not (yet) fixed by the spec; the scaffold decides as follows:
svc-<module> get all step and query permissions of the spec (like
svc-test-internal in C2-M5 14.2), not only those of section 11 of the caller.POST {identityApiUrl}/participants/{participantContextId}/credentials/request
with header x-api-key and body {issuerDid, holderPid, credentials: [{type}]}. The spec refers
to the identity interface of the IdentityHub without naming path and body./v1/directory/{bpn}, /v1/discovery/{bpn}, /v1/trust-list) and the
test endpoints /admin/... are not in the spec.testdata/agreements.json) maps agreementKey + version (C1-M2) to
agreementId (C3-M1).companyId;
the stand-in of C1-M2 takes the 7a ID for Beispiel Maschinenbau GmbH, so that the IDs of 7a appear
in the flows. demo.py does not depend on it. The app provider of scenario 3 is Pia, not Otto:
Otto, bound to the OSP company as in C2-M2 7.20, would approve his own offer (C8-M1 6.2, 403
own-offer; C8-M1 open question 5).POST /internal/..., e.g. C3-M1 findIdentifierMatches, C2-M3
rotateSubscriptionCredentials): the call convention names only path and query parameters; the
scaffold passes the fields of the body as further keyword arguments.Problem whatever the address of the neighbour.Apache License 2.0, see LICENSE. Third-party packages are listed in DEPENDENCIES; their status must be checked with the Eclipse Dash license tool before a release. JSON files cannot carry a licence header; they are covered by the licence of the repository.