Multiple organizations

← xrmverse documentation

The model is 1 organization = 1 database = 1 xrmd process. An installation generated by xrmd server init runs any number of organizations on one SQL Server: each has its own database, its own xrmd-<org> compose service, host port, administrator, token audience and CLI profile. Several organizations in one process are not supported.

xrmd server init --dir ./xrmd-server --org acme --org zenith   # acme on 5001, zenith on 5002
xrmd sdk install --dir ./xrmd-server                           # once per installation, if init did not install it
xrmd server up --dir ./xrmd-server
export XRMD_DEV_SIGNING_KEY=$(grep '^AUTH_DEV_SIGNING_KEY=' ./xrmd-server/.env | cut -d= -f2)
xrmd auth status --profile acme                 # WhoAmI on acme
xrmd server org add orbit --dir ./xrmd-server  # 5003; starts only xrmd-orbit
xrmd server status --dir ./xrmd-server             # one row per organization
xrmd server org remove orbit --dir ./xrmd-server   # stops and removes xrmd-orbit; the database is kept

Organization names. A name must match ^[a-z][a-z0-9]{0,31}$ and must not be master, tempdb, model or msdb; anything else is exit 2 with the rule (upper case is refused, never lower-cased). The name is used verbatim as the database name, the service name xrmd-<org>, Seed:OrgName and the CLI profile name. Without --org, server init creates one organization named xrmd.

Ports. --port is the first organization's host port (default 5001); every later organization takes the next port above the highest one in use. server org add --port <n> chooses it; a port already used by another organization is exit 2. Ports are bound to 127.0.0.1 unless XRMD_BIND=0.0.0.0 is set in .env. With the generated proxy (see Host names) the organizations publish no host port unless EXPOSE_PORTS=true; ORG_<ORG>_PORT stays in .env either way.

.env is the installation. server init, server org add and server org remove write .env first and then re-render docker-compose.yml from it; server up and server upgrade re-render it too, so a hand edit of .env is what runs. Never edit docker-compose.yml: it starts with # Generated by xrmd and is overwritten. A compose file without that header is never overwritten (exit 5; server init --force replaces it). The keys:

An organization listed in XRMD_ORGS without its keys, keys of an organization that is not listed, a duplicate port, a non-GUID administrator or an invalid name make every server command that reads the installation exit 5 with a message naming the key; nothing is rendered.

Adding and removing.

• server org add <name> [--admin-object-id <guid>] [--port <n>] adds the organization to .env, re-renders the compose file and runs docker compose up -d xrmd-<name>: only the new service starts, the others keep running. The database is created and seeded by the new process on its first start. --admin-object-id is required with Entra (dev default 00000000-0000-0000-0000-000000000001).
• server org remove <name> runs docker compose rm -s -f xrmd-<name>, removes the organization from .env and its CLI profile (a current profile on it moves to the first remaining organization). The database is kept. --drop-database also drops it, through DELETE /admin/orgs/<name> sent with the profile of the first remaining organization, after a confirmation (--yes skips it) asked before anything changes. An organization that is not part of the installation is exit 4; removing the last one is exit 5 (use xrmd server down, with --volumes to delete the data).
• org create <name> (Admin API) still creates a database, and now says that no process serves it and prints the xrmd server org add <name> --admin-object-id <guid> command that serves it (in JSON, a hint property).
• Legacy installations. Installations created before multi-organization support (a single xrmd service on Database=xrmverse, no XRMD_ORGS) are refused by server org add|remove with exit 5; server up|down|upgrade|logs keep working on them unchanged. To move one to the multi-organization layout, in its directory:
  1. xrmd org backup xrmverse while it runs (note the backup file).
  2. xrmd server down --dir <dir>, without --volumes: the SQL Server data and the backups stay.
  3. xrmd server init --org setup --force --dir <dir>. --force over an existing .env keeps its SA_PASSWORD and AUTH_DEV_SIGNING_KEY (and says so): SQL Server sets the sa password only when its data volume is first created, so a new password would lock the services out.
  4. xrmd server up --dir <dir>: up and upgrade run docker compose up -d --remove-orphans on a multi-organization installation, so the old xrmd container is removed instead of serving on.
  5. xrmd org restore <name> --file <backup file> (through the setup profile, now the current one), with <name> the old SEED_ORG_NAME (default xrmd) so that the seeded organization matches. The restore goes to a database no process serves yet: a restore over a served database is refused (409).
  6. xrmd server org add <name> --dir <dir> serves it; then xrmd server org remove setup --drop-database --dir <dir>.

Profiles and tokens. In dev mode server init and server org add create one CLI profile per organization, named as the organization (URL http://localhost:<port>/, or https://<host>/ when it has a public host, the organization's administrator as object id, key from XRMD_DEV_SIGNING_KEY); the first organization of init becomes the current profile. All organizations share one signing key, but each validates its own audience, so a token minted for one organization is rejected (401) by the others. With --auth entra no profile is created (the app registration is not known): init and org add print the xrmd auth login --profile <org> --url http://localhost:<port>/ --mode device-code … command to run, and the token audience the server expects.

Status, doctor, upgrade. With a multi-organization installation in --dir (default .), server status and server doctor query every organization through the profile named as it and print one row per organization (with -o json, an array of {org, url, health|error} or {org, url, checks|error}). Each probe gives up after 15 s; an organization without a profile, unreachable or timed out is reported in its row without stopping the others. status exits 1 unless every organization is Healthy, doctor exits 1 if any check is an error or an organization cannot be checked; stderr names them. An explicit --profile or --url queries that one server only. server upgrade restarts every organization, so it waits until every organization with a profile is healthy (5 minutes), whatever --profile says, and exits 1 naming the ones that are not, or those without a profile.

One process per database. At startup, before the migrations, xrmd takes the exclusive session applock xrmd-instance in its organization database and holds it on a dedicated connection for its whole lifetime. A second process on the same database (another service, a copied compose file, a process started by hand) fails startup with:

Another xrmd process already serves the database '<db>' (applock 'xrmd-instance' is held). Run one xrmd process per organization database.

The refused process neither migrates nor seeds, and the running one is not disturbed. If the lock connection is lost (SQL Server restart, failover, killed session), the process notices it within 10 s and takes the lock again, retrying every Startup:SqlRetryDelaySeconds up to Startup:SqlRetries times; if another process holds it by then, or the retries run out, it logs a Critical error and stops with exit code 1. It stops within about 10 s once another process holds the lock, and at most about 70 s (with the defaults) when SQL Server cannot be reached, so a process can serve without the lock for that long. DELETE /admin/orgs/<db> (org delete) and a restore over an existing database (org restore --overwrite) answer 409 The database '<db>' is served by another xrmd process; stop it first. while another process serves that database.

Shared volumes. files (file and image columns; blob keys are prefixed by the database name) and backups are shared by every organization; with the bundled SQL Server so are mssql-data and the .bak files of org backup (in backups, at /var/opt/mssql/backup in both the sqlserver and the xrmd-<org> containers).

External SQL Server. server init --sql-connection '<connection string>' uses an existing SQL Server instead of the bundled one: no sqlserver or backups-init service is generated and no SA_PASSWORD. The connection string must parse, set Server, and must not set Database/Initial Catalog (exit 2): each organization's connection string is SQL_CONNECTION plus ;Database=<org>. It is stored single-quoted in .env so that $ and # stay literal; it must not contain a single quote or a line break. The SQL Server edition, and its license, are then your own (see SQL Server editions).

• Minimum permissions of the login, at a process's startup: connect to master; CREATE DATABASE (dbcreator, or CREATE ANY DATABASE) when the organization's database does not exist yet; ALTER DATABASE … SET READ_COMMITTED_SNAPSHOT ON when RCSI is off (the database owner can; xrmd creates the database, so it owns it); then db_owner in the database (migrations and seed). The Admin API needs more for what it is used for: BACKUP DATABASE for org backup, CREATE DATABASE and RESTORE for org create and org restore, DROP DATABASE for org delete (and server org remove --drop-database).
• The generated installation uses one login for every organization (there is one SQL_CONNECTION). One SQL login per organization is possible only by running each xrmd process yourself, outside the generated compose file, with its own ConnectionStrings:Org: if a DBA creates the database beforehand with READ_COMMITTED_SNAPSHOT ON and makes that login its db_owner, the process needs neither CREATE DATABASE nor any server-level permission, and the Admin API calls above fail with SQL Server's permission error for anything outside that database.
• Backups: the .bak of org backup is written by SQL Server to Admin:BackupDirectory, a path on the SQL Server host: SQL_BACKUP_DIRECTORY in .env (rendered as Admin__BackupDirectory), default /var/opt/mssql/backup (a Linux SQL Server; on Windows set it, e.g. SQL_BACKUP_DIRECTORY=D:\Backups). The .files.tar archive is written by xrmd to the backups volume, mounted at /var/lib/xrmd/backups with Admin__FilesBackupDirectory pointing at it. org restore needs both: the .bak on the SQL Server host and the archive in that volume (or --no-files).

Known limitations.

• With Entra, xrmd auth login asks for the scope <--url>/.default. An organization whose ORG_<ORG>_AUDIENCE is not its URL (an Application ID URI such as api://…) gets a token for the wrong audience, and the server rejects it. Until the CLI can take the scope separately, the CLI works only with an organization whose App ID URI is its URL.
• UI_ENTRA_CLIENT_ID and UI_ENTRA_SCOPE (web admin sign-in with Entra) are shared by every organization, while each organization validates its own audience: the web admin signs in to the organization whose API scope UI_ENTRA_SCOPE names; for the others the server rejects the token.

Host names#

Item 2.19: each organization can have a public host, https://<org>.<domain> on port 443, as in Dataverse. server init --domain <domain> gives every organization <org>.<domain>; --public-host <org>=<host> gives one organization another host (ORG_<ORG>_HOST); server org add gives the new organization <name>.<domain>, or --public-host <host>. A host must be a lower-case DNS name with at least two labels, unique across the organizations (exit 2 otherwise, nothing written). With a host:

• Proxy. A generated Caddy service, proxy (caddy:2.11.9), terminates TLS (--tls acme|files|internal) on 443, redirects 80 to 443, and routes each host to xrmd-<org>:5000 over the compose network edge. Its Caddyfile is rendered from .env like the compose file, starts with # Generated by xrmd, and is never overwritten once that header is gone (exit 5). With --no-proxy --trusted-proxy <ip|cidr> (PROXY_MODE=external) your own proxy does this instead, and the ports are published on XRMD_BIND for it.
• Forwarded headers. Each organization honors X-Forwarded-Proto, -Host and -For only from the proxy (its fixed .10 address on edge, or TRUSTED_PROXIES) and only for its own host, so the URLs it generates (@odata.context, @odata.nextLink, the SOAP address, the file download Location, the OAuth metadata) use https://<host>. A forwarded host of another organization is ignored and logged once.
• Audience. The default audience becomes https://<host>/; ORG_<ORG>_AUDIENCE still overrides it.
• Changing organizations. server org add|remove rewrite the Caddyfile in place (the proxy reads it through a bind mount) and run caddy reload in the proxy (docker compose exec -T proxy …): the other organizations keep running. The first host starts the proxy alone (up -d); removing the last host removes it (rm -s -f proxy), but the network <project>_edge stays until docker network rm <project>_edge. When the proxy is not running (the stack is down) nothing is reloaded: the next server up starts it with the new file.
• Status and doctor. server status adds a publicUrl column; server doctor adds, per host, a dns check (the host resolves on the machine that runs the CLI) and a certificate check (what <host>:443 serves).
• Internal CA. With --tls internal, server up exports Caddy's root to <dir>/certs/caddy-root.crt and records it as CaFile in the dev profiles of the organizations with a host: the CLI trusts it for that host only.

The whole procedure, certificates, DNS and configurations for nginx, Traefik and IIS ARR are in Installing xrmverse.

Giving an existing installation its first host names. server org add --public-host on an installation without TLS_MODE exits 2 naming the key: the proxy settings come first. Two ways, after xrmd server down --dir <dir> (without --volumes):

1. Edit .env (keeps every key): set XRMD_DOMAIN=<domain> (or ORG_<ORG>_HOST per organization) and TLS_MODE (with TLS_ACME_EMAIL for acme; for files copy the pair to <dir>/certs/server.crt and server.key, mode 600). Run xrmd server up --dir <dir>: it renders the proxy and the Caddyfile. The dev profiles still point at http://localhost:<port>/, which no longer answers: re-create each with xrmd auth login --profile <org> --mode dev --url https://<host>/ --object-id <ORG_<ORG>_ADMIN_OBJECT_ID>, and with --tls internal run xrmd server up --dir <dir> once more so that they record the CA file.
2. xrmd server init --force with the host options (--domain/--public-host, --tls …). It rewrites .env from its options and keeps only SA_PASSWORD, AUTH_DEV_SIGNING_KEY, MSSQL_PID and XRMD_IMAGE. Everything else comes from the options again: pass the same --org list in the same order, --port (later organizations take the next ports, so ports that were not consecutive change), one --admin-object-id per organization (without it every organization gets the dev default 00000000-0000-0000-0000-000000000001, which is not the administrator seeded in an existing database), --auth/--tenant-id, and --sql-connection for an external SQL Server (without it the installation switches to the bundled one). ORG_<ORG>_AUDIENCE, SQL_BACKUP_DIRECTORY, UI_ENTRA_*, FILES_*, SEED_ADMIN_NAME and XRMD_BIND go back to their defaults: set them again. It re-creates the dev profiles with the public URLs.

Then create the DNS records init printed (or one per host), and with Entra change each API's Application ID URI to https://<host> (or keep ORG_<ORG>_AUDIENCE) and sign in again with --url https://<host>/.