Merge pull request #544 from lblod/hotfix/remove-suborganization-from-inactive-ckb-deinze

Remove suborganization from inactive ckb deinze

.gitignore

@@ -20,6 +20,7 @@ docker-compose.override.yml
 
 /config/migrations/private-data/*
 /config/migrations/local/*
+/config/migrations-triggering-indexing/local/*
 
 /config/proxy/nginx.htpasswd
 

CHANGELOG.md

@@ -0,0 +1,406 @@
+# Changelog
+## v1.31.3 (2025-05-08)
+### Backend
+ - Migrate active organizations linked to inactive CKB to the active one. [DL-6604]
+### Deploy notes
+```
+drc restart migrations-triggering-indexing && drc logs -ft --tail=200 migrations-triggering-indexing
+```
+
+## v1.31.2 (2025-04-02)
+### Backend
+- Extend the public producer to include: [CLBV-980]
+  - NIS codes
+  - Sites and statuses of representative organs
+  - Secondary sites with their contacts and addresses
+  - Site descriptions
+
+### Deploy notes
+```
+drc restart delta-producer-publication-graph-maintainer
+```
+
+## v1.31.1 (2025-04-01)
+### Backend
+- Removal of change events of 8 ckb's + update of their status [OP-3562]
+### Deploy Notes
+```
+drc restart migrations-triggering-indexing; drc logs -ft --tail=200 migrations-triggering-indexing
+drc restart migrations; drc logs -ft --tail=200 migrations
+```
+
+## v1.31.0 (2025-03-27)
+### Frontend
+- Bump to version [v1.30.0](https://github.com/lblod/frontend-organization-portal/blob/v1.30.0/CHANGELOG.md#v1300-2025-03-27)
+### Backend
+- Datafix: move status for municipalities and provinces to the `shared` graph such that it can be read by worship users [OP-3469]
+- Bump `triplestore` and `publication-triplestore` to `v1.3.0-rc.1`. [OP-2492] [OP-3547]
+- Add missing docker compose keys. [DL-6490]
+- Reorganize delta consumers config to harmonize with the ecosystem
+- Set up the dashboard app [OP-3103]
+- Datafix: correct date of change event of kerkfabriek st petrus [OP-3555]
+- Update OCMWv to VVMW [OP-3565] and back. It's technically a nil-operation. But you should run the migrations nevertheless.
+### Deploy notes
+```
+drc restart migrations-triggering-indexing
+drc restart migrations; drc logs -ft --tail=200 migrations
+drc pull frontend; drc up -d frontend
+drc up -d mandatarissen-consumer leidinggevenden-consumer worship-services-main-info-consumer worship-services-private-info-consumer
+```
+#### For upgrade databases
+[This README](https://github.com/Riadabd/upgrade-virtuoso) provides the necessary steps for upgrading the database. **NOTE**: This will involve shutting down the app for small period of time (around 30 minutes).
+#### For adding the missing keys
+```
+drc up -d search elasticsearch
+```
+
+#### Dashboard
+
+The dashboard requires some server specific configuration before it can be used.
+
+##### 1. Dashboard domains
+To configure the domain name that's used by the dashboard frontend, you can add it to the `VIRTUAL_HOST` and `LETSENCRYPT_HOST` environment variables of the identifier service.
+- DEV domain: dashboard.dev.app-organisation-portal.contacthub-dev.s.redhost.be
+- QA domain: dashboard.qa.app-organisation-portal.contacthub-dev.s.redhost.be
+- PROD domain: dashboard.app-organisation-portal.organisaties.s.redpencil.io
+
+##### 2. Dashboard users
+Before we can generate a user we need to add an application salt to the server. Add a `MU_APPLICATION_SALT` environment variable to the `dashboard-login` service in the docker-compose.override.yml file.
+
+Once this is done you can generate a user by running `mu script project-scripts generate-dashboard-login` and following the prompts. A new local migration will be generated which can be run by restarting the migrations service and which will insert the new user into the database.
+
+Store the login credentials as a comment in the docker-compose.override.yml file.
+
+##### 3. (Re)start the needed services
+`drc up -d frontend-dashboard dashboard-login identifier; drc restart dispatcher resource db`
+
+## v1.30.5 (2025-03-10)
+### Backend
+ - Added `http://www.w3.org/ns/prov#Location` to delta stream 'public' [DL-6496]
+### Deploy notes
+```
+drc restart delta-producer-publication-graph-maintainer
+drc exec delta-producer-background-jobs-initiator curl -X POST http://localhost/public/healing-jobs
+
+```
+
+## v1.30.6 (2025-03-25)
+### Backend
+- Flushed landingzones [DL-6552]
+### Deploy notes
+```
+drc restart migrations
+```
+
+## v1.30.5 (2025-03-10)
+### Backend
+ - Added `http://www.w3.org/ns/prov#Location` to delta stream 'public' [DL-6496]
+### Deploy notes
+```
+drc restart delta-producer-publication-graph-maintainer
+drc exec delta-producer-background-jobs-initiator curl -X POST http://localhost/public/healing-jobs
+
+```
+
+## v1.30.4 (2025-03-06)
+### Backend
+- Add missing codes to public producer [DL-6210] [DL-6449]
+#### Deploy commands
+```
+drc restart migrations delta-producer-publication-graph-maintainer
+drc exec delta-producer-background-jobs-initiator curl -X POST http://localhost/public/healing-jobs
+```
+
+## v1.30.3 (2025-03-04)
+### Frontend
+- Bump to [v1.29.1](https://github.com/lblod/frontend-organization-portal/releases/tag/v1.29.1)
+### Deploy notes
+- Bump frontend
+#### Deploy commands
+```
+drc pull frontend; drc up -d frontend
+```
+
+## v1.30.2 (2025-02-28)
+### Backend
+- Datafix: add new timed governing bodies for new legislature for AGBs [OP-3552]
+### Deploy notes
+- Restart the migration service
+- Start a healing job the public producer, or wait for it to kick in overnight
+#### Deploy commands
+```
+drc restart migrations; drc logs -ft --tail=200 migrations
+drc exec delta-producer-background-jobs-initiator curl -X POST http://localhost/public/healing-jobs
+```
+
+## v1.30.1 (2025-02-20)
+### Backend
+ - Bestuursorgaanclassificatiecode of VGC was in wrong graph, so it wouldn't export. [DL-6428]
+### Deploy notes
+```
+drc restart migrations
+```
+
+## v1.30.0 (2025-02-19)
+### Frontend
+- Bump to version [v1.29.0](https://github.com/lblod/frontend-organization-portal/releases/tag/v1.29.0)
+### Backend
+- Datafix: delete duplicate identifiers [OP-3157]
+- Feature: replace different relations between organisations by memberships [OP-3195 (epic), OP-2606, OP-3156, OP-3198, OP-3199, OP-3258, OP-3259, OP-3260, OP-3265, OP-3293, OP-3305]
+- Added `db-cleanup-service` to keep membership data backwards compatible [OP-3369]
+- Datafix: delete incorrect change event [OP-3363]
+- Feature: re-enable contact data editing [OP-3425]
+- Remove gemeente and provincie classifications from admin unit graph [OP-3393]
+- Remove several OCMW bcsd faciliteitgemeenten [OP-3535]
+- Completely remove AGB Stekene's data. [OP-3409]
+- Remove reasoner service, replaced by new consumer logic
+### Deploy Notes
+- Before pulling the changes
+  + Shut down the reasoner and thank it for its good service: `drc down reasoner`
+- Re-enable contact date editing:
+  + remove `EMBER_ENABLE_EDIT_CONTACT_DATA_FEATURE` flag in `docker-compose.override.yml`
+  + Stop and remove the `clb-contact-data-consumer` service
+  + Remove the `clb-contact-data-consumer` configuration in `docker-compose.override.yml`
+- Rebuild the `search` service before starting the re-indexing
+- To enable the membership functionality:
+  + Restart migration service
+  + Restart services for which the configuration was changed: `resource`,  `dispatcher` and `db`
+  + Start the `db-cleanup` service
+  + Pull and restart the frontend
+  + Perform a re-index using `reset-elastic.sh` script
+#### Deploy commands
+```
+drc down clb-contact-data-consumer
+drc up -d search
+drc restart migrations; drc logs -ft --tail=200 migrations
+drc restart migrations-triggering-indexing && drc logs -ft --tail=200 migrations-triggering-indexing
+drc restart resource dispatcher db
+drc up -d db-cleanup
+drc pull frontend; drc up -d frontend
+./scripts/reset-elastic.sh
+```
+
+## 1.29.2 (2025-02-17)
+### Data fix
+ - restore betrokken lokale besturen that have been accidentally flushed [OP-3546]
+
+## 1.29.1 (2025-02-14)
+### Backend
+ - re-init `worship-services-sensitive-consumer` [OP-3483]
+### Deploy notes
+:warning: Ensure first loket did a full re-sync again of this delta-stream.
+```
+drc stop worship-services-main-info-consumer worship-services-private-info-consumer
+drc retart migrations
+```
+Update `docker-compose.override.yml` to remove the config of the consumers and replace it by:
+```
+  worship-services-main-info-consumer:
+    environment:
+      DCR_SYNC_BASE_URL: "https://loket.lblod.info" # or another endpoint
+      DCR_SYNC_LOGIN_ENDPOINT: "https://loket.lblod.info/sync/worship-services-sensitive/login" # or another endpoint
+      DCR_SECRET_KEY: "key-of-the-producer"
+      DCR_LANDING_ZONE_DATABASE: "triplestore" # for the initial sync, we go directly to virtuoso
+      DCR_REMAPPING_DATABASE: "triplestore" # for the initial sync, we go directly to virtuoso
+      DCR_DISABLE_INITIAL_SYNC: "false"
+      DCR_DISABLE_DELTA_INGEST: "false"
+  worship-services-private-info-consumer:
+    environment:
+      DCR_SYNC_BASE_URL: "https://loket.lblod.info" # or another endpoint
+      DCR_SYNC_LOGIN_ENDPOINT: "https://loket.lblod.info/sync/worship-services-sensitive/login" # or another endpoint
+      DCR_SECRET_KEY: "key-of-the-producer"
+      DCR_LANDING_ZONE_DATABASE: "triplestore" # for the initial sync, we go directly to virtuoso
+      DCR_REMAPPING_DATABASE: "triplestore" # for the initial sync, we go directly to virtuoso
+      DCR_DISABLE_INITIAL_SYNC: "false"
+      DCR_DISABLE_DELTA_INGEST: "false"
+```
+Then
+```
+drc up -d db mandatarissen-consumer leidinggevenden-consumer worship-services-main-info-consumer worship-services-private-info-consumer
+# Wait until success of the previous step
+```
+Then, update `docker-compose.override.yml` to:
+```
+  worship-services-main-info-consumer:
+    environment:
+      DCR_SYNC_BASE_URL: "https://loket.lblod.info" # or another endpoint
+      DCR_SYNC_LOGIN_ENDPOINT: "https://loket.lblod.info/sync/worship-services-sensitive/login" # or another endpoint
+      DCR_SECRET_KEY: "key-of-the-producer"
+      DCR_LANDING_ZONE_DATABASE: "db"
+      DCR_REMAPPING_DATABASE: "db"
+      DCR_DISABLE_DELTA_INGEST: "false"
+      DCR_DISABLE_INITIAL_SYNC: "false"
+  worship-services-private-info-consumer:
+    environment:
+      DCR_SYNC_BASE_URL: "https://loket.lblod.info" # or another endpoint
+      DCR_SYNC_LOGIN_ENDPOINT: "https://loket.lblod.info/sync/worship-services-sensitive/login" # or another endpoint
+      DCR_SECRET_KEY: "key-of-the-producer"
+      DCR_LANDING_ZONE_DATABASE: "database"
+      DCR_REMAPPING_DATABASE: "database"
+      DCR_DISABLE_DELTA_INGEST: "false"
+      DCR_DISABLE_INITIAL_SYNC: "false"
+```
+```
+drc up -d
+sh scripts/reset-elastic.sh
+
+```
+
+## 1.29.0 (2025-02-07)
+### Backend
+#### Consumer
+- Upgraded `mandatarissen-consumer` [OP-3510]
+- Upgraded `leidinggevenden-consumer` [OP-3533], [OP-3511]
+- Upgraded `worship-services-sensitive-consumer` [OP-3337]
+
+### Deploy notes
+```
+drc down;
+```
+Update `docker-compose.override.yml` to remove the config of the consumers and replace it by:
+```
+  mandatarissen-consumer:
+    environment:
+      DCR_SYNC_BASE_URL: "https://loket.lblod.info" # or another endpoint
+      DCR_LANDING_ZONE_DATABASE: "triplestore" # for the initial sync, we go directly to virtuoso
+      DCR_REMAPPING_DATABASE: "triplestore" # for the initial sync, we go directly to virtuoso
+      DCR_DISABLE_DELTA_INGEST: "false"
+      DCR_DISABLE_INITIAL_SYNC: "false"
+  leidinggevenden-consumer:
+    environment:
+      DCR_SYNC_BASE_URL: "https://loket.lblod.info" # or another endpoint
+      DCR_LANDING_ZONE_DATABASE: "triplestore" # for the initial sync, we go directly to virtuoso
+      DCR_REMAPPING_DATABASE: "triplestore" # for the initial sync, we go directly to virtuoso
+      DCR_DISABLE_DELTA_INGEST: "false"
+      DCR_DISABLE_INITIAL_SYNC: "false"
+  worship-services-main-info-consumer:
+    environment:
+      DCR_SYNC_BASE_URL: "https://loket.lblod.info" # or another endpoint
+      DCR_SYNC_LOGIN_ENDPOINT: "https://loket.lblod.info/sync/worship-services-sensitive/login" # or another endpoint
+      DCR_SECRET_KEY: "key-of-the-producer"
+      DCR_LANDING_ZONE_DATABASE: "triplestore" # for the initial sync, we go directly to virtuoso
+      DCR_REMAPPING_DATABASE: "triplestore" # for the initial sync, we go directly to virtuoso
+      DCR_DISABLE_INITIAL_SYNC: "false"
+      DCR_DISABLE_DELTA_INGEST: "false"
+  worship-services-private-info-consumer:
+    environment:
+      DCR_SYNC_BASE_URL: "https://loket.lblod.info" # or another endpoint
+      DCR_SYNC_LOGIN_ENDPOINT: "https://loket.lblod.info/sync/worship-services-sensitive/login" # or another endpoint
+      DCR_SECRET_KEY: "key-of-the-producer"
+      DCR_LANDING_ZONE_DATABASE: "triplestore" # for the initial sync, we go directly to virtuoso
+      DCR_REMAPPING_DATABASE: "triplestore" # for the initial sync, we go directly to virtuoso
+      DCR_DISABLE_INITIAL_SYNC: "false"
+      DCR_DISABLE_DELTA_INGEST: "false"
+```
+Then:
+```
+drc up -d triplestore migrations
+drc up -d db mandatarissen-consumer leidinggevenden-consumer worship-services-main-info-consumer worship-services-private-info-consumer
+# Wait until success of the previous step
+```
+Then, update `docker-compose.override.yml` to:
+```
+  mandatarissen-consumer:
+    environment:
+      DCR_SYNC_BASE_URL: "https://loket.lblod.info" # or another endpoint
+      DCR_LANDING_ZONE_DATABASE: "db"
+      DCR_REMAPPING_DATABASE: "db"
+      DCR_DISABLE_DELTA_INGEST: "false"
+      DCR_DISABLE_INITIAL_SYNC: "false"
+  leidinggevenden-consumer:
+    environment:
+      DCR_SYNC_BASE_URL: "https://loket.lblod.info" # choose the correct endpoint
+      DCR_LANDING_ZONE_DATABASE: "db"
+      DCR_REMAPPING_DATABASE: "db"
+      DCR_DISABLE_DELTA_INGEST: "false"
+      DCR_DISABLE_INITIAL_SYNC: "false"
+  worship-services-main-info-consumer:
+    environment:
+      DCR_SYNC_BASE_URL: "https://loket.lblod.info" # or another endpoint
+      DCR_SYNC_LOGIN_ENDPOINT: "https://loket.lblod.info/sync/worship-services-sensitive/login" # or another endpoint
+      DCR_SECRET_KEY: "key-of-the-producer"
+      DCR_LANDING_ZONE_DATABASE: "db"
+      DCR_REMAPPING_DATABASE: "db"
+      DCR_DISABLE_DELTA_INGEST: "false"
+      DCR_DISABLE_INITIAL_SYNC: "false"
+  worship-services-private-info-consumer:
+    environment:
+      DCR_SYNC_BASE_URL: "https://loket.lblod.info" # or another endpoint
+      DCR_SYNC_LOGIN_ENDPOINT: "https://loket.lblod.info/sync/worship-services-sensitive/login" # or another endpoint
+      DCR_SECRET_KEY: "key-of-the-producer"
+      DCR_LANDING_ZONE_DATABASE: "database"
+      DCR_REMAPPING_DATABASE: "database"
+      DCR_DISABLE_DELTA_INGEST: "false"
+      DCR_DISABLE_INITIAL_SYNC: "false"
+```
+```
+drc up -d
+sh scripts/reset-elastic.sh
+```
+
+## 1.28.5 (2025-01-23)
+### Frontend
+- Bump to [v1.28.3](https://github.com/lblod/frontend-organization-portal/releases/tag/v1.28.3)
+### Backend
+- Datafix: correct merger change events for worship organisations (OP-3534)
+- Avoid municipality filter to include partial matches (OP-3509)
+### Deploy notes
+- `drc restart migrations-triggering-indexing; drc logs -ft --tail=200 migrations-triggering-indexing`
+- `drc pull frontend; drc up -d frontend`
+- Perform a full re-index by executing `./scripts/reset-elastic.sh`
+
+## 1.28.4 (2025-01-16)
+### Backend
+#### Data
+- Performance fix on 1.28.3 [DL-6377]
+### Deploy commands
+```
+drc restart migrations
+drc exec delta-producer-background-jobs-initiator curl -X POST http://localhost/public/healing-jobs # or wait for the healing to kick in
+```
+## 1.28.3 (2025-01-16)
+### Backend
+#### Data
+- Datafix. Fusiegemeentes had different bestuursfuncties attached to OCMW and Gemeente. [DL-6377]
+  However, business-rule-wise, these bestuursfuncties are shared.
+### Deploy commands
+```
+drc restart migrations
+drc exec delta-producer-background-jobs-initiator curl -X POST http://localhost/public/healing-jobs # or wait for the healing to kick in
+```
+## 1.28.2 (2025-01-13)
+### Backend
+ - Added "Opdrachthoudende vereniging met private deelname" in delta public (DL-6368)
+### Deploy commands
+```
+drc restart migrations
+drc exec delta-producer-background-jobs-initiator curl -X POST http://localhost/public/healing-jobs # or wait for the healing to kick in
+```
+## 1.28.1 (2025-01-11)
+### Backend
+- Import 2025 NIS codes and update werkingsgebieden (OP-3342)
+- Add merger change events for merged police zones (OP-3454)
+- Delete local involvements for worship services (OP-3501, OP-3512)
+- Move link between new municipalities and province to shared graph (OP-3513, OP-3514)
+
+### Deploy notes
+- Restart both migration services
+#### Docker commands
+- `drc restart migrations; drc logs -ft --tail=200 migrations`
+- `drc restart migrations-triggering-indexing; drc logs -ft --tail=200 migrations-triggering-indexing`
+
+## 1.28.0 (2025-01-02)
+### Backend
+- Bump frontend to `v1.28.0`
+- Add end date for governing bodies for municipalities and OCMWs and update status (OP-3348)
+- Update municipality and province name in organisation's addresses (OP-3410)
+- Add merger change events for merged municipalities and OCMWs (OP-3412)
+- Updates names for merged municipalities and OCMWs (OP-3456)
+
+### Deploy notes
+- Restart both migration services
+- Bump frontend
+#### Docker commands
+- `drc restart migrations; drc logs -ft --tail=200 migrations`
+- `drc restart migrations-triggering-indexing; drc logs -ft --tail=200 migrations-triggering-indexing`
+- `drc pull frontend; drc up -d frontend`

README.md

@@ -11,6 +11,145 @@ Backend for the organization portal application, based on the mu.semte.ch micros
 
 You can shut down using `docker-compose stop` and remove everything using `docker-compose rm`.
 
+#### Setup consumers
+
+The application depends on external data. You'll have to perform extra steps if you want this data to be available.
+These are multiple data sets, we'll try to be exhaustive on how to ingest these. Even though the procedures are similar.
+
+#### Setup mandatarissen sync
+Note: this is assuming you have never ran this before.
+```
+drc down;
+```
+Update `docker-compose.override.yml` to:
+```
+  mandatarissen-consumer:
+    environment:
+      DCR_SYNC_BASE_URL: "https://loket.lokaalbestuur.vlaanderen.be" # or another endpoint
+      DCR_LANDING_ZONE_DATABASE: "triplestore"
+      DCR_REMAPPING_DATABASE: "triplestore"
+      DCR_DISABLE_DELTA_INGEST: "false"
+      DCR_DISABLE_INITIAL_SYNC: "false"
+```
+Then:
+```
+drc up -d migrations
+drc up -d db mandatarissen-consumer
+# Wait until success of the previous step
+```
+Then, update `docker-compose.override.yml` to:
+```
+  mandatarissen-consumer:
+    environment:
+      DCR_SYNC_BASE_URL: "https://loket.lokaalbestuur.vlaanderen.be" # or another endpoint
+      DCR_LANDING_ZONE_DATABASE: "db"
+      DCR_REMAPPING_DATABASE: "db"
+      DCR_DISABLE_DELTA_INGEST: "false"
+      DCR_DISABLE_INITIAL_SYNC: "false"
+```
+Finally, put the stack back up and reindex elasticsearch
+```
+drc up -d
+sh scripts/reset-elastic.sh
+```
+
+#### Setup leidinggevenden sync (http://data.lblod.info/vocabularies/leidinggevenden/Functionaris)
+
+Note: this is assuming you have never ran this before.
+```
+drc down;
+```
+Update `docker-compose.override.yml` to:
+```
+  leidinggevenden-consumer:
+    environment:
+      DCR_SYNC_BASE_URL: "https://loket.lokaalbestuur.vlaanderen.be" # or another endpoint
+      DCR_LANDING_ZONE_DATABASE: "triplestore" # for the initial sync, we go directly to virtuoso
+      DCR_REMAPPING_DATABASE: "triplestore" # for the initial sync, we go directly to virtuoso
+      DCR_DISABLE_DELTA_INGEST: "false"
+      DCR_DISABLE_INITIAL_SYNC: "false"
+```
+Then:
+```
+drc up -d migrations
+drc up -d db leidinggevenden-consumer
+# Wait until success of the previous step
+```
+Then, update `docker-compose.override.yml` to:
+```
+  leidinggevenden-consumer:
+    environment:
+      DCR_SYNC_BASE_URL: "https://loket.lokaalbestuur.vlaanderen.be" # choose the correct endpoint
+      DCR_LANDING_ZONE_DATABASE: "db"
+      DCR_REMAPPING_DATABASE: "db"
+      DCR_DISABLE_DELTA_INGEST: "false"
+      DCR_DISABLE_INITIAL_SYNC: "false"
+```
+Finally, put the stack back up and reindex elasticsearch
+```
+drc up -d
+sh scripts/reset-elastic.sh
+```
+
+#### Setup worship mandatees sync
+
+Note: this is assuming you have never ran this before.
+```
+drc down;
+```
+Update `docker-compose.override.yml` to:
+```
+  worship-services-main-info-consumer:
+    environment:
+      DCR_SYNC_BASE_URL: "https://loket.lokaalbestuur.vlaanderen.be" # or another endpoint
+      DCR_SYNC_LOGIN_ENDPOINT: "https://loket.lokaalbestuur.vlaanderen.be/sync/worship-services-sensitive/login" # or another endpoint
+      DCR_SECRET_KEY: "key-of-the-producer"
+      DCR_LANDING_ZONE_DATABASE: "triplestore"
+      DCR_REMAPPING_DATABASE: "triplestore"
+      DCR_DISABLE_DELTA_INGEST: "false"
+      DCR_DISABLE_INITIAL_SYNC: "false"
+  worship-services-private-info-consumer:
+    environment:
+      DCR_SYNC_BASE_URL: "https://loket.lokaalbestuur.vlaanderen.be" # or another endpoint
+      DCR_SYNC_LOGIN_ENDPOINT: "https://loket.lokaalbestuur.vlaanderen.be/sync/worship-services-sensitive/login" # or another endpoint
+      DCR_SECRET_KEY: "key-of-the-producer"
+      DCR_LANDING_ZONE_DATABASE: "triplestore"
+      DCR_REMAPPING_DATABASE: "triplestore"
+      DCR_DISABLE_DELTA_INGEST: "false"
+      DCR_DISABLE_INITIAL_SYNC: "false"
+```
+Then:
+```
+drc up -d migrations
+drc up -d db worship-services-main-info-consumer worship-services-private-info-consumer
+# Wait until success of the previous step
+```
+Then, update `docker-compose.override.yml` to:
+```
+worship-services-main-info-consumer:
+    environment:
+      DCR_SYNC_BASE_URL: "https://loket.lokaalbestuur.vlaanderen.be" # or another endpoint
+      DCR_SYNC_LOGIN_ENDPOINT: "https://loket.lokaalbestuur.vlaanderen.be/sync/worship-services-sensitive/login" # or another endpoint
+      DCR_SECRET_KEY: "key-of-the-producer"
+      DCR_LANDING_ZONE_DATABASE: "db"
+      DCR_REMAPPING_DATABASE: "db"
+      DCR_DISABLE_DELTA_INGEST: "false"
+      DCR_DISABLE_INITIAL_SYNC: "false"
+  worship-services-private-info-consumer:
+    environment:
+      DCR_SYNC_BASE_URL: "https://loket.lokaalbestuur.vlaanderen.be" # or another endpoint
+      DCR_SYNC_LOGIN_ENDPOINT: "https://loket.lokaalbestuur.vlaanderen.be/sync/worship-services-sensitive/login" # or another endpoint
+      DCR_SECRET_KEY: "key-of-the-producer"
+      DCR_LANDING_ZONE_DATABASE: "db"
+      DCR_REMAPPING_DATABASE: "db"
+      DCR_DISABLE_DELTA_INGEST: "false"
+      DCR_DISABLE_INITIAL_SYNC: "false"
+```
+Finally, put the stack back up and reindex elasticsearch
+```
+drc up -d
+sh scripts/reset-elastic.sh
+```
 
 ### Setting up the delta-producers related services
 To make sure the app can share data, producers need to be set up. There is an intial sync, that is potentially very expensive, and must be started manually
@@ -46,3 +185,26 @@ To make sure the app can share data, producers need to be set up. There is an in
        )
      }
 ```
+
+## Configuring the dashboard
+### Accessing the dashboard from your local machine
+
+Since we use dispatcher v2, which dispatches on hostname, we'll have to update `/etc/hosts`.
+Add an entry similar to the following. Ensure the first part of the domain starts with `dashboard`.:
+
+```
+127.0.0.1 dashboard.localhost
+```
+
+### Creating new users
+For now, we use specific logins for the dashboard users. Each environment has its own passwords.
+
+To add a user, make sure to have [mu-cli](https://github.com/mu-semtech/mu-cli) installed first.
+Then in `docker-compose.override.yml`
+```
+  dashboard-login:
+    environment:
+      MU_APPLICATION_SALT: 'a_random_string_with_sufficient_entropy_hence_not_this_one'
+```
+You can now generate a user by running `mu script project-scripts generate-dashboard-login` and following the steps.
+Restart `migrations` and it should insert the new user into the database.

config/authorization/config.ex

@@ -79,6 +79,8 @@ defmodule Acl.UserGroups.Config do
     "http://www.w3.org/ns/org#ChangeEvent",
     "http://data.lblod.info/vocabularies/contacthub/AgentInPositie",
     "http://mu.semte.ch/vocabularies/ext/KboOrganisatie",
+    "http://www.w3.org/ns/org#Membership",
+    "http://www.w3.org/2006/time#ProperInterval"
   ]
 
   @shared_protected_type [
@@ -105,6 +107,19 @@ defmodule Acl.UserGroups.Config do
     }"
   end
 
+  defp can_access_dashboard() do
+    %AccessByQuery{
+      vars: [],
+      query: "PREFIX ext: <http://mu.semte.ch/vocabularies/ext/>
+        PREFIX mu: <http://mu.semte.ch/vocabularies/core/>
+        SELECT DISTINCT ?account WHERE {
+          <SESSION_ID> <http://mu.semte.ch/vocabularies/session/account> ?account.
+          ?account <http://mu.semte.ch/vocabularies/ext/sessionRole> ?session_role.
+          FILTER( ?session_role = \"dashboard-user\" )
+        }"
+      }
+  end
+
   def user_groups do
     [
       %GroupSpec{
@@ -204,6 +219,46 @@ defmodule Acl.UserGroups.Config do
           }
         ]
       },
+      # // dashboard users
+      %GroupSpec{
+        name: "dashboard-users",
+        useage: [:read],
+        access: can_access_dashboard(),
+        graphs: [
+          %GraphSpec{
+            graph: "http://mu.semte.ch/graphs/reports",
+            constraint: %ResourceConstraint{
+              resource_types: [
+                "http://lblod.data.gift/vocabularies/reporting/Report",
+                "http://open-services.net/ns/core#Error",
+                "http://www.semanticdesktop.org/ontologies/2007/03/22/nfo#DataContainer",
+                "http://www.semanticdesktop.org/ontologies/2007/03/22/nfo#FileDataObject"
+              ]
+            }
+          },
+          %GraphSpec{
+            graph: "http://mu.semte.ch/graphs/system/jobs",
+            constraint: %ResourceConstraint{
+              resource_types: [
+                "http://vocab.deri.ie/cogs#Job",
+                "http://www.semanticdesktop.org/ontologies/2007/03/22/nfo#DataContainer",
+                "http://www.semanticdesktop.org/ontologies/2007/03/22/nfo#FileDataObject"
+              ]
+            }
+          },
+          # OP stores jobs in multiple graphs, so we configure both until they are merged into one graph
+          %GraphSpec{
+            graph: "http://mu.semte.ch/graphs/jobs",
+            constraint: %ResourceConstraint{
+              resource_types: [
+                "http://vocab.deri.ie/cogs#Job",
+                "http://www.semanticdesktop.org/ontologies/2007/03/22/nfo#DataContainer",
+                "http://www.semanticdesktop.org/ontologies/2007/03/22/nfo#FileDataObject"
+              ]
+            }
+          },
+        ]
+      },
 
       # // CLEANUP
       #

config/clb-consumer/contact-data/config.js

@@ -1,30 +0,0 @@
-const BATCH_SIZE = parseInt(process.env.BATCH_SIZE) || 1000;
-const MU_CALL_SCOPE_ID_INITIAL_SYNC = process.env.MU_CALL_SCOPE_ID_INITIAL_SYNC || 'http://redpencil.data.gift/id/concept/muScope/deltas/consumer/initialSync';
-const BYPASS_MU_AUTH_FOR_EXPENSIVE_QUERIES = process.env.BYPASS_MU_AUTH_FOR_EXPENSIVE_QUERIES == 'true' ? true : false;
-const DIRECT_DATABASE_ENDPOINT = process.env.DIRECT_DATABASE_ENDPOINT || 'http://virtuoso:8890/sparql';
-const MAX_DB_RETRY_ATTEMPTS = parseInt(process.env.MAX_DB_RETRY_ATTEMPTS || 5);
-const MAX_REASONING_RETRY_ATTEMPTS = parseInt(process.env.MAX_REASONING_RETRY_ATTEMPTS || 5);
-const SLEEP_BETWEEN_BATCHES = parseInt(process.env.SLEEP_BETWEEN_BATCHES || 100);
-const SLEEP_TIME_AFTER_FAILED_REASONING_OPERATION = parseInt(process.env.SLEEP_TIME_AFTER_FAILED_REASONING_OPERATION || 10000);
-const SLEEP_TIME_AFTER_FAILED_DB_OPERATION = parseInt(process.env.SLEEP_TIME_AFTER_FAILED_DB_OPERATION || 60000);
-
-// Graphs
-const LANDING_ZONE_GRAPH = process.env.DCR_LANDING_ZONE_GRAPH || `http://mu.semte.ch/graphs/landing-zone/clb-contact-data`;
-
-// ENDPOINTS
-const LANDING_ZONE_DATABASE = process.env.DCR_LANDING_ZONE_DATABASE || 'database';
-const LANDING_ZONE_DATABASE_ENDPOINT = process.env.DCR_LANDING_ZONE_DATABASE_ENDPOINT || `http://${LANDING_ZONE_DATABASE}:8890/sparql`;
-
-module.exports = {
-  BATCH_SIZE,
-  MU_CALL_SCOPE_ID_INITIAL_SYNC,
-  BYPASS_MU_AUTH_FOR_EXPENSIVE_QUERIES,
-  DIRECT_DATABASE_ENDPOINT,
-  MAX_DB_RETRY_ATTEMPTS,
-  MAX_REASONING_RETRY_ATTEMPTS,
-  SLEEP_BETWEEN_BATCHES,
-  SLEEP_TIME_AFTER_FAILED_DB_OPERATION,
-  SLEEP_TIME_AFTER_FAILED_REASONING_OPERATION,
-  LANDING_ZONE_GRAPH,
-  LANDING_ZONE_DATABASE_ENDPOINT
-};

config/clb-consumer/contact-data/delta-context-config.js

@@ -1,268 +0,0 @@
-
-const PREFIXES = `
-PREFIX besluit: <http://data.vlaanderen.be/ns/besluit#>
-  PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
-  PREFIX lblodgeneriek: <https://data.lblod.info/vocabularies/generiek/>
-  PREFIX org: <http://www.w3.org/ns/org#>
-  PREFIX code: <http://lblod.data.gift/vocabularies/organisatie/>
-  PREFIX adms: <http://www.w3.org/ns/adms#>
-  PREFIX generiek: <https://data.vlaanderen.be/ns/generiek#>
-  PREFIX ere: <http://data.lblod.info/vocabularies/erediensten/>
-  PREFIX organisatie: <https://data.vlaanderen.be/ns/organisatie#>
-  PREFIX mu: <http://mu.semte.ch/vocabularies/core/>
-  PREFIX euvoc: <http://publications.europa.eu/ontology/euvoc#>
-  PREFIX prov: <http://www.w3.org/ns/prov#>
-  PREFIX schema: <http://schema.org/>
-  PREFIX locn: <http://www.w3.org/ns/locn#>
-  PREFIX ext: <http://mu.semte.ch/vocabularies/ext/>`
-
-const contextConfig = {
-  addTypes: {
-    scope: 'all', // 'inserts, 'deletes', 'all' or none. To add rdf:type to subjects of inserts, deletes or both
-    exhausitive: false, // true or false: find all types for a subject, even if one is already present in delta
-  },
-  contextQueries: [
-    {
-      trigger: { // subjectType or predicateValue
-        subjectType: "org:Site"
-      },
-      queryTemplate: (subject) => `
-        ${PREFIXES}
-        CONSTRUCT {
-          ${subject} ext:goesInGraph ?graphOne.
-          ${subject} ext:goesInGraph ?graphTwo.
-          ${subject} ext:goesInGraph ?graphThree.
-          ${subject} ext:goesInGraph ?graphFour.
-        } WHERE {
-          OPTIONAL {
-            GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
-              ?adminUnitOne org:hasPrimarySite ${subject}.
-            }
-            BIND(<http://mu.semte.ch/graphs/administrative-unit> AS ?graphOne)
-          }
-          OPTIONAL {
-            GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
-              ?adminUnitTwo org:hasSite ${subject}.
-            }
-            BIND(<http://mu.semte.ch/graphs/administrative-unit> AS ?graphTwo)
-          }
-          OPTIONAL {
-            GRAPH <http://mu.semte.ch/graphs/worship-service> {
-              ?adminUnitThree org:hasPrimarySite ${subject}.
-            }
-            BIND(<http://mu.semte.ch/graphs/worship-service> AS ?graphThree)
-          }
-          OPTIONAL {
-            GRAPH <http://mu.semte.ch/graphs/worship-service> {
-              ?adminUnitFour org:hasSite ${subject}.
-            }
-            BIND(<http://mu.semte.ch/graphs/worship-service> AS ?graphFour)
-          }
-        }`
-    },
-    {
-      trigger: { // subjectType or predicateValue
-        subjectType: "schema:ContactPoint"
-      },
-      queryTemplate: (subject) => `
-        ${PREFIXES}
-        CONSTRUCT {
-          ${subject} ext:goesInGraph ?graphOne.
-          ${subject} ext:goesInGraph ?graphTwo.
-        } WHERE {
-          OPTIONAL {
-            GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
-              ?siteOne org:siteAddress ${subject}.
-            }
-            BIND(<http://mu.semte.ch/graphs/administrative-unit> AS ?graphOne)
-          }
-          OPTIONAL {
-            GRAPH <http://mu.semte.ch/graphs/worship-service> {
-              ?siteTwo org:siteAddress ${subject}.
-            }
-            BIND(<http://mu.semte.ch/graphs/worship-service> AS ?graphTwo)
-          }
-        }`
-    },
-    {
-      trigger: { // subjectType or predicateValue
-        subjectType: "locn:Address"
-      },
-      queryTemplate: (subject) => `
-        ${PREFIXES}
-        CONSTRUCT {
-          ${subject} ext:goesInGraph ?graphOne.
-          ${subject} ext:goesInGraph ?graphTwo.
-        } WHERE {
-          OPTIONAL {
-            GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
-              ?siteOne organisatie:bestaatUit ${subject}.
-            }
-            BIND(<http://mu.semte.ch/graphs/administrative-unit> AS ?graphOne)
-          }
-          OPTIONAL {
-            GRAPH <http://mu.semte.ch/graphs/worship-service> {
-              ?siteTwo organisatie:bestaatUit ${subject}.
-            }
-            BIND(<http://mu.semte.ch/graphs/worship-service> AS ?graphTwo)
-          }
-        }`
-    },
-    //Predicate triggers to solve out of order deltas
-    {
-      trigger: { // subjectType or predicateValue
-        predicateValue: "org:hasSite"
-      },
-      //We assume that only administrative units are in the shared graph of OP due to the need of sharing the prefLabel with worships services
-      queryTemplate: (subject) => `
-        ${PREFIXES}
-        CONSTRUCT {
-          ${subject} ext:goesInGraph ?graphOne.
-          ${subject} ext:goesInGraph ?graphTwo.
-          ?site ?pSite ?oSite.
-          ?contact ?pContact ?oContact.
-          ?address ?pAddress ?oAddress.
-          ?addressContact ?pAddressContact ?oAddressContact.
-        } WHERE {
-          OPTIONAL {
-            GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
-              ${subject} a ?type.
-            }
-            BIND(<http://mu.semte.ch/graphs/administrative-unit> AS ?graphOne)
-          }
-          OPTIONAL {
-            GRAPH <http://mu.semte.ch/graphs/shared> {
-              ${subject} a ?type.
-            }
-            BIND(<http://mu.semte.ch/graphs/administrative-unit> AS ?graphOne)
-          }
-          OPTIONAL {
-            GRAPH <http://mu.semte.ch/graphs/worship-service> {
-              ${subject} a ?type.
-            }
-            BIND(<http://mu.semte.ch/graphs/worship-service> AS ?graphTwo)
-          }
-          ${subject} org:hasSite ?site.
-          ?site ?pSite ?oSite.
-          {
-            ?site org:siteAddress ?contact.
-            ?contact ?pContact ?oContact.
-            ?contact locn:address ?addressContact.
-            ?addressContact ?pAddressContact ?oAddressContact.
-          }
-          UNION
-          {
-            ?site org:siteAddress ?contact.
-            ?contact ?pContact ?oContact.
-          }
-          UNION
-          {
-            ?site organisatie:bestaatUit ?address.
-            ?address ?pAddress ?oAddress.
-          }        
-        }`
-    },
-    {
-      trigger: { // subjectType or predicateValue
-        predicateValue: "org:hasPrimarySite"
-      },
-      //We assume that only administrative units are in the shared graph of OP due to the need of sharing the prefLabel with worships services
-      queryTemplate: (subject) => `
-        ${PREFIXES}
-        CONSTRUCT {
-          ${subject} ext:goesInGraph ?graphOne.
-          ${subject} ext:goesInGraph ?graphTwo.
-          ?site ?pSite ?oSite.
-          ?contact ?pContact ?oContact.
-          ?address ?pAddress ?oAddress.
-          ?addressContact ?pAddressContact ?oAddressContact.
-        } WHERE {
-          OPTIONAL {
-            GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
-              ${subject} a ?type.
-            }
-            BIND(<http://mu.semte.ch/graphs/administrative-unit> AS ?graphOne)
-          }
-          OPTIONAL {
-            GRAPH <http://mu.semte.ch/graphs/shared> {
-              ${subject} a ?type.
-            }
-            BIND(<http://mu.semte.ch/graphs/administrative-unit> AS ?graphOne)
-          }
-          OPTIONAL {
-            GRAPH <http://mu.semte.ch/graphs/worship-service> {
-              ${subject} a ?type.
-            }
-            BIND(<http://mu.semte.ch/graphs/worship-service> AS ?graphTwo)
-          }
-          ${subject} org:hasPrimarySite ?site.
-          ?site ?pSite ?oSite.
-          {
-            ?site org:siteAddress ?contact.
-            ?contact ?pContact ?oContact.
-            ?contact locn:address ?addressContact.
-            ?addressContact ?pAddressContact ?oAddressContact.
-          }
-          UNION
-          {
-            ?site org:siteAddress ?contact.
-            ?contact ?pContact ?oContact.
-          }
-          UNION
-          {
-            ?site organisatie:bestaatUit ?address.
-            ?address ?pAddress ?oAddress.
-          }        
-        }`
-    },
-    {
-      trigger: { // subjectType or predicateValue
-        predicateValue: "org:siteAddress"
-      },
-      queryTemplate: (subject) => `
-        ${PREFIXES}
-        CONSTRUCT {
-          ?contact ?pContact ?oContact.
-          ?addressContact ?pAddressContact ?oAddressContact.
-        } WHERE {
-          ${subject} org:siteAddress ?contact.
-          ?contact ?pContact ?oContact.
-          OPTIONAL {
-            ?contact locn:address ?addressContact.
-            ?addressContact ?pAddressContact ?oAddressContact.
-          }
-        }`
-    },
-    {
-      trigger: { // subjectType or predicateValue
-        predicateValue: "locn:address"
-      },
-      queryTemplate: (subject) => `
-        ${PREFIXES}
-        CONSTRUCT {
-          ?addressContact ?pAddressContact ?oAddressContact.
-        } WHERE {
-          ${subject} locn:address ?addressContact.
-          ?addressContact ?pAddressContact ?oAddressContact.
-        }`
-    },
-    {
-      trigger: { // subjectType or predicateValue
-        predicateValue: "organisatie:bestaatUit"
-      },
-      queryTemplate: (subject) => `
-        ${PREFIXES}
-        CONSTRUCT {
-          ?address ?pAddress ?oAddress.
-        } WHERE {
-          ${subject} organisatie:bestaatUit ?address.
-          ?address ?pAddress ?oAddress.
-        }`
-    }
-  ]
-}
-
-module.exports = {
-  contextConfig,
-  PREFIXES
-};

config/clb-consumer/contact-data/delta-sync-dispatching.js

@@ -1,104 +0,0 @@
-const {
-  deleteFromPublicGraph,
-  insertIntoPublicGraph,
-  deleteFromSpecificGraphs,
-  insertIntoSpecificGraphs
-} = require('./util');
-
-const {
-  LANDING_ZONE_GRAPH
-} = require('./config')
-
-const publicTypes = [
-  '<http://lblod.data.gift/vocabularies/organisatie/TypeVestiging>'
-]
-
-/**
-* Dispatch the fetched information to a target graph.
-* Note: <share://file/data> will be ADDED to it's own graph.
-*   We take only care of adding them, not updating triples, this is a TODO
-* @param { mu, muAuthSudo } lib - The provided libraries from the host service.
-* @param { termObjectChangeSets: { deletes, inserts } } data - The fetched changes sets, which objects of serialized Terms
-*          [ {
-*              graph: "<http://foo>",
-*              subject: "<http://bar>",
-*              predicate: "<http://baz>",
-*              object: "<http://boom>^^<http://datatype>"
-*            }
-*         ]
-* @return {void} Nothing
-*/
-async function dispatch(lib, data) {
-  let { termObjectChangeSets, termObjectChangeSetsWithContext } = data;
-
-  // Both arrays are the same length, so we can zip them together for easier processing
-  zippedChangeSets = termObjectChangeSets.map((o, i) => ({
-    original: o,
-    withContext: termObjectChangeSetsWithContext && termObjectChangeSetsWithContext[i]
-  }));
-
-  console.log(`Received ${zippedChangeSets.length} change sets`)
-
-  for (let { original, withContext } of zippedChangeSets) {
-    
-    const {public: insertsOnPublic, target: insertsOnGraphs} = separateBetweenTargetAndPublic(original.inserts, withContext.inserts)
-    const {public: deletesOnPublic, target: deletesOnGraphs} = separateBetweenTargetAndPublic(original.deletes, withContext.deletes)
-
-    await deleteFromPublicGraph(lib, deletesOnPublic);
-    await deleteFromSpecificGraphs(lib, deletesOnGraphs);
-    await insertIntoPublicGraph(lib, insertsOnPublic);
-    await insertIntoSpecificGraphs(lib, insertsOnGraphs);
-
-  }
-}
-
-function separateBetweenTargetAndPublic(changes, context) {
-  const changesOnPublic = new Set();
-  const changesOnGraphs = {};
-  for(let change of changes) {
-    const subject = change.subject;
-    const contextTriples = context.filter((context) => context.subject === subject);
-    const graphTriple = contextTriples.find(
-      (context) => context.predicate === '<http://mu.semte.ch/vocabularies/ext/goesInGraph>' 
-        && context.object !== `<${LANDING_ZONE_GRAPH}>`
-    )
-    const otherContextTriples = context.filter(
-      (context) => context.predicate !== '<http://mu.semte.ch/vocabularies/ext/goesInGraph>' 
-        && context.predicate !== '<http://mu.semte.ch/vocabularies/ext/contextDataGoesInGraph>' 
-        && !(context.subject === subject && context.predicate === '<http://www.w3.org/1999/02/22-rdf-syntax-ns#type>')
-    );
-    if(graphTriple) {
-      const graph = graphTriple.object.slice(1,-1); // We have to slice it to remove the "<" and ">"
-      if(change.object !== '') {
-        if(!changesOnGraphs[graph]) {
-          changesOnGraphs[graph] = new Set();
-        }  
-        changesOnGraphs[graph].add(`${change.subject} ${change.predicate} ${change.object}.`)
-      }                    
-      for(let triple of otherContextTriples) {
-        if(triple.object === '') continue;
-        if(!changesOnGraphs[graph]) {
-          changesOnGraphs[graph] = new Set();
-        }  
-        changesOnGraphs[graph].add(`${triple.subject} ${triple.predicate} ${triple.object}.`)                                       
-      }
-    }
-    const typeTriple = contextTriples.find(
-      (context) => context.subject === subject 
-        && context.predicate === '<http://www.w3.org/1999/02/22-rdf-syntax-ns#type>'
-    );
-    if(!typeTriple) continue;
-    const type = typeTriple.object;
-    if(publicTypes.includes(type)) {
-      changesOnPublic.add(`${change.subject} ${change.predicate} ${change.object}.`)
-      for(let triple of otherContextTriples) {
-        changesOnPublic.add(`${triple.subject} ${triple.predicate} ${triple.object}.`)
-      }
-    }
-  }
-  return {public: changesOnPublic, target: changesOnGraphs}
-}
-
-module.exports = {
-  dispatch
-};

config/clb-consumer/contact-data/initial-sync-dispatching.js

@@ -1,69 +0,0 @@
-const { batchedDbUpdate, moveDataAfterInitialSync } = require('./util');
-const {
-  BYPASS_MU_AUTH_FOR_EXPENSIVE_QUERIES,
-  DIRECT_DATABASE_ENDPOINT,
-  MU_CALL_SCOPE_ID_INITIAL_SYNC,
-  BATCH_SIZE,
-  MAX_DB_RETRY_ATTEMPTS,
-  SLEEP_BETWEEN_BATCHES,
-  SLEEP_TIME_AFTER_FAILED_DB_OPERATION,
-  LANDING_ZONE_GRAPH,
-} = require('./config');
-
-const endpoint = BYPASS_MU_AUTH_FOR_EXPENSIVE_QUERIES ? DIRECT_DATABASE_ENDPOINT : process.env.MU_SPARQL_ENDPOINT;
-
-/**
-* Dispatch the fetched information to a target graph.
-* @param { mu, muAuthSudo } lib - The provided libraries from the host service.
-* @param { termObjects } data - The fetched quad information, which objects of serialized Terms
-*          [ {
-*              graph: "<http://foo>",
-*              subject: "<http://bar>",
-*              predicate: "<http://baz>",
-*              object: "<http://boom>^^<http://datatype>"
-*            }
-*         ]
-* @return {void} Nothing
-*/
-async function dispatch(lib, data) {
-  const { mu, muAuthSudo } = lib;
-  const { termObjects } = data;
-
-  // Steps:
-  // 1. Write the triples to the landing zone graph - without any mapping or filtering
-  // 2. One-time reasoning run with the full landing zone graph and write the results to the target graph (on-finish)
-
-  if (BYPASS_MU_AUTH_FOR_EXPENSIVE_QUERIES) {
-    console.warn(`Service configured to skip MU_AUTH!`);
-  }
-  console.log(`Using ${endpoint} to insert triples`);
-
-  if (termObjects.length) {
-    const originalInsertTriples = termObjects.map(o => `${o.subject} ${o.predicate} ${o.object}.`);
-
-    await batchedDbUpdate(
-      muAuthSudo.updateSudo,
-      LANDING_ZONE_GRAPH,
-      originalInsertTriples,
-      { 'mu-call-scope-id': MU_CALL_SCOPE_ID_INITIAL_SYNC },
-      endpoint,
-      BATCH_SIZE,
-      MAX_DB_RETRY_ATTEMPTS,
-      SLEEP_BETWEEN_BATCHES,
-      SLEEP_TIME_AFTER_FAILED_DB_OPERATION
-    );
-  }
-}
-
-async function onFinishInitialIngest(lib,data) {
-  const { muAuthSudo } = lib;
-
-  console.log(`!! On-finish triggered !!`);
-
-  await moveDataAfterInitialSync(muAuthSudo.updateSudo, endpoint)
-}
-
-module.exports = {
-  dispatch,
-  onFinishInitialIngest
-};

config/clb-consumer/contact-data/util.js

@@ -1,337 +0,0 @@
-const {
-  MAX_DB_RETRY_ATTEMPTS,
-  SLEEP_BETWEEN_BATCHES,
-  SLEEP_TIME_AFTER_FAILED_DB_OPERATION,
-  LANDING_ZONE_GRAPH,
-  BATCH_SIZE,
-} = require('./config');
-
-const prefixes = `
-PREFIX besluit: <http://data.vlaanderen.be/ns/besluit#>
-PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
-PREFIX lblodgeneriek: <https://data.lblod.info/vocabularies/generiek/>
-PREFIX org: <http://www.w3.org/ns/org#>
-PREFIX code: <http://lblod.data.gift/vocabularies/organisatie/>
-PREFIX adms: <http://www.w3.org/ns/adms#>
-PREFIX generiek: <https://data.vlaanderen.be/ns/generiek#>
-PREFIX ere: <http://data.lblod.info/vocabularies/erediensten/>
-PREFIX organisatie: <https://data.vlaanderen.be/ns/organisatie#>
-PREFIX mu: <http://mu.semte.ch/vocabularies/core/>
-PREFIX euvoc: <http://publications.europa.eu/ontology/euvoc#>
-PREFIX prov: <http://www.w3.org/ns/prov#>
-PREFIX schema: <http://schema.org/>
-PREFIX locn: <http://www.w3.org/ns/locn#>
-PREFIX foaf: <http://xmlns.com/foaf/0.1/>
-PREFIX ext:<http://mu.semte.ch/vocabularies/ext/>
-`
-
-async function batchedDbUpdate(
-  muUpdate,
-  graph,
-  triples,
-  extraHeaders,
-  endpoint,
-  batchSize,
-  maxAttempts,
-  sleepBetweenBatches = 1000,
-  sleepTimeOnFail = 1000,
-  operation = 'INSERT'
-) {
-  for (let i = 0; i < triples.length; i += batchSize) {
-    console.log(`Inserting triples in batch: ${i}-${i + batchSize}`);
-
-    const batch = triples.slice(i, i + batchSize).join('\n');
-
-    const insertCall = async () => {
-      await muUpdate(`
-${operation} DATA {
-GRAPH <${graph}> {
-${batch}
-}
-}
-`, extraHeaders, endpoint);
-    };
-
-    await operationWithRetry(insertCall, 0, maxAttempts, sleepTimeOnFail);
-
-    console.log(`Sleeping before next query execution: ${sleepBetweenBatches}`);
-    await new Promise(r => setTimeout(r, sleepBetweenBatches));
-  }
-}
-
-async function operationWithRetry(callback,
-  attempt,
-  maxAttempts,
-  sleepTimeOnFail) {
-  try {
-    if (typeof callback === "function")
-      return await callback();
-    else // Catch error from promise - not how I would do it normally, but allows re use of existing code.
-      return await callback;
-  }
-  catch (e) {
-    console.log(`Operation failed for ${callback.toString()}, attempt: ${attempt} of ${maxAttempts}`);
-    console.log(`Error: ${e}`);
-    console.log(`Sleeping ${sleepTimeOnFail} ms`);
-
-    if (attempt >= maxAttempts) {
-      console.log(`Max attempts reached for ${callback.toString()}, giving up`);
-      throw e;
-    }
-
-    await new Promise(r => setTimeout(r, sleepTimeOnFail));
-    return operationWithRetry(callback, ++attempt, maxAttempts, sleepTimeOnFail);
-  }
-}
-
-
-async function insertIntoPublicGraph(lib, statements) {
-  console.log(`Inserting ${statements.length} statements into public graph`);
-
-  await batchedDbUpdate(
-    lib.muAuthSudo.updateSudo,
-    'http://mu.semte.ch/graphs/public',
-    Array.from(statements),
-    {},
-    process.env.MU_SPARQL_ENDPOINT,
-    BATCH_SIZE,
-    MAX_DB_RETRY_ATTEMPTS,
-    SLEEP_BETWEEN_BATCHES,
-    SLEEP_TIME_AFTER_FAILED_DB_OPERATION,
-    'INSERT');
-}
-
-async function insertIntoSpecificGraphs(lib, statementsWithGraphs) {
-
-  for( let graph in statementsWithGraphs) {
-    console.log(`Inserting ${statementsWithGraphs[graph].length} statements into ${graph} graph`);
-    await batchedDbUpdate(
-      lib.muAuthSudo.updateSudo,
-      graph,
-      Array.from(statementsWithGraphs[graph]),
-      {},
-      process.env.MU_SPARQL_ENDPOINT,
-      BATCH_SIZE,
-      MAX_DB_RETRY_ATTEMPTS,
-      SLEEP_BETWEEN_BATCHES,
-      SLEEP_TIME_AFTER_FAILED_DB_OPERATION,
-      'INSERT');
-  }
-  
-}
-
-async function deleteFromPublicGraph(lib, statements) {
-  console.log(`Deleting ${statements.length} statements from public graph`);
-
-  await batchedDbUpdate(
-    lib.muAuthSudo.updateSudo,
-    'http://mu.semte.ch/graphs/public',
-    Array.from(statements),
-    {},
-    process.env.MU_SPARQL_ENDPOINT,
-    BATCH_SIZE,
-    MAX_DB_RETRY_ATTEMPTS,
-    SLEEP_BETWEEN_BATCHES,
-    SLEEP_TIME_AFTER_FAILED_DB_OPERATION,
-    'DELETE');
-}
-
-async function deleteFromSpecificGraphs(lib, statementsWithGraphs) {
-  
-
-  for( let graph in statementsWithGraphs) {
-    console.log(`Deleting ${statementsWithGraphs[graph].length} statements from ${graph} graph`);
-    await batchedDbUpdate(
-      lib.muAuthSudo.updateSudo,
-      graph,
-      Array.from(statementsWithGraphs[graph]),
-      {},
-      process.env.MU_SPARQL_ENDPOINT,
-      BATCH_SIZE,
-      MAX_DB_RETRY_ATTEMPTS,
-      SLEEP_BETWEEN_BATCHES,
-      SLEEP_TIME_AFTER_FAILED_DB_OPERATION,
-      'DELETE');
-  }
-  
-}
-
-
-async function moveDataAfterInitialSync(muUpdate, endpoint) {
-  // Move primary sites admin units
-  await muUpdate(`
-    ${prefixes}
-    INSERT {
-      GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
-        ?adminUnit org:hasPrimarySite ?site.
-        ?site ?pSite ?oSite.
-        ?contact ?pContact ?oContact.
-        ?address ?pAddress ?oAddress.
-        ?addressContact ?pAddressContact ?oAddressContact.
-      }
-    } WHERE {
-      GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
-        ?adminUnit a ?type.
-      }
-      GRAPH <${LANDING_ZONE_GRAPH}> {
-        ?adminUnit org:hasPrimarySite ?site.
-        ?site ?pSite ?oSite.
-        {
-          ?site org:siteAddress ?contact.
-          ?contact ?pContact ?oContact.
-          ?contact locn:address ?addressContact.
-          ?addressContact ?pAddressContact ?oAddressContact.
-        }
-        UNION
-        {
-          ?site org:siteAddress ?contact.
-          ?contact ?pContact ?oContact.
-        }
-        UNION
-        {
-          ?site organisatie:bestaatUit ?address.
-          ?address ?pAddress ?oAddress.
-        }
-      }        
-    }
-  `, undefined, endpoint)
-
-  // Move sites administrative units
-  await muUpdate(`
-    ${prefixes}
-    INSERT {
-      GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
-        ?adminUnit org:hasSite ?site.
-        ?site ?pSite ?oSite.
-        ?contact ?pContact ?oContact.
-        ?address ?pAddress ?oAddress.
-        ?addressContact ?pAddressContact ?oAddressContact.
-      }
-    } WHERE {
-      GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
-        ?adminUnit a ?type.
-      }
-      GRAPH <${LANDING_ZONE_GRAPH}> {
-        ?adminUnit org:hasSite ?site.
-        ?site ?pSite ?oSite.
-        {
-          ?site org:siteAddress ?contact.
-          ?contact ?pContact ?oContact.
-          ?contact locn:address ?addressContact.
-          ?addressContact ?pAddressContact ?oAddressContact.
-        }
-        UNION
-        {
-          ?site org:siteAddress ?contact.
-          ?contact ?pContact ?oContact.
-        }
-        UNION
-        {
-          ?site organisatie:bestaatUit ?address.
-          ?address ?pAddress ?oAddress.
-        }     
-      }   
-    }
-  `, undefined, endpoint)
-
-  // Move primary sites worships
-  await muUpdate(`
-    ${prefixes}
-    INSERT {
-      GRAPH <http://mu.semte.ch/graphs/worship-service> {
-        ?adminUnit org:hasPrimarySite ?site.
-        ?site ?pSite ?oSite.
-        ?contact ?pContact ?oContact.
-        ?address ?pAddress ?oAddress.
-        ?addressContact ?pAddressContact ?oAddressContact.
-      }
-    } WHERE {
-      GRAPH <http://mu.semte.ch/graphs/worship-service> {
-        ?adminUnit a ?type.
-      }
-      GRAPH <${LANDING_ZONE_GRAPH}> {
-        ?adminUnit org:hasPrimarySite ?site.
-        ?site ?pSite ?oSite.
-        {
-          ?site org:siteAddress ?contact.
-          ?contact ?pContact ?oContact.
-          ?contact locn:address ?addressContact.
-          ?addressContact ?pAddressContact ?oAddressContact.
-        }
-        UNION
-        {
-          ?site org:siteAddress ?contact.
-          ?contact ?pContact ?oContact.
-        }
-        UNION
-        {
-          ?site organisatie:bestaatUit ?address.
-          ?address ?pAddress ?oAddress.
-        }        
-      }
-    }
-  `, undefined, endpoint)
-
-  // Move sites worships
-  await muUpdate(`
-    ${prefixes}
-    INSERT {
-      GRAPH <http://mu.semte.ch/graphs/worship-service> {
-        ?adminUnit org:hasSite ?site.
-        ?site ?pSite ?oSite.
-        ?contact ?pContact ?oContact.
-        ?address ?pAddress ?oAddress.
-        ?addressContact ?pAddressContact ?oAddressContact.
-      }
-    } WHERE {
-      GRAPH <http://mu.semte.ch/graphs/worship-service> {
-        ?adminUnit a ?type.
-      }
-      GRAPH <${LANDING_ZONE_GRAPH}> {
-        ?adminUnit org:hasSite ?site.
-        ?site ?pSite ?oSite.
-        {
-          ?site org:siteAddress ?contact.
-          ?contact ?pContact ?oContact.
-          ?contact locn:address ?addressContact.
-          ?addressContact ?pAddressContact ?oAddressContact.
-        }
-        UNION
-        {
-          ?site org:siteAddress ?contact.
-          ?contact ?pContact ?oContact.
-        }
-        UNION
-        {
-          ?site organisatie:bestaatUit ?address.
-          ?address ?pAddress ?oAddress.
-        }        
-      }
-    }
-  `, undefined, endpoint)
-
-  // Move sites worships
-  await muUpdate(`
-    ${prefixes}
-    INSERT {
-      GRAPH <http://mu.semte.ch/graphs/public> {
-        ?type a <http://lblod.data.gift/vocabularies/organisatie/TypeVestiging>.
-        ?type ?a ?b.
-      }
-    } WHERE {
-      GRAPH <${LANDING_ZONE_GRAPH}> {
-        ?type a <http://lblod.data.gift/vocabularies/organisatie/TypeVestiging>.
-        ?type ?a ?b.
-      }
-    }
-  `, undefined, endpoint)
-}
-
-module.exports = {
-  batchedDbUpdate,
-  moveDataAfterInitialSync,
-  insertIntoPublicGraph,
-  deleteFromPublicGraph,
-  insertIntoSpecificGraphs,
-  deleteFromSpecificGraphs,
-};

config/db/virtuoso-production.ini

@@ -231,10 +231,12 @@ DefaultHost  = localhost:8890
 ;DefaultGraph           = http://localhost:8890/dataspace
 ;ImmutableGraphs        = http://localhost:8890/dataspace
 ResultSetMaxRows           = 1000000 ; 1 million
+MaxConstructTriples        = 1000000 ; More info on this addition here: https://github.com/redpencilio/docker-virtuoso/pull/29
 ;MaxQueryCostEstimationTime = 4000	; in seconds
 MaxQueryExecutionTime      = 1200	; in seconds
 DefaultQuery               = select distinct ?Concept where {[] a ?Concept} LIMIT 100
 DeferInferenceRulesInit    = 0	; controls inference rules loading
+MaxMemInUse                = 0 ; limits the amount of memory for construct dict (0=unlimited). More info on this addition here: https://github.com/redpencilio/docker-virtuoso/pull/29
 ;PingService            = http://rpc.pingthesemanticweb.com/
 
 [Plugins]

config/db/virtuoso.ini

@@ -231,10 +231,12 @@ DefaultHost  = localhost:8890
 ;DefaultGraph           = http://localhost:8890/dataspace
 ;ImmutableGraphs        = http://localhost:8890/dataspace
 ResultSetMaxRows           = 1000000 ; 1 million
+MaxConstructTriples        = 1000000 ; More info on this addition here: https://github.com/redpencilio/docker-virtuoso/pull/29
 ;MaxQueryCostEstimationTime = 4000	; in seconds
 MaxQueryExecutionTime      = 1200	; in seconds
 DefaultQuery               = select distinct ?Concept where {[] a ?Concept} LIMIT 100
 DeferInferenceRulesInit    = 0	; controls inference rules loading
+MaxMemInUse                = 0 ; limits the amount of memory for construct dict (0=unlimited). More info on this addition here: https://github.com/redpencilio/docker-virtuoso/pull/29
 ;PingService            = http://rpc.pingthesemanticweb.com/
 
 [Plugins]

config/delta-consumers/leidinggevenden/mapping/queries/dl2op/data-cleanup/phone.rq

@@ -0,0 +1,21 @@
+PREFIX schema: <http://schema.org/>
+
+CONSTRUCT {
+  ?s schema:telephone ?cleanedPhoneNumber .
+} WHERE {
+  ?s schema:telephone ?phoneNumber .
+
+  # Remove all non digits
+  BIND(REPLACE(?phoneNumber, """[^+\\d]""", "") AS ?scrapeResult)
+
+  # Replace leading 00 with +
+  BIND(REPLACE(?scrapeResult, """^00""", "+") AS ?replacedZeroZero)
+
+  BIND(REPLACE(?replacedZeroZero, """^0""", "+32") AS ?defaultBe)
+
+  BIND(REPLACE(?defaultBe, """\\+320""", "+32") AS ?extendedDefaultBe)
+
+  BIND(REPLACE(?extendedDefaultBe, """^320""", "+32") AS ?finalResult)
+
+  BIND(IF(REGEX(?finalResult, """^\\+((32\\d{8})|(324\\d{8})|((?!32)\\d+))$"""), CONCAT("tel:", ?finalResult), ?phoneNumber) AS ?cleanedPhoneNumber)
+}

config/delta-consumers/leidinggevenden/mapping/queries/dl2op/datetime-format/mandaat-einde.rq

@@ -0,0 +1,11 @@
+PREFIX mandaat: <http://data.vlaanderen.be/ns/mandaat#>
+PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
+
+CONSTRUCT {
+  ?s mandaat:einde ?endDateTime .
+} WHERE {
+  ?s mandaat:einde ?endDate .
+
+  BIND(SUBSTR(STR(?endDate), 1, 10) as ?endDateWithoutTimezone)
+  BIND(STRDT(CONCAT(?endDateWithoutTimezone, 'T02:00:00'), xsd:dateTime) as ?endDateTime)
+}

config/delta-consumers/leidinggevenden/mapping/queries/dl2op/datetime-format/mandaat-start.rq

@@ -0,0 +1,11 @@
+PREFIX mandaat: <http://data.vlaanderen.be/ns/mandaat#>
+PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
+
+CONSTRUCT {
+  ?s mandaat:start ?startDateTime .
+} WHERE {
+  ?s mandaat:start ?startDate .
+
+  BIND(SUBSTR(STR(?startDate), 1, 10) as ?startDateWithoutTimezone)
+  BIND(STRDT(CONCAT(?startDateWithoutTimezone, 'T02:00:00'), xsd:dateTime) as ?startDateTime)
+}

config/delta-consumers/leidinggevenden/mapping/queries/dl2op/inheritance/Agent.rq

@@ -0,0 +1,14 @@
+PREFIX dc_terms: <http://purl.org/dc/terms/>
+PREFIX lblodlg: <http://data.lblod.info/vocabularies/leidinggevenden/>
+PREFIX person: <http://www.w3.org/ns/person#>
+
+CONSTRUCT {
+  ?s a dc_terms:Agent .
+} WHERE {
+  ?s a ?type .
+
+  VALUES ?type {
+    lblodlg:Functionaris
+    person:Person
+  }
+}

config/delta-consumers/leidinggevenden/mapping/queries/dl2op/inheritance/AgentInPositie.rq

@@ -0,0 +1,12 @@
+PREFIX ch: <http://data.lblod.info/vocabularies/contacthub/>
+PREFIX lblodlg: <http://data.lblod.info/vocabularies/leidinggevenden/>
+
+CONSTRUCT {
+  ?s a ch:AgentInPositie .
+} WHERE {
+  ?s a ?type .
+
+  VALUES ?type {
+    lblodlg:Functionaris
+  }
+}

config/delta-consumers/leidinggevenden/mapping/queries/dl2op/inverse/holds-to-held-by.rq

@@ -0,0 +1,8 @@
+PREFIX org: <http://www.w3.org/ns/org#>
+
+CONSTRUCT {
+  ?o org:heldBy ?s.
+}
+WHERE {
+  ?s org:holds ?o.
+}

config/delta-consumers/leidinggevenden/mapping/queries/dl2op/pass/address.sparql

@@ -0,0 +1,10 @@
+CONSTRUCT {
+  ?s ?p ?o.
+} WHERE {
+  ?s a <http://www.w3.org/ns/locn#Address>;
+    ?p ?o.
+
+  FILTER NOT EXISTS {
+    ?s ?p "" .
+  }
+}

config/delta-consumers/leidinggevenden/mapping/queries/dl2op/pass/bestuursfunctie.sparql

@@ -0,0 +1,7 @@
+CONSTRUCT {
+  ?s <http://schema.org/contactPoint> ?contactpoint.
+}
+WHERE {
+  ?s a <http://data.lblod.info/vocabularies/leidinggevenden/Bestuursfunctie>.
+  ?s <http://schema.org/contactPoint> ?contactpoint.
+}

config/delta-consumers/leidinggevenden/mapping/queries/dl2op/pass/contactpoint.sparql

@@ -0,0 +1,11 @@
+CONSTRUCT {
+  ?s ?p ?o.
+} WHERE {
+  ?s a <http://schema.org/ContactPoint>;
+    ?p ?o.
+ FILTER (?p NOT IN (<http://schema.org/telephone>))
+
+ FILTER NOT EXISTS {
+   ?s ?p "" .
+ }
+}

config/delta-consumers/leidinggevenden/mapping/queries/dl2op/pass/functionaris.sparql

@@ -0,0 +1,14 @@
+PREFIX mandaat: <http://data.vlaanderen.be/ns/mandaat#>
+
+CONSTRUCT {
+  ?s ?p ?o.
+} WHERE {
+  ?s a <http://data.lblod.info/vocabularies/leidinggevenden/Functionaris>;
+    ?p ?o.
+
+  FILTER (?p NOT IN (mandaat:start, mandaat:einde))
+
+  FILTER NOT EXISTS {
+    ?s ?p "" .
+  }
+}

config/delta-consumers/leidinggevenden/mapping/queries/dl2op/pass/person.sparql

@@ -0,0 +1,16 @@
+CONSTRUCT {
+  ?s ?p ?o.
+} WHERE {
+  ?s a <http://www.w3.org/ns/person#Person>;
+    ?p ?o.
+
+  FILTER (?p IN (
+    <http://www.w3.org/1999/02/22-rdf-syntax-ns#type>,
+    <http://xmlns.com/foaf/0.1/familyName>,
+    <http://mu.semte.ch/vocabularies/core/uuid>
+    ))
+
+  FILTER NOT EXISTS {
+   ?s ?p "" .
+  }
+}

config/delta-consumers/leidinggevenden/mapping/queries/dl2op/predicate/person-foaf-name.sparql

@@ -0,0 +1,21 @@
+PREFIX foaf: <http://xmlns.com/foaf/0.1/>
+PREFIX persoon: <https://data.vlaanderen.be/ns/persoon#>
+PREFIX persoon_loket: <http://data.vlaanderen.be/ns/persoon#>
+PREFIX person: <http://www.w3.org/ns/person#>
+
+# DL and OP both use persoon:gebruikteVoornaam but with different semantics (!)
+
+# First name: persoon:gebruikteVoornaam => foaf:givenName
+# Alternatieve voornaam: foaf:name => persoon:gebruikteVoornaam
+
+# We do both in one query with an OPTIONAL because they need to happen simultaneously
+# not to end up in a loop
+
+# Part 1
+
+CONSTRUCT {
+  ?s persoon:gebruikteVoornaam ?name .
+} WHERE {
+  ?s a person:Person;
+    foaf:name ?name .
+}

config/delta-consumers/leidinggevenden/mapping/queries/dl2op/predicate/person-gebruikte-voornaam.sparql

@@ -0,0 +1,21 @@
+PREFIX foaf: <http://xmlns.com/foaf/0.1/>
+PREFIX persoon: <https://data.vlaanderen.be/ns/persoon#>
+PREFIX persoon_loket: <http://data.vlaanderen.be/ns/persoon#>
+PREFIX person: <http://www.w3.org/ns/person#>
+
+# DL and OP both use persoon:gebruikteVoornaam but with different semantics (!)
+
+# First name: persoon:gebruikteVoornaam => foaf:givenName
+# Alternatieve voornaam: foaf:name => persoon:gebruikteVoornaam
+
+# We do both in one query with an OPTIONAL because they need to happen simultaneously
+# not to end up in a loop
+
+# Part 2
+
+CONSTRUCT {
+  ?s foaf:givenName ?gebruikteVoornaam .
+} WHERE {
+  ?s a person:Person ;
+    persoon_loket:gebruikteVoornaam ?gebruikteVoornaam .
+}

config/delta-consumers/mandatarissen/mapping/queries/dl2op/datetime-format/mandaat-einde.rq

@@ -0,0 +1,11 @@
+PREFIX mandaat: <http://data.vlaanderen.be/ns/mandaat#>
+PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
+
+CONSTRUCT {
+  ?s mandaat:einde ?endDateTime .
+} WHERE {
+  ?s mandaat:einde ?endDate .
+
+  BIND(SUBSTR(STR(?endDate), 1, 10) as ?endDateWithoutTimezone)
+  BIND(STRDT(CONCAT(?endDateWithoutTimezone, 'T02:00:00'), xsd:dateTime) as ?endDateTime)
+}

config/delta-consumers/mandatarissen/mapping/queries/dl2op/datetime-format/mandaat-start.rq

@@ -0,0 +1,11 @@
+PREFIX mandaat: <http://data.vlaanderen.be/ns/mandaat#>
+PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
+
+CONSTRUCT {
+  ?s mandaat:start ?startDateTime .
+} WHERE {
+  ?s mandaat:start ?startDate .
+
+  BIND(SUBSTR(STR(?startDate), 1, 10) as ?startDateWithoutTimezone)
+  BIND(STRDT(CONCAT(?startDateWithoutTimezone, 'T02:00:00'), xsd:dateTime) as ?startDateTime)
+}

config/delta-consumers/mandatarissen/mapping/queries/dl2op/inheritance/Agent.rq

@@ -0,0 +1,14 @@
+PREFIX dc_terms: <http://purl.org/dc/terms/>
+PREFIX mandaat: <http://data.vlaanderen.be/ns/mandaat#>
+PREFIX person: <http://www.w3.org/ns/person#>
+
+CONSTRUCT {
+  ?s a dc_terms:Agent .
+} WHERE {
+  ?s a ?type .
+
+  VALUES ?type {
+    mandaat:Mandataris
+    person:Person
+  }
+}

config/delta-consumers/mandatarissen/mapping/queries/dl2op/inheritance/AgentInPositie.rq

@@ -0,0 +1,11 @@
+PREFIX ch: <http://data.lblod.info/vocabularies/contacthub/>
+PREFIX mandaat: <http://data.vlaanderen.be/ns/mandaat#>
+
+CONSTRUCT {
+  ?s a ch:AgentInPositie .
+} WHERE {
+  ?s a ?type .
+  VALUES ?type {
+    mandaat:Mandataris
+  }
+}

config/delta-consumers/mandatarissen/mapping/queries/dl2op/inverse/holds-to-held-by.rq

@@ -0,0 +1,8 @@
+PREFIX org: <http://www.w3.org/ns/org#>
+
+CONSTRUCT {
+  ?o org:heldBy ?s.
+}
+WHERE {
+  ?s org:holds ?o.
+}

config/delta-consumers/mandatarissen/mapping/queries/dl2op/pass/mandataris.sparql

@@ -0,0 +1,29 @@
+PREFIX mandaat: <http://data.vlaanderen.be/ns/mandaat#>
+
+CONSTRUCT {
+  ?s ?p ?o.
+} WHERE {
+  ?s a <http://data.vlaanderen.be/ns/mandaat#Mandataris>;
+    <http://www.w3.org/ns/org#holds> ?mandate;
+    ?p ?o.
+
+  ?orgaanInTime <http://www.w3.org/ns/org#hasPost> ?mandate ;
+    <http://data.vlaanderen.be/ns/mandaat#isTijdspecialisatieVan> ?orgaan .
+
+  ?orgaan <http://data.vlaanderen.be/ns/besluit#bestuurt> ?bestuur .
+  ?bestuur <http://data.vlaanderen.be/ns/besluit#classificatie> ?classification .
+
+
+  FILTER(?classification IN (
+    <http://data.vlaanderen.be/id/concept/BestuurseenheidClassificatieCode/5ab0e9b8a3b2ca7c5e000000>,
+    <http://data.vlaanderen.be/id/concept/BestuurseenheidClassificatieCode/5ab0e9b8a3b2ca7c5e000001>,
+    <http://data.vlaanderen.be/id/concept/BestuurseenheidClassificatieCode/5ab0e9b8a3b2ca7c5e000002>,
+    <http://data.vlaanderen.be/id/concept/BestuurseenheidClassificatieCode/5ab0e9b8a3b2ca7c5e000003>
+  ))
+
+  FILTER (?p NOT IN (mandaat:start, mandaat:einde))
+
+  FILTER NOT EXISTS {
+    ?s ?p "" .
+  }
+}

config/delta-consumers/mandatarissen/mapping/queries/dl2op/pass/person.sparql

@@ -0,0 +1,16 @@
+CONSTRUCT {
+  ?s ?p ?o.
+} WHERE {
+  ?s a <http://www.w3.org/ns/person#Person>;
+    ?p ?o.
+
+  FILTER (?p IN (
+    <http://www.w3.org/1999/02/22-rdf-syntax-ns#type>,
+    <http://xmlns.com/foaf/0.1/familyName>,
+    <http://mu.semte.ch/vocabularies/core/uuid>
+    ))
+
+  FILTER NOT EXISTS {
+   ?s ?p "" .
+  }
+}

config/delta-consumers/mandatarissen/mapping/queries/dl2op/predicate/person-foaf-name.rq

@@ -0,0 +1,21 @@
+PREFIX foaf: <http://xmlns.com/foaf/0.1/>
+PREFIX persoon: <https://data.vlaanderen.be/ns/persoon#>
+PREFIX persoon_loket: <http://data.vlaanderen.be/ns/persoon#>
+PREFIX person: <http://www.w3.org/ns/person#>
+
+# DL and OP both use persoon:gebruikteVoornaam but with different semantics (!)
+
+# First name: persoon:gebruikteVoornaam => foaf:givenName
+# Alternatieve voornaam: foaf:name => persoon:gebruikteVoornaam
+
+# We do both in one query with an OPTIONAL because they need to happen simultaneously
+# not to end up in a loop
+
+# Part 1
+
+CONSTRUCT {
+  ?s persoon:gebruikteVoornaam ?name .
+} WHERE {
+  ?s a person:Person;
+    foaf:name ?name .
+}

config/delta-consumers/mandatarissen/mapping/queries/dl2op/predicate/person-gebruikte-voornaam.rq

@@ -0,0 +1,22 @@
+
+PREFIX foaf: <http://xmlns.com/foaf/0.1/>
+PREFIX persoon: <https://data.vlaanderen.be/ns/persoon#>
+PREFIX persoon_loket: <http://data.vlaanderen.be/ns/persoon#>
+PREFIX person: <http://www.w3.org/ns/person#>
+
+# DL and OP both use persoon:gebruikteVoornaam but with different semantics (!)
+
+# First name: persoon:gebruikteVoornaam => foaf:givenName
+# Alternatieve voornaam: foaf:name => persoon:gebruikteVoornaam
+
+# We do both in one query with an OPTIONAL because they need to happen simultaneously
+# not to end up in a loop
+
+# Part 2
+
+CONSTRUCT {
+  ?s foaf:givenName ?gebruikteVoornaam .
+} WHERE {
+  ?s a person:Person ;
+    persoon_loket:gebruikteVoornaam ?gebruikteVoornaam .
+}

config/delta-consumers/worship-services-main-info/mapping/queries/dl2op/data-cleanup/phone.rq

@@ -0,0 +1,21 @@
+PREFIX schema: <http://schema.org/>
+
+CONSTRUCT {
+  ?s schema:telephone ?cleanedPhoneNumber .
+} WHERE {
+  ?s schema:telephone ?phoneNumber .
+
+  # Remove all non digits
+  BIND(REPLACE(?phoneNumber, """[^+\\d]""", "") AS ?scrapeResult)
+
+  # Replace leading 00 with +
+  BIND(REPLACE(?scrapeResult, """^00""", "+") AS ?replacedZeroZero)
+
+  BIND(REPLACE(?replacedZeroZero, """^0""", "+32") AS ?defaultBe)
+
+  BIND(REPLACE(?defaultBe, """\\+320""", "+32") AS ?extendedDefaultBe)
+
+  BIND(REPLACE(?extendedDefaultBe, """^320""", "+32") AS ?finalResult)
+
+  BIND(IF(REGEX(?finalResult, """^\\+((32\\d{8})|(324\\d{8})|((?!32)\\d+))$"""), CONCAT("tel:", ?finalResult), ?phoneNumber) AS ?cleanedPhoneNumber)
+}

config/delta-consumers/worship-services-main-info/mapping/queries/dl2op/date-to-datetime/contacthub-eindedatum.rq

@@ -0,0 +1,10 @@
+PREFIX ch: <http://data.lblod.info/vocabularies/contacthub/>
+PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
+
+CONSTRUCT {
+  ?s ch:eindedatum ?endDateTime .
+} WHERE {
+  ?s ch:eindedatum ?endDate .
+
+  BIND(STRDT(CONCAT(STR(?endDate), 'T02:00:00'), xsd:dateTime) as ?endDateTime)
+}

config/delta-consumers/worship-services-main-info/mapping/queries/dl2op/date-to-datetime/contacthub-startdatum.rq

@@ -0,0 +1,10 @@
+PREFIX ch: <http://data.lblod.info/vocabularies/contacthub/>
+PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
+
+CONSTRUCT {
+  ?s ch:startdatum ?startDateTime .
+} WHERE {
+  ?s ch:startdatum ?startDate .
+
+  BIND(STRDT(CONCAT(STR(?startDate), 'T02:00:00'), xsd:dateTime) as ?startDateTime)
+}

config/delta-consumers/worship-services-main-info/mapping/queries/dl2op/date-to-datetime/ere-geplande-einddatum-aanstelling.rq

@@ -0,0 +1,10 @@
+PREFIX ere: <http://data.lblod.info/vocabularies/erediensten/>
+PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
+
+CONSTRUCT {
+  ?s ere:geplandeEinddatumAanstelling ?dateTime .
+} WHERE {
+  ?s ere:geplandeEinddatumAanstelling ?date .
+
+  BIND(STRDT(CONCAT(STR(?date), 'T02:00:00'), xsd:dateTime) as ?dateTime)
+}

config/delta-consumers/worship-services-main-info/mapping/queries/dl2op/inheritance/Agent.rq

@@ -0,0 +1,20 @@
+PREFIX ch: <http://data.lblod.info/vocabularies/contacthub/>
+PREFIX dc_terms: <http://purl.org/dc/terms/>
+PREFIX ere: <http://data.lblod.info/vocabularies/erediensten/>
+PREFIX lblodlg: <http://data.lblod.info/vocabularies/leidinggevenden/>
+PREFIX mandaat: <http://data.vlaanderen.be/ns/mandaat#>
+PREFIX person: <http://www.w3.org/ns/person#>
+
+CONSTRUCT {
+  ?s a dc_terms:Agent .
+} WHERE {
+  ?s a ?type .
+
+  VALUES ?type {
+    lblodlg:Functionaris
+    mandaat:Mandataris
+    ere:EredienstMandataris
+    ere:RolBedienaar
+    person:Person
+  }
+}

config/delta-consumers/worship-services-main-info/mapping/queries/dl2op/inheritance/AgentInPositie.rq

@@ -0,0 +1,17 @@
+PREFIX ch: <http://data.lblod.info/vocabularies/contacthub/>
+PREFIX ere: <http://data.lblod.info/vocabularies/erediensten/>
+PREFIX lblodlg: <http://data.lblod.info/vocabularies/leidinggevenden/>
+PREFIX mandaat: <http://data.vlaanderen.be/ns/mandaat#>
+
+CONSTRUCT {
+  ?s a ch:AgentInPositie .
+} WHERE {
+  ?s a ?type .
+
+  VALUES ?type {
+    lblodlg:Functionaris
+    mandaat:Mandataris
+    ere:EredienstMandataris
+    ere:RolBedienaar
+  }
+}

config/delta-consumers/worship-services-main-info/mapping/queries/dl2op/inverse/held-by-to-holds.rq

@@ -0,0 +1,8 @@
+PREFIX org: <http://www.w3.org/ns/org#>
+
+CONSTRUCT {
+  ?o org:holds ?s.
+}
+WHERE {
+  ?s org:heldBy ?o.
+}

config/delta-consumers/worship-services-main-info/mapping/queries/dl2op/inverse/holds-to-held-by.rq

@@ -0,0 +1,8 @@
+PREFIX org: <http://www.w3.org/ns/org#>
+
+CONSTRUCT {
+  ?o org:heldBy ?s.
+}
+WHERE {
+  ?s org:holds ?o.
+}

config/delta-consumers/worship-services-main-info/mapping/queries/dl2op/pass/address.sparql

@@ -0,0 +1,11 @@
+
+CONSTRUCT {
+  ?s ?p ?o.
+} WHERE {
+  ?s a <http://www.w3.org/ns/locn#Address>;
+    ?p ?o.
+
+  FILTER NOT EXISTS {
+    ?s ?p "" .
+  }
+}

config/delta-consumers/worship-services-main-info/mapping/queries/dl2op/pass/bestuursfunctie.sparql

@@ -0,0 +1,7 @@
+CONSTRUCT {
+  ?s <http://schema.org/contactPoint> ?contactpoint.
+}
+WHERE {
+  ?s a <http://data.lblod.info/vocabularies/leidinggevenden/Bestuursfunctie>.
+  ?s <http://schema.org/contactPoint> ?contactpoint.
+}

config/delta-consumers/worship-services-main-info/mapping/queries/dl2op/pass/contactpoint.sparql

@@ -0,0 +1,12 @@
+CONSTRUCT {
+  ?s ?p ?o.
+} WHERE {
+  ?s a <http://schema.org/ContactPoint>;
+    ?p ?o.
+
+ FILTER (?p NOT IN (<http://schema.org/telephone>))
+
+ FILTER NOT EXISTS {
+   ?s ?p "" .
+ }
+}

config/delta-consumers/worship-services-main-info/mapping/queries/dl2op/pass/eredienst-mandataris.sparql

@@ -0,0 +1,15 @@
+PREFIX ch: <http://data.lblod.info/vocabularies/contacthub/>
+PREFIX ere: <http://data.lblod.info/vocabularies/erediensten/>
+
+CONSTRUCT {
+  ?s ?p ?o.
+} WHERE {
+  ?s a <http://data.lblod.info/vocabularies/erediensten/EredienstMandataris>;
+    ?p ?o.
+
+ FILTER (?p NOT IN (ch:startdatum, ch:eindedatum, ere:geplandeEinddatumAanstelling))
+
+ FILTER NOT EXISTS {
+   ?s ?p "" .
+ }
+}

config/delta-consumers/worship-services-main-info/mapping/queries/dl2op/pass/person.sparql

@@ -0,0 +1,16 @@
+CONSTRUCT {
+  ?s ?p ?o.
+} WHERE {
+  ?s a <http://www.w3.org/ns/person#Person>;
+    ?p ?o.
+
+  FILTER (?p IN (
+    <http://www.w3.org/1999/02/22-rdf-syntax-ns#type>,
+    <http://mu.semte.ch/vocabularies/core/uuid>,
+    <http://xmlns.com/foaf/0.1/familyName>
+  ))
+
+  FILTER NOT EXISTS {
+   ?s ?p "" .
+  }
+}

config/delta-consumers/worship-services-main-info/mapping/queries/dl2op/pass/rol-bedienaar.sparql

@@ -0,0 +1,10 @@
+CONSTRUCT {
+  ?s ?p ?o.
+} WHERE {
+  ?s a <http://data.lblod.info/vocabularies/erediensten/RolBedienaar>;
+    ?p ?o.
+
+ FILTER NOT EXISTS {
+   ?s ?p "" .
+ }
+}

config/delta-consumers/worship-services-main-info/mapping/queries/dl2op/predicate/gebruikte-voornaam.rq

@@ -0,0 +1,14 @@
+PREFIX foaf: <http://xmlns.com/foaf/0.1/>
+PREFIX persoon: <https://data.vlaanderen.be/ns/persoon#>
+PREFIX person: <http://www.w3.org/ns/person#>
+
+# DL and OP both use persoon:gebruikteVoornaam but with different semantics (!)
+
+# Alternatieve voornaam: foaf:name => persoon:gebruikteVoornaam
+
+CONSTRUCT {
+  ?s persoon:gebruikteVoornaam ?name .
+} WHERE {
+  ?s a person:Person ;
+    foaf:name ?name .
+}

config/delta-consumers/worship-services-main-info/mapping/queries/dl2op/predicate/given-name.rq

@@ -0,0 +1,14 @@
+PREFIX foaf: <http://xmlns.com/foaf/0.1/>
+PREFIX persoon_loket: <http://data.vlaanderen.be/ns/persoon#>
+PREFIX person: <http://www.w3.org/ns/person#>
+
+# DL and OP both use persoon:gebruikteVoornaam but with different semantics (!)
+
+# First name: persoon:gebruikteVoornaam => foaf:givenName
+
+CONSTRUCT {
+  ?s foaf:givenName ?gebruikteVoornaam .
+} WHERE {
+  ?s a person:Person ;
+    persoon_loket:gebruikteVoornaam ?gebruikteVoornaam .
+}

config/delta-consumers/worship-services-private-info/mapping/queries/dl2op/pass/geboorte.sparql

@@ -0,0 +1,10 @@
+CONSTRUCT {
+  ?s ?p ?o.
+} WHERE {
+  ?s a <http://data.vlaanderen.be/ns/persoon#Geboorte>;
+    ?p ?o.
+
+ FILTER NOT EXISTS {
+   ?s ?p "" .
+ }
+}

config/delta-consumers/worship-services-private-info/mapping/queries/dl2op/persoon-http-to-https/Geboorte.rq

@@ -0,0 +1,10 @@
+PREFIX persoon: <https://data.vlaanderen.be/ns/persoon#>
+PREFIX persoon_loket: <http://data.vlaanderen.be/ns/persoon#>
+
+# Map from http persoon to https persoon namespace
+
+CONSTRUCT {
+  ?s a persoon:Geboorte
+} WHERE {
+  ?s a persoon_loket:Geboorte .
+}

config/delta-consumers/worship-services-private-info/mapping/queries/dl2op/persoon-http-to-https/datum.rq

@@ -0,0 +1,10 @@
+PREFIX persoon: <https://data.vlaanderen.be/ns/persoon#>
+PREFIX persoon_loket: <http://data.vlaanderen.be/ns/persoon#>
+
+# Map from http persoon to https persoon namespace
+
+CONSTRUCT {
+  ?s persoon:datum ?o .
+} WHERE {
+  ?s persoon_loket:datum ?o .
+}

config/delta-consumers/worship-services-private-info/mapping/queries/dl2op/persoon-http-to-https/geslacht.rq

@@ -0,0 +1,10 @@
+PREFIX persoon: <https://data.vlaanderen.be/ns/persoon#>
+PREFIX persoon_loket: <http://data.vlaanderen.be/ns/persoon#>
+
+# Map from http persoon to https persoon namespace
+
+CONSTRUCT {
+  ?s persoon:geslacht ?o .
+} WHERE {
+  ?s persoon_loket:geslacht ?o .
+}

config/delta-consumers/worship-services-private-info/mapping/queries/dl2op/persoon-http-to-https/heeftGeboorte.rq

@@ -0,0 +1,10 @@
+PREFIX persoon: <https://data.vlaanderen.be/ns/persoon#>
+PREFIX persoon_loket: <http://data.vlaanderen.be/ns/persoon#>
+
+# Map from http persoon to https persoon namespace
+
+CONSTRUCT {
+  ?s persoon:heeftGeboorte ?o .
+} WHERE {
+  ?s persoon_loket:heeftGeboorte ?o .
+}

config/delta-consumers/worship-services-private-info/mapping/queries/dl2op/persoon-http-to-https/heeftNationaliteit.rq

@@ -0,0 +1,10 @@
+PREFIX persoon: <https://data.vlaanderen.be/ns/persoon#>
+PREFIX persoon_loket: <http://data.vlaanderen.be/ns/persoon#>
+
+# Map from http persoon to https persoon namespace
+
+CONSTRUCT {
+  ?s persoon:heeftNationaliteit ?o .
+} WHERE {
+  ?s persoon_loket:heeftNationaliteit ?o .
+}

config/delta-consumers/worship-services-private-info/mapping/queries/dl2op/persoon-http-to-https/registratie.rq

@@ -0,0 +1,10 @@
+PREFIX persoon: <https://data.vlaanderen.be/ns/persoon#>
+PREFIX persoon_loket: <http://data.vlaanderen.be/ns/persoon#>
+
+# Map from http persoon to https persoon namespace
+
+CONSTRUCT {
+  ?s persoon:registratie ?o .
+} WHERE {
+  ?s persoon_loket:registratie ?o .
+}

config/delta-consumers/worship-services-private-info/mapping/queries/dl2op/rrn/identifier.rq

@@ -0,0 +1,19 @@
+PREFIX generiek: <https://data.vlaanderen.be/ns/generiek#>
+PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
+PREFIX mu: <http://mu.semte.ch/vocabularies/core/>
+
+CONSTRUCT {
+  ?identifier
+    generiek:gestructureerdeIdentificator ?structuredIdentifier ;
+    skos:notation "Rijksregisternummer" .
+
+  ?structuredIdentifier a generiek:GestructureerdeIdentificator ;
+    mu:uuid ?uuid ;
+    generiek:lokaleIdentificator ?notation .
+}
+WHERE {
+    ?identifier skos:notation ?notation.
+
+    BIND(MD5(CONCAT(STR(?identifier),"STRUCTURED_IDENTIFIER")) as ?uuid)
+    BIND(IRI(CONCAT("http://data.lblod.info/id/gestructureerdeIdentificatoren/", ?uuid)) AS ?structuredIdentifier)
+}

config/delta-consumers/worship-services-private-info/mapping/queries/dl2op/rrn/registratie.rq

@@ -0,0 +1,10 @@
+PREFIX adms: <http://www.w3.org/ns/adms#>
+PREFIX persoon: <https://data.vlaanderen.be/ns/persoon#>
+
+# identifier property in the export is only used for RRN.
+CONSTRUCT {
+  ?person persoon:registratie ?rrn .
+}
+WHERE {
+  ?person adms:identifier ?rrn .
+}

config/delta-producer/public/export.json

@@ -157,6 +157,9 @@
         "http://www.w3.org/2004/02/skos/core#prefLabel",
         "http://www.w3.org/ns/regorg#legalName",
         "http://www.w3.org/2004/02/skos/core#altLabel",
+        "http://www.w3.org/ns/regorg#orgStatus",
+        "http://www.w3.org/ns/org#hasPrimarySite",
+        "http://www.w3.org/ns/org#hasSite",
         "http://www.w3.org/ns/org#linkedTo",
         "http://data.lblod.info/vocabularies/erediensten/typeEredienst",
         "http://www.w3.org/ns/org#classification",
@@ -204,7 +207,28 @@
         "http://mu.semte.ch/vocabularies/core/uuid",
         "http://www.w3.org/ns/org#siteAddress",
         "https://data.vlaanderen.be/ns/organisatie#bestaatUit",
-        "http://data.lblod.info/vocabularies/erediensten/vestigingstype"
+        "http://data.lblod.info/vocabularies/erediensten/vestigingstype",
+        "http://purl.org/dc/terms/description"
+      ]
+    },
+    {
+      "type": "http://www.w3.org/ns/org#Site",
+      "graphsFilter": [
+        "http://mu.semte.ch/graphs/shared",
+        "http://mu.semte.ch/graphs/worship-service",
+        "http://mu.semte.ch/graphs/administrative-unit"
+      ],
+      "pathToConceptScheme": [
+        "^http://www.w3.org/ns/org#hasSite",
+        "http://www.w3.org/ns/org#classification",
+        "http://www.w3.org/2004/02/skos/core#inScheme"
+      ],
+      "properties": [
+        "http://mu.semte.ch/vocabularies/core/uuid",
+        "http://www.w3.org/ns/org#siteAddress",
+        "https://data.vlaanderen.be/ns/organisatie#bestaatUit",
+        "http://data.lblod.info/vocabularies/erediensten/vestigingstype",
+        "http://purl.org/dc/terms/description"
       ]
     },
     {
@@ -234,6 +258,33 @@
         "http://purl.org/dc/terms/source"
       ]
     },
+    {
+      "type": "http://www.w3.org/ns/locn#Address",
+      "graphsFilter": [
+        "http://mu.semte.ch/graphs/shared",
+        "http://mu.semte.ch/graphs/worship-service",
+        "http://mu.semte.ch/graphs/administrative-unit"
+      ],
+      "pathToConceptScheme": [
+        "^https://data.vlaanderen.be/ns/organisatie#bestaatUit",
+        "^http://www.w3.org/ns/org#hasSite",
+        "http://www.w3.org/ns/org#classification",
+        "http://www.w3.org/2004/02/skos/core#inScheme"
+      ],
+      "properties": [
+        "http://mu.semte.ch/vocabularies/core/uuid",
+        "https://data.vlaanderen.be/ns/adres#Adresvoorstelling.huisnummer",
+        "https://data.vlaanderen.be/ns/adres#Adresvoorstelling.busnummer",
+        "http://www.w3.org/ns/locn#thoroughfare",
+        "http://www.w3.org/ns/locn#postCode",
+        "https://data.vlaanderen.be/ns/adres#gemeentenaam",
+        "http://www.w3.org/ns/locn#adminUnitL2",
+        "https://data.vlaanderen.be/ns/adres#land",
+        "http://www.w3.org/ns/locn#fullAddress",
+        "https://data.vlaanderen.be/ns/adres#verwijstNaar",
+        "http://purl.org/dc/terms/source"
+      ]
+    },
     {
       "type": "http://schema.org/ContactPoint",
       "graphsFilter": [
@@ -258,6 +309,30 @@
         "http://schema.org/telephone"
       ]
     },
+    {
+      "type": "http://schema.org/ContactPoint",
+      "graphsFilter": [
+        "http://mu.semte.ch/graphs/shared",
+        "http://mu.semte.ch/graphs/worship-service",
+        "http://mu.semte.ch/graphs/administrative-unit"
+      ],
+      "pathToConceptScheme": [
+        "^http://www.w3.org/ns/org#siteAddress",
+        "^http://www.w3.org/ns/org#hasSite",
+        "http://www.w3.org/ns/org#classification",
+        "http://www.w3.org/2004/02/skos/core#inScheme"
+      ],
+      "properties": [
+        "http://mu.semte.ch/vocabularies/core/uuid",
+        "http://www.w3.org/ns/locn#address",
+        "http://xmlns.com/foaf/0.1/page",
+        "http://schema.org/faxNumber",
+        "http://schema.org/contactType",
+        "http://xmlns.com/foaf/0.1/page",
+        "http://schema.org/email",
+        "http://schema.org/telephone"
+      ]
+    },
     {
       "type": "http://www.w3.org/ns/adms#Identifier",
       "graphsFilter": [
@@ -497,6 +572,108 @@
         "http://mu.semte.ch/vocabularies/core/uuid",
         "http://www.w3.org/2004/02/skos/core#prefLabel"
       ]
+    },
+    {
+      "type": "http://lblod.data.gift/vocabularies/organisatie/MandatarisStatusCode",
+      "graphsFilter": [
+        "http://mu.semte.ch/graphs/public"
+      ],
+      "pathToConceptScheme": [
+        "http://www.w3.org/2004/02/skos/core#inScheme"
+      ],
+      "properties": [
+        "http://mu.semte.ch/vocabularies/core/uuid",
+        "http://www.w3.org/2004/02/skos/core#prefLabel"
+      ]
+    },
+    {
+      "type": "http://lblod.data.gift/vocabularies/organisatie/BestuursfunctieCode",
+      "graphsFilter": [
+        "http://mu.semte.ch/graphs/public"
+      ],
+      "pathToConceptScheme": [
+        "http://www.w3.org/2004/02/skos/core#inScheme"
+      ],
+      "properties": [
+        "http://mu.semte.ch/vocabularies/core/uuid",
+        "http://www.w3.org/2004/02/skos/core#prefLabel"
+      ]
+    },
+    {
+      "type": "http://data.lblod.info/vocabularies/leidinggevenden/FunctionarisStatusCode",
+      "graphsFilter": [
+        "http://mu.semte.ch/graphs/public"
+      ],
+      "pathToConceptScheme": [
+        "http://www.w3.org/2004/02/skos/core#inScheme"
+      ],
+      "properties": [
+        "http://mu.semte.ch/vocabularies/core/uuid",
+        "http://www.w3.org/2004/02/skos/core#prefLabel"
+      ]
+    },
+    {
+      "type": "http://mu.semte.ch/vocabularies/ext/OrganizationClassificationCode",
+      "graphsFilter": [
+        "http://mu.semte.ch/graphs/public"
+      ],
+      "pathToConceptScheme": [
+        "http://www.w3.org/2004/02/skos/core#inScheme"
+      ],
+      "properties": [
+        "http://mu.semte.ch/vocabularies/core/uuid",
+        "http://www.w3.org/2004/02/skos/core#prefLabel"
+      ]
+    },
+    {
+      "type": "http://www.w3.org/ns/prov#Location",
+      "graphsFilter": [
+        "http://mu.semte.ch/graphs/shared",
+        "http://mu.semte.ch/graphs/administrative-unit"
+      ],
+      "hasRegexGraphsFilter": false,
+      "properties": [
+        "http://www.w3.org/2000/01/rdf-schema#label",
+        "http://mu.semte.ch/vocabularies/core/uuid",
+        "http://mu.semte.ch/vocabularies/ext/werkingsgebiedNiveau",
+        "http://www.opengis.net/ont/geosparql#sfWithin",
+        "http://www.w3.org/2004/02/skos/core#exactMatch"
+      ]
+    },
+    {
+      "type": "http://www.w3.org/2004/02/skos/core#Concept",
+      "graphsFilter": [
+        "http://mu.semte.ch/graphs/public"
+      ],
+      "pathToConceptScheme": [
+        "http://www.w3.org/2004/02/skos/core#inScheme"
+      ],
+      "properties": [
+        "http://mu.semte.ch/vocabularies/core/uuid",
+        "http://www.w3.org/2004/02/skos/core#altLabel",
+        "http://www.w3.org/2004/02/skos/core#broadMatch",
+        "http://www.w3.org/2004/02/skos/core#broader",
+        "http://www.w3.org/2004/02/skos/core#exactMatch",
+        "http://www.w3.org/2004/02/skos/core#inScheme",
+        "http://www.w3.org/2004/02/skos/core#narrowMatch",
+        "http://www.w3.org/2004/02/skos/core#narrower",
+        "http://www.w3.org/2004/02/skos/core#notation",
+        "http://www.w3.org/2004/02/skos/core#prefLabel",
+        "http://www.w3.org/2004/02/skos/core#relatedMatch",
+        "http://www.w3.org/2004/02/skos/core#topConceptOf",
+        "https://schema.org/endDate",
+        "https://schema.org/startDate"
+      ]
+    },
+    {
+      "type": "http://publications.europa.eu/ontology/euvoc#Country",
+      "graphsFilter": [
+        "http://mu.semte.ch/graphs/public"
+      ],
+      "properties": [
+        "http://www.w3.org/2004/02/skos/core#prefLabel",
+        "http://www.w3.org/2000/01/rdf-schema#label"
+      ]
     }
   ]
 }

config/dev-migrations/20241003152155-remove-invalid-generic-memberships.sparql

@@ -0,0 +1,31 @@
+PREFIX org: <http://www.w3.org/ns/org#>
+
+DELETE {
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?membership ?p ?o .
+  }
+} WHERE {
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?membership a org:Membership ;
+    org:member ?member ;
+    org:organization ?organization ;
+    org:role <http://data.lblod.info/id/rollen/4ec7d5c39bdc4e84b4174379b9e22ad8> ; # has a relation with
+    ?p ?o .
+
+    ?organization org:classification ?organizationClass .
+    FILTER (?organizationClass IN
+            (
+             <http://data.vlaanderen.be/id/concept/BestuurseenheidClassificatieCode/ea446861-2c51-45fa-afd3-4e4a37b71562> , # HVZ
+             <http://data.vlaanderen.be/id/concept/BestuurseenheidClassificatieCode/80310756-ce0a-4a1b-9b8e-7c01b6cc7a2d> , # APB
+             <http://data.vlaanderen.be/id/concept/BestuurseenheidClassificatieCode/d01bb1f6-2439-4e33-9c25-1fc295de2e71> , # Dienstverlenende vereniging
+             <http://data.vlaanderen.be/id/concept/BestuurseenheidClassificatieCode/cd93f147-3ece-4308-acab-5c5ada3ec63d> , # Opdrachthoudende vereniging
+             <http://data.vlaanderen.be/id/concept/BestuurseenheidClassificatieCode/4b8450cf-a326-4c66-9e63-b4ec10acc7f6> , # Opdrachthoudende vereniging met private deelname
+             <http://data.vlaanderen.be/id/concept/BestuurseenheidClassificatieCode/b156b67f-c5f4-4584-9b30-4c090be02fdc> , # Projectvereniging
+             <http://data.vlaanderen.be/id/concept/BestuurseenheidClassificatieCode/a3922c6d-425b-474f-9a02-ffb71a436bfc> , # Politiezone
+             <http://data.vlaanderen.be/id/concept/BestuurseenheidClassificatieCode/34b5af85-dc9f-468f-9e03-ef89b174c267> , # Autonome verzorgingsinstelling
+             <http://data.vlaanderen.be/id/concept/BestuurseenheidClassificatieCode/36a82ba0-7ff1-4697-a9dd-2e94df73b721> , # AGB
+             <http://data.vlaanderen.be/id/concept/BestuurseenheidClassificatieCode/e8294b73-87c9-4fa2-9441-1937350763c9> # Welzijnsvereniging
+             )
+            )
+  }
+}

config/dispatcher/dispatcher.ex

@@ -1,364 +1,432 @@
 defmodule Dispatcher do
   use Matcher
 
-  define_accept_types [
+  define_accept_types(
     html: ["text/html", "application/xhtml+html"],
     json: ["application/json", "application/vnd.api+json"],
     upload: ["multipart/form-data"],
     sparql_json: ["application/sparql-results+json"],
-    any: [ "*/*" ],
-  ]
-
-  define_layers [ :api_services, :api, :frontend, :not_found ]
+    any: ["*/*"]
+  )
 
+  define_layers([:api_services, :api, :frontend, :not_found])
 
   ###############################################################
   # domain.json
   ###############################################################
 
-  match "/people/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/people/"
+  match "/people/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/people/")
   end
 
-  match "/agents-in-position/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/agents-in-position/"
+  match "/agents-in-position/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/agents-in-position/")
   end
 
-  match "/posts/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/posts/"
+  match "/posts/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/posts/")
   end
 
-  match "/mandatories/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/mandatories/"
+  match "/mandatories/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/mandatories/")
   end
 
-  match "/mandatory-status-codes/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/mandatory-status-codes/"
+  match "/mandatory-status-codes/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/mandatory-status-codes/")
   end
 
-  match "/worship-mandatories/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/worship-mandatories/"
+  match "/worship-mandatories/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/worship-mandatories/")
   end
 
-  match "/half-elections/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/half-elections/"
+  match "/half-elections/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/half-elections/")
   end
 
-  match "/contact-points/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/contact-points/"
+  match "/contact-points/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/contact-points/")
   end
 
-  match "/mandates/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/mandates/"
+  match "/mandates/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/mandates/")
   end
 
-  match "/board-positions/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/board-positions/"
+  match "/board-positions/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/board-positions/")
   end
 
-  match "/board-position-codes/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/board-position-codes/"
+  match "/board-position-codes/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/board-position-codes/")
   end
 
-  match "/functionaries/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/functionaries/"
+  match "/functionaries/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/functionaries/")
   end
 
-  match "/agent-status-codes/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/agent-status-codes/"
+  match "/agent-status-codes/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/agent-status-codes/")
   end
 
-  match "/roles/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/roles/"
+  match "/roles/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/roles/")
   end
 
-  match "/organizations/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/organizations/"
+  match "/organizations/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/organizations/")
   end
 
-  match "/registered-organizations/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/registered-organizations/"
+  match "/registered-organizations/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/registered-organizations/")
   end
 
-  match "/administrative-units/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/administrative-units/"
+  match "/administrative-units/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/administrative-units/")
   end
 
-  match "/administrative-unit-classification-codes/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/administrative-unit-classification-codes/"
+  match "/administrative-unit-classification-codes/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/administrative-unit-classification-codes/")
   end
 
-  match "/organization-classification-codes/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/organization-classification-codes/"
+  match "/organization-classification-codes/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/organization-classification-codes/")
   end
 
-  match "/registered-organization-classification-codes/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/registered-organization-classification-codes/"
+  match "/registered-organization-classification-codes/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/registered-organization-classification-codes/")
   end
 
-  match "/worship-administrative-units/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/worship-administrative-units/"
+  match "/worship-administrative-units/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/worship-administrative-units/")
   end
 
-  match "/worship-services/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/worship-services/"
+  match "/worship-services/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/worship-services/")
   end
 
-  match "/recognized-worship-types/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/recognized-worship-types/"
+  match "/recognized-worship-types/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/recognized-worship-types/")
   end
 
-  match "/central-worship-services/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/central-worship-services/"
+  match "/central-worship-services/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/central-worship-services/")
   end
 
-  match "/representative-bodies/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/representative-bodies/"
+  match "/representative-bodies/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/representative-bodies/")
   end
 
-  match "/governing-bodies/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/governing-bodies/"
+  match "/governing-bodies/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/governing-bodies/")
   end
 
-  match "/governing-body-classification-codes/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/governing-body-classification-codes/"
+  match "/governing-body-classification-codes/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/governing-body-classification-codes/")
   end
 
-  match "/local-involvements/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/local-involvements/"
+  match "/local-involvements/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/local-involvements/")
   end
 
-  match "/identifiers/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/identifiers/"
+  match "/identifiers/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/identifiers/")
   end
 
-  match "/structured-identifiers/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/structured-identifiers/"
+  match "/structured-identifiers/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/structured-identifiers/")
   end
 
-  match "/addresses/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/addresses/"
+  match "/addresses/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/addresses/")
   end
 
-  match "/sites/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/sites/"
+  match "/sites/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/sites/")
   end
 
-  match "/change-events/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/change-events/"
+  match "/change-events/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/change-events/")
   end
 
-  match "/change-event-types/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/change-event-types/"
+  match "/change-event-types/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/change-event-types/")
   end
 
-  match "/change-event-results/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/change-event-results/"
+  match "/change-event-results/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/change-event-results/")
   end
 
-  match "/organization-status-codes/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/organization-status-codes/"
+  match "/organization-status-codes/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/organization-status-codes/")
   end
 
-  match "/locations/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/locations/"
+  match "/locations/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/locations/")
   end
 
-  match "/involvement-types/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/involvement-types/"
+  match "/involvement-types/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/involvement-types/")
   end
 
-  match "/ministers/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/ministers/"
+  match "/ministers/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/ministers/")
   end
 
-  match "/minister-conditions/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/minister-conditions/"
+  match "/minister-conditions/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/minister-conditions/")
   end
 
-  match "/dates-of-birth/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/dates-of-birth/"
+  match "/dates-of-birth/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/dates-of-birth/")
   end
 
-  match "/nationalities/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/nationalities/"
+  match "/nationalities/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/nationalities/")
   end
 
-  match "/gender-codes/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/gender-codes/"
-  end
-  match "/concepts/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/concepts/"
+  match "/gender-codes/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/gender-codes/")
   end
 
-  match "/minister-positions/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/minister-positions/"
+  match "/concepts/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/concepts/")
   end
 
-  match "/minister-position-functions/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/minister-position-functions/"
+  match "/minister-positions/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/minister-positions/")
   end
 
-  match "/financing-codes/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/financing-codes/"
+  match "/minister-position-functions/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/minister-position-functions/")
   end
 
-  match "/minister-condition-criterions/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/minister-condition-criterions/"
+  match "/financing-codes/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/financing-codes/")
   end
 
-  match "/document-types-criterions/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/document-types-criterions/"
+  match "/minister-condition-criterions/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/minister-condition-criterions/")
   end
 
-  match "/site-types/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/site-types/"
+  match "/document-types-criterions/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/document-types-criterions/")
   end
 
-  match "/request-reasons/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/request-reasons/"
+  match "/site-types/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/site-types/")
   end
 
-  match "/decisions/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/decisions/"
+  match "/request-reasons/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/request-reasons/")
   end
 
-  match "/decision-activities/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/decision-activities/"
+  match "/decisions/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/decisions/")
+  end
+
+  match "/decision-activities/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/decision-activities/")
+  end
+
+  match "/memberships/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/memberships/")
+  end
+
+  match "/membership-roles/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/membership-roles/")
   end
 
   # There is currently no logic to invalidate the kboOrganization cache when wegwijse updates the content
-  match "/kbo-organizations/*path", %{ accept: [:any], layer: :api} do
-    Proxy.forward conn, path, "http://resource/kbo-organizations/"
+  match "/kbo-organizations/*path", %{accept: [:any], layer: :api} do
+    Proxy.forward(conn, path, "http://resource/kbo-organizations/")
+  end
+
+  # Dashboard
+  match "/jobs/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/jobs/")
+  end
+
+  match "/tasks/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/tasks/")
+  end
+
+  match "/data-containers/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/data-containers/")
+  end
+
+  match "/job-errors/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/job-errors/")
+  end
+
+  match "/reports/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://resource/reports/")
+  end
+
+  match "/log-entries/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://resource/log-entries/")
+  end
+
+  match "/log-levels/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://resource/log-levels/")
+  end
+
+  match "/status-codes/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://resource/status-codes/")
+  end
+
+  match "/log-sources/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://resource/log-sources/")
+  end
+
+  match "/remote-data-objects/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://resource/remote-data-objects/")
   end
 
   ###############
   # LOGIN
   ###############
 
-  match "/accounts", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, [], "http://resource/accounts/"
-  end
-  match "/accounts/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://accountdetail/accounts/"
-  end
-  match "/active-role/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://roles/"
-  end
-  match "/groups/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://resource/groups/"
+  match "/accounts", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, [], "http://resource/accounts/")
   end
 
-  match "/mock/sessions/*path", %{ accept: [:any], layer: :api} do
-    Proxy.forward conn, path, "http://mocklogin/sessions/"
+  match "/accounts/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://accountdetail/accounts/")
   end
 
-  match "/sessions/*path", %{ accept: [:any], layer: :api} do
-    Proxy.forward conn, path, "http://login/sessions/"
+  match "/active-role/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://roles/")
+  end
+
+  match "/groups/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://resource/groups/")
+  end
+
+  match "/mock/sessions/*path", %{accept: [:any], layer: :api} do
+    Proxy.forward(conn, path, "http://mocklogin/sessions/")
+  end
+
+  match "/sessions/*path", %{ reverse_host: ["dashboard" | _rest] } do
+    Proxy.forward conn, path, "http://dashboard-login/sessions/"
+  end
+
+  match "/sessions/*path", %{accept: [:any], layer: :api} do
+    Proxy.forward(conn, path, "http://login/sessions/")
   end
 
   ###############################################################
   # SEARCH
   ###############################################################
 
-  match "/search/*path", %{  accept: %{ json: true }, layer: :api_services} do
-    Proxy.forward conn, path, "http://search/"
+  match "/search/*path", %{accept: %{json: true}, layer: :api_services} do
+    Proxy.forward(conn, path, "http://search/")
   end
 
   ###############
   # API SERVICES
   ###############
 
-  match "/v3/api-docs/*path", %{ layer: :api_services, accept: %{ json: true } } do
-    forward conn, path, "http://kalliope-api/v3/api-docs/"
+  match "/v3/api-docs/*path", %{layer: :api_services, accept: %{json: true}} do
+    forward(conn, path, "http://kalliope-api/v3/api-docs/")
   end
 
-  match "/consolidated/*path", %{ layer: :api_services, accept: %{ json: true } } do
-    forward conn, path, "http://kalliope-api/consolidated/"
+  match "/consolidated/*path", %{layer: :api_services, accept: %{json: true}} do
+    forward(conn, path, "http://kalliope-api/consolidated/")
   end
 
-  match "/adresses-register/*path", %{ layer: :api_services, accept: %{ json: true } } do
-    forward conn, path, "http://adressenregister/"
+  match "/adresses-register/*path", %{layer: :api_services, accept: %{json: true}} do
+    forward(conn, path, "http://adressenregister/")
   end
 
-  match "/worship/person-information-updates/*path", %{ layer: :api_services, accept: %{ json: true } } do
-    forward conn, path, "http://privacy-worship/person-information-updates"
+  match "/worship/person-information-updates/*path", %{
+    layer: :api_services,
+    accept: %{json: true}
+  } do
+    forward(conn, path, "http://privacy-worship/person-information-updates")
   end
 
-  match "/worship/person-information-requests/*path", %{ layer: :api_services, accept: %{ json: true } } do
-    forward conn, path, "http://privacy-worship/person-information-requests"
+  match "/worship/person-information-requests/*path", %{
+    layer: :api_services,
+    accept: %{json: true}
+  } do
+    forward(conn, path, "http://privacy-worship/person-information-requests")
   end
 
-  match "/worship/person-information-ask/*path", %{ layer: :api_services, accept: %{ json: true } } do
-    forward conn, path, "http://privacy-worship/person-information-ask/"
+  match "/worship/person-information-ask/*path", %{layer: :api_services, accept: %{json: true}} do
+    forward(conn, path, "http://privacy-worship/person-information-ask/")
   end
 
-  match "/worship/person-information-validate-ssn/*path", %{ layer: :api_services, accept: %{ json: true } } do
-    forward conn, path, "http://privacy-worship/person-information-validate-ssn/"
+  match "/worship/person-information-validate-ssn/*path", %{
+    layer: :api_services,
+    accept: %{json: true}
+  } do
+    forward(conn, path, "http://privacy-worship/person-information-validate-ssn/")
   end
 
-  match "/person-information-updates/*path", %{ layer: :api_services, accept: %{ json: true } } do
-    forward conn, path, "http://privacy-unit/person-information-updates"
+  match "/person-information-updates/*path", %{layer: :api_services, accept: %{json: true}} do
+    forward(conn, path, "http://privacy-unit/person-information-updates")
   end
 
-  match "/person-information-requests/*path", %{ layer: :api_services, accept: %{ json: true } } do
-    forward conn, path, "http://privacy-unit/person-information-requests"
+  match "/person-information-requests/*path", %{layer: :api_services, accept: %{json: true}} do
+    forward(conn, path, "http://privacy-unit/person-information-requests")
   end
 
-  match "/person-information-ask/*path", %{ layer: :api_services, accept: %{ json: true } } do
-    forward conn, path, "http://privacy-unit/person-information-ask/"
+  match "/person-information-ask/*path", %{layer: :api_services, accept: %{json: true}} do
+    forward(conn, path, "http://privacy-unit/person-information-ask/")
   end
 
-  match "/person-information-validate-ssn/*path", %{ layer: :api_services, accept: %{ json: true } } do
-    forward conn, path, "http://privacy-unit/person-information-validate-ssn/"
+  match "/person-information-validate-ssn/*path", %{layer: :api_services, accept: %{json: true}} do
+    forward(conn, path, "http://privacy-unit/person-information-validate-ssn/")
   end
 
-  get "/uri-info/*path", %{ layer: :api_services, accept: %{ json: true } } do
-    forward conn, path, "http://uri-info/"
+  get "/uri-info/*path", %{layer: :api_services, accept: %{json: true}} do
+    forward(conn, path, "http://uri-info/")
   end
 
-  post "/construct-organization-relationships/*path", %{ layer: :api_services, accept: %{ json: true } } do
-    forward conn, path, "http://construct-organization-relationships/create-relationships/"
+  post "/construct-organization-relationships/*path", %{
+    layer: :api_services,
+    accept: %{json: true}
+  } do
+    forward(conn, path, "http://construct-organization-relationships/create-relationships/")
   end
 
-  post "/kbo-data-sync/*path", %{ layer: :api_services, accept: %{ json: true } } do
-    forward conn, path, "http://kbo-data-sync/sync-kbo-data/"
+  post "/kbo-data-sync/*path", %{layer: :api_services, accept: %{json: true}} do
+    forward(conn, path, "http://kbo-data-sync/sync-kbo-data/")
   end
 
-  get "/ldes/*path", %{ layer: :api_services } do
-    forward conn, path, "http://ldes-backend/"
+  get "/ldes/*path", %{layer: :api_services} do
+    forward(conn, path, "http://ldes-backend/")
   end
 
   #################################################################
   # ERROR REPORT
   #################################################################
 
-  match "/error-reports/*path", %{ layer: :api_services, accept: %{ json: true } } do
-    forward conn, path, "http://error-report-service/error-reports/"
+  match "/error-reports/*path", %{layer: :api_services, accept: %{json: true}} do
+    forward(conn, path, "http://error-report-service/error-reports/")
   end
 
   #################################################################
   # FILES
   #################################################################
 
-  get "/files/:id/download", %{ accept: [:any], } do
-    Proxy.forward conn, [], "http://file/files/" <> id <> "/download"
+  get "/files/:id/download", %{accept: [:any]} do
+    Proxy.forward(conn, [], "http://file/files/" <> id <> "/download")
   end
 
-  get "/files/*path" , %{ layer: :api_services, accept: %{ json: true } } do
-    Proxy.forward conn, path, "http://resource/files/"
+  get "/files/*path", %{layer: :api_services, accept: %{json: true}} do
+    Proxy.forward(conn, path, "http://resource/files/")
   end
 
   #################################################################
   # DCAT
   #################################################################
 
-  get "/datasets/*path", %{ layer: :api_services, accept: %{ json: true } } do
-    Proxy.forward conn, path, "http://resource/datasets/"
+  get "/datasets/*path", %{layer: :api_services, accept: %{json: true}} do
+    Proxy.forward(conn, path, "http://resource/datasets/")
   end
 
-  get "/distributions/*path", %{ layer: :api_services, accept: %{ json: true } } do
-    Proxy.forward conn, path, "http://resource/distributions/"
+  get "/distributions/*path", %{layer: :api_services, accept: %{json: true}} do
+    Proxy.forward(conn, path, "http://resource/distributions/")
   end
 
   #################################################################
@@ -398,44 +466,56 @@ defmodule Dispatcher do
   #################################################################
 
   get "/download-exports/*path" do
-    Proxy.forward conn, path, "http://download-exports/"
+    Proxy.forward(conn, path, "http://download-exports/")
   end
 
   ###############################################################
   # frontend layer
   ###############################################################
 
-  match "/assets/*path", %{ layer: :api } do
-    Proxy.forward conn, path, "http://frontend/assets/"
+  get "/assets/*path",  %{ reverse_host: ["dashboard" | _rest], layer: :api }  do
+    forward conn, path, "http://frontend-dashboard/assets/"
   end
 
-  match "/@appuniversum/*path", %{ layer: :api } do
-    Proxy.forward conn, path, "http://frontend/@appuniversum/"
+  get "/@appuniversum/*path", %{ reverse_host: ["dashboard" | _rest], layer: :api} do
+    forward conn, path, "http://frontend-dashboard/@appuniversum/"
   end
 
-  match "/*path", %{ accept: [:html], layer: :api } do
-    Proxy.forward conn, [], "http://frontend/index.html"
+  match "/*_path", %{ reverse_host: ["dashboard" | _rest], layer: :api } do
+    # *_path allows a path to be supplied, but will not yield
+    # an error that we don't use the path variable.
+    forward conn, [], "http://frontend-dashboard/index.html"
   end
 
-  match "/*_path", %{ layer: :frontend } do
-    Proxy.forward conn, [], "http://frontend/index.html"
+  match "/assets/*path", %{layer: :api} do
+    Proxy.forward(conn, path, "http://frontend/assets/")
+  end
+
+  match "/@appuniversum/*path", %{layer: :api} do
+    Proxy.forward(conn, path, "http://frontend/@appuniversum/")
+  end
+
+  match "/*path", %{accept: [:html], layer: :api} do
+    Proxy.forward(conn, [], "http://frontend/index.html")
+  end
+
+  match "/*_path", %{layer: :frontend} do
+    Proxy.forward(conn, [], "http://frontend/index.html")
   end
 
   ###############################################################
   # sparql endpoint
   ###############################################################
 
-  post "/sparql/*path", %{ layer: :api_services, accept: %{ sparql_json: true } } do
-    forward conn, path, "http://db:8890/sparql/"
+  post "/sparql/*path", %{layer: :api_services, accept: %{sparql_json: true}} do
+    forward(conn, path, "http://db:8890/sparql/")
   end
 
   ###############################################################
   # errors
   ###############################################################
 
-  match "/*_path", %{ accept: [:any], layer: :not_found} do
-    send_resp( conn, 404, "{\"error\": {\"code\": 404}")
+  match "/*_path", %{accept: [:any], layer: :not_found} do
+    send_resp(conn, 404, "{\"error\": {\"code\": 404}")
   end
-
-
 end

config/loket-consumer/leidinggevenden-dispatching/config.js

@@ -1,32 +0,0 @@
-const BATCH_SIZE = parseInt(process.env.BATCH_SIZE) || 100;
-const MU_CALL_SCOPE_ID_INITIAL_SYNC = process.env.MU_CALL_SCOPE_ID_INITIAL_SYNC || 'http://redpencil.data.gift/id/concept/muScope/deltas/consumer/initialSync';
-const BYPASS_MU_AUTH_FOR_EXPENSIVE_QUERIES = process.env.BYPASS_MU_AUTH_FOR_EXPENSIVE_QUERIES == 'true' ? true : false;
-const DIRECT_DATABASE_ENDPOINT = process.env.DIRECT_DATABASE_ENDPOINT || 'http://triplestore:8890/sparql';
-const MAX_DB_RETRY_ATTEMPTS = parseInt(process.env.MAX_DB_RETRY_ATTEMPTS || 5);
-const MAX_REASONING_RETRY_ATTEMPTS = parseInt(process.env.MAX_REASONING_RETRY_ATTEMPTS || 5);
-const SLEEP_BETWEEN_BATCHES = parseInt(process.env.SLEEP_BETWEEN_BATCHES || 1000);
-const SLEEP_TIME_AFTER_FAILED_REASONING_OPERATION = parseInt(process.env.SLEEP_TIME_AFTER_FAILED_REASONING_OPERATION || 10000);
-const SLEEP_TIME_AFTER_FAILED_DB_OPERATION = parseInt(process.env.SLEEP_TIME_AFTER_FAILED_DB_OPERATION || 60000);
-
-// Graphs
-const TARGET_GRAPH = process.env.TARGET_GRAPH || `http://mu.semte.ch/graphs/administrative-unit`;
-const LANDING_ZONE_GRAPH = process.env.DCR_LANDING_ZONE_GRAPH || `http://mu.semte.ch/graphs/landing-zone/leidinggevenden`;
-
-// ENDPOINTS
-const LANDING_ZONE_DATABASE = process.env.DCR_LANDING_ZONE_DATABASE || 'db';
-const LANDING_ZONE_DATABASE_ENDPOINT = process.env.DCR_LANDING_ZONE_DATABASE_ENDPOINT || `http://${LANDING_ZONE_DATABASE}:8890/sparql`;
-
-module.exports = {
-  BATCH_SIZE,
-  MU_CALL_SCOPE_ID_INITIAL_SYNC,
-  BYPASS_MU_AUTH_FOR_EXPENSIVE_QUERIES,
-  DIRECT_DATABASE_ENDPOINT,
-  MAX_DB_RETRY_ATTEMPTS,
-  MAX_REASONING_RETRY_ATTEMPTS,
-  SLEEP_BETWEEN_BATCHES,
-  SLEEP_TIME_AFTER_FAILED_DB_OPERATION,
-  SLEEP_TIME_AFTER_FAILED_REASONING_OPERATION,
-  LANDING_ZONE_GRAPH,
-  TARGET_GRAPH,
-  LANDING_ZONE_DATABASE_ENDPOINT
-};

config/loket-consumer/leidinggevenden-dispatching/delta-context-config.js

@@ -1,72 +0,0 @@
-
-const PREFIXES = `
-PREFIX org: <http://www.w3.org/ns/org#>
-PREFIX mandaat: <http://data.vlaanderen.be/ns/mandaat#>
-PREFIX lblodlg: <http://data.lblod.info/vocabularies/leidinggevenden/>
-PREFIX besluit: <http://data.vlaanderen.be/ns/besluit#>`
-
-const contextConfig = {
-  addTypes: {
-    scope: 'all', // 'inserts, 'deletes', 'all' or none. To add rdf:type to subjects of inserts, deletes or both
-    exhausitive: false, // true or false: find all types for a subject, even if one is already present in delta
-  },
-  contextQueries: [
-    // Case Leidnggevenden
-    // They can be triggered when being inserted or when their link to a bestuursfunctie is added
-    {
-      trigger: { // subjectType or predicateValue
-        subjectType: "lblodlg:Functionaris"
-      },
-      queryTemplate: (subject) => `
-        ${PREFIXES}
-        CONSTRUCT {
-          ?functionaris org:holds ?bestuursfunctie ;
-            ?p ?o .
-          ?orgaanInTime lblodlg:heeftBestuursfunctie ?bestuursfunctie ;
-            mandaat:isTijdspecialisatieVan ?orgaan .
-          ?orgaan besluit:bestuurt ?eenheid .
-          ?eenheid besluit:classificatie ?classification .
-        } WHERE {
-          GRAPH <http://mu.semte.ch/graphs/landing-zone/leidinggevenden> {
-            VALUES ?functionaris { ${subject} }
-            ?functionaris org:holds ?bestuursfunctie ;
-              ?p ?o .
-            ?orgaanInTime lblodlg:heeftBestuursfunctie ?bestuursfunctie ;
-              mandaat:isTijdspecialisatieVan ?orgaan .
-            ?orgaan besluit:bestuurt ?eenheid .
-            ?eenheid besluit:classificatie ?classification .
-          }
-        }`
-    },
-    {
-      trigger: {
-        predicateValue: "org:holds"
-      },
-      queryTemplate: (subject) => `
-        ${PREFIXES}
-        CONSTRUCT {
-          ?functionaris org:holds ?bestuursfunctie ;
-            ?p ?o .
-          ?orgaanInTime lblodlg:heeftBestuursfunctie ?bestuursfunctie ;
-            mandaat:isTijdspecialisatieVan ?orgaan .
-          ?orgaan besluit:bestuurt ?eenheid .
-          ?eenheid besluit:classificatie ?classification .
-        } WHERE {
-          GRAPH <http://mu.semte.ch/graphs/landing-zone/leidinggevenden> {
-            VALUES ?functionaris { ${subject} }
-            ?functionaris org:holds ?bestuursfunctie ;
-              ?p ?o .
-            ?orgaanInTime lblodlg:heeftBestuursfunctie ?bestuursfunctie ;
-              mandaat:isTijdspecialisatieVan ?orgaan .
-            ?orgaan besluit:bestuurt ?eenheid .
-            ?eenheid besluit:classificatie ?classification .
-          }
-        }`
-    }
-  ]
-}
-
-module.exports = {
-  contextConfig,
-  PREFIXES
-};

config/loket-consumer/leidinggevenden-dispatching/delta-sync-dispatching.js

@@ -1,85 +0,0 @@
-const {
-  transformStatements,
-  deleteFromTargetGraph,
-  insertIntoTargetGraph,
-} = require('./util');
-
-/**
-* Dispatch the fetched information to a target graph.
-* Note: <share://file/data> will be ADDED to it's own graph.
-*   We take only care of adding them, not updating triples, this is a TODO
-* @param { mu, muAuthSudo } lib - The provided libraries from the host service.
-* @param { termObjectChangeSets: { deletes, inserts } } data - The fetched changes sets, which objects of serialized Terms
-*          [ {
-*              graph: "<http://foo>",
-*              subject: "<http://bar>",
-*              predicate: "<http://baz>",
-*              object: "<http://boom>^^<http://datatype>"
-*            }
-*         ]
-* @return {void} Nothing
-*/
-async function dispatch(lib, data) {
-  const { mu, fetch } = lib;
-  let { termObjectChangeSets, termObjectChangeSetsWithContext } = data;
-
-  // Both arrays are the same length, so we can zip them together for easier processing
-  zippedChangeSets = termObjectChangeSets.map((o, i) => ({
-    original: o,
-    withContext: termObjectChangeSetsWithContext[i]
-  }));
-
-  console.log(`Received ${zippedChangeSets.length} change sets`)
-
-  for (let { original, withContext } of zippedChangeSets) {
-    let originalDeletes, originalInserts, insertsWithContext, deletesWithContext;
-    try {
-      originalInserts = original.inserts.map(o => `${o.subject} ${o.predicate} ${o.object}.`);
-      originalDeletes = original.deletes.map(o => `${o.subject} ${o.predicate} ${o.object}.`);
-    } catch (e) {
-      console.log(`Error while processing change set: ${e}`);
-      console.log(`Original change set: ${JSON.stringify(original)}`);
-      throw e;
-    }
-
-    // Extra context needed for mapping from DL to OP model and filtering based on type.
-    try {
-      insertsWithContext = withContext.inserts.map(o => `${o.subject} ${o.predicate} ${o.object}.`);
-      deletesWithContext = withContext.deletes.map(o => `${o.subject} ${o.predicate} ${o.object}.`);
-    } catch (e) {
-      console.log(`Error while processing change set with context: ${e}`);
-      console.log(`Change set with context: ${JSON.stringify(withContext)}`);
-      throw e;
-    }
-
-    if (originalDeletes.length) {
-      // Map deletes from DL to OP model
-      const transformedDeletes = await transformStatements(fetch, deletesWithContext);
-
-      if (!transformedDeletes.length) {
-        console.log(`Warn: Delete statements mapped to empty result.`);
-        console.log(`Input: ${deletesWithContext}`);
-        console.log(`Output: ${transformedDeletes}`);
-      } else {
-        await deleteFromTargetGraph(lib, transformedDeletes);
-      }
-    }
-
-    if (originalInserts.length) {
-      // Map inserts from DL to OP model
-      const transformedInserts = await transformStatements(fetch, insertsWithContext);
-
-      if (!transformedInserts.length) {
-        console.log(`Warn: Insert statements mapped to empty result.`);
-        console.log(`Input: ${insertsWithContext}`);
-        console.log(`Output: ${transformedInserts}`);
-      } else {
-        await insertIntoTargetGraph(lib, transformedInserts);
-      }
-    }
-  }
-}
-
-module.exports = {
-  dispatch
-};

config/loket-consumer/leidinggevenden-dispatching/initial-sync-dispatching.js

@@ -1,85 +0,0 @@
-const { batchedDbUpdate, transformLandingZoneGraph } = require('./util');
-const {
-  BYPASS_MU_AUTH_FOR_EXPENSIVE_QUERIES,
-  DIRECT_DATABASE_ENDPOINT,
-  MU_CALL_SCOPE_ID_INITIAL_SYNC,
-  BATCH_SIZE,
-  MAX_DB_RETRY_ATTEMPTS,
-  SLEEP_BETWEEN_BATCHES,
-  SLEEP_TIME_AFTER_FAILED_DB_OPERATION,
-  TARGET_GRAPH,
-  LANDING_ZONE_GRAPH,
-  LANDING_ZONE_DATABASE_ENDPOINT
-} = require('./config');
-
-const endpoint = BYPASS_MU_AUTH_FOR_EXPENSIVE_QUERIES ? DIRECT_DATABASE_ENDPOINT : process.env.MU_SPARQL_ENDPOINT;
-
-/**
-* Dispatch the fetched information to a target graph.
-* @param { mu, muAuthSudo } lib - The provided libraries from the host service.
-* @param { termObjects } data - The fetched quad information, which objects of serialized Terms
-*          [ {
-*              graph: "<http://foo>",
-*              subject: "<http://bar>",
-*              predicate: "<http://baz>",
-*              object: "<http://boom>^^<http://datatype>"
-*            }
-*         ]
-* @return {void} Nothing
-*/
-async function dispatch(lib, data) {
-  const { mu, muAuthSudo } = lib;
-  const { termObjects } = data;
-
-  // Steps:
-  // 1. Write the triples to the landing zone graph - withouth any mapping or filtering
-  // 2. One-time reasoning run with the full landing zone graph and write the results to the target graph (on-finish)
-
-  if (BYPASS_MU_AUTH_FOR_EXPENSIVE_QUERIES) {
-    console.warn(`Service configured to skip MU_AUTH!`);
-  }
-  console.log(`Using ${endpoint} to insert triples`);
-
-  if (termObjects.length) {
-    const originalInsertTriples = termObjects.map(o => `${o.subject} ${o.predicate} ${o.object}.`);
-
-    await batchedDbUpdate(
-      muAuthSudo.updateSudo,
-      LANDING_ZONE_GRAPH,
-      originalInsertTriples,
-      { 'mu-call-scope-id': MU_CALL_SCOPE_ID_INITIAL_SYNC },
-      LANDING_ZONE_DATABASE_ENDPOINT,
-      BATCH_SIZE,
-      MAX_DB_RETRY_ATTEMPTS,
-      SLEEP_BETWEEN_BATCHES,
-      SLEEP_TIME_AFTER_FAILED_DB_OPERATION
-    );
-  }
-}
-
-async function onFinishInitialIngest(lib) {
-  const { muAuthSudo, fetch } = lib;
-
-  console.log(`!! On-finish triggered !!`);
-
-  const transformedInsertTriples = await transformLandingZoneGraph(fetch, endpoint);
-
-  console.log(`Transformed ${transformedInsertTriples.length} triples`);
-
-  await batchedDbUpdate(
-    muAuthSudo.updateSudo,
-    TARGET_GRAPH,
-    transformedInsertTriples,
-    { 'mu-call-scope-id': MU_CALL_SCOPE_ID_INITIAL_SYNC },
-    endpoint,
-    BATCH_SIZE,
-    MAX_DB_RETRY_ATTEMPTS,
-    SLEEP_BETWEEN_BATCHES,
-    SLEEP_TIME_AFTER_FAILED_DB_OPERATION
-  );
-}
-
-module.exports = {
-  dispatch,
-  onFinishInitialIngest
-};

config/loket-consumer/leidinggevenden-dispatching/util.js

@@ -1,168 +0,0 @@
-const {
-  MAX_REASONING_RETRY_ATTEMPTS,
-  SLEEP_TIME_AFTER_FAILED_REASONING_OPERATION,
-  MAX_DB_RETRY_ATTEMPTS,
-  SLEEP_BETWEEN_BATCHES,
-  SLEEP_TIME_AFTER_FAILED_DB_OPERATION,
-  TARGET_GRAPH,
-  LANDING_ZONE_GRAPH,
-  BATCH_SIZE,
-} = require('./config');
-
-async function batchedDbUpdate(
-  muUpdate,
-  graph,
-  triples,
-  extraHeaders,
-  endpoint,
-  batchSize,
-  maxAttempts,
-  sleepBetweenBatches = 1000,
-  sleepTimeOnFail = 1000,
-  operation = 'INSERT'
-) {
-
-  for (let i = 0; i < triples.length; i += batchSize) {
-    console.log(`Inserting triples in batch: ${i}-${i + batchSize}`);
-
-    const batch = triples.slice(i, i + batchSize).join('\n');
-
-    const insertCall = async () => {
-      await muUpdate(`
-${operation} DATA {
-GRAPH <${graph}> {
-${batch}
-}
-}
-`, extraHeaders, endpoint);
-    };
-
-    await operationWithRetry(insertCall, 0, maxAttempts, sleepTimeOnFail);
-
-    console.log(`Sleeping before next query execution: ${sleepBetweenBatches}`);
-    await new Promise(r => setTimeout(r, sleepBetweenBatches));
-  }
-}
-
-async function operationWithRetry(callback,
-  attempt,
-  maxAttempts,
-  sleepTimeOnFail) {
-  try {
-    if (typeof callback === "function")
-      return await callback();
-    else // Catch error from promise - not how I would do it normally, but allows re use of existing code.
-      return await callback;
-  }
-  catch (e) {
-    console.log(`Operation failed for ${callback.toString()}, attempt: ${attempt} of ${maxAttempts}`);
-    console.log(`Error: ${e}`);
-    console.log(`Sleeping ${sleepTimeOnFail} ms`);
-
-    if (attempt >= maxAttempts) {
-      console.log(`Max attempts reached for ${callback.toString()}, giving up`);
-      throw e;
-    }
-
-    await new Promise(r => setTimeout(r, sleepTimeOnFail));
-    return operationWithRetry(callback, ++attempt, maxAttempts, sleepTimeOnFail);
-  }
-}
-
-/**
- * Splits an array into two parts, a part that passes and a part that fails a predicate function.
- * Credits: https://github.com/benjay10
- * @public
- * @function partition
- * @param {Array} arr - Array to be partitioned
- * @param {Function} fn - Function that accepts single argument: an element of the array, and should return a truthy or falsy value.
- * @returns {Object} Object that contains keys passes and fails, each representing an array with elemets that pass or fail the predicate respectively
- */
-function partition(arr, fn) {
-  let passes = [], fails = [];
-  arr.forEach((item) => (fn(item) ? passes : fails).push(item));
-  return { passes, fails };
-}
-
-/**
- * Send triples to reasoning service for conversion
- *
- */
-async function transformTriples(fetch, triples, mapping) {
-  return await operationWithRetry(mainConversion(fetch, triples, mapping), 0,
-    MAX_REASONING_RETRY_ATTEMPTS, SLEEP_TIME_AFTER_FAILED_REASONING_OPERATION);
-}
-
-async function mainConversion(fetch, triples, mapping) {
-  let formdata = new URLSearchParams();
-  formdata.append("data", triples);
-
-  let requestOptions = {
-    method: 'POST',
-    body: formdata,
-    redirect: 'follow'
-  };
-
-  const response = await fetch(`http://reasoner/reason/dl2op/${mapping}`, requestOptions)
-  return await response.text();
-}
-
-async function transformStatements(fetch, triples, mapping = 'main') {
-  console.log(`Received ${JSON.stringify(triples)} to transform`);
-  const transformedTriples = await transformTriples(fetch, triples.join('\n'), mapping);
-  statements = transformedTriples ? transformedTriples.replace(/\n{2,}/g, '').split('\n') : [];
-  console.log(`CONVERSION: FROM ${triples.length} triples to ${statements.length}`);
-  return statements;
-}
-
-async function deleteFromTargetGraph(lib, statements) {
-  console.log(`Deleting ${statements.length} statements from target graph`);
-  console.log(`Statements:  ${JSON.stringify(statements)}`)
-  await batchedDbUpdate(
-    lib.muAuthSudo.updateSudo,
-    TARGET_GRAPH,
-    statements,
-    {},
-    process.env.MU_SPARQL_ENDPOINT,
-    BATCH_SIZE,
-    MAX_DB_RETRY_ATTEMPTS,
-    SLEEP_BETWEEN_BATCHES,
-    SLEEP_TIME_AFTER_FAILED_DB_OPERATION,
-    'DELETE');
-}
-
-async function insertIntoTargetGraph(lib, statements) {
-  console.log(`Inserting ${statements.length} statements into target graph`);
-  console.log(`Statements:  ${JSON.stringify(statements)}`)
-
-  await batchedDbUpdate(
-    lib.muAuthSudo.updateSudo,
-    TARGET_GRAPH,
-    statements,
-    {},
-    process.env.MU_SPARQL_ENDPOINT,
-    BATCH_SIZE,
-    MAX_DB_RETRY_ATTEMPTS,
-    SLEEP_BETWEEN_BATCHES,
-    SLEEP_TIME_AFTER_FAILED_DB_OPERATION,
-    'INSERT');
-}
-
-async function transformLandingZoneGraph(fetch, endpoint, mapping = 'main') {
-  console.log(`Transforming landing zone graph: ${LANDING_ZONE_GRAPH}`);
-
-  const response = await fetch(`http://reasoner/reason/dl2op/${mapping}?data=${encodeURIComponent(`${endpoint}?default-graph-uri=&query=CONSTRUCT+%7B%0D%0A%3Fs+%3Fp+%3Fo%0D%0A%7D+WHERE+%7B%0D%0A+GRAPH+<${LANDING_ZONE_GRAPH}>+%7B%0D%0A%3Fs+%3Fp+%3Fo%0D%0A%7D%0D%0A%7D&should-sponge=&format=text%2Fplain&timeout=0&run=+Run+Query`)}`);
-  const text = await response.text();
-  const statements = text.replace(/\n{2,}/g, '').split('\n');
-
-  return statements;
-}
-
-module.exports = {
-  batchedDbUpdate,
-  partition,
-  transformStatements,
-  transformLandingZoneGraph,
-  deleteFromTargetGraph,
-  insertIntoTargetGraph
-};

config/loket-consumer/mandatarissen-dispatching/config.js

@@ -1,33 +0,0 @@
-const BATCH_SIZE = parseInt(process.env.BATCH_SIZE) || 100;
-const MU_CALL_SCOPE_ID_INITIAL_SYNC = process.env.MU_CALL_SCOPE_ID_INITIAL_SYNC || 'http://redpencil.data.gift/id/concept/muScope/deltas/consumer/initialSync';
-const BYPASS_MU_AUTH_FOR_EXPENSIVE_QUERIES = process.env.BYPASS_MU_AUTH_FOR_EXPENSIVE_QUERIES == 'true' ? true : false;
-const DIRECT_DATABASE_ENDPOINT = process.env.DIRECT_DATABASE_ENDPOINT || 'http://triplestore:8890/sparql';
-const MAX_DB_RETRY_ATTEMPTS = parseInt(process.env.MAX_DB_RETRY_ATTEMPTS || 5);
-const MAX_REASONING_RETRY_ATTEMPTS = parseInt(process.env.MAX_REASONING_RETRY_ATTEMPTS || 5);
-const SLEEP_BETWEEN_BATCHES = parseInt(process.env.SLEEP_BETWEEN_BATCHES || 1000);
-const SLEEP_TIME_AFTER_FAILED_REASONING_OPERATION = parseInt(process.env.SLEEP_TIME_AFTER_FAILED_REASONING_OPERATION || 10000);
-const SLEEP_TIME_AFTER_FAILED_DB_OPERATION = parseInt(process.env.SLEEP_TIME_AFTER_FAILED_DB_OPERATION || 60000);
-
-// Graphs
-const TARGET_GRAPH = process.env.TARGET_GRAPH || `http://mu.semte.ch/graphs/administrative-unit`;
-const LANDING_ZONE_GRAPH = process.env.DCR_LANDING_ZONE_GRAPH || `http://mu.semte.ch/graphs/landing-zone/leidinggevenden`;
-
-// ENDPOINTS
-const LANDING_ZONE_DATABASE = process.env.DCR_LANDING_ZONE_DATABASE || 'db';
-const LANDING_ZONE_DATABASE_ENDPOINT = process.env.DCR_LANDING_ZONE_DATABASE_ENDPOINT || `http://${LANDING_ZONE_DATABASE}:8890/sparql`;
-
-module.exports = {
-  BATCH_SIZE,
-  MU_CALL_SCOPE_ID_INITIAL_SYNC,
-  BYPASS_MU_AUTH_FOR_EXPENSIVE_QUERIES,
-  DIRECT_DATABASE_ENDPOINT,
-  MAX_DB_RETRY_ATTEMPTS,
-  MAX_REASONING_RETRY_ATTEMPTS,
-  SLEEP_BETWEEN_BATCHES,
-  SLEEP_TIME_AFTER_FAILED_DB_OPERATION,
-  SLEEP_TIME_AFTER_FAILED_REASONING_OPERATION,
-  LANDING_ZONE_GRAPH,
-  LANDING_ZONE_DATABASE_ENDPOINT,
-  TARGET_GRAPH
-};
-

config/loket-consumer/mandatarissen-dispatching/delta-context-config.js

@@ -1,72 +0,0 @@
-
-const PREFIXES = `
-PREFIX org: <http://www.w3.org/ns/org#>
-PREFIX mandaat: <http://data.vlaanderen.be/ns/mandaat#>
-PREFIX lblodlg: <http://data.lblod.info/vocabularies/leidinggevenden/>
-PREFIX besluit: <http://data.vlaanderen.be/ns/besluit#>`
-
-const contextConfig = {
-  addTypes: {
-    scope: 'all', // 'inserts, 'deletes', 'all' or none. To add rdf:type to subjects of inserts, deletes or both
-    exhausitive: false, // true or false: find all types for a subject, even if one is already present in delta
-  },
-  contextQueries: [
-    // Case Mandatarissen
-    // They can be triggered when being inserted or when their link to a mandate is added
-    {
-      trigger: { // subjectType or predicateValue
-        subjectType: "mandaat:Mandataris"
-      },
-      queryTemplate: (subject) => `
-        ${PREFIXES}
-        CONSTRUCT {
-          ?mandataris org:holds ?mandate ;
-            ?p ?o .
-          ?orgaanInTime org:hasPost ?mandate ;
-            mandaat:isTijdspecialisatieVan ?orgaan .
-          ?orgaan besluit:bestuurt ?eenheid .
-          ?eenheid besluit:classificatie ?classification .
-        } WHERE {
-          GRAPH <http://mu.semte.ch/graphs/landing-zone/mandatarissen> {
-            VALUES ?mandataris { ${subject} }
-            ?mandataris org:holds ?mandate ;
-              ?p ?o .
-            ?orgaanInTime org:hasPost ?mandate ;
-              mandaat:isTijdspecialisatieVan ?orgaan .
-            ?orgaan besluit:bestuurt ?eenheid .
-            ?eenheid besluit:classificatie ?classification .
-          }
-        }`
-    },
-    {
-      trigger: {
-        predicateValue: "org:holds"
-      },
-      queryTemplate: (subject) => `
-        ${PREFIXES}
-        CONSTRUCT {
-          ?mandataris org:holds ?mandate ;
-            ?p ?o .
-          ?orgaanInTime org:hasPost ?mandate ;
-            mandaat:isTijdspecialisatieVan ?orgaan .
-          ?orgaan besluit:bestuurt ?eenheid .
-          ?eenheid besluit:classificatie ?classification .
-        } WHERE {
-          GRAPH <http://mu.semte.ch/graphs/landing-zone/mandatarissen> {
-            VALUES ?mandataris { ${subject} }
-            ?mandataris org:holds ?mandate ;
-              ?p ?o .
-            ?orgaanInTime org:hasPost ?mandate ;
-              mandaat:isTijdspecialisatieVan ?orgaan .
-            ?orgaan besluit:bestuurt ?eenheid .
-            ?eenheid besluit:classificatie ?classification .
-          }
-        }`
-    }
-  ]
-}
-
-module.exports = {
-  contextConfig,
-  PREFIXES
-};

config/loket-consumer/mandatarissen-dispatching/delta-sync-dispatching.js

@@ -1,85 +0,0 @@
-const {
-  transformStatements,
-  deleteFromTargetGraph,
-  insertIntoTargetGraph,
-} = require('./util');
-
-/**
-* Dispatch the fetched information to a target graph.
-* Note: <share://file/data> will be ADDED to it's own graph.
-*   We take only care of adding them, not updating triples, this is a TODO
-* @param { mu, muAuthSudo } lib - The provided libraries from the host service.
-* @param { termObjectChangeSets: { deletes, inserts } } data - The fetched changes sets, which objects of serialized Terms
-*          [ {
-*              graph: "<http://foo>",
-*              subject: "<http://bar>",
-*              predicate: "<http://baz>",
-*              object: "<http://boom>^^<http://datatype>"
-*            }
-*         ]
-* @return {void} Nothing
-*/
-async function dispatch(lib, data) {
-  const { mu, fetch } = lib;
-  let { termObjectChangeSets, termObjectChangeSetsWithContext } = data;
-
-  // Both arrays are the same length, so we can zip them together for easier processing
-  zippedChangeSets = termObjectChangeSets.map((o, i) => ({
-    original: o,
-    withContext: termObjectChangeSetsWithContext[i]
-  }));
-
-  console.log(`Received ${zippedChangeSets.length} change sets`)
-
-  for (let { original, withContext } of zippedChangeSets) {
-    let originalDeletes, originalInserts, insertsWithContext, deletesWithContext;
-    try {
-      originalInserts = original.inserts.map(o => `${o.subject} ${o.predicate} ${o.object}.`);
-      originalDeletes = original.deletes.map(o => `${o.subject} ${o.predicate} ${o.object}.`);
-    } catch (e) {
-      console.log(`Error while processing change set: ${e}`);
-      console.log(`Original change set: ${JSON.stringify(original)}`);
-      throw e;
-    }
-
-    // Extra context needed for mapping from DL to OP model and filtering based on type.
-    try {
-      insertsWithContext = withContext.inserts.map(o => `${o.subject} ${o.predicate} ${o.object}.`);
-      deletesWithContext = withContext.deletes.map(o => `${o.subject} ${o.predicate} ${o.object}.`);
-    } catch (e) {
-      console.log(`Error while processing change set with context: ${e}`);
-      console.log(`Change set with context: ${JSON.stringify(withContext)}`);
-      throw e;
-    }
-
-    if (originalDeletes.length) {
-      // Map deletes from DL to OP model
-      const transformedDeletes = await transformStatements(fetch, deletesWithContext);
-
-      if (!transformedDeletes.length) {
-        console.log(`Warn: Delete statements mapped to empty result.`);
-        console.log(`Input: ${deletesWithContext}`);
-        console.log(`Output: ${transformedDeletes}`);
-      } else {
-        await deleteFromTargetGraph(lib, transformedDeletes);
-      }
-    }
-
-    if (originalInserts.length) {
-      // Map inserts from DL to OP model
-      const transformedInserts = await transformStatements(fetch, insertsWithContext);
-
-      if (!transformedInserts.length) {
-        console.log(`Warn: Insert statements mapped to empty result.`);
-        console.log(`Input: ${insertsWithContext}`);
-        console.log(`Output: ${transformedInserts}`);
-      } else {
-        await insertIntoTargetGraph(lib, transformedInserts);
-      }
-    }
-  }
-}
-
-module.exports = {
-  dispatch
-};

config/loket-consumer/mandatarissen-dispatching/initial-sync-dispatching.js

@@ -1,85 +0,0 @@
-const { batchedDbUpdate, transformLandingZoneGraph } = require('./util');
-const {
-  BYPASS_MU_AUTH_FOR_EXPENSIVE_QUERIES,
-  DIRECT_DATABASE_ENDPOINT,
-  MU_CALL_SCOPE_ID_INITIAL_SYNC,
-  BATCH_SIZE,
-  MAX_DB_RETRY_ATTEMPTS,
-  SLEEP_BETWEEN_BATCHES,
-  SLEEP_TIME_AFTER_FAILED_DB_OPERATION,
-  TARGET_GRAPH,
-  LANDING_ZONE_GRAPH,
-  LANDING_ZONE_DATABASE_ENDPOINT
-} = require('./config');
-
-const endpoint = BYPASS_MU_AUTH_FOR_EXPENSIVE_QUERIES ? DIRECT_DATABASE_ENDPOINT : process.env.MU_SPARQL_ENDPOINT;
-
-/**
-* Dispatch the fetched information to a target graph.
-* @param { mu, muAuthSudo } lib - The provided libraries from the host service.
-* @param { termObjects } data - The fetched quad information, which objects of serialized Terms
-*          [ {
-*              graph: "<http://foo>",
-*              subject: "<http://bar>",
-*              predicate: "<http://baz>",
-*              object: "<http://boom>^^<http://datatype>"
-*            }
-*         ]
-* @return {void} Nothing
-*/
-async function dispatch(lib, data) {
-  const { mu, muAuthSudo } = lib;
-  const { termObjects } = data;
-
-  // Steps:
-  // 1. Write the triples to the landing zone graph - withouth any mapping or filtering
-  // 2. One-time reasoning run with the full landing zone graph and write the results to the target graph (on-finish)
-
-  if (BYPASS_MU_AUTH_FOR_EXPENSIVE_QUERIES) {
-    console.warn(`Service configured to skip MU_AUTH!`);
-  }
-  console.log(`Using ${endpoint} to insert triples`);
-
-  if (termObjects.length) {
-    const originalInsertTriples = termObjects.map(o => `${o.subject} ${o.predicate} ${o.object}.`);
-
-    await batchedDbUpdate(
-      muAuthSudo.updateSudo,
-      LANDING_ZONE_GRAPH,
-      originalInsertTriples,
-      { 'mu-call-scope-id': MU_CALL_SCOPE_ID_INITIAL_SYNC },
-      LANDING_ZONE_DATABASE_ENDPOINT,
-      BATCH_SIZE,
-      MAX_DB_RETRY_ATTEMPTS,
-      SLEEP_BETWEEN_BATCHES,
-      SLEEP_TIME_AFTER_FAILED_DB_OPERATION
-    );
-  }
-}
-
-async function onFinishInitialIngest(lib) {
-  const { muAuthSudo, fetch } = lib;
-
-  console.log(`!! On-finish triggered !!`);
-
-  const transformedInsertTriples = await transformLandingZoneGraph(fetch, endpoint);
-
-  console.log(`Transformed ${transformedInsertTriples.length} triples`);
-
-  await batchedDbUpdate(
-    muAuthSudo.updateSudo,
-    TARGET_GRAPH,
-    transformedInsertTriples,
-    { 'mu-call-scope-id': MU_CALL_SCOPE_ID_INITIAL_SYNC },
-    endpoint,
-    BATCH_SIZE,
-    MAX_DB_RETRY_ATTEMPTS,
-    SLEEP_BETWEEN_BATCHES,
-    SLEEP_TIME_AFTER_FAILED_DB_OPERATION
-  );
-}
-
-module.exports = {
-  dispatch,
-  onFinishInitialIngest
-};

config/loket-consumer/mandatarissen-dispatching/util.js

@@ -1,168 +0,0 @@
-const {
-  MAX_REASONING_RETRY_ATTEMPTS,
-  SLEEP_TIME_AFTER_FAILED_REASONING_OPERATION,
-  MAX_DB_RETRY_ATTEMPTS,
-  SLEEP_BETWEEN_BATCHES,
-  SLEEP_TIME_AFTER_FAILED_DB_OPERATION,
-  TARGET_GRAPH,
-  LANDING_ZONE_GRAPH,
-  BATCH_SIZE
-} = require('./config');
-
-async function batchedDbUpdate(
-  muUpdate,
-  graph,
-  triples,
-  extraHeaders,
-  endpoint,
-  batchSize,
-  maxAttempts,
-  sleepBetweenBatches = 1000,
-  sleepTimeOnFail = 1000,
-  operation = 'INSERT'
-) {
-
-  for (let i = 0; i < triples.length; i += batchSize) {
-    console.log(`Inserting triples in batch: ${i}-${i + batchSize}`);
-
-    const batch = triples.slice(i, i + batchSize).join('\n');
-
-    const insertCall = async () => {
-      await muUpdate(`
-${operation} DATA {
-GRAPH <${graph}> {
-${batch}
-}
-}
-`, extraHeaders, endpoint);
-    };
-
-    await operationWithRetry(insertCall, 0, maxAttempts, sleepTimeOnFail);
-
-    console.log(`Sleeping before next query execution: ${sleepBetweenBatches}`);
-    await new Promise(r => setTimeout(r, sleepBetweenBatches));
-  }
-}
-
-async function operationWithRetry(callback,
-  attempt,
-  maxAttempts,
-  sleepTimeOnFail) {
-  try {
-    if (typeof callback === "function")
-      return await callback();
-    else // Catch error from promise - not how I would do it normally, but allows re use of existing code.
-      return await callback;
-  }
-  catch (e) {
-    console.log(`Operation failed for ${callback.toString()}, attempt: ${attempt} of ${maxAttempts}`);
-    console.log(`Error: ${e}`);
-    console.log(`Sleeping ${sleepTimeOnFail} ms`);
-
-    if (attempt >= maxAttempts) {
-      console.log(`Max attempts reached for ${callback.toString()}, giving up`);
-      throw e;
-    }
-
-    await new Promise(r => setTimeout(r, sleepTimeOnFail));
-    return operationWithRetry(callback, ++attempt, maxAttempts, sleepTimeOnFail);
-  }
-}
-
-/**
- * Splits an array into two parts, a part that passes and a part that fails a predicate function.
- * Credits: https://github.com/benjay10
- * @public
- * @function partition
- * @param {Array} arr - Array to be partitioned
- * @param {Function} fn - Function that accepts single argument: an element of the array, and should return a truthy or falsy value.
- * @returns {Object} Object that contains keys passes and fails, each representing an array with elemets that pass or fail the predicate respectively
- */
-function partition(arr, fn) {
-  let passes = [], fails = [];
-  arr.forEach((item) => (fn(item) ? passes : fails).push(item));
-  return { passes, fails };
-}
-
-/**
- * Send triples to reasoning service for conversion
- *
- */
-async function transformTriples(fetch, triples, mapping) {
-  return await operationWithRetry(mainConversion(fetch, triples, mapping), 0,
-    MAX_REASONING_RETRY_ATTEMPTS, SLEEP_TIME_AFTER_FAILED_REASONING_OPERATION);
-}
-
-async function mainConversion(fetch, triples, mapping) {
-  let formdata = new URLSearchParams();
-  formdata.append("data", triples);
-
-  let requestOptions = {
-    method: 'POST',
-    body: formdata,
-    redirect: 'follow'
-  };
-
-  const response = await fetch(`http://reasoner/reason/dl2op/${mapping}`, requestOptions)
-  return await response.text();
-}
-
-async function transformStatements(fetch, triples, mapping = 'main') {
-  console.log(`Received ${JSON.stringify(triples)} to transform`);
-  const transformedTriples = await transformTriples(fetch, triples.join('\n'), mapping);
-  statements = transformedTriples ? transformedTriples.replace(/\n{2,}/g, '').split('\n') : [];
-  console.log(`CONVERSION: FROM ${triples.length} triples to ${statements.length}`);
-  return statements;
-}
-
-async function deleteFromTargetGraph(lib, statements) {
-  console.log(`Deleting ${statements.length} statements from target graph`);
-  console.log(`Statements:  ${JSON.stringify(statements)}`)
-  await batchedDbUpdate(
-    lib.muAuthSudo.updateSudo,
-    TARGET_GRAPH,
-    statements,
-    {},
-    process.env.MU_SPARQL_ENDPOINT,
-    BATCH_SIZE,
-    MAX_DB_RETRY_ATTEMPTS,
-    SLEEP_BETWEEN_BATCHES,
-    SLEEP_TIME_AFTER_FAILED_DB_OPERATION,
-    'DELETE');
-}
-
-async function insertIntoTargetGraph(lib, statements) {
-  console.log(`Inserting ${statements.length} statements into target graph`);
-  console.log(`Statements:  ${JSON.stringify(statements)}`)
-
-  await batchedDbUpdate(
-    lib.muAuthSudo.updateSudo,
-    TARGET_GRAPH,
-    statements,
-    {},
-    process.env.MU_SPARQL_ENDPOINT,
-    BATCH_SIZE,
-    MAX_DB_RETRY_ATTEMPTS,
-    SLEEP_BETWEEN_BATCHES,
-    SLEEP_TIME_AFTER_FAILED_DB_OPERATION,
-    'INSERT');
-}
-
-async function transformLandingZoneGraph(fetch, endpoint, mapping = 'main') {
-  console.log(`Transforming landing zone graph: ${LANDING_ZONE_GRAPH}`);
-
-  const response = await fetch(`http://reasoner/reason/dl2op/${mapping}?data=${encodeURIComponent(`${endpoint}?default-graph-uri=&query=CONSTRUCT+%7B%0D%0A%3Fs+%3Fp+%3Fo%0D%0A%7D+WHERE+%7B%0D%0A+GRAPH+<${LANDING_ZONE_GRAPH}>+%7B%0D%0A%3Fs+%3Fp+%3Fo%0D%0A%7D%0D%0A%7D&should-sponge=&format=text%2Fplain&timeout=0&run=+Run+Query`)}`);
-  const text = await response.text();
-  const statements = text.replace(/\n{2,}/g, '').split('\n');
-
-  return statements;
-}
-
-module.exports = {
-  batchedDbUpdate,
-  partition,
-  transformStatements,
-  transformLandingZoneGraph,
-  deleteFromTargetGraph,
-  insertIntoTargetGraph
-};

config/loket-consumer/worship-services-sensitive-dispatching/config.js

@@ -1,40 +0,0 @@
-const BATCH_SIZE = parseInt(process.env.BATCH_SIZE) || 100;
-const MU_CALL_SCOPE_ID_INITIAL_SYNC = process.env.MU_CALL_SCOPE_ID_INITIAL_SYNC || 'http://redpencil.data.gift/id/concept/muScope/deltas/consumer/initialSync';
-const BYPASS_MU_AUTH_FOR_EXPENSIVE_QUERIES = process.env.BYPASS_MU_AUTH_FOR_EXPENSIVE_QUERIES == 'true' ? true : false;
-const DIRECT_DATABASE_ENDPOINT = process.env.DIRECT_DATABASE_ENDPOINT || 'http://triplestore:8890/sparql';
-const MAX_DB_RETRY_ATTEMPTS = parseInt(process.env.MAX_DB_RETRY_ATTEMPTS || 5);
-const MAX_REASONING_RETRY_ATTEMPTS = parseInt(process.env.MAX_REASONING_RETRY_ATTEMPTS || 5);
-const SLEEP_BETWEEN_BATCHES = parseInt(process.env.SLEEP_BETWEEN_BATCHES || 1000);
-const SLEEP_TIME_AFTER_FAILED_REASONING_OPERATION = parseInt(process.env.SLEEP_TIME_AFTER_FAILED_REASONING_OPERATION || 10000);
-const SLEEP_TIME_AFTER_FAILED_DB_OPERATION = parseInt(process.env.SLEEP_TIME_AFTER_FAILED_DB_OPERATION || 60000);
-
-// Graphs
-const TARGET_GRAPH = process.env.TARGET_GRAPH || `http://mu.semte.ch/graphs/worship-service`;
-const PRIVACY_SENSITIVE_GRAPH = process.env.PRIVACY_SENSITIVE_GRAPH || `http://mu.semte.ch/graphs/worship-privacy-centric-graph`;
-const LANDING_ZONE_GRAPH = process.env.DCR_LANDING_ZONE_GRAPH || `http://mu.semte.ch/graphs/landing-zone/leidinggevenden`;
-
-// Mappings
-const MAIN_INFO_MAPPING = "main-info";
-const PRIVATE_INFO_MAPPING = "private-info";
-
-// ENDPOINTS
-const LANDING_ZONE_DATABASE = process.env.DCR_LANDING_ZONE_DATABASE || 'db';
-const LANDING_ZONE_DATABASE_ENDPOINT = process.env.DCR_LANDING_ZONE_DATABASE_ENDPOINT || `http://${LANDING_ZONE_DATABASE}:8890/sparql`;
-
-module.exports = {
-  BATCH_SIZE,
-  MU_CALL_SCOPE_ID_INITIAL_SYNC,
-  BYPASS_MU_AUTH_FOR_EXPENSIVE_QUERIES,
-  DIRECT_DATABASE_ENDPOINT,
-  MAX_DB_RETRY_ATTEMPTS,
-  MAX_REASONING_RETRY_ATTEMPTS,
-  SLEEP_BETWEEN_BATCHES,
-  SLEEP_TIME_AFTER_FAILED_DB_OPERATION,
-  SLEEP_TIME_AFTER_FAILED_REASONING_OPERATION,
-  LANDING_ZONE_GRAPH,
-  LANDING_ZONE_DATABASE_ENDPOINT,
-  TARGET_GRAPH,
-  PRIVACY_SENSITIVE_GRAPH,
-  MAIN_INFO_MAPPING,
-  PRIVATE_INFO_MAPPING
-};

config/loket-consumer/worship-services-sensitive-dispatching/delta-context-config.js

@@ -1,11 +0,0 @@
-const contextConfig = {
-  addTypes: {
-    scope: 'all', // 'inserts, 'deletes', 'all' or none. To add rdf:type to subjects of inserts, deletes or both
-    exhausitive: false, // true or false: find all types for a subject, even if one is already present in delta
-  },
-  contextQueries: []
-}
-
-module.exports = {
-  contextConfig
-};

config/loket-consumer/worship-services-sensitive-dispatching/delta-sync-dispatching.js

@@ -1,114 +0,0 @@
-const {
-  transformStatements,
-  deleteFromTargetGraph,
-  insertIntoTargetGraph,
-  deleteFromPrivateGraph,
-  insertIntoPrivateGraph
-} = require('./util');
-
-const {
-  MAIN_INFO_MAPPING,
-  PRIVATE_INFO_MAPPING
-} = require('./config');
-
-/**
-* Dispatch the fetched information to a target graph.
-* Note: <share://file/data> will be ADDED to it's own graph.
-*   We take only care of adding them, not updating triples, this is a TODO
-* @param { mu, muAuthSudo } lib - The provided libraries from the host service.
-* @param { termObjectChangeSets: { deletes, inserts } } data - The fetched changes sets, which objects of serialized Terms
-*          [ {
-*              graph: "<http://foo>",
-*              subject: "<http://bar>",
-*              predicate: "<http://baz>",
-*              object: "<http://boom>^^<http://datatype>"
-*            }
-*         ]
-* @return {void} Nothing
-*/
-async function dispatch(lib, data) {
-  const { mu, fetch } = lib;
-  let { termObjectChangeSets, termObjectChangeSetsWithContext } = data;
-
-  // Both arrays are the same length, so we can zip them together for easier processing
-  zippedChangeSets = termObjectChangeSets.map((o, i) => ({
-    original: o,
-    withContext: termObjectChangeSetsWithContext[i]
-  }));
-
-  console.log(`Received ${zippedChangeSets.length} change sets`)
-
-  for (let { original, withContext } of zippedChangeSets) {
-    let originalDeletes, originalInserts, insertsWithContext, deletesWithContext;
-    try {
-      originalInserts = original.inserts.map(o => `${o.subject} ${o.predicate} ${o.object}.`);
-      originalDeletes = original.deletes.map(o => `${o.subject} ${o.predicate} ${o.object}.`);
-    } catch (e) {
-      console.log(`Error while processing change set: ${e}`);
-      console.log(`Original change set: ${JSON.stringify(original)}`);
-      throw e;
-    }
-
-    // Extra context needed for mapping from DL to OP model and filtering based on type.
-    try {
-      insertsWithContext = withContext.inserts.map(o => `${o.subject} ${o.predicate} ${o.object}.`);
-      deletesWithContext = withContext.deletes.map(o => `${o.subject} ${o.predicate} ${o.object}.`);
-    } catch (e) {
-      console.log(`Error while processing change set with context: ${e}`);
-      console.log(`Change set with context: ${JSON.stringify(withContext)}`);
-      throw e;
-    }
-
-    if (originalDeletes.length) {
-      // Map main deletes from DL to OP model
-      const transformedMainDeletes = await transformStatements(fetch, deletesWithContext, MAIN_INFO_MAPPING);
-
-      if (!transformedMainDeletes.length) {
-        console.log(`Warn: Delete main statements mapped to empty result.`);
-        console.log(`Input: ${deletesWithContext}`);
-        console.log(`Output: ${transformedMainDeletes}`);
-      } else {
-        await deleteFromTargetGraph(lib, transformedMainDeletes);
-      }
-
-      // Map private deletes from DL to OP model
-      const transformedPrivateDeletes = await transformStatements(fetch, deletesWithContext, PRIVATE_INFO_MAPPING);
-
-      if (!transformedPrivateDeletes.length) {
-        console.log(`Warn: Delete private statements mapped to empty result.`);
-        console.log(`Input: ${deletesWithContext}`);
-        console.log(`Output: ${transformedPrivateDeletes}`);
-      } else {
-        await deleteFromPrivateGraph(lib, transformedPrivateDeletes);
-      }
-    }
-
-    if (originalInserts.length) {
-      // Map main inserts from DL to OP model
-      const transformedMainInserts = await transformStatements(fetch, insertsWithContext, MAIN_INFO_MAPPING);
-
-      if (!transformedMainInserts.length) {
-        console.log(`Warn: Insert statements mapped to empty result.`);
-        console.log(`Input: ${insertsWithContext}`);
-        console.log(`Output: ${transformedMainInserts}`);
-      } else {
-        await insertIntoTargetGraph(lib, transformedMainInserts);
-      }
-
-      // Map private inserts from DL to OP model
-      const transformedPrivateInserts = await transformStatements(fetch, insertsWithContext, PRIVATE_INFO_MAPPING);
-
-      if (!transformedPrivateInserts.length) {
-        console.log(`Warn: Insert statements mapped to empty result.`);
-        console.log(`Input: ${insertsWithContext}`);
-        console.log(`Output: ${transformedPrivateInserts}`);
-      } else {
-        await insertIntoPrivateGraph(lib, transformedPrivateInserts);
-      }
-    }
-  }
-}
-
-module.exports = {
-  dispatch
-};

config/loket-consumer/worship-services-sensitive-dispatching/initial-sync-dispatching.js

@@ -1,106 +0,0 @@
-const { batchedDbUpdate, transformLandingZoneGraph } = require('./util');
-const {
-  BYPASS_MU_AUTH_FOR_EXPENSIVE_QUERIES,
-  DIRECT_DATABASE_ENDPOINT,
-  MU_CALL_SCOPE_ID_INITIAL_SYNC,
-  BATCH_SIZE,
-  MAX_DB_RETRY_ATTEMPTS,
-  SLEEP_BETWEEN_BATCHES,
-  SLEEP_TIME_AFTER_FAILED_DB_OPERATION,
-  TARGET_GRAPH,
-  PRIVACY_SENSITIVE_GRAPH,
-  LANDING_ZONE_GRAPH,
-  LANDING_ZONE_DATABASE_ENDPOINT,
-  MAIN_INFO_MAPPING,
-  PRIVATE_INFO_MAPPING
-} = require('./config');
-
-const endpoint = BYPASS_MU_AUTH_FOR_EXPENSIVE_QUERIES ? DIRECT_DATABASE_ENDPOINT : process.env.MU_SPARQL_ENDPOINT;
-
-/**
-* Dispatch the fetched information to a target graph.
-* @param { mu, muAuthSudo } lib - The provided libraries from the host service.
-* @param { termObjects } data - The fetched quad information, which objects of serialized Terms
-*          [ {
-*              graph: "<http://foo>",
-*              subject: "<http://bar>",
-*              predicate: "<http://baz>",
-*              object: "<http://boom>^^<http://datatype>"
-*            }
-*         ]
-* @return {void} Nothing
-*/
-async function dispatch(lib, data) {
-  const { mu, muAuthSudo } = lib;
-  const { termObjects } = data;
-
-  // Steps:
-  // 1. Write the triples to the landing zone graph - withouth any mapping or filtering
-  // 2. One-time reasoning run with the full landing zone graph and write the results to the target graph (on-finish)
-
-  if (BYPASS_MU_AUTH_FOR_EXPENSIVE_QUERIES) {
-    console.warn(`Service configured to skip MU_AUTH!`);
-  }
-  console.log(`Using ${endpoint} to insert triples`);
-
-  if (termObjects.length) {
-    const originalInsertTriples = termObjects.map(o => `${o.subject} ${o.predicate} ${o.object}.`);
-
-    await batchedDbUpdate(
-      muAuthSudo.updateSudo,
-      LANDING_ZONE_GRAPH,
-      originalInsertTriples,
-      { 'mu-call-scope-id': MU_CALL_SCOPE_ID_INITIAL_SYNC },
-      LANDING_ZONE_DATABASE_ENDPOINT,
-      BATCH_SIZE,
-      MAX_DB_RETRY_ATTEMPTS,
-      SLEEP_BETWEEN_BATCHES,
-      SLEEP_TIME_AFTER_FAILED_DB_OPERATION
-    );
-  }
-}
-
-async function onFinishInitialIngest(lib) {
-  const { muAuthSudo, fetch } = lib;
-
-  console.log(`!! On-finish triggered !!`);
-
-  const transformedMainInsertTriples = await transformLandingZoneGraph(fetch, endpoint, MAIN_INFO_MAPPING);
-  console.log(`Transformed ${transformedMainInsertTriples.length} main triples`);
-
-  if (transformedMainInsertTriples.length) {
-    await batchedDbUpdate(
-      muAuthSudo.updateSudo,
-      TARGET_GRAPH,
-      transformedMainInsertTriples,
-      { 'mu-call-scope-id': MU_CALL_SCOPE_ID_INITIAL_SYNC },
-      endpoint,
-      BATCH_SIZE,
-      MAX_DB_RETRY_ATTEMPTS,
-      SLEEP_BETWEEN_BATCHES,
-      SLEEP_TIME_AFTER_FAILED_DB_OPERATION
-    );
-  }
-
-  const transformedPrivateInsertTriples = await transformLandingZoneGraph(fetch, endpoint, PRIVATE_INFO_MAPPING);
-  console.log(`Transformed ${transformedPrivateInsertTriples.length} private triples`);
-
-  if (transformedPrivateInsertTriples.length) {
-    await batchedDbUpdate(
-      muAuthSudo.updateSudo,
-      PRIVACY_SENSITIVE_GRAPH,
-      transformedPrivateInsertTriples,
-      { 'mu-call-scope-id': MU_CALL_SCOPE_ID_INITIAL_SYNC },
-      endpoint,
-      BATCH_SIZE,
-      MAX_DB_RETRY_ATTEMPTS,
-      SLEEP_BETWEEN_BATCHES,
-      SLEEP_TIME_AFTER_FAILED_DB_OPERATION
-    );
-  }
-}
-
-module.exports = {
-  dispatch,
-  onFinishInitialIngest
-};

config/loket-consumer/worship-services-sensitive-dispatching/util.js

@@ -1,204 +0,0 @@
-const {
-  MAX_REASONING_RETRY_ATTEMPTS,
-  SLEEP_TIME_AFTER_FAILED_REASONING_OPERATION,
-  MAX_DB_RETRY_ATTEMPTS,
-  SLEEP_BETWEEN_BATCHES,
-  SLEEP_TIME_AFTER_FAILED_DB_OPERATION,
-  TARGET_GRAPH,
-  PRIVACY_SENSITIVE_GRAPH,
-  LANDING_ZONE_GRAPH,
-  BATCH_SIZE
-} = require('./config');
-
-async function batchedDbUpdate(
-  muUpdate,
-  graph,
-  triples,
-  extraHeaders,
-  endpoint,
-  batchSize,
-  maxAttempts,
-  sleepBetweenBatches = 1000,
-  sleepTimeOnFail = 1000,
-  operation = 'INSERT'
-) {
-
-  for (let i = 0; i < triples.length; i += batchSize) {
-    console.log(`Inserting triples in batch: ${i}-${i + batchSize}`);
-
-    const batch = triples.slice(i, i + batchSize).join('\n');
-
-    const insertCall = async () => {
-      await muUpdate(`
-${operation} DATA {
-GRAPH <${graph}> {
-${batch}
-}
-}
-`, extraHeaders, endpoint);
-    };
-
-    await operationWithRetry(insertCall, 0, maxAttempts, sleepTimeOnFail);
-
-    console.log(`Sleeping before next query execution: ${sleepBetweenBatches}`);
-    await new Promise(r => setTimeout(r, sleepBetweenBatches));
-  }
-}
-
-async function operationWithRetry(callback,
-  attempt,
-  maxAttempts,
-  sleepTimeOnFail) {
-  try {
-    if (typeof callback === "function")
-      return await callback();
-    else // Catch error from promise - not how I would do it normally, but allows re use of existing code.
-      return await callback;
-  }
-  catch (e) {
-    console.log(`Operation failed for ${callback.toString()}, attempt: ${attempt} of ${maxAttempts}`);
-    console.log(`Error: ${e}`);
-    console.log(`Sleeping ${sleepTimeOnFail} ms`);
-
-    if (attempt >= maxAttempts) {
-      console.log(`Max attempts reached for ${callback.toString()}, giving up`);
-      throw e;
-    }
-
-    await new Promise(r => setTimeout(r, sleepTimeOnFail));
-    return operationWithRetry(callback, ++attempt, maxAttempts, sleepTimeOnFail);
-  }
-}
-
-/**
- * Splits an array into two parts, a part that passes and a part that fails a predicate function.
- * Credits: https://github.com/benjay10
- * @public
- * @function partition
- * @param {Array} arr - Array to be partitioned
- * @param {Function} fn - Function that accepts single argument: an element of the array, and should return a truthy or falsy value.
- * @returns {Object} Object that contains keys passes and fails, each representing an array with elemets that pass or fail the predicate respectively
- */
-function partition(arr, fn) {
-  let passes = [], fails = [];
-  arr.forEach((item) => (fn(item) ? passes : fails).push(item));
-  return { passes, fails };
-}
-
-/**
- * Send triples to reasoning service for conversion
- *
- */
-async function transformTriples(fetch, triples, mapping) {
-  return await operationWithRetry(mainConversion(fetch, triples, mapping), 0,
-    MAX_REASONING_RETRY_ATTEMPTS, SLEEP_TIME_AFTER_FAILED_REASONING_OPERATION);
-}
-
-async function mainConversion(fetch, triples, mapping) {
-  let formdata = new URLSearchParams();
-  formdata.append("data", triples);
-
-  let requestOptions = {
-    method: 'POST',
-    body: formdata,
-    redirect: 'follow'
-  };
-
-  const response = await fetch(`http://reasoner/reason/dl2op/${mapping}`, requestOptions)
-  return await response.text();
-}
-
-async function transformStatements(fetch, triples, mapping = 'main') {
-  console.log(`Received ${JSON.stringify(triples)} to transform`);
-  const transformedTriples = await transformTriples(fetch, triples.join('\n'), mapping);
-  statements = transformedTriples ? transformedTriples.replace(/\n{2,}/g, '').split('\n') : [];
-  console.log(`CONVERSION: FROM ${triples.length} triples to ${statements.length}`);
-  return statements;
-}
-
-async function deleteFromTargetGraph(lib, statements) {
-  console.log(`Deleting ${statements.length} statements from target graph`);
-  console.log(`Statements:  ${JSON.stringify(statements)}`)
-  await batchedDbUpdate(
-    lib.muAuthSudo.updateSudo,
-    TARGET_GRAPH,
-    statements,
-    {},
-    process.env.MU_SPARQL_ENDPOINT,
-    BATCH_SIZE,
-    MAX_DB_RETRY_ATTEMPTS,
-    SLEEP_BETWEEN_BATCHES,
-    SLEEP_TIME_AFTER_FAILED_DB_OPERATION,
-    'DELETE');
-}
-
-async function deleteFromPrivateGraph(lib, statements) {
-  console.log(`Deleting ${statements.length} statements from target graph`);
-  console.log(`Statements:  ${JSON.stringify(statements)}`)
-  await batchedDbUpdate(
-    lib.muAuthSudo.updateSudo,
-    PRIVACY_SENSITIVE_GRAPH,
-    statements,
-    {},
-    process.env.MU_SPARQL_ENDPOINT,
-    BATCH_SIZE,
-    MAX_DB_RETRY_ATTEMPTS,
-    SLEEP_BETWEEN_BATCHES,
-    SLEEP_TIME_AFTER_FAILED_DB_OPERATION,
-    'DELETE');
-}
-
-async function insertIntoTargetGraph(lib, statements) {
-  console.log(`Inserting ${statements.length} statements into target graph`);
-  console.log(`Statements:  ${JSON.stringify(statements)}`)
-
-  await batchedDbUpdate(
-    lib.muAuthSudo.updateSudo,
-    TARGET_GRAPH,
-    statements,
-    {},
-    process.env.MU_SPARQL_ENDPOINT,
-    BATCH_SIZE,
-    MAX_DB_RETRY_ATTEMPTS,
-    SLEEP_BETWEEN_BATCHES,
-    SLEEP_TIME_AFTER_FAILED_DB_OPERATION,
-    'INSERT');
-}
-
-async function insertIntoPrivateGraph(lib, statements) {
-  console.log(`Inserting ${statements.length} statements into target graph`);
-  console.log(`Statements:  ${JSON.stringify(statements)}`)
-
-  await batchedDbUpdate(
-    lib.muAuthSudo.updateSudo,
-    PRIVACY_SENSITIVE_GRAPH,
-    statements,
-    {},
-    process.env.MU_SPARQL_ENDPOINT,
-    BATCH_SIZE,
-    MAX_DB_RETRY_ATTEMPTS,
-    SLEEP_BETWEEN_BATCHES,
-    SLEEP_TIME_AFTER_FAILED_DB_OPERATION,
-    'INSERT');
-}
-
-async function transformLandingZoneGraph(fetch, endpoint, mapping = 'main') {
-  console.log(`Transforming landing zone graph: ${LANDING_ZONE_GRAPH}`);
-
-  const response = await fetch(`http://reasoner/reason/dl2op/${mapping}?data=${encodeURIComponent(`${endpoint}?default-graph-uri=&query=CONSTRUCT+%7B%0D%0A%3Fs+%3Fp+%3Fo%0D%0A%7D+WHERE+%7B%0D%0A+GRAPH+<${LANDING_ZONE_GRAPH}>+%7B%0D%0A%3Fs+%3Fp+%3Fo%0D%0A%7D%0D%0A%7D&should-sponge=&format=text%2Fplain&timeout=0&run=+Run+Query`)}`);
-  const text = await response.text();
-  const statements = text.replace(/\n{2,}/g, '').split('\n');
-
-  return statements;
-}
-
-module.exports = {
-  batchedDbUpdate,
-  partition,
-  transformStatements,
-  transformLandingZoneGraph,
-  deleteFromTargetGraph,
-  insertIntoTargetGraph,
-  deleteFromPrivateGraph,
-  insertIntoPrivateGraph
-};

config/migrations-triggering-indexing/2024/20241126155527-rename-merged-municipalities-and-ocmws.sparql

@@ -0,0 +1,33 @@
+PREFIX besluit: <http://data.vlaanderen.be/ns/besluit#>
+PREFIX regorg: <http://www.w3.org/ns/regorg#>
+PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
+
+DELETE {
+  GRAPH ?g {
+    ?organisation skos:prefLabel ?oldLabel;
+                  regorg:legalName ?oldName.
+  }
+} INSERT {
+  GRAPH ?g {
+    ?organisation skos:prefLabel ?newName;
+                  regorg:legalName ?newName.
+  }
+} WHERE {
+  GRAPH ?g {
+    ?organisation a besluit:Bestuurseenheid;
+                  skos:prefLabel ?oldLabel;
+                  regorg:legalName ?oldName.
+  }
+  VALUES ?g {
+    <http://mu.semte.ch/graphs/administrative-unit>
+    <http://mu.semte.ch/graphs/shared>
+  }
+  VALUES (?organisation ?newName) {
+    (<http://data.lblod.info/id/bestuurseenheden/99da98a7a0087d3429b084ebfc4eb5d488c593790d4d5af7253982a2e21a6a5f> """Bilzen-Hoeselt""")
+    (<http://data.lblod.info/id/bestuurseenheden/9ae900a5447b7d727ca6496910220d4389aba7f1869923f1bbf9729bdeca28e2> """Bilzen-Hoeselt""")
+    (<http://data.lblod.info/id/bestuurseenheden/05099fa1f6524b8b994d86f61549455d2c00b2a956d5308683ac2d1f810fc729> """Tessenderlo-Ham""")
+    (<http://data.lblod.info/id/bestuurseenheden/42a43591e0db1dca9432f480f0f49f9bd4056c2b131e2fc997497130f5e099d0> """Tessenderlo-Ham""")
+    (<http://data.lblod.info/id/bestuurseenheden/104f32d7fb8d4b8b61b71717301656f136fe046eabaf126fb3325896b5c2d625> """Tongeren-Borgloon""")
+    (<http://data.lblod.info/id/bestuurseenheden/ab684633d605d93dbbe6b9ea40667e2bcf03a0856cafe1825e95b7829ed502a3> """Tongeren-Borgloon""")
+  }
+}

config/migrations-triggering-indexing/2024/20241203144827-mergers-2025-update-organisation-status.sparql

@@ -0,0 +1,102 @@
+PREFIX regorg: <http://www.w3.org/ns/regorg#>
+
+DELETE {
+  GRAPH ?g {
+    ?org regorg:orgStatus ?oldStatus.
+  }
+} INSERT {
+  GRAPH ?g {
+    ?org  regorg:orgStatus <http://lblod.data.gift/concepts/d02c4e12bf88d2fdf5123b07f29c9311>.
+  }
+} WHERE {
+  GRAPH ?g {
+    ?org regorg:orgStatus ?oldStatus.
+  }
+  VALUES ?g {
+    <http://mu.semte.ch/graphs/administrative-unit>
+  }
+  VALUES ?org {
+    <http://data.lblod.info/id/bestuurseenheden/e41ffa04f94bb450a79793020e70d55b5fee5033a5280277998608a9d0913117>
+    <http://data.lblod.info/id/bestuurseenheden/7125fcbfb7a08f8c01efbed115cea602db249c04afe9b5a2cc298cc7949cd040>
+
+    <http://data.lblod.info/id/bestuurseenheden/4f0eb4436c88cf831c35f84e7c34ce32f9ee4e99c5139aff62990e5e531aa1e7>
+    <http://data.lblod.info/id/bestuurseenheden/6af55cecaea3621f53bb32417a36ed6e3d41b2aa5b9f6d62ab3d80cc8ec11539>
+    <http://data.lblod.info/id/bestuurseenheden/c362abc58ac4579ff417824ce620962ac57bc344b34fe8f51d21b35ef54da36d>
+    <http://data.lblod.info/id/bestuurseenheden/72bc672ceb6894d8dd31dc2dff2faad2fd93f4b9f5347d556d092516eaa30aa3>
+    <http://data.lblod.info/id/bestuurseenheden/d3c76fd84794dad5e12c9eed6d6332dc5e5c99daf0e4ddd7ceebf13d5d6c0ccd>
+    <http://data.lblod.info/id/bestuurseenheden/bbc23313f4c8585a0c51f15f920d77123356535cd1cad9af38f5707892c7088d>
+
+    <http://data.lblod.info/id/bestuurseenheden/a3bd0845853278f478f90b14436d3efa99e73249a84d462f0ddc2e5b6e37a156>
+    <http://data.lblod.info/id/bestuurseenheden/785e8d5516378c7382ad0e7e356d2301496d810c6cd46d72ccebb455d3ac525e>
+
+    <http://data.lblod.info/id/bestuurseenheden/05441122597e0b20b61a8968ea1247c07f9014aad1f1f0709d0b1234e3dfbc2f>
+    <http://data.lblod.info/id/bestuurseenheden/3834647b5e95ea20f5ffe5a12a09fbd3d7fdf0d187fab385cb7b841422176ad6>
+
+    <http://data.lblod.info/id/bestuurseenheden/0327a51548f73607f8a5ec11b36711a3c96703686ad93a3d632718c135c295db>
+    <http://data.lblod.info/id/bestuurseenheden/e39bc997aa6dd9f240277735efd745b6a0422614d2b36cf01825c86b1b91a9ee>
+    <http://data.lblod.info/id/bestuurseenheden/0a2d980559eb9f3e0820386541274a72692f495204cd6d951ea1907ecc65829b>
+    <http://data.lblod.info/id/bestuurseenheden/144c75e403c5084af57f5c0c86252b8759b56baf9694326abbcb5ea40ab69f1a>
+
+    <http://data.lblod.info/id/bestuurseenheden/00eb231f51bd6b4f5dcc6536b2d09a174ea8583f5bf28b10bc4fc769a07e511d>
+    <http://data.lblod.info/id/bestuurseenheden/c69192b973a3ca11531b9657b3ee20aa6688fa33ea1ef1392310fec751980ab2>
+    <http://data.lblod.info/id/bestuurseenheden/3eb8c9fae32d02359dcd7b22e2a74e67a5b48388df31ad819c27c688fedd10b0>
+    <http://data.lblod.info/id/bestuurseenheden/ba8a253d98ea7d2ab1afc43e5e2cd145a12ccd86b68e71d023e3c13a83d7498f>
+    <http://data.lblod.info/id/bestuurseenheden/76c45d60f05bf8afc8c71859342b69cac6a57ee2b320b900a60afd6ba9b8eb9d>
+    <http://data.lblod.info/id/bestuurseenheden/5d21ccef2f5aa3ccbf3a74c6f25ebe9f0b1ef5a97b87f3d79c86ae9d6d857df8>
+
+    <http://data.lblod.info/id/bestuurseenheden/af8969752f6b28c66b6bc402d7987fa38774901ac72b95c5cb7976570487c3c9>
+    <http://data.lblod.info/id/bestuurseenheden/fb1be873c4b31e391613dfae8e68edd694b1fdf126eeecb502b1e5cad6f2f682>
+
+    <http://data.lblod.info/id/bestuurseenheden/f6b131de5e40a0dfd4fc93fedf3b95c9bf156ece718b87fe00469dea2564b3fb>
+    <http://data.lblod.info/id/bestuurseenheden/3e077115eeb70b649d44e2ebbb218f2cfcac5d84e130c88584f0ec23f7cebbf9>
+
+    <http://data.lblod.info/id/bestuurseenheden/1313d4a58f9ecf52cc7e274e3549a759b35e731973cc9f5e07562e5650f594dd>
+    <http://data.lblod.info/id/bestuurseenheden/7c6b38fa6fc65879cb674a654617de171adfcfacba086fb918da3749dbaa43b5>
+
+    <http://data.lblod.info/id/bestuurseenheden/c5dbb08e35e2d090a05d119fef4cc161b5ee1f322698cb8ea3c8c6643a521cf2>
+    <http://data.lblod.info/id/bestuurseenheden/f2a9a2392d5e4c0f992a84a0871d048a7b91b14681e9980a63aa6ba4b1f7eb8c>
+
+    <http://data.lblod.info/id/bestuurseenheden/c0b6b5cf198cd939251dff8ed052177cfff245074c6b8d43394c8c97f7b6e945>
+    <http://data.lblod.info/id/bestuurseenheden/8ef1e4a43efd913e6b09b0ddea344b7b5d723a344ad559389a55ae1ff0bebc8f>
+    <http://data.lblod.info/id/bestuurseenheden/04895013fe6301f32aa46deae98abfb833a717ece5a33b6c453674c6d0f4cc5e>
+    <http://data.lblod.info/id/bestuurseenheden/f56f068943a99059f5bfc3b905fe8848a25a47cfa30fb3ed3d294b47815b2255>
+
+    <http://data.lblod.info/id/bestuurseenheden/5a4b2b4f1de1f3b91b0348a7eb6d6aa0ef9420b8ec31374970c9ffe933a79515>
+    <http://data.lblod.info/id/bestuurseenheden/3f57a17fbcdfc13def910c54948bec11d71d3f75946b0336fffc0587bf18d783>
+
+    <http://data.lblod.info/id/bestuurseenheden/2564e21e3650f91625189ccb7eb055e47754a0633c54c7582a899171fef60c52>
+    <http://data.lblod.info/id/bestuurseenheden/60daf27c2cf12dd123447aa300dd7320890391dbd8dd6b4dd7ec02258f8281b3>
+  }
+}
+;
+DELETE {
+  GRAPH ?g {
+    ?org regorg:orgStatus ?oldStatus.
+  }
+} INSERT {
+  GRAPH ?g {
+    ?org  regorg:orgStatus <http://lblod.data.gift/concepts/63cc561de9188d64ba5840a42ae8f0d6>.
+  }
+} WHERE {
+  GRAPH ?g {
+    ?org regorg:orgStatus ?oldStatus.
+  }
+  VALUES ?g {
+    <http://mu.semte.ch/graphs/administrative-unit>
+  }
+  VALUES ?org {
+    <http://data.lblod.info/id/bestuurseenheden/4b44e6f1-113b-4692-b149-44a889b215f2>
+
+    <http://data.lblod.info/id/bestuurseenheden/1ca65d65-54ff-4b44-b750-bd70c91191af>
+    <http://data.lblod.info/id/bestuurseenheden/7d53f659-3a3b-44b1-9e77-3ea067678c0e>
+
+    <http://data.lblod.info/id/bestuurseenheden/b8bb293d-aa22-4b43-bda4-0b6af76e9493>
+    <http://data.lblod.info/id/bestuurseenheden/5d94b2fd-60ee-4e56-a1f0-a586d596adf6>
+
+    <http://data.lblod.info/id/bestuurseenheden/19483103-318e-435a-aa37-45e485406ee9>
+    <http://data.lblod.info/id/bestuurseenheden/fd41f573-7d9a-4d9f-b7d0-d5b6114d1622>
+
+    <http://data.lblod.info/id/bestuurseenheden/add1e4eb-9ec7-4ea6-af82-335aa76b7d48>
+    <http://data.lblod.info/id/bestuurseenheden/650378e7-1bee-4737-91ff-5b20ac4623cf>
+  }
+}

config/migrations-triggering-indexing/2024/20241212174128-archive-agb-stekene.sparql

@@ -0,0 +1,15 @@
+DELETE {
+  GRAPH ?g {
+    <http://data.lblod.info/id/bestuurseenheden/8d6609b4-1a20-46b8-97ed-150c6d53acd6> <http://www.w3.org/ns/regorg#orgStatus> ?orgStatus .
+  }
+}
+INSERT {
+  GRAPH ?g {
+    <http://data.lblod.info/id/bestuurseenheden/8d6609b4-1a20-46b8-97ed-150c6d53acd6> <http://www.w3.org/ns/regorg#orgStatus> <http://lblod.data.gift/concepts/d02c4e12bf88d2fdf5123b07f29c9311> .
+  }
+}
+WHERE {
+  GRAPH ?g {
+    <http://data.lblod.info/id/bestuurseenheden/8d6609b4-1a20-46b8-97ed-150c6d53acd6> <http://www.w3.org/ns/regorg#orgStatus> ?orgStatus .
+  }
+}

config/migrations-triggering-indexing/2024/20241217082404-mergers-2025--update-addresses/20241217082404-change-zwijndrecht-to-province-east-flanders.sparql

@@ -0,0 +1,27 @@
+PREFIX locn: <http://www.w3.org/ns/locn#>
+PREFIX adres: <https://data.vlaanderen.be/ns/adres#>
+PREFIX organisatie: <https://data.vlaanderen.be/ns/organisatie#>
+PREFIX org: <http://www.w3.org/ns/org#>
+
+DELETE {
+  GRAPH ?g {
+    ?address locn:adminUnitL2 "Antwerpen".
+  }
+} INSERT {
+  GRAPH ?g {
+    ?address locn:adminUnitL2 "Oost-Vlaanderen".
+  }
+} WHERE {
+  GRAPH ?g {
+    ?address a locn:Address;
+             adres:gemeentenaam "Zwijndrecht";
+             locn:adminUnitL2 "Antwerpen"
+    FILTER (EXISTS { ?site organisatie:bestaatUit ?address. } ||
+            EXISTS { ?site org:siteAddress/locn:address ?address. })
+
+  }
+  VALUES ?g {
+    <http://mu.semte.ch/graphs/worship-service>
+    <http://mu.semte.ch/graphs/administrative-unit>
+  }
+}

config/migrations-triggering-indexing/2024/20241217082404-mergers-2025--update-addresses/20241217082944-update-address-municipality-to-merged-municipality-name.sparql

@@ -0,0 +1,51 @@
+PREFIX adres: <https://data.vlaanderen.be/ns/adres#>
+PREFIX locn: <http://www.w3.org/ns/locn#>
+PREFIX organisatie: <https://data.vlaanderen.be/ns/organisatie#>
+PREFIX org: <http://www.w3.org/ns/org#>
+
+DELETE {
+  GRAPH ?g {
+    ?address adres:gemeentenaam ?oldMunicipality.
+  }
+}
+INSERT {
+  GRAPH ?g {
+    ?address adres:gemeentenaam ?newMunicipality.
+  }
+}
+WHERE {
+  GRAPH ?g {
+    ?address a locn:Address;
+             adres:gemeentenaam ?oldMunicipality.
+    FILTER (EXISTS { ?site organisatie:bestaatUit ?address. } ||
+            EXISTS { ?site org:siteAddress/locn:address ?address. })
+  }
+  VALUES ?g {
+    <http://mu.semte.ch/graphs/administrative-unit>
+    <http://mu.semte.ch/graphs/worship-service>
+  }
+  VALUES (?oldMunicipality ?newMunicipality) {
+    ("Borsbeek" "Antwerpen")
+    ("Galmaarden" "Pajottegem")
+    ("Gooik" "Pajottegem")
+    ("Herne" "Pajottegem")
+    ("Ruiselede" "Wingene")
+    ("Meulebeke" "Tielt")
+    ("De Pinte" "Nazareth-De Pinte")
+    ("Nazareth" "Nazareth-De Pinte")
+    ("Wachtebeke" "Lochristi")
+    ("Moerbeke" "Lokeren")
+    ("Melle" "Merelbeke-Melle")
+    ("Merelbeke" "Merelbeke-Melle")
+    ("Beveren" "Beveren-Kruibeke-Zwijndrecht")
+    ("Kruibeke" "Beveren-Kruibeke-Zwijndrecht")
+    ("Zwijndrecht" "Beveren-Kruibeke-Zwijndrecht")
+    ("Bilzen" "Bilzen-Hoeselt")
+    ("Hoeselt" "Bilzen-Hoeselt")
+    ("Borgloon" "Tongeren-Borgloon")
+    ("Tongeren" "Tongeren-Borgloon")
+    ("Ham" "Tessenderlo-Ham")
+    ("Tessenderlo" "Tessenderlo-Ham")
+    ("Kortessem" "Hasselt")
+  }
+}

config/migrations-triggering-indexing/2024/20241217082404-mergers-2025--update-addresses/20241217083314-regenerate-full-address-triple-for-changed-addresses.sparql

@@ -0,0 +1,92 @@
+PREFIX locn: <http://www.w3.org/ns/locn#>
+PREFIX adres: <https://data.vlaanderen.be/ns/adres#>
+PREFIX organisatie: <https://data.vlaanderen.be/ns/organisatie#>
+PREFIX org: <http://www.w3.org/ns/org#>
+
+DELETE {
+  GRAPH ?g {
+    ?address locn:fullAddress ?fullAddress.
+  }
+} INSERT {
+  GRAPH ?g {
+    ?address locn:fullAddress ?newFullAddress.
+  }
+} WHERE {
+  GRAPH ?g {
+    ?address locn:fullAddress ?fullAddress;
+             locn:thoroughfare ?street;
+             adres:Adresvoorstelling.huisnummer ?buildingNo;
+             locn:postCode ?postalCode;
+             adres:gemeentenaam ?municipality;
+             adres:land ?country.
+    FILTER NOT EXISTS { ?address adres:Adresvoorstelling.busnummer ?boxNo. }
+    FILTER (EXISTS { ?site organisatie:bestaatUit ?address. } ||
+            EXISTS { ?site org:siteAddress/locn:address ?address. })
+    FILTER (?municipality IN (
+                              "Antwerpen",
+                              "Pajottegem",
+                              "Wingene",
+                              "Tielt",
+                              "Nazareth-De Pinte",
+                              "Lochristi",
+                              "Lokeren",
+                              "Merelbeke-Melle",
+                              "Beveren-Kruibeke-Zwijndrecht",
+                              "Bilzen-Hoeselt",
+                              "Tongeren-Borgloon",
+                              "Tessenderlo-Ham",
+                              "Hasselt"
+                              ))
+    BIND (CONCAT(?street, " ", ?buildingNo, ", ", ?postalCode, " ", ?municipality, ", ", ?country) AS ?newFullAddress)
+    FILTER (?fullAddress != ?newFullAddress)
+  }
+
+  VALUES ?g {
+    <http://mu.semte.ch/graphs/administrative-unit>
+    <http://mu.semte.ch/graphs/worship-service>
+  }
+}
+;
+DELETE {
+  GRAPH ?g {
+    ?address locn:fullAddress ?fullAddress.
+  }
+} INSERT {
+  GRAPH ?g {
+    ?address locn:fullAddress ?newFullAddress.
+  }
+} WHERE {
+  GRAPH ?g {
+    ?address locn:fullAddress ?fullAddress;
+             locn:thoroughfare ?street;
+             adres:Adresvoorstelling.huisnummer ?buildingNo;
+             adres:Adresvoorstelling.busnummer ?boxNo;
+             locn:postCode ?postalCode;
+             adres:gemeentenaam ?municipality;
+             adres:land ?country.
+    FILTER (EXISTS { ?site organisatie:bestaatUit ?address. } ||
+            EXISTS { ?site org:siteAddress/locn:address ?address. })
+    FILTER (?municipality IN (
+                              "Antwerpen",
+                              "Pajottegem",
+                              "Wingene",
+                              "Tielt",
+                              "Nazareth-De Pinte",
+                              "Lochristi",
+                              "Lokeren",
+                              "Merelbeke-Melle",
+                              "Beveren-Kruibeke-Zwijndrecht",
+                              "Bilzen-Hoeselt",
+                              "Tongeren-Borgloon",
+                              "Tessenderlo-Ham",
+                              "Hasselt"
+                              ))
+    BIND (CONCAT(?street, " ", ?buildingNo, " ", ?boxNo, ", ", ?postalCode, " ", ?municipality, ", ", ?country) AS ?newFullAddress)
+    FILTER (?fullAddress != ?newFullAddress)
+  }
+
+  VALUES ?g {
+    <http://mu.semte.ch/graphs/administrative-unit>
+    <http://mu.semte.ch/graphs/worship-service>
+  }
+}

config/migrations-triggering-indexing/2025/20250108194417-mergers-2025-police-zones--deactivate-merged-organisations.sparql

@@ -0,0 +1,25 @@
+PREFIX regorg: <http://www.w3.org/ns/regorg#>
+
+DELETE {
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?s regorg:orgStatus ?activeStatus.
+  }
+} INSERT {
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?s regorg:orgStatus ?notActiveStatus.
+  }
+} WHERE {
+  VALUES ?s {
+    <http://data.lblod.info/id/bestuurseenheden/150440a8bb763d6f53d24ca1c6f1460cb5817c8a261fd90e74aadac3292c3efe>
+    <http://data.lblod.info/id/bestuurseenheden/41d31a2eb697af977aadf64c73e0bcffb094e0f20e4017fe3b49c507079c0976>
+
+    <http://data.lblod.info/id/bestuurseenheden/330516b323c00851cf4b93e6d5d7a2ef4591f68ae6e71ad9c1f432ee70521bb2>
+    <http://data.lblod.info/id/bestuurseenheden/7dd1250996188607bb1e28df1ebea0b943e8db59c08ea01e3c57672531c43ec6>
+    <http://data.lblod.info/id/bestuurseenheden/541c1603eb8c01968f95d1f262854134499e00d135cd7c9147d307ccbd1e20fd>
+
+    <http://data.lblod.info/id/bestuurseenheden/4deae42b2b25bff407f5abc7cbb6d62eb02afaf2892086341c324fdf2c0e6e92>
+    <http://data.lblod.info/id/bestuurseenheden/b7af3483cb5d0021c6f5864a433b211fce30246ef685621d104b1951915c21c4>
+  }
+  BIND (<http://lblod.data.gift/concepts/d02c4e12bf88d2fdf5123b07f29c9311> AS ?notActiveStatus)
+  BIND (<http://lblod.data.gift/concepts/63cc561de9188d64ba5840a42ae8f0d6> AS ?activeStatus)
+}

config/migrations-triggering-indexing/2025/20250109200145-archive-agb-stekene.sparql

@@ -0,0 +1,196 @@
+DELETE {
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?bestuursorganen ?bestuursorganenP ?bestuursorganenO .
+  }
+}
+WHERE {
+  VALUES ?bestuursorganen {
+    <http://data.lblod.info/id/bestuursorganen/24d8e602a8ddbe0adf3dea8c40277706>
+    <http://data.lblod.info/id/bestuursorganen/dad1789d9b9f8f6cb3c81cc2ca88154f>
+    <http://data.lblod.info/id/bestuursorganen/b90d16f2c8043bc20b88da232c0389fc>
+    <http://data.lblod.info/id/bestuursorganen/2f27a688e1728c1906a4bbd8670d7810>
+  }
+
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?bestuursorganen ?bestuursorganenP ?bestuursorganenO .
+  }
+}
+
+;
+
+DELETE {
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?bestuursorganenInTijd ?bestuursorganenInTijdP ?bestuursorganenInTijdO .
+  }
+}
+WHERE {
+  VALUES ?bestuursorganenInTijd {
+    <http://data.lblod.info/id/bestuursorganen/c5bb7bca7363109b94b58b1aef6ed59c>
+    <http://data.lblod.info/id/bestuursorganen/8d43a29c6ed4a8af0cd43d4aba2c7014>
+    <http://data.lblod.info/id/bestuursorganen/f54979f47464bae1cc001788aedb8b86>
+    <http://data.lblod.info/id/bestuursorganen/c9a7d9130e1dd8f26ddb8c96fdd94c64>
+  }
+
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?bestuursorganenInTijd ?bestuursorganenInTijdP ?bestuursorganenInTijdO .
+  }
+}
+
+;
+
+DELETE {
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?identifier ?identifierP ?identifierO .
+  }
+}
+WHERE {
+  VALUES ?identifier {
+    <http://data.lblod.info/id/identificatoren/99554be7-29d2-403a-a7d5-bda842b4bd86>
+    <http://data.lblod.info/id/identificatoren/19847e1b-a2bf-48a1-872e-baf8d4efaa12>
+    <http://data.lblod.info/id/identificatoren/b2123d27-b6bc-4e70-b594-27d35c696713>
+  }
+
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?identifier ?identifierP ?identifierO .
+  }
+}
+
+;
+
+DELETE {
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?structuredIdentifier ?structuredIdentifierP ?structuredIdentifierO .
+  }
+}
+WHERE {
+  VALUES ?structuredIdentifier {
+    <http://data.lblod.info/id/gestructureerdeIdentificatoren/920f98a2-1785-4de1-81ed-5859c224dcb8>
+    <http://data.lblod.info/id/gestructureerdeIdentificatoren/5e1082d8-7751-4bed-a53f-457b347f7f28>
+    <http://data.lblod.info/id/gestructureerdeIdentificatoren/2e978403-d3ef-40ab-a9f9-3a4ae6a35609>
+  }
+
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?structuredIdentifier ?structuredIdentifierP ?structuredIdentifierO .
+  }
+}
+
+;
+
+DELETE {
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?site ?siteP ?siteO .
+  }
+}
+WHERE {
+  VALUES ?site {
+    <http://data.lblod.info/id/vestigingen/daa79855-1de7-4479-8a2e-a8d89db55ee3>
+  }
+
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?site ?siteP ?siteO .
+  }
+}
+
+;
+
+DELETE {
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?address ?addressP ?addressO .
+  }
+}
+WHERE {
+  VALUES ?address {
+    <http://data.lblod.info/id/adressen/df102cb4-8abb-4588-a199-1ce22c211812>
+  }
+
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?address ?addressP ?addressO .
+  }
+}
+
+;
+
+DELETE {
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?contactPoint ?contactPointP ?contactPointO .
+  }
+}
+WHERE {
+  VALUES ?contactPoint {
+    <http://data.lblod.info/id/contact-punten/042e8816-2e9d-45a1-89f6-dcf805882af7>
+    <http://data.lblod.info/id/contact-punten/364d8e1a-cb18-4970-aeaf-5f6f307370ad>
+  }
+
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?contactPoint ?contactPointP ?contactPointO .
+  }
+}
+
+;
+
+DELETE {
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?parentOrganization <http://www.w3.org/ns/org#hasSubOrganization> ?organization .
+  }
+}
+WHERE {
+  BIND(<http://data.lblod.info/id/bestuurseenheden/8d6609b4-1a20-46b8-97ed-150c6d53acd6> AS ?organization)
+  BIND(<http://data.lblod.info/id/bestuurseenheden/6025a5d1ca2262a784f002edd8ce9ca9073ae3d5ebc6b6b5531f05a29e9250af> AS ?parentOrganization)
+
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?parentOrganization <http://www.w3.org/ns/org#hasSubOrganization> ?organization .
+  }
+}
+
+;
+
+DELETE {
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?parentOrganization <http://mu.semte.ch/vocabularies/ext/isFounderOfOrganization> ?organization .
+  }
+}
+WHERE {
+  BIND(<http://data.lblod.info/id/bestuurseenheden/8d6609b4-1a20-46b8-97ed-150c6d53acd6> AS ?organization)
+  BIND(<http://data.lblod.info/id/bestuurseenheden/6025a5d1ca2262a784f002edd8ce9ca9073ae3d5ebc6b6b5531f05a29e9250af> AS ?parentOrganization)
+
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?parentOrganization <http://mu.semte.ch/vocabularies/ext/isFounderOfOrganization> ?organization .
+  }
+}
+
+;
+
+DELETE {
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?membership ?p ?o.
+  }
+}
+WHERE {
+  BIND(<http://data.lblod.info/id/bestuurseenheden/8d6609b4-1a20-46b8-97ed-150c6d53acd6> AS ?organization)
+
+  VALUES ?membershipP {
+    <http://www.w3.org/ns/org#organization>
+    <http://www.w3.org/ns/org#member>
+  }
+
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?membership a <http://www.w3.org/ns/org#Membership> ;
+      ?membershipP ?organization ;
+      ?p ?o .
+  }
+}
+
+;
+
+DELETE {
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?organization ?organizationP ?organizationO .
+  }
+}
+WHERE {
+  BIND(<http://data.lblod.info/id/bestuurseenheden/8d6609b4-1a20-46b8-97ed-150c6d53acd6> AS ?organization)
+
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?organization ?organizationP ?organizationO .
+  }
+}

config/migrations-triggering-indexing/2025/20250321162045-change-ocmwv-to-vvmw.sparql

@@ -0,0 +1,18 @@
+DELETE {
+  GRAPH <http://mu.semte.ch/graphs/public> {
+    ?s <http://www.w3.org/2004/02/skos/core#prefLabel> ?label
+  }
+}
+INSERT {
+  GRAPH <http://mu.semte.ch/graphs/public> {
+    ?s <http://www.w3.org/2004/02/skos/core#prefLabel> "VVMW".
+  }
+}
+WHERE {
+  VALUES ?s {
+    <http://data.vlaanderen.be/id/concept/BestuurseenheidClassificatieCode/cc4e2d67-603b-4784-9b61-e50bac1ec089>
+  }
+  GRAPH <http://mu.semte.ch/graphs/public> {
+    ?s <http://www.w3.org/2004/02/skos/core#prefLabel> ?label.
+  }
+}

config/migrations-triggering-indexing/2025/20250321171109-change-vvmw-to-ocmwv.sparql

@@ -0,0 +1,18 @@
+DELETE {
+  GRAPH <http://mu.semte.ch/graphs/public> {
+    ?s <http://www.w3.org/2004/02/skos/core#prefLabel> ?label
+  }
+}
+INSERT {
+  GRAPH <http://mu.semte.ch/graphs/public> {
+    ?s <http://www.w3.org/2004/02/skos/core#prefLabel> "OCMW vereniging".
+  }
+}
+WHERE {
+  VALUES ?s {
+    <http://data.vlaanderen.be/id/concept/BestuurseenheidClassificatieCode/cc4e2d67-603b-4784-9b61-e50bac1ec089>
+  }
+  GRAPH <http://mu.semte.ch/graphs/public> {
+    ?s <http://www.w3.org/2004/02/skos/core#prefLabel> ?label.
+  }
+}

config/migrations-triggering-indexing/2025/20250327135401-update-ckb-statuses.sparql

@@ -0,0 +1,42 @@
+PREFIX regorg: <http://www.w3.org/ns/regorg#>
+
+DELETE {
+  GRAPH <http://mu.semte.ch/graphs/worship-service> {
+    ?s regorg:orgStatus ?activeStatus.
+  }
+} INSERT {
+  GRAPH <http://mu.semte.ch/graphs/worship-service> {
+    ?s regorg:orgStatus ?notActiveStatus.
+  }
+} WHERE {
+  VALUES ?s {
+    <http://data.lblod.info/id/centraleBesturenVanDeEredienst/ed0914f40dfed83ca44af845a2bf4d81>
+    <http://data.lblod.info/id/centraleBesturenVanDeEredienst/9e0e8f060ca954da6263243642433f44>
+    <http://data.lblod.info/id/centraleBesturenVanDeEredienst/b93ae92a8eb4912125094d98b8b4841d>
+    <http://data.lblod.info/id/centraleBesturenVanDeEredienst/988321712204bd0104fb62e6f5dbe104>
+    <http://data.lblod.info/id/centraleBesturenVanDeEredienst/70e058024a3cd58f76005d645b892e01>
+    <http://data.lblod.info/id/centraleBesturenVanDeEredienst/cf4add06db226861767559753fb50206>
+    <http://data.lblod.info/id/centraleBesturenVanDeEredienst/5442c1374ed159347bc0c64518a9b1a9>
+  }
+  BIND (<http://lblod.data.gift/concepts/d02c4e12bf88d2fdf5123b07f29c9311> AS ?notActiveStatus)
+  BIND (<http://lblod.data.gift/concepts/63cc561de9188d64ba5840a42ae8f0d6> AS ?activeStatus)
+}
+
+;
+
+DELETE {
+  GRAPH <http://mu.semte.ch/graphs/worship-service> {
+    ?s regorg:orgStatus ?notActiveStatus.
+  }
+} INSERT {
+  GRAPH <http://mu.semte.ch/graphs/worship-service> {
+    ?s regorg:orgStatus ?activeStatus.
+  }
+} WHERE {
+  VALUES ?s {
+    <http://data.lblod.info/id/centraleBesturenVanDeEredienst/e12c4f9f7167cd0074a97a2f658d1d51>
+
+  }
+  BIND (<http://lblod.data.gift/concepts/d02c4e12bf88d2fdf5123b07f29c9311> AS ?notActiveStatus)
+  BIND (<http://lblod.data.gift/concepts/63cc561de9188d64ba5840a42ae8f0d6> AS ?activeStatus)
+}

config/migrations-triggering-indexing/2025/20250429103515-remove-has-suborganization-from-inactive-ckb-deinze.sparql

@@ -0,0 +1,55 @@
+PREFIX org: <http://www.w3.org/ns/org#>
+PREFIX mu: <http://mu.semte.ch/vocabularies/core/>
+
+INSERT {
+  GRAPH <http://mu.semte.ch/graphs/worship-service> {
+    ?activeOrg
+      <http://www.w3.org/ns/org#hasSubOrganization> ?sub .
+
+    ?membership a org:Membership ;
+      mu:uuid ?uuid ;
+      org:organization ?activeOrg ;
+      org:member ?sub ;
+      org:role <http://data.lblod.info/id/rollen/4ec7d5c39bdc4e84b4174379b9e22ad8> .
+  }
+}
+WHERE {
+  BIND(<http://data.lblod.info/id/centraleBesturenVanDeEredienst/643ddbd9448ca43828702527af7bfbb8> AS ?inactiveOrg)
+  BIND(<http://data.lblod.info/id/centraleBesturenVanDeEredienst/7f5475cfb202d12f54779f046441c9e1> AS ?activeOrg)
+
+  GRAPH <http://mu.semte.ch/graphs/worship-service> {
+    ?inactiveOrg
+      <http://www.w3.org/ns/org#hasSubOrganization> ?sub .
+  }
+
+  FILTER NOT EXISTS {
+   ?activeOrg
+     <http://www.w3.org/ns/org#hasSubOrganization> ?sub .
+  }
+
+  BIND(SHA1(CONCAT(STR(?activeOrg), STR(?sub), "sub organization")) as ?uuid)
+  BIND(IRI(CONCAT("http://data.lblod.info/id/lidmaatschap/", ?uuid)) AS ?membership)
+}
+
+;
+
+DELETE {
+  GRAPH <http://mu.semte.ch/graphs/worship-service> {
+    ?inactiveOrg
+      <http://www.w3.org/ns/org#hasSubOrganization> ?sub .
+
+    ?membership ?membershipP ?membershipO .
+  }
+}
+WHERE {
+  BIND(<http://data.lblod.info/id/centraleBesturenVanDeEredienst/643ddbd9448ca43828702527af7bfbb8> AS ?inactiveOrg)
+
+  GRAPH <http://mu.semte.ch/graphs/worship-service> {
+    ?inactiveOrg
+      <http://www.w3.org/ns/org#hasSubOrganization> ?sub .
+
+    ?membership a org:Membership ;
+      org:organization ?inactiveOrg ;
+      ?membershipP ?membershipO .
+  }
+}

config/migrations-triggering-indexing/20250114093158-fix-merger-change-event-sint_martinus-st_amandus.sparql

@@ -0,0 +1,23 @@
+PREFIX regorg: <http://www.w3.org/ns/regorg#>
+PREFIX org: <http://www.w3.org/ns/org#>
+
+DELETE {
+  GRAPH <http://mu.semte.ch/graphs/worship-service> {
+    ?stAmandus regorg:orgStatus ?oldStatus.
+  }
+} INSERT {
+  GRAPH <http://mu.semte.ch/graphs/worship-service> {
+    ?stAmandus regorg:orgStatus ?notActiveStatus.
+
+    ?mergerChangeEvent org:originalOrganization ?stAmandus.
+  }
+} WHERE {
+  GRAPH <http://mu.semte.ch/graphs/worship-service> {
+    ?stAmandus regorg:orgStatus ?oldStatus.
+  }
+  VALUES ?stAmandus {
+    <http://data.lblod.info/id/besturenVanDeEredienst/b18a5d3943087d509d76dcceeba102c3>
+  }
+  BIND (<http://data.lblod.info/id/veranderingsgebeurtenissen/677E445E29299CC832892A0F> AS ?mergerChangeEvent)
+  BIND (<http://lblod.data.gift/concepts/d02c4e12bf88d2fdf5123b07f29c9311> AS ?notActiveStatus)
+}

config/migrations-triggering-indexing/local/README.md

@@ -0,0 +1,3 @@
+# Motivation
+
+This folder is designed to contain migration files specific to a specific deployed instance of the app. These files will be ignored by git. The typical use case for this folder includes migrations containing sensitive data or those related to a re-sync of consumers. Migrations that should be applied to all instances of the app should be stored outside of this folder, but within the 'migrations-triggering-indexing' folder.

config/migrations/2024/20240704112300-related-organizations-as-memberships/20240704112500-codelist-lidmaatschap-rollen.graph

@@ -0,0 +1 @@
+http://mu.semte.ch/graphs/public

config/migrations/2024/20240704112300-related-organizations-as-memberships/20240704112500-codelist-lidmaatschap-rollen.ttl

@@ -0,0 +1,31 @@
+@prefix org: <http://www.w3.org/ns/org#> .
+@prefix owl: <http://www.w3.org/2002/07/owl#> .
+@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
+@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
+@prefix schema: <http://schema.org/> .
+@prefix skos: <http://www.w3.org/2004/02/skos/core#> .
+@prefix mu: <http://mu.semte.ch/vocabularies/core/> .
+
+<https://data.vlaanderen.be/id/conceptscheme/lidmaatschap-rollen> a skos:ConceptScheme;
+  mu:uuid "4755a6afc9044ff289d872092e202472";
+  skos:prefLabel "lidmaatschap rollen".
+
+<http://data.lblod.info/id/rollen/93c48754610c45e6bd9a894d2720a53d> a org:Role;
+  mu:uuid "93c48754610c45e6bd9a894d2720a53d";
+  skos:topConceptOf <https://data.vlaanderen.be/id/conceptscheme/lidmaatschap-rollen>;
+  skos:prefLabel "lid".
+
+<http://data.lblod.info/id/rollen/73d5e1cf250d42fab15926771f07505a> a org:Role;
+  mu:uuid "73d5e1cf250d42fab15926771f07505a";
+  skos:prefLabel "stichtend lid";
+  skos:broader <http://data.lblod.info/id/rollen/93c48754610c45e6bd9a894d2720a53d>.
+
+<http://data.lblod.info/id/rollen/2152eb830b1143bfb97a7dd9596d6c63> a org:Role;
+  mu:uuid "2152eb830b1143bfb97a7dd9596d6c63";
+  skos:prefLabel "participant";
+  skos:broader <http://data.lblod.info/id/rollen/93c48754610c45e6bd9a894d2720a53d>.
+
+<http://data.lblod.info/id/rollen/4ec7d5c39bdc4e84b4174379b9e22ad8> a org:Role;
+  mu:uuid "4ec7d5c39bdc4e84b4174379b9e22ad8";
+  skos:prefLabel "gerelateerd";
+  skos:broader <http://data.lblod.info/id/rollen/93c48754610c45e6bd9a894d2720a53d>.

config/migrations/2024/20240704112300-related-organizations-as-memberships/20240704112600-founding-members.sparql

@@ -0,0 +1,35 @@
+PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
+PREFIX org: <http://www.w3.org/ns/org#>
+PREFIX mu: <http://mu.semte.ch/vocabularies/core/>
+PREFIX ext: <http://mu.semte.ch/vocabularies/ext/>
+
+DELETE {
+  GRAPH ?g {
+    ?foundingOrg ext:isFounderOfOrganization ?foundedOrg .
+  }
+}
+INSERT {
+  GRAPH ?g {
+    ?foundingOrg org:memberOf ?foundedOrg .
+
+    ?membership a org:Membership ;
+      mu:uuid ?uuid ;
+      org:organization ?foundedOrg ;
+      org:member ?foundingOrg ;
+      org:role <http://data.lblod.info/id/rollen/73d5e1cf250d42fab15926771f07505a> . # stichtend lid
+  }
+}
+WHERE {
+    VALUES ?g {
+      <http://mu.semte.ch/graphs/worship-service>
+      <http://mu.semte.ch/graphs/administrative-unit>
+      <http://mu.semte.ch/graphs/shared>
+    }
+
+    GRAPH ?g {
+      ?foundingOrg ext:isFounderOfOrganization ?foundedOrg .
+    }
+
+  BIND(SHA1(CONCAT(STR(?foundingOrg), STR(?foundedOrg), "founded by")) as ?uuid)
+  BIND(IRI(CONCAT("http://data.lblod.info/id/lidmaatschap/", ?uuid)) AS ?membership)
+}

config/migrations/2024/20240704112300-related-organizations-as-memberships/20240704112700-participants.sparql

@@ -0,0 +1,32 @@
+PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
+PREFIX org: <http://www.w3.org/ns/org#>
+PREFIX mu: <http://mu.semte.ch/vocabularies/core/>
+
+DELETE {
+  GRAPH ?g {
+    ?memberOrganization org:memberOf ?organization .
+  }
+}
+INSERT {
+  GRAPH ?g {
+    ?membership a org:Membership ;
+      mu:uuid ?uuid ;
+      org:organization ?organization ;
+      org:member ?memberOrganization ;
+      org:role <http://data.lblod.info/id/rollen/2152eb830b1143bfb97a7dd9596d6c63> . # participant
+  }
+}
+WHERE {
+    VALUES ?g {
+      <http://mu.semte.ch/graphs/worship-service>
+      <http://mu.semte.ch/graphs/administrative-unit>
+      <http://mu.semte.ch/graphs/shared>
+    }
+
+    GRAPH ?g {
+      ?memberOrganization org:memberOf ?organization .
+    }
+
+  BIND(SHA1(CONCAT(STR(?memberOrganization), STR(?organization), "member of")) as ?uuid)
+  BIND(IRI(CONCAT("http://data.lblod.info/id/lidmaatschap/", ?uuid)) AS ?membership)
+}

config/migrations/2024/20240704112300-related-organizations-as-memberships/20240704112800-related-members.sparql

@@ -0,0 +1,26 @@
+PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
+PREFIX org: <http://www.w3.org/ns/org#>
+PREFIX mu: <http://mu.semte.ch/vocabularies/core/>
+
+INSERT {
+  GRAPH ?g {
+    ?membership a org:Membership ;
+      mu:uuid ?uuid ;
+      org:organization ?organization ;
+      org:member ?subOrganization ;
+      org:role <http://data.lblod.info/id/rollen/4ec7d5c39bdc4e84b4174379b9e22ad8> . # gerelateerd
+  }
+} WHERE {
+    VALUES ?g {
+      <http://mu.semte.ch/graphs/worship-service>
+      <http://mu.semte.ch/graphs/administrative-unit>
+      <http://mu.semte.ch/graphs/shared>
+    }
+
+    GRAPH ?g {
+      ?organization org:hasSubOrganization ?subOrganization.
+    }
+
+  BIND(SHA1(CONCAT(STR(?organization), STR(?subOrganization), "sub organization")) as ?uuid)
+  BIND(IRI(CONCAT("http://data.lblod.info/id/lidmaatschap/", ?uuid)) AS ?membership)
+}

config/migrations/2024/20240704112300-related-organizations-as-memberships/20240716151700-associated-with.sparql

@@ -0,0 +1,27 @@
+PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
+PREFIX org: <http://www.w3.org/ns/org#>
+PREFIX mu: <http://mu.semte.ch/vocabularies/core/>
+PREFIX ext: <http://mu.semte.ch/vocabularies/ext/>
+
+INSERT {
+  GRAPH ?g {
+    ?membership a org:Membership ;
+      mu:uuid ?uuid ;
+      org:organization ?linkedOrg ;
+      org:member ?org ;
+      org:role <http://data.lblod.info/id/rollen/4ec7d5c39bdc4e84b4174379b9e22ad8> . # gerelateerd
+  }
+} WHERE {
+    VALUES ?g {
+      <http://mu.semte.ch/graphs/worship-service>
+      <http://mu.semte.ch/graphs/administrative-unit>
+      <http://mu.semte.ch/graphs/shared>
+    }
+
+    GRAPH ?g {
+      ?org org:linkedTo ?linkedOrg .
+    }
+
+  BIND(SHA1(CONCAT(STR(?org), STR(?linkedOrg), "associated with")) as ?uuid)
+  BIND(IRI(CONCAT("http://data.lblod.info/id/lidmaatschap/", ?uuid)) AS ?membership)
+}

config/migrations/2024/20240905164200-remove-https-BestuurseenheidClassificatieCode.sparql

@@ -0,0 +1,13 @@
+PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
+
+DELETE DATA {
+  GRAPH <http://mu.semte.ch/graphs/public> {
+    <http://data.vlaanderen.be/id/concept/BestuurseenheidClassificatieCode/36a82ba0-7ff1-4697-a9dd-2e94df73b721>
+      skos:inScheme <https://data.vlaanderen.be/id/conceptscheme/BestuurseenheidClassificatieCode> ;
+      skos:topConceptOf <https://data.vlaanderen.be/id/conceptscheme/BestuurseenheidClassificatieCode> .
+    <http://data.vlaanderen.be/id/concept/BestuurseenheidClassificatieCode/80310756-ce0a-4a1b-9b8e-7c01b6cc7a2d>
+      skos:inScheme <https://data.vlaanderen.be/id/conceptscheme/BestuurseenheidClassificatieCode> ;
+      skos:topConceptOf <https://data.vlaanderen.be/id/conceptscheme/BestuurseenheidClassificatieCode> .
+  }
+}
+