Release v1.41.5

Post-rebase cleanup: docker-compose.yml still pointed at frontend 1.40.4
instead of the actual release version 1.40.4-1.

.env.example

@@ -1 +0,0 @@
-COMPOSE_FILE=docker-compose.yml:docker-compose.dev.yml

.gitignore

@@ -20,6 +20,7 @@ docker-compose.override.yml
 
 /config/migrations/private-data/*
 /config/migrations/local/*
+/config/migrations-triggering-indexing/local/*
 
 /config/proxy/nginx.htpasswd
 

CHANGELOG.md

@@ -0,0 +1,924 @@
+# Changelog
+## Unreleased
+- Derive an organization's reference region(s) from its werkingsgebied; bump scope-of-operation to 0.2.0 [OP-3799]
+- Frontend [v1.40.1], [v1.40.0], [v1.39.0] ,[v1.38.0]
+- Add new organization types (interlokale vereniging, vervoerregioraad, zorgraad, bosgroep, woonmaatschappij); add werkingsgebied (dct:spatial) to registered organizations [OP-3828]
+- Rename generic organization types to "Vereniging algemeen" / "Vennootschap algemeen" [OP-3828]
+- Werkingsgebied (dct:spatial) for private OCMW associations [OP-3832]
+- Search organizations by vendor [OP-3808]
+- Add organization types "Regionaal zorgplatform" (altLabel "Regionale zorgzone") and residual "Andere" [OP-3844]
+- Move organizations from "Vereniging algemeen" / "Vennootschap algemeen" to "Andere"; delete those codes [OP-3844]
+- Bump construct-organization-relationships [OP-3844]
+- Frontend [v1.40.5](https://github.com/lblod/frontend-organization-portal/blob/v1.40.5/CHANGELOG.md) [OP-3844] [OP-3808] [OP-3849]
+- Consume and serve provenance properties on harvested Worship positions [DL-7437]
+- Frontend [v1.41.0](https://github.com/lblod/frontend-organization-portal/blob/v1.41.0/CHANGELOG.md) [DL-7438]
+- Keep werkingsgebied filter up to date via db-cleanup service [OP-3841]
+- Restrict available imam roles based on worship organisation status [DL-7485]
+  - IMPORTANT NOTE: Make sure first the [DL-7485] loket migration has completed + data has flown from loket to OP
+
+### Deploy notes
+```
+drc restart migrations
+drc up -d frontend scope-of-operation
+drc restart cache resource
+```
+
+```
+drc pull construct-organization-relationships && drc up -d construct-organization-relationships
+drc restart delta-producer-publication-graph-maintainer
+scripts/reset-elastic.sh
+drc restart migrations frontend db resource cache
+# reindex elastic search:
+/bin/bash scripts/reset-elastic.sh
+drc restart migrations search
+drc up -d frontend
+```
+
+## v1.41.5
+- Add the value politiecollege to the existing dropdown list beslissingsorgaan [DL-7473]
+- Bump construct-organization-relationships [DL-7473]
+
+### Deploy notes
+```
+drc pull construct-organization-relationships && drc up -d construct-organization-relationships
+drc restart migrations cache
+drc exec delta-producer-background-jobs-initiator curl -X POST http://localhost/public/healing-jobs
+
+```
+
+## v1.41.4
+- Fix broken lmb mandataris mappin query [OP-3867]
+
+### Deploy notes
+```
+drc restart migrations mandatarissen-consumer
+
+```
+
+## v1.41.3
+- Fix betrokken lokale besturen links via migration [OP-3882]
+
+### Deploy notes
+```
+drc restart migrations
+drc restart cache resource
+```
+
+## v1.41.2
+- Fix betrokken lokale besturen links via migration [OP-3877]
+- Add timeout to producer jobs to prevent them staying stuck forever [OP-3815]
+
+### Deploy notes
+In the docker-compose.override.yml, on PROD only, set the `error-alert` environment variable `EMAIL_TO:` to `support+lblod@redpencil.io`.
+
+Then:
+```
+drc up -d error-alert delta-producer-background-jobs-initiator
+drc restart migrations
+drc restart cache resource
+```
+
+## v1.41.1
+- Added service to link addresses to addressregister uris [OP-3795]
+- Site type changes [OP-3818]
+- Added Juridische vorm [OP-3816] [OP-3820]
+- Frontend [v1.40.4-1]
+- bump construct-organization-relationships [OP-3810]
+- "In Oprichting" bestuursorgaan lifecycle [OP-3810]
+- Added "werkingsgebied" filter [OP-3812]
+- Inhoudelijk thema [OP-3817]
+- Add a new "Wijkkantoor" site type [OP-3837]
+- cleanup local involvement herne [OP-3858]
+
+### Deploy notes
+```
+drc up -d link-adressenregister-uri frontend construct-organization-relationships
+drc restart resource migrations dispatcher
+```
+
+```
+drc restart migrations
+drc restart cache resource
+```
+
+```
+drc restart migrations frontend db resource cache
+# reindex elastic search:
+/bin/bash scripts/reset-elastic.sh
+```
+
+```
+drc restart migrations
+drc exec delta-producer-background-jobs-initiator curl -X POST http://localhost/public/healing-jobs
+drc up -d construct-organization-relationships
+```
+
+## v1.41.0
+ - added explicit healtchecks [DL-7466]
+
+### Deploy notes
+```
+drc restart dispatcher
+```
+
+## v1.40.0
+- Query to check missing address URI's [OP-3784]
+- Create new query 'organisations shown in OP frontend' in Dashboard [OP-3774]
+- Moved AGB BRUGGE to correct graph [OP-3790]
+- Bump acm-login service [DL-7346]
+- Frontend [v1.37.0](https://github.com/lblod/frontend-organization-portal/blob/99e0ca22617d17cbc8a23dfa2f6d31691fed90b9/CHANGELOG.md#v1370-2026-05-27)
+
+### Deploy notes
+```
+drc restart report-generation
+drc restart migrations
+scripts/reset-elastic.sh
+drc up -d login login-dashboard frontend
+```
+
+## 1.39.1 (2026-04-24)
+- OP-3771: Adds labels for: EredienstBeroepen and TypeBetrokkenheid
+- Bump to kbo-data-sync [OP-3765]
+- Added administrativeUnits report with strekking [OP-3766]
+- Cleanup local involvement for oud heverlee [OP-3757]
+- Add column to query organizations in Dashboard [OP-3758]
+- Query list change events of organisations in Dashboard [OP-3760]
+- Disable SPARQL endpoint
+- Prepare worship election period: backfill missing 2026-2029 governing bodies + mandates, seed fresh 2029-2032 period, and clean up `hhttp://` Secretaris typo from the 2022 central-worship migrations and bumped construct-organization-relationships to 1.1.1[OP-3776] and [DL-7294]
+- Cleanup local involvements for Sint Laureins [OP-3775]
+- sparql-parser [DL-6575]
+
+### Deploy notes
+```
+drc pull kbo-data-sync db resource && drc up -d kbo-data-sync db resource
+drc restart report-generation
+drc restart migrations && drc logs -ft --tail=200 migrations
+drc up -d delta-producer-publication-graph-maintainer
+drc restart dispatcher
+```
+
+## 1.39.0 (2026-04-02)
+- Add KBO organizations report to report generation service [OP-3738]
+- Add organizations detail report (addresses and contact info) to report generation service [OP-3738]
+- Query organisations: organisational data [OP-3761]
+
+### Deploy notes
+```
+drc restart report-generation
+```
+
+## 1.38.4 (2026-02-26)
+### Backend
+- Bump frontend [OP-3735]
+
+### Deploy instructions
+```
+drc pull frontend && drc up -d frontend
+```
+
+## 1.38.3 (2026-02-26)
+### Backend
+- Related organisations: Duplicate relations between municipalities and OCMWs [OP-3743]
+
+### Deploy instructions
+```
+drc restart migrations cache
+```
+
+## 1.38.2 (2026-02-20)
+### Backend
+- Remove deprecated organizations-public-info producer [OP-3581]
+- bump loket-error-alert-service [DL-7142]
+- bump adressenregister [OP-3721]
+- Aalter can be selected twice in editing "werkingsgebied" when making a new Politiezone [OP-3724]
+- bump frontend [OP-3726]
+- Add related organizations report to report generation service [OP-3739]
+
+### Deploy instructions
+```
+# removing producer
+drc restart dispatcher delta-producer-dump-file-publisher delta-producer-background-jobs-initiator delta-producer-publication-graph-maintainer jobs-controller migrations cache
+drc up -d publication-triplestore-migrations error-alert adressenregister frontend
+rm -r data/files/deltas/organizations-public-info
+rm -r data/files/delta-producer-dumps/dump-organizations-public-info
+
+# new related organizations report
+drc pull report-generation; drc up -d report-generation
+```
+
+### Deploy instructions
+## 1.38.1 (2026-01-30)
+### General
+- Bump frontend to v1.36.3 [OP-3428]
+- [OP-3720]: Delta background job having errors/getting stuck, send an email
+
+### Deploy instructions
+#### For error-alert and deliver-email-service
+Per environment a specfic configuration for the email service similar to loket config.
+Ask colleague for specfic credentials.
+
+```
+   deliver-email-service:
+     environment:
+      EMAIL_PROTOCOL: "smtp"
+      #WELL_KNOWN_SERVICE: "smtp"
+      EMAIL_HOST: "Fill in Email host adress"
+      EMAIL_ADDRESS: "Fill in email"
+      EMAIL_PASSWORD: "Fill in email password"
+      EMAIL_PORT: "Fill in port"
+      SECURE_CONNECTION: "true"
+      EMAIL_CRON_PATTERN: "*/1 * * * *"
+     restart: "no"
+
+   error-alert:
+    environment:
+      EMAIL_FROM: "Fill in email adress of sender
+      EMAIL_TO: "Fill in email adress of receiver"
+```
+
+#### Deploy instructions
+```
+drc restart migrations deltanotifier
+drc up -d error-alert deliver-email-service frontend
+```
+
+## 1.38.0 (2026-01-08)
+### General
+ - [OP-3590]: OP is now master of "aantal houders" of mandates
+ - [OP-3699]: Update aantal houders for some mandates Hasselt and Antwerp
+ - [OP-3680]: Indicate whether a worship service is a customer of ReligioPoint or Loket voor Lokale Besturen
+- Bump frontend to v1.36.2 [OP-3623] [OP-3681] [OP-3703] [OP-2712]
+
+### Deploy instructions
+#### For the vendor-management-consumer
+Per environment a slightly different configuration is needed. Update the `docker-compose.override.yml` accordingly.
+##### development
+```
+  vendor-management-consumer:
+    environment:
+      DCR_SYNC_BASE_URL: "https://dev.loket.lblod.info/"
+      DCR_SYNC_LOGIN_ENDPOINT: "https://dev.loket.lblod.info/sync/vendor-management/login" # Add DCR_SECRET_KEY in docker-compose.override.yml
+      DCR_DISABLE_INITIAL_SYNC: "false"
+      DCR_DISABLE_DELTA_INGEST: "false"
+      DCR_SECRET_KEY: "SECRECT KEY TO ASK"
+```
+##### qa
+```
+  vendor-management-consumer:
+    environment:
+      DCR_SYNC_BASE_URL: "https://loket.lblod.info/"
+      DCR_SYNC_LOGIN_ENDPOINT: "https://loket.lblod.info/sync/vendor-management/login" # Add DCR_SECRET_KEY in docker-compose.override.yml
+      DCR_DISABLE_INITIAL_SYNC: "false"
+      DCR_DISABLE_DELTA_INGEST: "false"
+      DCR_SECRET_KEY: "SECRECT KEY TO ASK"
+```
+##### production
+```
+  vendor-management-consumer:
+    environment:
+      DCR_SYNC_BASE_URL: "https://loket.lokaalbestuur.vlaanderen.be/"
+      DCR_SYNC_LOGIN_ENDPOINT: "https://loket.lokaalbestuur.vlaanderen.be/sync/vendor-management/login" # Add DCR_SECRET_KEY in docker-compose.override.yml
+      DCR_DISABLE_INITIAL_SYNC: "false"
+      DCR_DISABLE_DELTA_INGEST: "false"
+      DCR_SECRET_KEY: "SECRECT KEY TO ASK"
+```
+
+#### Deploy instructions
+```
+drc restart migrations
+drc restart delta-producer-publication-graph-maintainer resource
+drc pull vendor-management-consumer; drc up -d vendor-management-consumer
+drc restart db resource
+drc pull frontend && drc up -d frontend
+```
+
+## 1.37.3 (2025-11-14)
+### Backend
+- Added cleanup job to clean duplicated data of municipalities and provinces [OP-3676]
+- Delete local involvements for worship services [OP-3693]
+
+### Deploy notes
+```
+# cleanup job
+drc restart migrations
+drc exec db-cleanup curl -X GET "http://localhost/runCronJob?cronJobID=8ffd9f11-db25-430d-a66a-31fc8b393d5f"
+```
+
+## 1.37.2 (2025-11-13)
+- Bump frontend to v1.36.0
+### Deploy notes
+```
+drc up -d frontend
+```
+## 1.37.1 (2025-11-04)
+### Bakend
+- Include v1.35.2 hotfix to the 1.37.x version. Already deployed so no deploy instructions
+## 1.37.0 (2025-10-24)
+### Backend
+- Add municipalities and werkingsgebieden in the area of Brussel [OP-3669]
+- Updated mu-search configuration with source [OP-3670]
+
+### Deploy notes
+```
+drc restart migrations-triggering-indexing; drc logs -ft --tail=200 migrations-triggering-indexing
+drc up -d frontend
+scripts/reset-elastic.sh
+```
+
+## 1.36.0 (2025-10-03)
+### Dashboard
+- Enable ACM/IDM for the dashboard [OP-3674]
+
+#### QA
+Configure the environment variables for the QA ACM/IDM environment:
+
+```yml
+  frontend-dashboard:
+    environment:
+      EMBER_ACMIDM_CLIENT_ID: "03382bf4-4886-4fb6-8602-945ad3080a7a"
+      EMBER_ACMIDM_BASE_URL: "https://authenticatie-ti.vlaanderen.be/op/v1/auth"
+      EMBER_ACMIDM_REDIRECT_URL: "https://dashboard.organisaties.abb.lblod.info/authorization/callback"
+      EMBER_ACMIDM_LOGOUT_URL: "https://authenticatie-ti.vlaanderen.be/op/v1/logout"
+      EMBER_ACMIDM_SCOPE: "openid vo profile abb_organisatieportaal"
+
+  login-dashboard:
+    environment:
+      MU_APPLICATION_AUTH_DISCOVERY_URL: "https://authenticatie-ti.vlaanderen.be/op"
+      MU_APPLICATION_AUTH_CLIENT_ID: "03382bf4-4886-4fb6-8602-945ad3080a7a"
+      MU_APPLICATION_AUTH_REDIRECT_URI: "https://dashboard.organisaties.abb.lblod.info/authorization/callback"
+      MU_APPLICATION_AUTH_CLIENT_SECRET: "snip" # see ticket for secret
+
+```
+
+Link the new domain name to QA (and remove from dev)
+
+#### PROD
+Configure the environment variables for the PROD ACM/IDM environment once the values are known.
+
+Link the new domain name to PROD + DNS config to the server
+
+#### CLI
+```
+drc restart migrations
+drc restart dispatcher db
+drc up -d --remove-orphans
+```
+
+### Frontend
+- Bump to [v1.34.1](https://github.com/lblod/frontend-organization-portal/releases/tag/v1.34.1) [OP-3638]
+- Remove province for addresses in the Brussels area [OP-3638] & [OP-3648]
+
+### Deploy notes
+```
+drc pull frontend; drc up -d frontend
+drc restart migrations-triggering-indexing; drc logs -ft --tail=200 migrations-triggering-indexing
+```
+
+## v1.35.2 (2025-11-04)
+### Backend
+- Fix camel case name of municipality Heist-op-den-Berg in addresses [OP-3694]
+### Deploy notes
+```
+drc restart migrations-triggering-indexing
+```
+
+## v1.35.1 (2025-09-15)
+### Backend
+- Produce memberships and their roles in organizations and public producers [OP-3658] & [OP-3661]
+### Deploy notes
+```
+drc restart delta-producer-publication-graph-maintainer
+```
+
+## v1.35.0 (2025-08-04)
+### Frontend
+- Bump to [v1.34.0](https://github.com/lblod/frontend-organization-portal/releases/tag/v1.34.0)
+### Backend
+- Add a service to manage organisation's their scope of operation [OP-3205]
+- datamodel: Allow a location to be within multiple other locations [OP-3205]
+- datafix: set scope of operation for worship administrative units [OP-3626]
+- Clean addresses linked to both sites and contact points of mandatarissen and addresses that are in the wrong graph (admin unit graph but linked to a contact point in the worship graph) [DL-6701]
+- datafix: cleanup of general memberships if specific ones exist [OP-3640]
+- datafix: cleanup of duplicate memberships [OP-3634]
+- Fix KBO organizations' date formats [OP-3560]
+
+
+### Deploy notes
+```
+drc pull scope-of-operation; drc up -d scope-of-operation
+drc restart db resource dispatcher
+drc pull frontend; drc up -d frontend
+drc restart migrations; drc logs -ft --tail=200 migrations
+drc up -d kbo-data-sync
+```
+
+The labels for new location resources created when setting the scope of operation for worship services are not necessarily alphabetically sorted. To correct this run the `correct-location-labels` project script using [mu-cli](https://github.com/mu-semtech/mu-cli) and restart the migrations service once more to execute the generated local migrations.
+
+```
+mu script project-scripts correct-location-labels
+drc restart migrations; drc logs -ft --tail=200 migrations
+```
+
+## v1.34.0 (2025-07-11)
+### Frontend
+- Bump to version [v1.33.0](https://github.com/lblod/frontend-organization-portal/releases/tag/v1.33.0)
+### Backend
+- Add Brussels Hoofdstedelijk Gewest as province and location, update province of addresses in Brussel's area, update resource config to add `skos:note` on organizations [OP-3624]
+- Add Brussel as a municipality [OP-3628]
+- Revert Brussel as province and municipality and update back province of addresses in Brussel's area [OP-3637]
+- Bump delta-notifier service to most recent version [OP-3190]
+- Add URI to resource's organization responses
+- Fix query for organizations report [OP-3636]
+- Bump report-generation-service to most recent version
+- Write migration to harmonize date formats of change events [OP-3559]
+### Deploy notes
+```
+drc pull deltanotifier; drc up -d deltanotifier
+drc restart migrations-triggering-indexing resource
+drc pull frontend; drc up -d frontend
+drc pull report-generation; drc up -d report-generation
+drc restart migrations; drc logs -ft --tail=200 migrations
+```
+
+## v1.33.3 (2025-07-04)
+### Backend
+- datafix: re-add Havenbedrijf Antwerpen as an AGB [OP-3639]
+### Deploy notes
+```
+drc restart migrations; drc logs -ft --tail=200 migrations
+drc exec delta-producer-background-jobs-initiator curl -X POST http://localhost/public/healing-jobs
+```
+
+## v1.33.2 (2025-06-20)
+### Backend
+- datafix: correct name for locations whose municipalities kept their URI in the 2025 mergers [OP-3627]
+### Deploy notes
+```
+drc restart migrations-triggering-indexing && drc logs -ft --tail=200
+```
+
+## v1.33.1 (2025-06-16)
+### Backend
+- Added Vlaams Parlement as bestuurseenheid. [DL-6686]
+### Deploy Notes
+```
+drc restart migrations;
+drc exec delta-producer-background-jobs-initiator curl -X POST http://localhost/public/healing-jobs
+```
+
+## v1.33.0 (2025-06-05)
+### Frontend
+- Bump to version [v1.32.0](https://github.com/lblod/frontend-organization-portal/releases/tag/v1.32.0)
+### Backend
+- datafix: move memberships involving provinces and municipalities to shared graph [OP-3619]
+### Deploy notes
+```
+drc restart migrations; drc logs -ft --tail=200 migrations
+drc exec db-cleanup curl -X POST "http://localhost/cleanup"
+drc pull frontend; drc up -d frontend
+```
+
+## v1.32.1 (2025-06-02)
+### Backend
+- Added Kabinet Crevits as bestuurseenheid. [DL-6655]
+### Deploy Notes
+```
+drc restart migrations;
+drc exec delta-producer-background-jobs-initiator curl -X POST http://localhost/public/healing-jobs
+```
+
+## v1.32.0 (2025-05-23)
+### Frontend
+- Bump to version [v1.31.0](https://github.com/lblod/frontend-organization-portal/releases/tag/v1.31.0)
+### Backend
+- Fix KBO statuses not being displayed [OP-3584]
+- datafix: update NIS2019 end dates and link additional NIS2025 to werkingsgebieden [part of OP-3566]
+- datafix: correct date of change event of ckb olen [OP-3594]
+- datafix: move memberships from shared to administrative unit graph
+- datafix: cleanup werkingsgbieden data [OP-3566]
+  + Move werkingsgebieden to public graph
+  + Link district werkingsgebieden to Antwerp municipality werkingsgebied
+  + Add missing werkingsgebied for Borsbeek district
+- Write migration to extract from existing primary sites the province URI [OP-3597]
+- datafix: update status kerkfabriek weelde [OP-3598]
+- datafix: delete change event and update status protestantse kerk oostende [OP-3604]
+- datafix: update change event date islamic assocation [OP-3602]
+### Deploy Notes
+```
+drc pull frontend; drc up -d frontend
+drc restart migrations; drc logs -ft --tail=200 migrations
+drc restart migrations-triggering-indexing && drc logs -ft --tail=200 migrations-triggering-indexing
+drc up -d kbo-data-sync
+drc exec kbo-data-sync curl -X POST http://localhost/sync-all-kbo-data
+
+drc restart delta-producer-publication-graph-maintainer
+drc exec delta-producer-background-jobs-initiator curl -X POST http://localhost/public/healing-jobs # or wait until nightly healing kicks in
+drc exec delta-producer-background-jobs-initiator curl -X POST http://localhost/organizations/healing-jobs # or wait until nightly healing kicks in
+drc exec db-cleanup curl -X POST "http://localhost/cleanup"
+```
+
+## v1.31.3 (2025-05-08)
+### Backend
+ - Migrate active organizations linked to inactive CKB to the active one. [DL-6604]
+### Deploy notes
+```
+drc restart migrations-triggering-indexing && drc logs -ft --tail=200 migrations-triggering-indexing
+```
+
+## v1.31.2 (2025-04-02)
+### Backend
+- Extend the public producer to include: [CLBV-980]
+  - NIS codes
+  - Sites and statuses of representative organs
+  - Secondary sites with their contacts and addresses
+  - Site descriptions
+
+### Deploy notes
+```
+drc restart delta-producer-publication-graph-maintainer
+```
+
+## v1.31.1 (2025-04-01)
+### Backend
+- Removal of change events of 8 ckb's + update of their status [OP-3562]
+### Deploy Notes
+```
+drc restart migrations-triggering-indexing; drc logs -ft --tail=200 migrations-triggering-indexing
+drc restart migrations; drc logs -ft --tail=200 migrations
+```
+
+## v1.31.0 (2025-03-27)
+### Frontend
+- Bump to version [v1.30.0](https://github.com/lblod/frontend-organization-portal/blob/v1.30.0/CHANGELOG.md#v1300-2025-03-27)
+### Backend
+- Datafix: move status for municipalities and provinces to the `shared` graph such that it can be read by worship users [OP-3469]
+- Bump `triplestore` and `publication-triplestore` to `v1.3.0-rc.1`. [OP-2492] [OP-3547]
+- Add missing docker compose keys. [DL-6490]
+- Reorganize delta consumers config to harmonize with the ecosystem
+- Set up the dashboard app [OP-3103]
+- Datafix: correct date of change event of kerkfabriek st petrus [OP-3555]
+- Update OCMWv to VVMW [OP-3565] and back. It's technically a nil-operation. But you should run the migrations nevertheless.
+### Deploy notes
+```
+drc restart migrations-triggering-indexing
+drc restart migrations; drc logs -ft --tail=200 migrations
+drc pull frontend; drc up -d frontend
+drc up -d mandatarissen-consumer leidinggevenden-consumer worship-services-main-info-consumer worship-services-private-info-consumer
+```
+#### For upgrade databases
+[This README](https://github.com/Riadabd/upgrade-virtuoso) provides the necessary steps for upgrading the database. **NOTE**: This will involve shutting down the app for small period of time (around 30 minutes).
+#### For adding the missing keys
+```
+drc up -d search elasticsearch
+```
+
+#### Dashboard
+
+The dashboard requires some server specific configuration before it can be used.
+
+##### 1. Dashboard domains
+To configure the domain name that's used by the dashboard frontend, you can add it to the `VIRTUAL_HOST` and `LETSENCRYPT_HOST` environment variables of the identifier service.
+- DEV domain: dashboard.dev.app-organisation-portal.contacthub-dev.s.redhost.be
+- QA domain: dashboard.qa.app-organisation-portal.contacthub-dev.s.redhost.be
+- PROD domain: dashboard.app-organisation-portal.organisaties.s.redpencil.io
+
+##### 2. Dashboard users
+Before we can generate a user we need to add an application salt to the server. Add a `MU_APPLICATION_SALT` environment variable to the `dashboard-login` service in the docker-compose.override.yml file.
+
+Once this is done you can generate a user by running `mu script project-scripts generate-dashboard-login` and following the prompts. A new local migration will be generated which can be run by restarting the migrations service and which will insert the new user into the database.
+
+Store the login credentials as a comment in the docker-compose.override.yml file.
+
+##### 3. (Re)start the needed services
+`drc up -d frontend-dashboard dashboard-login identifier; drc restart dispatcher resource db`
+
+## v1.30.5 (2025-03-10)
+### Backend
+ - Added `http://www.w3.org/ns/prov#Location` to delta stream 'public' [DL-6496]
+### Deploy notes
+```
+drc restart delta-producer-publication-graph-maintainer
+drc exec delta-producer-background-jobs-initiator curl -X POST http://localhost/public/healing-jobs
+
+```
+
+## v1.30.6 (2025-03-25)
+### Backend
+- Flushed landingzones [DL-6552]
+### Deploy notes
+```
+drc restart migrations
+```
+
+## v1.30.5 (2025-03-10)
+### Backend
+ - Added `http://www.w3.org/ns/prov#Location` to delta stream 'public' [DL-6496]
+### Deploy notes
+```
+drc restart delta-producer-publication-graph-maintainer
+drc exec delta-producer-background-jobs-initiator curl -X POST http://localhost/public/healing-jobs
+
+```
+
+## v1.30.4 (2025-03-06)
+### Backend
+- Add missing codes to public producer [DL-6210] [DL-6449]
+#### Deploy commands
+```
+drc restart migrations delta-producer-publication-graph-maintainer
+drc exec delta-producer-background-jobs-initiator curl -X POST http://localhost/public/healing-jobs
+```
+
+## v1.30.3 (2025-03-04)
+### Frontend
+- Bump to [v1.29.1](https://github.com/lblod/frontend-organization-portal/releases/tag/v1.29.1)
+### Deploy notes
+- Bump frontend
+#### Deploy commands
+```
+drc pull frontend; drc up -d frontend
+```
+
+## v1.30.2 (2025-02-28)
+### Backend
+- Datafix: add new timed governing bodies for new legislature for AGBs [OP-3552]
+### Deploy notes
+- Restart the migration service
+- Start a healing job the public producer, or wait for it to kick in overnight
+#### Deploy commands
+```
+drc restart migrations; drc logs -ft --tail=200 migrations
+drc exec delta-producer-background-jobs-initiator curl -X POST http://localhost/public/healing-jobs
+```
+
+## v1.30.1 (2025-02-20)
+### Backend
+ - Bestuursorgaanclassificatiecode of VGC was in wrong graph, so it wouldn't export. [DL-6428]
+### Deploy notes
+```
+drc restart migrations
+```
+
+## v1.30.0 (2025-02-19)
+### Frontend
+- Bump to version [v1.29.0](https://github.com/lblod/frontend-organization-portal/releases/tag/v1.29.0)
+### Backend
+- Datafix: delete duplicate identifiers [OP-3157]
+- Feature: replace different relations between organisations by memberships [OP-3195 (epic), OP-2606, OP-3156, OP-3198, OP-3199, OP-3258, OP-3259, OP-3260, OP-3265, OP-3293, OP-3305]
+- Added `db-cleanup-service` to keep membership data backwards compatible [OP-3369]
+- Datafix: delete incorrect change event [OP-3363]
+- Feature: re-enable contact data editing [OP-3425]
+- Remove gemeente and provincie classifications from admin unit graph [OP-3393]
+- Remove several OCMW bcsd faciliteitgemeenten [OP-3535]
+- Completely remove AGB Stekene's data. [OP-3409]
+- Remove reasoner service, replaced by new consumer logic
+### Deploy Notes
+- Before pulling the changes
+  + Shut down the reasoner and thank it for its good service: `drc down reasoner`
+- Re-enable contact date editing:
+  + remove `EMBER_ENABLE_EDIT_CONTACT_DATA_FEATURE` flag in `docker-compose.override.yml`
+  + Stop and remove the `clb-contact-data-consumer` service
+  + Remove the `clb-contact-data-consumer` configuration in `docker-compose.override.yml`
+- Rebuild the `search` service before starting the re-indexing
+- To enable the membership functionality:
+  + Restart migration service
+  + Restart services for which the configuration was changed: `resource`,  `dispatcher` and `db`
+  + Start the `db-cleanup` service
+  + Pull and restart the frontend
+  + Perform a re-index using `reset-elastic.sh` script
+#### Deploy commands
+```
+drc down clb-contact-data-consumer
+drc up -d search
+drc restart migrations; drc logs -ft --tail=200 migrations
+drc restart migrations-triggering-indexing && drc logs -ft --tail=200 migrations-triggering-indexing
+drc restart resource dispatcher db
+drc up -d db-cleanup
+drc pull frontend; drc up -d frontend
+./scripts/reset-elastic.sh
+```
+
+## 1.29.2 (2025-02-17)
+### Data fix
+ - restore betrokken lokale besturen that have been accidentally flushed [OP-3546]
+
+## 1.29.1 (2025-02-14)
+### Backend
+ - re-init `worship-services-sensitive-consumer` [OP-3483]
+### Deploy notes
+:warning: Ensure first loket did a full re-sync again of this delta-stream.
+```
+drc stop worship-services-main-info-consumer worship-services-private-info-consumer
+drc retart migrations
+```
+Update `docker-compose.override.yml` to remove the config of the consumers and replace it by:
+```
+  worship-services-main-info-consumer:
+    environment:
+      DCR_SYNC_BASE_URL: "https://loket.lblod.info" # or another endpoint
+      DCR_SYNC_LOGIN_ENDPOINT: "https://loket.lblod.info/sync/worship-services-sensitive/login" # or another endpoint
+      DCR_SECRET_KEY: "key-of-the-producer"
+      DCR_LANDING_ZONE_DATABASE: "triplestore" # for the initial sync, we go directly to virtuoso
+      DCR_REMAPPING_DATABASE: "triplestore" # for the initial sync, we go directly to virtuoso
+      DCR_DISABLE_INITIAL_SYNC: "false"
+      DCR_DISABLE_DELTA_INGEST: "false"
+  worship-services-private-info-consumer:
+    environment:
+      DCR_SYNC_BASE_URL: "https://loket.lblod.info" # or another endpoint
+      DCR_SYNC_LOGIN_ENDPOINT: "https://loket.lblod.info/sync/worship-services-sensitive/login" # or another endpoint
+      DCR_SECRET_KEY: "key-of-the-producer"
+      DCR_LANDING_ZONE_DATABASE: "triplestore" # for the initial sync, we go directly to virtuoso
+      DCR_REMAPPING_DATABASE: "triplestore" # for the initial sync, we go directly to virtuoso
+      DCR_DISABLE_INITIAL_SYNC: "false"
+      DCR_DISABLE_DELTA_INGEST: "false"
+```
+Then
+```
+drc up -d db mandatarissen-consumer leidinggevenden-consumer worship-services-main-info-consumer worship-services-private-info-consumer
+# Wait until success of the previous step
+```
+Then, update `docker-compose.override.yml` to:
+```
+  worship-services-main-info-consumer:
+    environment:
+      DCR_SYNC_BASE_URL: "https://loket.lblod.info" # or another endpoint
+      DCR_SYNC_LOGIN_ENDPOINT: "https://loket.lblod.info/sync/worship-services-sensitive/login" # or another endpoint
+      DCR_SECRET_KEY: "key-of-the-producer"
+      DCR_LANDING_ZONE_DATABASE: "db"
+      DCR_REMAPPING_DATABASE: "db"
+      DCR_DISABLE_DELTA_INGEST: "false"
+      DCR_DISABLE_INITIAL_SYNC: "false"
+  worship-services-private-info-consumer:
+    environment:
+      DCR_SYNC_BASE_URL: "https://loket.lblod.info" # or another endpoint
+      DCR_SYNC_LOGIN_ENDPOINT: "https://loket.lblod.info/sync/worship-services-sensitive/login" # or another endpoint
+      DCR_SECRET_KEY: "key-of-the-producer"
+      DCR_LANDING_ZONE_DATABASE: "database"
+      DCR_REMAPPING_DATABASE: "database"
+      DCR_DISABLE_DELTA_INGEST: "false"
+      DCR_DISABLE_INITIAL_SYNC: "false"
+```
+```
+drc up -d
+sh scripts/reset-elastic.sh
+
+```
+
+## 1.29.0 (2025-02-07)
+### Backend
+#### Consumer
+- Upgraded `mandatarissen-consumer` [OP-3510]
+- Upgraded `leidinggevenden-consumer` [OP-3533], [OP-3511]
+- Upgraded `worship-services-sensitive-consumer` [OP-3337]
+
+### Deploy notes
+```
+drc down;
+```
+Update `docker-compose.override.yml` to remove the config of the consumers and replace it by:
+```
+  mandatarissen-consumer:
+    environment:
+      DCR_SYNC_BASE_URL: "https://loket.lblod.info" # or another endpoint
+      DCR_LANDING_ZONE_DATABASE: "triplestore" # for the initial sync, we go directly to virtuoso
+      DCR_REMAPPING_DATABASE: "triplestore" # for the initial sync, we go directly to virtuoso
+      DCR_DISABLE_DELTA_INGEST: "false"
+      DCR_DISABLE_INITIAL_SYNC: "false"
+  leidinggevenden-consumer:
+    environment:
+      DCR_SYNC_BASE_URL: "https://loket.lblod.info" # or another endpoint
+      DCR_LANDING_ZONE_DATABASE: "triplestore" # for the initial sync, we go directly to virtuoso
+      DCR_REMAPPING_DATABASE: "triplestore" # for the initial sync, we go directly to virtuoso
+      DCR_DISABLE_DELTA_INGEST: "false"
+      DCR_DISABLE_INITIAL_SYNC: "false"
+  worship-services-main-info-consumer:
+    environment:
+      DCR_SYNC_BASE_URL: "https://loket.lblod.info" # or another endpoint
+      DCR_SYNC_LOGIN_ENDPOINT: "https://loket.lblod.info/sync/worship-services-sensitive/login" # or another endpoint
+      DCR_SECRET_KEY: "key-of-the-producer"
+      DCR_LANDING_ZONE_DATABASE: "triplestore" # for the initial sync, we go directly to virtuoso
+      DCR_REMAPPING_DATABASE: "triplestore" # for the initial sync, we go directly to virtuoso
+      DCR_DISABLE_INITIAL_SYNC: "false"
+      DCR_DISABLE_DELTA_INGEST: "false"
+  worship-services-private-info-consumer:
+    environment:
+      DCR_SYNC_BASE_URL: "https://loket.lblod.info" # or another endpoint
+      DCR_SYNC_LOGIN_ENDPOINT: "https://loket.lblod.info/sync/worship-services-sensitive/login" # or another endpoint
+      DCR_SECRET_KEY: "key-of-the-producer"
+      DCR_LANDING_ZONE_DATABASE: "triplestore" # for the initial sync, we go directly to virtuoso
+      DCR_REMAPPING_DATABASE: "triplestore" # for the initial sync, we go directly to virtuoso
+      DCR_DISABLE_INITIAL_SYNC: "false"
+      DCR_DISABLE_DELTA_INGEST: "false"
+```
+Then:
+```
+drc up -d triplestore migrations
+drc up -d db mandatarissen-consumer leidinggevenden-consumer worship-services-main-info-consumer worship-services-private-info-consumer
+# Wait until success of the previous step
+```
+Then, update `docker-compose.override.yml` to:
+```
+  mandatarissen-consumer:
+    environment:
+      DCR_SYNC_BASE_URL: "https://loket.lblod.info" # or another endpoint
+      DCR_LANDING_ZONE_DATABASE: "db"
+      DCR_REMAPPING_DATABASE: "db"
+      DCR_DISABLE_DELTA_INGEST: "false"
+      DCR_DISABLE_INITIAL_SYNC: "false"
+  leidinggevenden-consumer:
+    environment:
+      DCR_SYNC_BASE_URL: "https://loket.lblod.info" # choose the correct endpoint
+      DCR_LANDING_ZONE_DATABASE: "db"
+      DCR_REMAPPING_DATABASE: "db"
+      DCR_DISABLE_DELTA_INGEST: "false"
+      DCR_DISABLE_INITIAL_SYNC: "false"
+  worship-services-main-info-consumer:
+    environment:
+      DCR_SYNC_BASE_URL: "https://loket.lblod.info" # or another endpoint
+      DCR_SYNC_LOGIN_ENDPOINT: "https://loket.lblod.info/sync/worship-services-sensitive/login" # or another endpoint
+      DCR_SECRET_KEY: "key-of-the-producer"
+      DCR_LANDING_ZONE_DATABASE: "db"
+      DCR_REMAPPING_DATABASE: "db"
+      DCR_DISABLE_DELTA_INGEST: "false"
+      DCR_DISABLE_INITIAL_SYNC: "false"
+  worship-services-private-info-consumer:
+    environment:
+      DCR_SYNC_BASE_URL: "https://loket.lblod.info" # or another endpoint
+      DCR_SYNC_LOGIN_ENDPOINT: "https://loket.lblod.info/sync/worship-services-sensitive/login" # or another endpoint
+      DCR_SECRET_KEY: "key-of-the-producer"
+      DCR_LANDING_ZONE_DATABASE: "database"
+      DCR_REMAPPING_DATABASE: "database"
+      DCR_DISABLE_DELTA_INGEST: "false"
+      DCR_DISABLE_INITIAL_SYNC: "false"
+```
+```
+drc up -d
+sh scripts/reset-elastic.sh
+```
+
+## 1.28.5 (2025-01-23)
+### Frontend
+- Bump to [v1.28.3](https://github.com/lblod/frontend-organization-portal/releases/tag/v1.28.3)
+### Backend
+- Datafix: correct merger change events for worship organisations (OP-3534)
+- Avoid municipality filter to include partial matches (OP-3509)
+### Deploy notes
+- `drc restart migrations-triggering-indexing; drc logs -ft --tail=200 migrations-triggering-indexing`
+- `drc pull frontend; drc up -d frontend`
+- Perform a full re-index by executing `./scripts/reset-elastic.sh`
+
+## 1.28.4 (2025-01-16)
+### Backend
+#### Data
+- Performance fix on 1.28.3 [DL-6377]
+### Deploy commands
+```
+drc restart migrations
+drc exec delta-producer-background-jobs-initiator curl -X POST http://localhost/public/healing-jobs # or wait for the healing to kick in
+```
+## 1.28.3 (2025-01-16)
+### Backend
+#### Data
+- Datafix. Fusiegemeentes had different bestuursfuncties attached to OCMW and Gemeente. [DL-6377]
+  However, business-rule-wise, these bestuursfuncties are shared.
+### Deploy commands
+```
+drc restart migrations
+drc exec delta-producer-background-jobs-initiator curl -X POST http://localhost/public/healing-jobs # or wait for the healing to kick in
+```
+## 1.28.2 (2025-01-13)
+### Backend
+ - Added "Opdrachthoudende vereniging met private deelname" in delta public (DL-6368)
+### Deploy commands
+```
+drc restart migrations
+drc exec delta-producer-background-jobs-initiator curl -X POST http://localhost/public/healing-jobs # or wait for the healing to kick in
+```
+## 1.28.1 (2025-01-11)
+### Backend
+- Import 2025 NIS codes and update werkingsgebieden (OP-3342)
+- Add merger change events for merged police zones (OP-3454)
+- Delete local involvements for worship services (OP-3501, OP-3512)
+- Move link between new municipalities and province to shared graph (OP-3513, OP-3514)
+
+### Deploy notes
+- Restart both migration services
+#### Docker commands
+- `drc restart migrations; drc logs -ft --tail=200 migrations`
+- `drc restart migrations-triggering-indexing; drc logs -ft --tail=200 migrations-triggering-indexing`
+
+## 1.28.0 (2025-01-02)
+### Backend
+- Bump frontend to `v1.28.0`
+- Add end date for governing bodies for municipalities and OCMWs and update status (OP-3348)
+- Update municipality and province name in organisation's addresses (OP-3410)
+- Add merger change events for merged municipalities and OCMWs (OP-3412)
+- Updates names for merged municipalities and OCMWs (OP-3456)
+
+### Deploy notes
+- Restart both migration services
+- Bump frontend
+#### Docker commands
+- `drc restart migrations; drc logs -ft --tail=200 migrations`
+- `drc restart migrations-triggering-indexing; drc logs -ft --tail=200 migrations-triggering-indexing`
+- `drc pull frontend; drc up -d frontend`

README.md

@@ -2,15 +2,208 @@
 
 Backend for the organization portal application, based on the mu.semte.ch microservices stack.
 
-## How to
-
 ### Boot up the system in DEV environment
 
-    cd /path/to/mu-project
-    docker-compose -f docker-compose.yml -f docker-compose.dev.yml up
+    cd /path/to/app-organization-portal
+    docker compose -f docker-compose.yml -f docker-compose.dev.yml up
 
-You can shut down using `docker-compose stop` and remove everything using `docker-compose rm`.
+You can shut down using `docker compose stop` and remove everything using `docker compose rm`.
 
+You need to wait until the migrations are ready.
+
+After that, you need to trigger the ES index manually.
+
+You can do it by:
+```
+# Assumes you are in root of app-organization-portal
+/bin/bash scripts/reset-elastic.sh 
+```
+And follow the steps. It takes a bit, from an install from scratch, about 10 mins. (Mileage may vary)
+
+
+#### Tricks to make it a bit easier.
+If you think typing the full command takes too long every time, run this in your shell:
+
+```
+# Assumes you are in root of app-organization-portal
+echo "COMPOSE_FILE=docker-compose.yml:docker-compose.dev.yml:docker-compose.override.yml" > .env
+touch docker-compose.override.yml
+```
+Now you can just run `docker compose up` and it will take the config files into account.
+If you want to go a step further, and still think (like most of us) that `docker compose` is too long to type, you can alias it:
+```
+echo "alias drc='docker compose'" >> ~/.bashrc
+echo "alias drcl='docker compose logs -ft --tail=100'" >> ~/.bashrc
+```
+Now everywhere `drc up -d` will work as shorthand for `docker compose up -d`.
+
+#### Setup consumers
+
+The application depends on external data. You'll have to perform extra steps if you want this data to be available.
+These are multiple data sets, we'll try to be exhaustive on how to ingest these. Even though the procedures are similar.
+
+#### Setup mandatarissen sync
+Note: this is assuming you have never ran this before.
+```
+drc down;
+```
+Update `docker-compose.override.yml` to:
+```
+  mandatarissen-consumer:
+    environment:
+      DCR_SYNC_BASE_URL: "https://loket.lokaalbestuur.vlaanderen.be" # or another endpoint
+      DCR_LANDING_ZONE_DATABASE: "triplestore"
+      DCR_REMAPPING_DATABASE: "triplestore"
+      DCR_DISABLE_DELTA_INGEST: "false"
+      DCR_DISABLE_INITIAL_SYNC: "false"
+```
+Then:
+```
+drc up -d migrations
+drc up -d db mandatarissen-consumer
+# Wait until success of the previous step
+```
+Then, update `docker-compose.override.yml` to:
+```
+  mandatarissen-consumer:
+    environment:
+      DCR_SYNC_BASE_URL: "https://loket.lokaalbestuur.vlaanderen.be" # or another endpoint
+      DCR_LANDING_ZONE_DATABASE: "db"
+      DCR_REMAPPING_DATABASE: "db"
+      DCR_DISABLE_DELTA_INGEST: "false"
+      DCR_DISABLE_INITIAL_SYNC: "false"
+```
+Finally, put the stack back up and reindex elasticsearch
+```
+drc up -d
+sh scripts/reset-elastic.sh
+```
+
+#### Setup leidinggevenden sync (http://data.lblod.info/vocabularies/leidinggevenden/Functionaris)
+
+Note: this is assuming you have never ran this before.
+```
+drc down;
+```
+Update `docker-compose.override.yml` to:
+```
+  leidinggevenden-consumer:
+    environment:
+      DCR_SYNC_BASE_URL: "https://loket.lokaalbestuur.vlaanderen.be" # or another endpoint
+      DCR_LANDING_ZONE_DATABASE: "triplestore" # for the initial sync, we go directly to virtuoso
+      DCR_REMAPPING_DATABASE: "triplestore" # for the initial sync, we go directly to virtuoso
+      DCR_DISABLE_DELTA_INGEST: "false"
+      DCR_DISABLE_INITIAL_SYNC: "false"
+```
+Then:
+```
+drc up -d migrations
+drc up -d db leidinggevenden-consumer
+# Wait until success of the previous step
+```
+Then, update `docker-compose.override.yml` to:
+```
+  leidinggevenden-consumer:
+    environment:
+      DCR_SYNC_BASE_URL: "https://loket.lokaalbestuur.vlaanderen.be" # choose the correct endpoint
+      DCR_LANDING_ZONE_DATABASE: "db"
+      DCR_REMAPPING_DATABASE: "db"
+      DCR_DISABLE_DELTA_INGEST: "false"
+      DCR_DISABLE_INITIAL_SYNC: "false"
+```
+Finally, put the stack back up and reindex elasticsearch
+```
+drc up -d
+sh scripts/reset-elastic.sh
+```
+
+#### Setup worship mandatees sync
+
+Note: this is assuming you have never ran this before.
+```
+drc down;
+```
+Update `docker-compose.override.yml` to:
+```
+  worship-services-main-info-consumer:
+    environment:
+      DCR_SYNC_BASE_URL: "https://loket.lokaalbestuur.vlaanderen.be" # or another endpoint
+      DCR_SYNC_LOGIN_ENDPOINT: "https://loket.lokaalbestuur.vlaanderen.be/sync/worship-services-sensitive/login" # or another endpoint
+      DCR_SECRET_KEY: "key-of-the-producer"
+      DCR_LANDING_ZONE_DATABASE: "triplestore"
+      DCR_REMAPPING_DATABASE: "triplestore"
+      DCR_DISABLE_DELTA_INGEST: "false"
+      DCR_DISABLE_INITIAL_SYNC: "false"
+  worship-services-private-info-consumer:
+    environment:
+      DCR_SYNC_BASE_URL: "https://loket.lokaalbestuur.vlaanderen.be" # or another endpoint
+      DCR_SYNC_LOGIN_ENDPOINT: "https://loket.lokaalbestuur.vlaanderen.be/sync/worship-services-sensitive/login" # or another endpoint
+      DCR_SECRET_KEY: "key-of-the-producer"
+      DCR_LANDING_ZONE_DATABASE: "triplestore"
+      DCR_REMAPPING_DATABASE: "triplestore"
+      DCR_DISABLE_DELTA_INGEST: "false"
+      DCR_DISABLE_INITIAL_SYNC: "false"
+```
+Then:
+```
+drc up -d migrations
+drc up -d db worship-services-main-info-consumer worship-services-private-info-consumer
+# Wait until success of the previous step
+```
+Then, update `docker-compose.override.yml` to:
+```
+worship-services-main-info-consumer:
+    environment:
+      DCR_SYNC_BASE_URL: "https://loket.lokaalbestuur.vlaanderen.be" # or another endpoint
+      DCR_SYNC_LOGIN_ENDPOINT: "https://loket.lokaalbestuur.vlaanderen.be/sync/worship-services-sensitive/login" # or another endpoint
+      DCR_SECRET_KEY: "key-of-the-producer"
+      DCR_LANDING_ZONE_DATABASE: "db"
+      DCR_REMAPPING_DATABASE: "db"
+      DCR_DISABLE_DELTA_INGEST: "false"
+      DCR_DISABLE_INITIAL_SYNC: "false"
+  worship-services-private-info-consumer:
+    environment:
+      DCR_SYNC_BASE_URL: "https://loket.lokaalbestuur.vlaanderen.be" # or another endpoint
+      DCR_SYNC_LOGIN_ENDPOINT: "https://loket.lokaalbestuur.vlaanderen.be/sync/worship-services-sensitive/login" # or another endpoint
+      DCR_SECRET_KEY: "key-of-the-producer"
+      DCR_LANDING_ZONE_DATABASE: "db"
+      DCR_REMAPPING_DATABASE: "db"
+      DCR_DISABLE_DELTA_INGEST: "false"
+      DCR_DISABLE_INITIAL_SYNC: "false"
+```
+Finally, put the stack back up and reindex elasticsearch
+```
+drc up -d
+sh scripts/reset-elastic.sh
+```
+
+#### Setup vendors sync
+Note: this is assuming you have never ran this before.
+```
+drc down;
+```
+Update `docker-compose.override.yml` to:
+```
+  vendor-management-consumer:
+    environment:
+      DCR_SYNC_BASE_URL: "https://dev.loket.lblod.info/" # or another endpoint
+      BATCH_SIZE: "500"
+      DCR_SECRET_KEY: "<key>>" # fill in key for selected endpoint
+      DCR_SYNC_LOGIN_ENDPOINT: "https://dev.loket.lblod.info/sync/vendor-management/login" # login for selected endpoint
+      DCR_DISABLE_DELTA_INGEST: "false"
+      DCR_DISABLE_INITIAL_SYNC: "false"
+```
+Then:
+```
+drc up -d migrations
+drc up -d db vendor-management-consumer
+# Wait until success of the previous step
+
+Finally, put the stack back up and reindex elasticsearch:
+```
+drc up -d
+sh scripts/reset-elastic.sh
+```
 
 ### Setting up the delta-producers related services
 To make sure the app can share data, producers need to be set up. There is an intial sync, that is potentially very expensive, and must be started manually

config/authorization/config.ex

@@ -1,217 +0,0 @@
-alias Acl.Accessibility.Always, as: AlwaysAccessible
-alias Acl.GraphSpec.Constraint.Resource, as: ResourceConstraint
-alias Acl.GraphSpec.Constraint.ResourceFormat, as: ResourceFormatConstraint
-alias Acl.Accessibility.ByQuery, as: AccessByQuery
-alias Acl.GraphSpec, as: GraphSpec
-alias Acl.GroupSpec, as: GroupSpec
-alias Acl.GroupSpec.GraphCleanup, as: GraphCleanup
-
-defmodule Acl.UserGroups.Config do
-  @error_type [
-    "https://docs.oasis-open-projects.org/oslc-op/core/v3.0/os/core-vocab.html#Error"
-  ]
-
-  @worship_type [
-    "http://data.lblod.info/vocabularies/erediensten/EredienstMandataris",
-    "http://data.lblod.info/vocabularies/erediensten/BestuurVanDeEredienst",
-    "http://data.lblod.info/vocabularies/erediensten/EredienstBestuurseenheid",
-    "http://data.lblod.info/vocabularies/erediensten/RolBedienaar",
-    "http://data.lblod.info/vocabularies/erediensten/VoorwaardenBedienaar",
-    "http://data.lblod.info/vocabularies/erediensten/PositieBedienaar",
-    "http://data.lblod.info/vocabularies/erediensten/BetrokkenLokaleBesturen",
-    "http://data.lblod.info/vocabularies/erediensten/VerbondenJuridischeStructuren",
-    "http://data.lblod.info/vocabularies/erediensten/CentraalBestuurVanDeEredienst",
-    "http://data.lblod.info/vocabularies/erediensten/RepresentatiefOrgaan"
-  ]
-
-  @public_type [
-    "http://www.w3.org/ns/org#Role",
-    "http://data.vlaanderen.be/ns/besluit#Bestuurseenheid",
-    "http://xmlns.com/foaf/0.1/Person",
-    "http://xmlns.com/foaf/0.1/OnlineAccount",
-    "http://www.semanticdesktop.org/ontologies/2007/03/22/nfo#FileDataObject",
-    "http://www.w3.org/ns/dcat#Dataset",
-    "http://www.w3.org/ns/dcat#Distribution",
-    "http://www.w3.org/ns/dcat#Catalog",
-    "http://mu.semte.ch/vocabularies/ext/GeslachtCode",
-    "http://data.lblod.info/vocabularies/leidinggevenden/FunctionarisStatusCode",
-    "http://lblod.data.gift/vocabularies/organisatie/OrganisatieStatusCode",
-    "http://lblod.data.gift/vocabularies/organisatie/BestuursfunctieCode",
-    "http://lblod.data.gift/vocabularies/organisatie/BestuursorgaanClassificatieCode",
-    "http://lblod.data.gift/vocabularies/organisatie/MandatarisStatusCode",
-    "http://lblod.data.gift/vocabularies/organisatie/BestuurseenheidClassificatieCode",
-    "http://mu.semte.ch/vocabularies/ext/GeregistreerdeOrganisatieClassificatieCode",
-    "http://lblod.data.gift/vocabularies/organisatie/TypeEredienst",
-    "http://www.w3.org/2004/02/skos/core#ConceptScheme",
-    "http://lblod.data.gift/vocabularies/organisatie/BedienaarFinancieringCode",
-    "http://www.w3.org/2004/02/skos/core#Concept",
-    "http://publications.europa.eu/ontology/euvoc#Country",
-    "http://lblod.data.gift/vocabularies/organisatie/TypeVestiging",
-    "http://lblod.data.gift/vocabularies/organisatie/HelftVerkiezing",
-    "http://lblod.data.gift/vocabularies/organisatie/Veranderingsgebeurtenis",
-    "http://lblod.data.gift/vocabularies/organisatie/TypeBetrokkenheid",
-    "http://lblod.data.gift/vocabularies/organisatie/VoorwaardenBedienaarCriterium",
-    "http://lblod.data.gift/vocabularies/organisatie/BedienaarCriteriumBewijsstuk",
-    "http://lblod.data.gift/vocabularies/organisatie/EredienstBeroepen",
-    "http://lblod.data.gift/vocabularies/organisatie/Rechtsvormtype"
-  ]
-
-  @org_type [
-    "http://purl.org/dc/terms/Agent",
-    "http://www.w3.org/ns/org#Organization",
-    "http://www.w3.org/ns/regorg#RegisteredOrganization",
-    "http://data.vlaanderen.be/ns/besluit#Besluit",
-    "https://data.vlaanderen.be/ns/besluitvorming#Beslissingsactiviteit",
-    "http://www.w3.org/ns/org#Site",
-    "http://www.w3.org/ns/locn#Address",
-    "http://www.w3.org/ns/person#Person",
-    "http://schema.org/ContactPoint",
-    "http://data.vlaanderen.be/ns/besluit#Bestuurseenheid",
-    "http://data.vlaanderen.be/ns/besluit#Bestuursorgaan",
-    "http://www.w3.org/ns/adms#Identifier",
-    "https://data.vlaanderen.be/ns/generiek#GestructureerdeIdentificator",
-    "http://data.vlaanderen.be/ns/mandaat#Mandaat",
-    "http://data.vlaanderen.be/ns/mandaat#Mandataris",
-    "http://data.lblod.info/vocabularies/leidinggevenden/Functionaris",
-    "http://data.lblod.info/vocabularies/leidinggevenden/Bestuursfunctie",
-    "http://lblod.data.gift/vocabularies/organisatie/VeranderingsgebeurtenisResultaat",
-    "http://www.w3.org/ns/org#Post",
-    "http://www.w3.org/ns/org#ChangeEvent",
-    "http://data.lblod.info/vocabularies/contacthub/AgentInPositie",
-    "http://mu.semte.ch/vocabularies/ext/KboOrganisatie",
-  ]
-
-  @shared_protected_type [
-    "http://www.w3.org/ns/prov#Location",
-    "http://xmlns.com/foaf/0.1/Image",
-    "http://www.semanticdesktop.org/ontologies/2007/03/22/nfo#FileDataObject"
-  ]
-
-  defp access_by_role(group_string) do
-    %AccessByQuery{
-      #      vars: ["session_group","session_role"], todo we might have to  create multiple graph depending on role
-      vars: [],
-      query: sparql_query_for_access_role(group_string)
-    }
-  end
-
-  defp sparql_query_for_access_role(group_string) do
-    "PREFIX ext: <http://mu.semte.ch/vocabularies/ext/>
-    PREFIX mu: <http://mu.semte.ch/vocabularies/core/>
-    SELECT distinct ?session_group ?session_role WHERE {
-      <SESSION_ID> ext:sessionGroup/mu:uuid ?session_group;
-                   ext:activeSessionRole ?session_role.
-      FILTER( ?session_role = \"#{group_string}\" )
-    }"
-  end
-
-  def user_groups do
-    [
-      %GroupSpec{
-        name: "acmidm-lezer",
-        useage: [:read],
-        access: access_by_role("ABBOrganisatiePortaalGebruiker-lezer"),
-        graphs: [
-          %GraphSpec{
-            graph: "http://mu.semte.ch/graphs/administrative-unit",
-            constraint: %ResourceConstraint{
-              resource_types: @org_type
-            }
-          }
-        ]
-      },
-      %GroupSpec{
-        name: "acmidm-editeerder",
-        useage: [:read, :write, :read_for_write],
-        access: access_by_role("ABBOrganisatiePortaalGebruiker-editeerder"),
-        graphs: [
-          %GraphSpec{
-            graph: "http://mu.semte.ch/graphs/administrative-unit",
-            constraint: %ResourceConstraint{
-              resource_types: @org_type ++ @error_type
-            }
-          }
-        ]
-      },
-      %GroupSpec{
-        name: "acmidm-beheerder",
-        useage: [:read, :write, :read_for_write],
-        access: access_by_role("ABBOrganisatiePortaalGebruiker-beheerder"),
-        graphs: [
-          %GraphSpec{
-            graph: "http://mu.semte.ch/graphs/administrative-unit",
-            constraint: %ResourceConstraint{
-              resource_types: @org_type ++ @error_type
-            }
-          }
-        ]
-      },
-      %GroupSpec{
-        name: "acmidm-worship-lezer",
-        useage: [:read],
-        access: access_by_role("ABBOrganisatiePortaalErediensten-lezer"),
-        graphs: [
-          %GraphSpec{
-            graph: "http://mu.semte.ch/graphs/worship-service",
-            constraint: %ResourceConstraint{
-              resource_types: @org_type ++ @worship_type
-            }
-          }
-        ]
-      },
-      %GroupSpec{
-        name: "acmidm-worship-editeerder",
-        useage: [:read, :write, :read_for_write],
-        access: access_by_role("ABBOrganisatiePortaalErediensten-editeerder"),
-        graphs: [
-          %GraphSpec{
-            graph: "http://mu.semte.ch/graphs/worship-service",
-            constraint: %ResourceConstraint{
-              resource_types: @org_type ++ @error_type ++ @worship_type
-            }
-          }
-        ]
-      },
-      %GroupSpec{
-        name: "acmidm-worship-beheerder",
-        useage: [:read, :write, :read_for_write],
-        access: access_by_role("ABBOrganisatiePortaalErediensten-beheerder"),
-        graphs: [
-          %GraphSpec{
-            graph: "http://mu.semte.ch/graphs/worship-service",
-            constraint: %ResourceConstraint{
-              resource_types: @org_type ++ @error_type ++ @worship_type
-            }
-          }
-        ]
-      },
-      %GroupSpec{
-        name: "public",
-        useage: [:read],
-        access: %AlwaysAccessible{},
-        graphs: [
-          %GraphSpec{
-            graph: "http://mu.semte.ch/graphs/public",
-            constraint: %ResourceConstraint{
-              resource_types: @public_type
-            }
-          },
-          %GraphSpec{
-            graph: "http://mu.semte.ch/graphs/shared",
-            constraint: %ResourceConstraint{
-              resource_types: @shared_protected_type ++ @org_type
-            }
-          }
-        ]
-      },
-
-      # // CLEANUP
-      #
-      %GraphCleanup{
-        originating_graph: "http://mu.semte.ch/application",
-        useage: [:read, :write],
-        name: "clean"
-      }
-    ]
-  end
-end

config/authorization/config.lisp

@@ -0,0 +1,369 @@
+;;;;;;;;;;;;;;;;;;;
+;;; delta messenger
+(in-package :delta-messenger)
+
+(add-delta-logger)
+(add-delta-messenger "http://deltanotifier/")
+
+;;;;;;;;;;;;;;;;;
+;;; configuration
+(in-package :client)
+(setf *log-sparql-query-roundtrip* t)
+(setf *backend* "http://triplestore:8890/sparql")
+
+(in-package :server)
+(setf *log-incoming-requests-p* nil)
+
+;;;;;;;;;;;;;;;;;
+;;; access rights
+(in-package :acl)
+
+(defparameter *access-specifications* nil
+  "All known ACCESS specifications.")
+
+(defparameter *graphs* nil
+  "All known GRAPH-SPECIFICATION instances.")
+
+(defparameter *rights* nil
+  "All known GRANT instances connecting ACCESS-SPECIFICATION to GRAPH.")
+
+;; Prefixes used in the constraints below (not in the SPARQL queries)
+(define-prefixes
+    :org "http://www.w3.org/ns/org#"
+    :locn "http://www.w3.org/ns/locn#"
+    :schema "http://schema.org/"
+    :regorg "http://www.w3.org/ns/regorg#"
+    :person "http://www.w3.org/ns/person#"
+    :adms "http://www.w3.org/ns/adms#"
+    :prov "http://www.w3.org/ns/prov#"
+    :generiek "https://data.vlaanderen.be/ns/generiek#"
+    :mandaat "http://data.vlaanderen.be/ns/mandaat#"
+    :besluit "http://data.vlaanderen.be/ns/besluit#"
+    :lblodlg "http://data.lblod.info/vocabularies/leidinggevenden/"
+    :dc_terms "http://purl.org/dc/terms/"
+    :foaf "http://xmlns.com/foaf/0.1/"
+    :skos "http://www.w3.org/2004/02/skos/core#"
+    :euvoc "http://publications.europa.eu/ontology/euvoc#"
+    :ere "http://data.lblod.info/vocabularies/erediensten/"
+    :ch "http://data.lblod.info/vocabularies/contacthub/"
+    :code "http://lblod.data.gift/vocabularies/organisatie/"
+    :ext "http://mu.semte.ch/vocabularies/ext/"
+    :time "http://www.w3.org/2006/time#"
+    :oasis "https://docs.oasis-open-projects.org/oslc-op/core/v3.0/os/core-vocab.html#"
+    :nfo "http://www.semanticdesktop.org/ontologies/2007/03/22/nfo#"
+    :dcat "http://www.w3.org/ns/dcat#"
+    :besluitvorming "https://data.vlaanderen.be/ns/besluitvorming#"
+    :reporting "http://lblod.data.gift/vocabularies/reporting/"
+    :oslc "http://open-services.net/ns/core#"
+    :cogs "http://vocab.deri.ie/cogs#"
+  )
+
+(type-cache::add-type-for-prefix "http://mu.semte.ch/sessions/" "http://mu.semte.ch/vocabularies/session/Session")
+
+(define-graph acmidm-lezer ("http://mu.semte.ch/graphs/administrative-unit")
+  ("dc_terms:Agent" -> _)
+  ("org:Organization" -> _)
+  ("regorg:RegisteredOrganization" -> _)
+  ("besluit:Besluit" -> _)
+  ("besluitvorming:Beslissingsactiviteit" -> _)
+  ("org:Site" -> _)
+  ("locn:Address" -> _)
+  ("person:Person" -> _)
+  ("schema:ContactPoint" -> _)
+  ("besluit:Bestuurseenheid" -> _)
+  ("besluit:Bestuursorgaan" -> _)
+  ("adms:Identifier" -> _)
+  ("generiek:GestructureerdeIdentificator" -> _)
+  ("mandaat:Mandaat" -> _)
+  ("mandaat:Mandataris" -> _)
+  ("lblodlg:Functionaris" -> _)
+  ("lblodlg:Bestuursfunctie" -> _)
+  ("code:VeranderingsgebeurtenisResultaat" -> _)
+  ("org:Post" -> _)
+  ("org:ChangeEvent" -> _)
+  ("ch:AgentInPositie" -> _)
+  ("ext:KboOrganisatie" -> _)
+  ("org:Membership" -> _)
+  ("time:ProperInterval" -> _))
+
+(define-graph acmidm-vendor ("http://mu.semte.ch/graphs/vendor-management-data")
+  ("ext:Vendor" -> _))
+
+(define-graph acmidm-editeerder ("http://mu.semte.ch/graphs/administrative-unit")
+  ("oasis:Error" -> _)
+  ("dc_terms:Agent" -> _)
+  ("org:Organization" -> _)
+  ("regorg:RegisteredOrganization" -> _)
+  ("besluit:Besluit" -> _)
+  ("besluitvorming:Beslissingsactiviteit" -> _)
+  ("org:Site" -> _)
+  ("locn:Address" -> _)
+  ("person:Person" -> _)
+  ("schema:ContactPoint" -> _)
+  ("besluit:Bestuurseenheid" -> _)
+  ("besluit:Bestuursorgaan" -> _)
+  ("adms:Identifier" -> _)
+  ("generiek:GestructureerdeIdentificator" -> _)
+  ("mandaat:Mandaat" -> _)
+  ("mandaat:Mandataris" -> _)
+  ("lblodlg:Functionaris" -> _)
+  ("lblodlg:Bestuursfunctie" -> _)
+  ("code:VeranderingsgebeurtenisResultaat" -> _)
+  ("org:Post" -> _)
+  ("org:ChangeEvent" -> _)
+  ("ch:AgentInPositie" -> _)
+  ("ext:KboOrganisatie" -> _)
+  ("org:Membership" -> _)
+  ("time:ProperInterval" -> _))
+
+(define-graph acmidm-worship-lezer ("http://mu.semte.ch/graphs/worship-service")
+  ("ere:EredienstMandataris" -> _)
+  ("ere:BestuurVanDeEredienst" -> _)
+  ("ere:EredienstBestuurseenheid" -> _)
+  ("ere:RolBedienaar" -> _)
+  ("ere:VoorwaardenBedienaar" -> _)
+  ("ere:PositieBedienaar" -> _)
+  ("ere:BetrokkenLokaleBesturen" -> _)
+  ("ere:VerbondenJuridischeStructuren" -> _)
+  ("ere:CentraalBestuurVanDeEredienst" -> _)
+  ("ere:RepresentatiefOrgaan" -> _)
+  ("dc_terms:Agent" -> _)
+  ("org:Organization" -> _)
+  ("regorg:RegisteredOrganization" -> _)
+  ("besluit:Besluit" -> _)
+  ("besluitvorming:Beslissingsactiviteit" -> _)
+  ("org:Site" -> _)
+  ("locn:Address" -> _)
+  ("person:Person" -> _)
+  ("schema:ContactPoint" -> _)
+  ("besluit:Bestuurseenheid" -> _)
+  ("besluit:Bestuursorgaan" -> _)
+  ("adms:Identifier" -> _)
+  ("generiek:GestructureerdeIdentificator" -> _)
+  ("mandaat:Mandaat" -> _)
+  ("mandaat:Mandataris" -> _)
+  ("lblodlg:Functionaris" -> _)
+  ("lblodlg:Bestuursfunctie" -> _)
+  ("code:VeranderingsgebeurtenisResultaat" -> _)
+  ("org:Post" -> _)
+  ("org:ChangeEvent" -> _)
+  ("ch:AgentInPositie" -> _)
+  ("ext:KboOrganisatie" -> _)
+  ("org:Membership" -> _)
+  ("time:ProperInterval" -> _))
+
+(define-graph acmidm-worship-editeerder ("http://mu.semte.ch/graphs/worship-service")
+  ("oasis:Error" -> _)
+  ("ere:EredienstMandataris" -> _)
+  ("ere:BestuurVanDeEredienst" -> _)
+  ("ere:EredienstBestuurseenheid" -> _)
+  ("ere:RolBedienaar" -> _)
+  ("ere:VoorwaardenBedienaar" -> _)
+  ("ere:PositieBedienaar" -> _)
+  ("ere:BetrokkenLokaleBesturen" -> _)
+  ("ere:VerbondenJuridischeStructuren" -> _)
+  ("ere:CentraalBestuurVanDeEredienst" -> _)
+  ("ere:RepresentatiefOrgaan" -> _)
+  ("dc_terms:Agent" -> _)
+  ("org:Organization" -> _)
+  ("regorg:RegisteredOrganization" -> _)
+  ("besluit:Besluit" -> _)
+  ("besluitvorming:Beslissingsactiviteit" -> _)
+  ("org:Site" -> _)
+  ("locn:Address" -> _)
+  ("person:Person" -> _)
+  ("schema:ContactPoint" -> _)
+  ("besluit:Bestuurseenheid" -> _)
+  ("besluit:Bestuursorgaan" -> _)
+  ("adms:Identifier" -> _)
+  ("generiek:GestructureerdeIdentificator" -> _)
+  ("mandaat:Mandaat" -> _)
+  ("mandaat:Mandataris" -> _)
+  ("lblodlg:Functionaris" -> _)
+  ("lblodlg:Bestuursfunctie" -> _)
+  ("code:VeranderingsgebeurtenisResultaat" -> _)
+  ("org:Post" -> _)
+  ("org:ChangeEvent" -> _)
+  ("ch:AgentInPositie" -> _)
+  ("ext:KboOrganisatie" -> _)
+  ("org:Membership" -> _)
+  ("time:ProperInterval" -> _))
+
+(define-graph public ("http://mu.semte.ch/graphs/public")
+  ("org:Role" -> _)
+  ("besluit:Bestuurseenheid" -> _)
+  ("foaf:Person" -> _)
+  ("foaf:OnlineAccount" -> _)
+  ("nfo:FileDataObject" -> _)
+  ("dcat:Dataset" -> _)
+  ("dcat:Distribution" -> _)
+  ("dcat:Catalog" -> _)
+  ("ext:GeslachtCode" -> _)
+  ("lblodlg:FunctionarisStatusCode" -> _)
+  ("code:OrganisatieStatusCode" -> _)
+  ("code:BestuursfunctieCode" -> _)
+  ("code:BestuursorgaanClassificatieCode" -> _)
+  ("code:MandatarisStatusCode" -> _)
+  ("code:BestuurseenheidClassificatieCode" -> _)
+  ("ext:GeregistreerdeOrganisatieClassificatieCode" -> _)
+  ("code:TypeEredienst" -> _)
+  ("skos:ConceptScheme" -> _)
+  ("code:BedienaarFinancieringCode" -> _)
+  ("skos:Concept" -> _)
+  ("euvoc:Country" -> _)
+  ("code:TypeVestiging" -> _)
+  ("code:HelftVerkiezing" -> _)
+  ("code:Veranderingsgebeurtenis" -> _)
+  ("code:TypeBetrokkenheid" -> _)
+  ("code:VoorwaardenBedienaarCriterium" -> _)
+  ("code:BedienaarCriteriumBewijsstuk" -> _)
+  ("code:EredienstBeroepen" -> _)
+  ("code:Rechtsvormtype" -> _)
+  ("prov:Location" -> _)
+  ("code:inhoudelijkThema" -> _))
+
+(define-graph shared ("http://mu.semte.ch/graphs/shared")
+  ("foaf:Image" -> _)
+  ("nfo:FileDataObject" -> _)
+  ("dc_terms:Agent" -> _)
+  ("org:Organization" -> _)
+  ("regorg:RegisteredOrganization" -> _)
+  ("besluit:Besluit" -> _)
+  ("besluitvorming:Beslissingsactiviteit" -> _)
+  ("org:Site" -> _)
+  ("locn:Address" -> _)
+  ("person:Person" -> _)
+  ("schema:ContactPoint" -> _)
+  ("besluit:Bestuurseenheid" -> _)
+  ("besluit:Bestuursorgaan" -> _)
+  ("adms:Identifier" -> _)
+  ("generiek:GestructureerdeIdentificator" -> _)
+  ("mandaat:Mandaat" -> _)
+  ("mandaat:Mandataris" -> _)
+  ("lblodlg:Functionaris" -> _)
+  ("lblodlg:Bestuursfunctie" -> _)
+  ("code:VeranderingsgebeurtenisResultaat" -> _)
+  ("org:Post" -> _)
+  ("org:ChangeEvent" -> _)
+  ("ch:AgentInPositie" -> _)
+  ("ext:KboOrganisatie" -> _)
+  ("org:Membership" -> _)
+  ("time:ProperInterval" -> _))
+
+(define-graph reports ("http://mu.semte.ch/graphs/reports")
+  ("reporting:Report" -> _)
+  ("oslc:Error" -> _)
+  ("nfo:DataContainer" -> _)
+  ("nfo:FileDataObject" -> _))
+
+(define-graph system-jobs ("http://mu.semte.ch/graphs/system/jobs")
+  ("cogs:Job" -> _)
+  ("nfo:DataContainer" -> _)
+  ("nfo:FileDataObject" -> _))
+
+(define-graph jobs ("http://mu.semte.ch/graphs/jobs")
+  ("cogs:Job" -> _)
+  ("nfo:DataContainer" -> _)
+  ("nfo:FileDataObject" -> _))
+
+(supply-allowed-group "public")
+
+(supply-allowed-group "o-admin-rwf"
+  :parameters ()
+  :query "PREFIX ext: <http://mu.semte.ch/vocabularies/ext/>
+    SELECT DISTINCT ?session WHERE {
+      <SESSION_ID> ext:sessionRole \"LoketLB-AdminDashboardOrganisatiePortaalGebruiker\" .
+    }")
+
+(supply-allowed-group "ABBOrganisatiePortaalGebruiker-lezer"
+  :parameters ()
+  :query "PREFIX ext: <http://mu.semte.ch/vocabularies/ext/>
+    PREFIX mu: <http://mu.semte.ch/vocabularies/core/>
+    SELECT DISTINCT ?session_group ?session_role WHERE {
+      <SESSION_ID> ext:sessionGroup/mu:uuid ?session_group;
+                   ext:activeSessionRole ?session_role.
+      FILTER( ?session_role = \"ABBOrganisatiePortaalGebruiker-lezer\" )
+    }")
+
+(supply-allowed-group "ABBOrganisatiePortaalGebruiker-editeerder"
+  :parameters ()
+  :query "PREFIX ext: <http://mu.semte.ch/vocabularies/ext/>
+    PREFIX mu: <http://mu.semte.ch/vocabularies/core/>
+    SELECT DISTINCT ?session_group ?session_role WHERE {
+      <SESSION_ID> ext:sessionGroup/mu:uuid ?session_group;
+                   ext:activeSessionRole ?session_role.
+      FILTER( ?session_role = \"ABBOrganisatiePortaalGebruiker-editeerder\" )
+    }")
+
+(supply-allowed-group "ABBOrganisatiePortaalGebruiker-beheerder"
+  :parameters ()
+  :query "PREFIX ext: <http://mu.semte.ch/vocabularies/ext/>
+    PREFIX mu: <http://mu.semte.ch/vocabularies/core/>
+    SELECT DISTINCT ?session_group ?session_role WHERE {
+      <SESSION_ID> ext:sessionGroup/mu:uuid ?session_group;
+                   ext:activeSessionRole ?session_role.
+      FILTER( ?session_role = \"ABBOrganisatiePortaalGebruiker-beheerder\" )
+    }")
+
+
+(supply-allowed-group "ABBOrganisatiePortaalErediensten-lezer"
+  :parameters ()
+  :query "PREFIX ext: <http://mu.semte.ch/vocabularies/ext/>
+    PREFIX mu: <http://mu.semte.ch/vocabularies/core/>
+    SELECT DISTINCT ?session_group ?session_role WHERE {
+      <SESSION_ID> ext:sessionGroup/mu:uuid ?session_group;
+                   ext:activeSessionRole ?session_role.
+      FILTER( ?session_role = \"ABBOrganisatiePortaalErediensten-lezer\" )
+    }")
+
+(supply-allowed-group "ABBOrganisatiePortaalErediensten-editeerder"
+  :parameters ()
+  :query "PREFIX ext: <http://mu.semte.ch/vocabularies/ext/>
+    PREFIX mu: <http://mu.semte.ch/vocabularies/core/>
+    SELECT DISTINCT ?session_group ?session_role WHERE {
+      <SESSION_ID> ext:sessionGroup/mu:uuid ?session_group;
+                   ext:activeSessionRole ?session_role.
+      FILTER( ?session_role = \"ABBOrganisatiePortaalErediensten-editeerder\" )
+    }")
+
+(supply-allowed-group "ABBOrganisatiePortaalErediensten-beheerder"
+  :parameters ()
+  :query "PREFIX ext: <http://mu.semte.ch/vocabularies/ext/>
+    PREFIX mu: <http://mu.semte.ch/vocabularies/core/>
+    SELECT DISTINCT ?session_group ?session_role WHERE {
+      <SESSION_ID> ext:sessionGroup/mu:uuid ?session_group;
+                   ext:activeSessionRole ?session_role.
+      FILTER( ?session_role = \"ABBOrganisatiePortaalErediensten-beheerder\" )
+    }")
+
+(grant (read)
+  :to-graph (acmidm-lezer acmidm-vendor)
+  :for-allowed-group "ABBOrganisatiePortaalGebruiker-lezer")
+
+(grant (read write)
+  :to-graph (acmidm-editeerder acmidm-vendor)
+  :for-allowed-group "ABBOrganisatiePortaalGebruiker-editeerder")
+
+(grant (read write)
+  :to-graph (acmidm-editeerder acmidm-vendor)
+  :for-allowed-group "ABBOrganisatiePortaalGebruiker-beheerder")
+
+(grant (read)
+  :to-graph (acmidm-worship-lezer acmidm-vendor)
+  :for-allowed-group "ABBOrganisatiePortaalErediensten-lezer")
+
+(grant (read write)
+  :to-graph (acmidm-worship-editeerder acmidm-vendor)
+  :for-allowed-group "ABBOrganisatiePortaalErediensten-editeerder")
+
+(grant (read write)
+  :to-graph (acmidm-worship-editeerder acmidm-vendor)
+  :for-allowed-group "ABBOrganisatiePortaalErediensten-beheerder")
+
+(grant (read)
+  :to-graph (public shared)
+  :for-allowed-group "public")
+
+(grant (read write)
+  :to-graph (reports jobs system-jobs)
+  :for-allowed-group "o-admin-rwf")

config/authorization/delta.ex

@@ -1,5 +0,0 @@
-defmodule Delta.Config do
-  def targets do
-    [ "http://deltanotifier" ]
-  end
-end

config/clb-consumer/contact-data/config.js

@@ -1,30 +0,0 @@
-const BATCH_SIZE = parseInt(process.env.BATCH_SIZE) || 1000;
-const MU_CALL_SCOPE_ID_INITIAL_SYNC = process.env.MU_CALL_SCOPE_ID_INITIAL_SYNC || 'http://redpencil.data.gift/id/concept/muScope/deltas/consumer/initialSync';
-const BYPASS_MU_AUTH_FOR_EXPENSIVE_QUERIES = process.env.BYPASS_MU_AUTH_FOR_EXPENSIVE_QUERIES == 'true' ? true : false;
-const DIRECT_DATABASE_ENDPOINT = process.env.DIRECT_DATABASE_ENDPOINT || 'http://virtuoso:8890/sparql';
-const MAX_DB_RETRY_ATTEMPTS = parseInt(process.env.MAX_DB_RETRY_ATTEMPTS || 5);
-const MAX_REASONING_RETRY_ATTEMPTS = parseInt(process.env.MAX_REASONING_RETRY_ATTEMPTS || 5);
-const SLEEP_BETWEEN_BATCHES = parseInt(process.env.SLEEP_BETWEEN_BATCHES || 100);
-const SLEEP_TIME_AFTER_FAILED_REASONING_OPERATION = parseInt(process.env.SLEEP_TIME_AFTER_FAILED_REASONING_OPERATION || 10000);
-const SLEEP_TIME_AFTER_FAILED_DB_OPERATION = parseInt(process.env.SLEEP_TIME_AFTER_FAILED_DB_OPERATION || 60000);
-
-// Graphs
-const LANDING_ZONE_GRAPH = process.env.DCR_LANDING_ZONE_GRAPH || `http://mu.semte.ch/graphs/landing-zone/clb-contact-data`;
-
-// ENDPOINTS
-const LANDING_ZONE_DATABASE = process.env.DCR_LANDING_ZONE_DATABASE || 'database';
-const LANDING_ZONE_DATABASE_ENDPOINT = process.env.DCR_LANDING_ZONE_DATABASE_ENDPOINT || `http://${LANDING_ZONE_DATABASE}:8890/sparql`;
-
-module.exports = {
-  BATCH_SIZE,
-  MU_CALL_SCOPE_ID_INITIAL_SYNC,
-  BYPASS_MU_AUTH_FOR_EXPENSIVE_QUERIES,
-  DIRECT_DATABASE_ENDPOINT,
-  MAX_DB_RETRY_ATTEMPTS,
-  MAX_REASONING_RETRY_ATTEMPTS,
-  SLEEP_BETWEEN_BATCHES,
-  SLEEP_TIME_AFTER_FAILED_DB_OPERATION,
-  SLEEP_TIME_AFTER_FAILED_REASONING_OPERATION,
-  LANDING_ZONE_GRAPH,
-  LANDING_ZONE_DATABASE_ENDPOINT
-};

config/clb-consumer/contact-data/delta-context-config.js

@@ -1,268 +0,0 @@
-
-const PREFIXES = `
-PREFIX besluit: <http://data.vlaanderen.be/ns/besluit#>
-  PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
-  PREFIX lblodgeneriek: <https://data.lblod.info/vocabularies/generiek/>
-  PREFIX org: <http://www.w3.org/ns/org#>
-  PREFIX code: <http://lblod.data.gift/vocabularies/organisatie/>
-  PREFIX adms: <http://www.w3.org/ns/adms#>
-  PREFIX generiek: <https://data.vlaanderen.be/ns/generiek#>
-  PREFIX ere: <http://data.lblod.info/vocabularies/erediensten/>
-  PREFIX organisatie: <https://data.vlaanderen.be/ns/organisatie#>
-  PREFIX mu: <http://mu.semte.ch/vocabularies/core/>
-  PREFIX euvoc: <http://publications.europa.eu/ontology/euvoc#>
-  PREFIX prov: <http://www.w3.org/ns/prov#>
-  PREFIX schema: <http://schema.org/>
-  PREFIX locn: <http://www.w3.org/ns/locn#>
-  PREFIX ext: <http://mu.semte.ch/vocabularies/ext/>`
-
-const contextConfig = {
-  addTypes: {
-    scope: 'all', // 'inserts, 'deletes', 'all' or none. To add rdf:type to subjects of inserts, deletes or both
-    exhausitive: false, // true or false: find all types for a subject, even if one is already present in delta
-  },
-  contextQueries: [
-    {
-      trigger: { // subjectType or predicateValue
-        subjectType: "org:Site"
-      },
-      queryTemplate: (subject) => `
-        ${PREFIXES}
-        CONSTRUCT {
-          ${subject} ext:goesInGraph ?graphOne.
-          ${subject} ext:goesInGraph ?graphTwo.
-          ${subject} ext:goesInGraph ?graphThree.
-          ${subject} ext:goesInGraph ?graphFour.
-        } WHERE {
-          OPTIONAL {
-            GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
-              ?adminUnitOne org:hasPrimarySite ${subject}.
-            }
-            BIND(<http://mu.semte.ch/graphs/administrative-unit> AS ?graphOne)
-          }
-          OPTIONAL {
-            GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
-              ?adminUnitTwo org:hasSite ${subject}.
-            }
-            BIND(<http://mu.semte.ch/graphs/administrative-unit> AS ?graphTwo)
-          }
-          OPTIONAL {
-            GRAPH <http://mu.semte.ch/graphs/worship-service> {
-              ?adminUnitThree org:hasPrimarySite ${subject}.
-            }
-            BIND(<http://mu.semte.ch/graphs/worship-service> AS ?graphThree)
-          }
-          OPTIONAL {
-            GRAPH <http://mu.semte.ch/graphs/worship-service> {
-              ?adminUnitFour org:hasSite ${subject}.
-            }
-            BIND(<http://mu.semte.ch/graphs/worship-service> AS ?graphFour)
-          }
-        }`
-    },
-    {
-      trigger: { // subjectType or predicateValue
-        subjectType: "schema:ContactPoint"
-      },
-      queryTemplate: (subject) => `
-        ${PREFIXES}
-        CONSTRUCT {
-          ${subject} ext:goesInGraph ?graphOne.
-          ${subject} ext:goesInGraph ?graphTwo.
-        } WHERE {
-          OPTIONAL {
-            GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
-              ?siteOne org:siteAddress ${subject}.
-            }
-            BIND(<http://mu.semte.ch/graphs/administrative-unit> AS ?graphOne)
-          }
-          OPTIONAL {
-            GRAPH <http://mu.semte.ch/graphs/worship-service> {
-              ?siteTwo org:siteAddress ${subject}.
-            }
-            BIND(<http://mu.semte.ch/graphs/worship-service> AS ?graphTwo)
-          }
-        }`
-    },
-    {
-      trigger: { // subjectType or predicateValue
-        subjectType: "locn:Address"
-      },
-      queryTemplate: (subject) => `
-        ${PREFIXES}
-        CONSTRUCT {
-          ${subject} ext:goesInGraph ?graphOne.
-          ${subject} ext:goesInGraph ?graphTwo.
-        } WHERE {
-          OPTIONAL {
-            GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
-              ?siteOne organisatie:bestaatUit ${subject}.
-            }
-            BIND(<http://mu.semte.ch/graphs/administrative-unit> AS ?graphOne)
-          }
-          OPTIONAL {
-            GRAPH <http://mu.semte.ch/graphs/worship-service> {
-              ?siteTwo organisatie:bestaatUit ${subject}.
-            }
-            BIND(<http://mu.semte.ch/graphs/worship-service> AS ?graphTwo)
-          }
-        }`
-    },
-    //Predicate triggers to solve out of order deltas
-    {
-      trigger: { // subjectType or predicateValue
-        predicateValue: "org:hasSite"
-      },
-      //We assume that only administrative units are in the shared graph of OP due to the need of sharing the prefLabel with worships services
-      queryTemplate: (subject) => `
-        ${PREFIXES}
-        CONSTRUCT {
-          ${subject} ext:goesInGraph ?graphOne.
-          ${subject} ext:goesInGraph ?graphTwo.
-          ?site ?pSite ?oSite.
-          ?contact ?pContact ?oContact.
-          ?address ?pAddress ?oAddress.
-          ?addressContact ?pAddressContact ?oAddressContact.
-        } WHERE {
-          OPTIONAL {
-            GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
-              ${subject} a ?type.
-            }
-            BIND(<http://mu.semte.ch/graphs/administrative-unit> AS ?graphOne)
-          }
-          OPTIONAL {
-            GRAPH <http://mu.semte.ch/graphs/shared> {
-              ${subject} a ?type.
-            }
-            BIND(<http://mu.semte.ch/graphs/administrative-unit> AS ?graphOne)
-          }
-          OPTIONAL {
-            GRAPH <http://mu.semte.ch/graphs/worship-service> {
-              ${subject} a ?type.
-            }
-            BIND(<http://mu.semte.ch/graphs/worship-service> AS ?graphTwo)
-          }
-          ${subject} org:hasSite ?site.
-          ?site ?pSite ?oSite.
-          {
-            ?site org:siteAddress ?contact.
-            ?contact ?pContact ?oContact.
-            ?contact locn:address ?addressContact.
-            ?addressContact ?pAddressContact ?oAddressContact.
-          }
-          UNION
-          {
-            ?site org:siteAddress ?contact.
-            ?contact ?pContact ?oContact.
-          }
-          UNION
-          {
-            ?site organisatie:bestaatUit ?address.
-            ?address ?pAddress ?oAddress.
-          }        
-        }`
-    },
-    {
-      trigger: { // subjectType or predicateValue
-        predicateValue: "org:hasPrimarySite"
-      },
-      //We assume that only administrative units are in the shared graph of OP due to the need of sharing the prefLabel with worships services
-      queryTemplate: (subject) => `
-        ${PREFIXES}
-        CONSTRUCT {
-          ${subject} ext:goesInGraph ?graphOne.
-          ${subject} ext:goesInGraph ?graphTwo.
-          ?site ?pSite ?oSite.
-          ?contact ?pContact ?oContact.
-          ?address ?pAddress ?oAddress.
-          ?addressContact ?pAddressContact ?oAddressContact.
-        } WHERE {
-          OPTIONAL {
-            GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
-              ${subject} a ?type.
-            }
-            BIND(<http://mu.semte.ch/graphs/administrative-unit> AS ?graphOne)
-          }
-          OPTIONAL {
-            GRAPH <http://mu.semte.ch/graphs/shared> {
-              ${subject} a ?type.
-            }
-            BIND(<http://mu.semte.ch/graphs/administrative-unit> AS ?graphOne)
-          }
-          OPTIONAL {
-            GRAPH <http://mu.semte.ch/graphs/worship-service> {
-              ${subject} a ?type.
-            }
-            BIND(<http://mu.semte.ch/graphs/worship-service> AS ?graphTwo)
-          }
-          ${subject} org:hasPrimarySite ?site.
-          ?site ?pSite ?oSite.
-          {
-            ?site org:siteAddress ?contact.
-            ?contact ?pContact ?oContact.
-            ?contact locn:address ?addressContact.
-            ?addressContact ?pAddressContact ?oAddressContact.
-          }
-          UNION
-          {
-            ?site org:siteAddress ?contact.
-            ?contact ?pContact ?oContact.
-          }
-          UNION
-          {
-            ?site organisatie:bestaatUit ?address.
-            ?address ?pAddress ?oAddress.
-          }        
-        }`
-    },
-    {
-      trigger: { // subjectType or predicateValue
-        predicateValue: "org:siteAddress"
-      },
-      queryTemplate: (subject) => `
-        ${PREFIXES}
-        CONSTRUCT {
-          ?contact ?pContact ?oContact.
-          ?addressContact ?pAddressContact ?oAddressContact.
-        } WHERE {
-          ${subject} org:siteAddress ?contact.
-          ?contact ?pContact ?oContact.
-          OPTIONAL {
-            ?contact locn:address ?addressContact.
-            ?addressContact ?pAddressContact ?oAddressContact.
-          }
-        }`
-    },
-    {
-      trigger: { // subjectType or predicateValue
-        predicateValue: "locn:address"
-      },
-      queryTemplate: (subject) => `
-        ${PREFIXES}
-        CONSTRUCT {
-          ?addressContact ?pAddressContact ?oAddressContact.
-        } WHERE {
-          ${subject} locn:address ?addressContact.
-          ?addressContact ?pAddressContact ?oAddressContact.
-        }`
-    },
-    {
-      trigger: { // subjectType or predicateValue
-        predicateValue: "organisatie:bestaatUit"
-      },
-      queryTemplate: (subject) => `
-        ${PREFIXES}
-        CONSTRUCT {
-          ?address ?pAddress ?oAddress.
-        } WHERE {
-          ${subject} organisatie:bestaatUit ?address.
-          ?address ?pAddress ?oAddress.
-        }`
-    }
-  ]
-}
-
-module.exports = {
-  contextConfig,
-  PREFIXES
-};

config/clb-consumer/contact-data/delta-sync-dispatching.js

@@ -1,104 +0,0 @@
-const {
-  deleteFromPublicGraph,
-  insertIntoPublicGraph,
-  deleteFromSpecificGraphs,
-  insertIntoSpecificGraphs
-} = require('./util');
-
-const {
-  LANDING_ZONE_GRAPH
-} = require('./config')
-
-const publicTypes = [
-  '<http://lblod.data.gift/vocabularies/organisatie/TypeVestiging>'
-]
-
-/**
-* Dispatch the fetched information to a target graph.
-* Note: <share://file/data> will be ADDED to it's own graph.
-*   We take only care of adding them, not updating triples, this is a TODO
-* @param { mu, muAuthSudo } lib - The provided libraries from the host service.
-* @param { termObjectChangeSets: { deletes, inserts } } data - The fetched changes sets, which objects of serialized Terms
-*          [ {
-*              graph: "<http://foo>",
-*              subject: "<http://bar>",
-*              predicate: "<http://baz>",
-*              object: "<http://boom>^^<http://datatype>"
-*            }
-*         ]
-* @return {void} Nothing
-*/
-async function dispatch(lib, data) {
-  let { termObjectChangeSets, termObjectChangeSetsWithContext } = data;
-
-  // Both arrays are the same length, so we can zip them together for easier processing
-  zippedChangeSets = termObjectChangeSets.map((o, i) => ({
-    original: o,
-    withContext: termObjectChangeSetsWithContext && termObjectChangeSetsWithContext[i]
-  }));
-
-  console.log(`Received ${zippedChangeSets.length} change sets`)
-
-  for (let { original, withContext } of zippedChangeSets) {
-    
-    const {public: insertsOnPublic, target: insertsOnGraphs} = separateBetweenTargetAndPublic(original.inserts, withContext.inserts)
-    const {public: deletesOnPublic, target: deletesOnGraphs} = separateBetweenTargetAndPublic(original.deletes, withContext.deletes)
-
-    await deleteFromPublicGraph(lib, deletesOnPublic);
-    await deleteFromSpecificGraphs(lib, deletesOnGraphs);
-    await insertIntoPublicGraph(lib, insertsOnPublic);
-    await insertIntoSpecificGraphs(lib, insertsOnGraphs);
-
-  }
-}
-
-function separateBetweenTargetAndPublic(changes, context) {
-  const changesOnPublic = new Set();
-  const changesOnGraphs = {};
-  for(let change of changes) {
-    const subject = change.subject;
-    const contextTriples = context.filter((context) => context.subject === subject);
-    const graphTriple = contextTriples.find(
-      (context) => context.predicate === '<http://mu.semte.ch/vocabularies/ext/goesInGraph>' 
-        && context.object !== `<${LANDING_ZONE_GRAPH}>`
-    )
-    const otherContextTriples = context.filter(
-      (context) => context.predicate !== '<http://mu.semte.ch/vocabularies/ext/goesInGraph>' 
-        && context.predicate !== '<http://mu.semte.ch/vocabularies/ext/contextDataGoesInGraph>' 
-        && !(context.subject === subject && context.predicate === '<http://www.w3.org/1999/02/22-rdf-syntax-ns#type>')
-    );
-    if(graphTriple) {
-      const graph = graphTriple.object.slice(1,-1); // We have to slice it to remove the "<" and ">"
-      if(change.object !== '') {
-        if(!changesOnGraphs[graph]) {
-          changesOnGraphs[graph] = new Set();
-        }  
-        changesOnGraphs[graph].add(`${change.subject} ${change.predicate} ${change.object}.`)
-      }                    
-      for(let triple of otherContextTriples) {
-        if(triple.object === '') continue;
-        if(!changesOnGraphs[graph]) {
-          changesOnGraphs[graph] = new Set();
-        }  
-        changesOnGraphs[graph].add(`${triple.subject} ${triple.predicate} ${triple.object}.`)                                       
-      }
-    }
-    const typeTriple = contextTriples.find(
-      (context) => context.subject === subject 
-        && context.predicate === '<http://www.w3.org/1999/02/22-rdf-syntax-ns#type>'
-    );
-    if(!typeTriple) continue;
-    const type = typeTriple.object;
-    if(publicTypes.includes(type)) {
-      changesOnPublic.add(`${change.subject} ${change.predicate} ${change.object}.`)
-      for(let triple of otherContextTriples) {
-        changesOnPublic.add(`${triple.subject} ${triple.predicate} ${triple.object}.`)
-      }
-    }
-  }
-  return {public: changesOnPublic, target: changesOnGraphs}
-}
-
-module.exports = {
-  dispatch
-};

config/clb-consumer/contact-data/initial-sync-dispatching.js

@@ -1,69 +0,0 @@
-const { batchedDbUpdate, moveDataAfterInitialSync } = require('./util');
-const {
-  BYPASS_MU_AUTH_FOR_EXPENSIVE_QUERIES,
-  DIRECT_DATABASE_ENDPOINT,
-  MU_CALL_SCOPE_ID_INITIAL_SYNC,
-  BATCH_SIZE,
-  MAX_DB_RETRY_ATTEMPTS,
-  SLEEP_BETWEEN_BATCHES,
-  SLEEP_TIME_AFTER_FAILED_DB_OPERATION,
-  LANDING_ZONE_GRAPH,
-} = require('./config');
-
-const endpoint = BYPASS_MU_AUTH_FOR_EXPENSIVE_QUERIES ? DIRECT_DATABASE_ENDPOINT : process.env.MU_SPARQL_ENDPOINT;
-
-/**
-* Dispatch the fetched information to a target graph.
-* @param { mu, muAuthSudo } lib - The provided libraries from the host service.
-* @param { termObjects } data - The fetched quad information, which objects of serialized Terms
-*          [ {
-*              graph: "<http://foo>",
-*              subject: "<http://bar>",
-*              predicate: "<http://baz>",
-*              object: "<http://boom>^^<http://datatype>"
-*            }
-*         ]
-* @return {void} Nothing
-*/
-async function dispatch(lib, data) {
-  const { mu, muAuthSudo } = lib;
-  const { termObjects } = data;
-
-  // Steps:
-  // 1. Write the triples to the landing zone graph - without any mapping or filtering
-  // 2. One-time reasoning run with the full landing zone graph and write the results to the target graph (on-finish)
-
-  if (BYPASS_MU_AUTH_FOR_EXPENSIVE_QUERIES) {
-    console.warn(`Service configured to skip MU_AUTH!`);
-  }
-  console.log(`Using ${endpoint} to insert triples`);
-
-  if (termObjects.length) {
-    const originalInsertTriples = termObjects.map(o => `${o.subject} ${o.predicate} ${o.object}.`);
-
-    await batchedDbUpdate(
-      muAuthSudo.updateSudo,
-      LANDING_ZONE_GRAPH,
-      originalInsertTriples,
-      { 'mu-call-scope-id': MU_CALL_SCOPE_ID_INITIAL_SYNC },
-      endpoint,
-      BATCH_SIZE,
-      MAX_DB_RETRY_ATTEMPTS,
-      SLEEP_BETWEEN_BATCHES,
-      SLEEP_TIME_AFTER_FAILED_DB_OPERATION
-    );
-  }
-}
-
-async function onFinishInitialIngest(lib,data) {
-  const { muAuthSudo } = lib;
-
-  console.log(`!! On-finish triggered !!`);
-
-  await moveDataAfterInitialSync(muAuthSudo.updateSudo, endpoint)
-}
-
-module.exports = {
-  dispatch,
-  onFinishInitialIngest
-};

config/clb-consumer/contact-data/util.js

@@ -1,337 +0,0 @@
-const {
-  MAX_DB_RETRY_ATTEMPTS,
-  SLEEP_BETWEEN_BATCHES,
-  SLEEP_TIME_AFTER_FAILED_DB_OPERATION,
-  LANDING_ZONE_GRAPH,
-  BATCH_SIZE,
-} = require('./config');
-
-const prefixes = `
-PREFIX besluit: <http://data.vlaanderen.be/ns/besluit#>
-PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
-PREFIX lblodgeneriek: <https://data.lblod.info/vocabularies/generiek/>
-PREFIX org: <http://www.w3.org/ns/org#>
-PREFIX code: <http://lblod.data.gift/vocabularies/organisatie/>
-PREFIX adms: <http://www.w3.org/ns/adms#>
-PREFIX generiek: <https://data.vlaanderen.be/ns/generiek#>
-PREFIX ere: <http://data.lblod.info/vocabularies/erediensten/>
-PREFIX organisatie: <https://data.vlaanderen.be/ns/organisatie#>
-PREFIX mu: <http://mu.semte.ch/vocabularies/core/>
-PREFIX euvoc: <http://publications.europa.eu/ontology/euvoc#>
-PREFIX prov: <http://www.w3.org/ns/prov#>
-PREFIX schema: <http://schema.org/>
-PREFIX locn: <http://www.w3.org/ns/locn#>
-PREFIX foaf: <http://xmlns.com/foaf/0.1/>
-PREFIX ext:<http://mu.semte.ch/vocabularies/ext/>
-`
-
-async function batchedDbUpdate(
-  muUpdate,
-  graph,
-  triples,
-  extraHeaders,
-  endpoint,
-  batchSize,
-  maxAttempts,
-  sleepBetweenBatches = 1000,
-  sleepTimeOnFail = 1000,
-  operation = 'INSERT'
-) {
-  for (let i = 0; i < triples.length; i += batchSize) {
-    console.log(`Inserting triples in batch: ${i}-${i + batchSize}`);
-
-    const batch = triples.slice(i, i + batchSize).join('\n');
-
-    const insertCall = async () => {
-      await muUpdate(`
-${operation} DATA {
-GRAPH <${graph}> {
-${batch}
-}
-}
-`, extraHeaders, endpoint);
-    };
-
-    await operationWithRetry(insertCall, 0, maxAttempts, sleepTimeOnFail);
-
-    console.log(`Sleeping before next query execution: ${sleepBetweenBatches}`);
-    await new Promise(r => setTimeout(r, sleepBetweenBatches));
-  }
-}
-
-async function operationWithRetry(callback,
-  attempt,
-  maxAttempts,
-  sleepTimeOnFail) {
-  try {
-    if (typeof callback === "function")
-      return await callback();
-    else // Catch error from promise - not how I would do it normally, but allows re use of existing code.
-      return await callback;
-  }
-  catch (e) {
-    console.log(`Operation failed for ${callback.toString()}, attempt: ${attempt} of ${maxAttempts}`);
-    console.log(`Error: ${e}`);
-    console.log(`Sleeping ${sleepTimeOnFail} ms`);
-
-    if (attempt >= maxAttempts) {
-      console.log(`Max attempts reached for ${callback.toString()}, giving up`);
-      throw e;
-    }
-
-    await new Promise(r => setTimeout(r, sleepTimeOnFail));
-    return operationWithRetry(callback, ++attempt, maxAttempts, sleepTimeOnFail);
-  }
-}
-
-
-async function insertIntoPublicGraph(lib, statements) {
-  console.log(`Inserting ${statements.length} statements into public graph`);
-
-  await batchedDbUpdate(
-    lib.muAuthSudo.updateSudo,
-    'http://mu.semte.ch/graphs/public',
-    Array.from(statements),
-    {},
-    process.env.MU_SPARQL_ENDPOINT,
-    BATCH_SIZE,
-    MAX_DB_RETRY_ATTEMPTS,
-    SLEEP_BETWEEN_BATCHES,
-    SLEEP_TIME_AFTER_FAILED_DB_OPERATION,
-    'INSERT');
-}
-
-async function insertIntoSpecificGraphs(lib, statementsWithGraphs) {
-
-  for( let graph in statementsWithGraphs) {
-    console.log(`Inserting ${statementsWithGraphs[graph].length} statements into ${graph} graph`);
-    await batchedDbUpdate(
-      lib.muAuthSudo.updateSudo,
-      graph,
-      Array.from(statementsWithGraphs[graph]),
-      {},
-      process.env.MU_SPARQL_ENDPOINT,
-      BATCH_SIZE,
-      MAX_DB_RETRY_ATTEMPTS,
-      SLEEP_BETWEEN_BATCHES,
-      SLEEP_TIME_AFTER_FAILED_DB_OPERATION,
-      'INSERT');
-  }
-  
-}
-
-async function deleteFromPublicGraph(lib, statements) {
-  console.log(`Deleting ${statements.length} statements from public graph`);
-
-  await batchedDbUpdate(
-    lib.muAuthSudo.updateSudo,
-    'http://mu.semte.ch/graphs/public',
-    Array.from(statements),
-    {},
-    process.env.MU_SPARQL_ENDPOINT,
-    BATCH_SIZE,
-    MAX_DB_RETRY_ATTEMPTS,
-    SLEEP_BETWEEN_BATCHES,
-    SLEEP_TIME_AFTER_FAILED_DB_OPERATION,
-    'DELETE');
-}
-
-async function deleteFromSpecificGraphs(lib, statementsWithGraphs) {
-  
-
-  for( let graph in statementsWithGraphs) {
-    console.log(`Deleting ${statementsWithGraphs[graph].length} statements from ${graph} graph`);
-    await batchedDbUpdate(
-      lib.muAuthSudo.updateSudo,
-      graph,
-      Array.from(statementsWithGraphs[graph]),
-      {},
-      process.env.MU_SPARQL_ENDPOINT,
-      BATCH_SIZE,
-      MAX_DB_RETRY_ATTEMPTS,
-      SLEEP_BETWEEN_BATCHES,
-      SLEEP_TIME_AFTER_FAILED_DB_OPERATION,
-      'DELETE');
-  }
-  
-}
-
-
-async function moveDataAfterInitialSync(muUpdate, endpoint) {
-  // Move primary sites admin units
-  await muUpdate(`
-    ${prefixes}
-    INSERT {
-      GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
-        ?adminUnit org:hasPrimarySite ?site.
-        ?site ?pSite ?oSite.
-        ?contact ?pContact ?oContact.
-        ?address ?pAddress ?oAddress.
-        ?addressContact ?pAddressContact ?oAddressContact.
-      }
-    } WHERE {
-      GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
-        ?adminUnit a ?type.
-      }
-      GRAPH <${LANDING_ZONE_GRAPH}> {
-        ?adminUnit org:hasPrimarySite ?site.
-        ?site ?pSite ?oSite.
-        {
-          ?site org:siteAddress ?contact.
-          ?contact ?pContact ?oContact.
-          ?contact locn:address ?addressContact.
-          ?addressContact ?pAddressContact ?oAddressContact.
-        }
-        UNION
-        {
-          ?site org:siteAddress ?contact.
-          ?contact ?pContact ?oContact.
-        }
-        UNION
-        {
-          ?site organisatie:bestaatUit ?address.
-          ?address ?pAddress ?oAddress.
-        }
-      }        
-    }
-  `, undefined, endpoint)
-
-  // Move sites administrative units
-  await muUpdate(`
-    ${prefixes}
-    INSERT {
-      GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
-        ?adminUnit org:hasSite ?site.
-        ?site ?pSite ?oSite.
-        ?contact ?pContact ?oContact.
-        ?address ?pAddress ?oAddress.
-        ?addressContact ?pAddressContact ?oAddressContact.
-      }
-    } WHERE {
-      GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
-        ?adminUnit a ?type.
-      }
-      GRAPH <${LANDING_ZONE_GRAPH}> {
-        ?adminUnit org:hasSite ?site.
-        ?site ?pSite ?oSite.
-        {
-          ?site org:siteAddress ?contact.
-          ?contact ?pContact ?oContact.
-          ?contact locn:address ?addressContact.
-          ?addressContact ?pAddressContact ?oAddressContact.
-        }
-        UNION
-        {
-          ?site org:siteAddress ?contact.
-          ?contact ?pContact ?oContact.
-        }
-        UNION
-        {
-          ?site organisatie:bestaatUit ?address.
-          ?address ?pAddress ?oAddress.
-        }     
-      }   
-    }
-  `, undefined, endpoint)
-
-  // Move primary sites worships
-  await muUpdate(`
-    ${prefixes}
-    INSERT {
-      GRAPH <http://mu.semte.ch/graphs/worship-service> {
-        ?adminUnit org:hasPrimarySite ?site.
-        ?site ?pSite ?oSite.
-        ?contact ?pContact ?oContact.
-        ?address ?pAddress ?oAddress.
-        ?addressContact ?pAddressContact ?oAddressContact.
-      }
-    } WHERE {
-      GRAPH <http://mu.semte.ch/graphs/worship-service> {
-        ?adminUnit a ?type.
-      }
-      GRAPH <${LANDING_ZONE_GRAPH}> {
-        ?adminUnit org:hasPrimarySite ?site.
-        ?site ?pSite ?oSite.
-        {
-          ?site org:siteAddress ?contact.
-          ?contact ?pContact ?oContact.
-          ?contact locn:address ?addressContact.
-          ?addressContact ?pAddressContact ?oAddressContact.
-        }
-        UNION
-        {
-          ?site org:siteAddress ?contact.
-          ?contact ?pContact ?oContact.
-        }
-        UNION
-        {
-          ?site organisatie:bestaatUit ?address.
-          ?address ?pAddress ?oAddress.
-        }        
-      }
-    }
-  `, undefined, endpoint)
-
-  // Move sites worships
-  await muUpdate(`
-    ${prefixes}
-    INSERT {
-      GRAPH <http://mu.semte.ch/graphs/worship-service> {
-        ?adminUnit org:hasSite ?site.
-        ?site ?pSite ?oSite.
-        ?contact ?pContact ?oContact.
-        ?address ?pAddress ?oAddress.
-        ?addressContact ?pAddressContact ?oAddressContact.
-      }
-    } WHERE {
-      GRAPH <http://mu.semte.ch/graphs/worship-service> {
-        ?adminUnit a ?type.
-      }
-      GRAPH <${LANDING_ZONE_GRAPH}> {
-        ?adminUnit org:hasSite ?site.
-        ?site ?pSite ?oSite.
-        {
-          ?site org:siteAddress ?contact.
-          ?contact ?pContact ?oContact.
-          ?contact locn:address ?addressContact.
-          ?addressContact ?pAddressContact ?oAddressContact.
-        }
-        UNION
-        {
-          ?site org:siteAddress ?contact.
-          ?contact ?pContact ?oContact.
-        }
-        UNION
-        {
-          ?site organisatie:bestaatUit ?address.
-          ?address ?pAddress ?oAddress.
-        }        
-      }
-    }
-  `, undefined, endpoint)
-
-  // Move sites worships
-  await muUpdate(`
-    ${prefixes}
-    INSERT {
-      GRAPH <http://mu.semte.ch/graphs/public> {
-        ?type a <http://lblod.data.gift/vocabularies/organisatie/TypeVestiging>.
-        ?type ?a ?b.
-      }
-    } WHERE {
-      GRAPH <${LANDING_ZONE_GRAPH}> {
-        ?type a <http://lblod.data.gift/vocabularies/organisatie/TypeVestiging>.
-        ?type ?a ?b.
-      }
-    }
-  `, undefined, endpoint)
-}
-
-module.exports = {
-  batchedDbUpdate,
-  moveDataAfterInitialSync,
-  insertIntoPublicGraph,
-  deleteFromPublicGraph,
-  insertIntoSpecificGraphs,
-  deleteFromSpecificGraphs,
-};

config/db/virtuoso-production.ini

@@ -231,10 +231,12 @@ DefaultHost  = localhost:8890
 ;DefaultGraph           = http://localhost:8890/dataspace
 ;ImmutableGraphs        = http://localhost:8890/dataspace
 ResultSetMaxRows           = 1000000 ; 1 million
+MaxConstructTriples        = 1000000 ; More info on this addition here: https://github.com/redpencilio/docker-virtuoso/pull/29
 ;MaxQueryCostEstimationTime = 4000	; in seconds
 MaxQueryExecutionTime      = 1200	; in seconds
 DefaultQuery               = select distinct ?Concept where {[] a ?Concept} LIMIT 100
 DeferInferenceRulesInit    = 0	; controls inference rules loading
+MaxMemInUse                = 0 ; limits the amount of memory for construct dict (0=unlimited). More info on this addition here: https://github.com/redpencilio/docker-virtuoso/pull/29
 ;PingService            = http://rpc.pingthesemanticweb.com/
 
 [Plugins]

config/db/virtuoso.ini

@@ -231,10 +231,12 @@ DefaultHost  = localhost:8890
 ;DefaultGraph           = http://localhost:8890/dataspace
 ;ImmutableGraphs        = http://localhost:8890/dataspace
 ResultSetMaxRows           = 1000000 ; 1 million
+MaxConstructTriples        = 1000000 ; More info on this addition here: https://github.com/redpencilio/docker-virtuoso/pull/29
 ;MaxQueryCostEstimationTime = 4000	; in seconds
 MaxQueryExecutionTime      = 1200	; in seconds
 DefaultQuery               = select distinct ?Concept where {[] a ?Concept} LIMIT 100
 DeferInferenceRulesInit    = 0	; controls inference rules loading
+MaxMemInUse                = 0 ; limits the amount of memory for construct dict (0=unlimited). More info on this addition here: https://github.com/redpencilio/docker-virtuoso/pull/29
 ;PingService            = http://rpc.pingthesemanticweb.com/
 
 [Plugins]

config/delta-consumers/leidinggevenden/mapping/queries/dl2op/data-cleanup/phone.rq

@@ -0,0 +1,21 @@
+PREFIX schema: <http://schema.org/>
+
+CONSTRUCT {
+  ?s schema:telephone ?cleanedPhoneNumber .
+} WHERE {
+  ?s schema:telephone ?phoneNumber .
+
+  # Remove all non digits
+  BIND(REPLACE(?phoneNumber, """[^+\\d]""", "") AS ?scrapeResult)
+
+  # Replace leading 00 with +
+  BIND(REPLACE(?scrapeResult, """^00""", "+") AS ?replacedZeroZero)
+
+  BIND(REPLACE(?replacedZeroZero, """^0""", "+32") AS ?defaultBe)
+
+  BIND(REPLACE(?defaultBe, """\\+320""", "+32") AS ?extendedDefaultBe)
+
+  BIND(REPLACE(?extendedDefaultBe, """^320""", "+32") AS ?finalResult)
+
+  BIND(IF(REGEX(?finalResult, """^\\+((32\\d{8})|(324\\d{8})|((?!32)\\d+))$"""), CONCAT("tel:", ?finalResult), ?phoneNumber) AS ?cleanedPhoneNumber)
+}

config/delta-consumers/leidinggevenden/mapping/queries/dl2op/datetime-format/mandaat-einde.rq

@@ -0,0 +1,11 @@
+PREFIX mandaat: <http://data.vlaanderen.be/ns/mandaat#>
+PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
+
+CONSTRUCT {
+  ?s mandaat:einde ?endDateTime .
+} WHERE {
+  ?s mandaat:einde ?endDate .
+
+  BIND(SUBSTR(STR(?endDate), 1, 10) as ?endDateWithoutTimezone)
+  BIND(STRDT(CONCAT(?endDateWithoutTimezone, 'T02:00:00'), xsd:dateTime) as ?endDateTime)
+}

config/delta-consumers/leidinggevenden/mapping/queries/dl2op/datetime-format/mandaat-start.rq

@@ -0,0 +1,11 @@
+PREFIX mandaat: <http://data.vlaanderen.be/ns/mandaat#>
+PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
+
+CONSTRUCT {
+  ?s mandaat:start ?startDateTime .
+} WHERE {
+  ?s mandaat:start ?startDate .
+
+  BIND(SUBSTR(STR(?startDate), 1, 10) as ?startDateWithoutTimezone)
+  BIND(STRDT(CONCAT(?startDateWithoutTimezone, 'T02:00:00'), xsd:dateTime) as ?startDateTime)
+}

config/delta-consumers/leidinggevenden/mapping/queries/dl2op/inheritance/Agent.rq

@@ -0,0 +1,14 @@
+PREFIX dc_terms: <http://purl.org/dc/terms/>
+PREFIX lblodlg: <http://data.lblod.info/vocabularies/leidinggevenden/>
+PREFIX person: <http://www.w3.org/ns/person#>
+
+CONSTRUCT {
+  ?s a dc_terms:Agent .
+} WHERE {
+  ?s a ?type .
+
+  VALUES ?type {
+    lblodlg:Functionaris
+    person:Person
+  }
+}

config/delta-consumers/leidinggevenden/mapping/queries/dl2op/inheritance/AgentInPositie.rq

@@ -0,0 +1,12 @@
+PREFIX ch: <http://data.lblod.info/vocabularies/contacthub/>
+PREFIX lblodlg: <http://data.lblod.info/vocabularies/leidinggevenden/>
+
+CONSTRUCT {
+  ?s a ch:AgentInPositie .
+} WHERE {
+  ?s a ?type .
+
+  VALUES ?type {
+    lblodlg:Functionaris
+  }
+}

config/delta-consumers/leidinggevenden/mapping/queries/dl2op/inverse/holds-to-held-by.rq

@@ -0,0 +1,8 @@
+PREFIX org: <http://www.w3.org/ns/org#>
+
+CONSTRUCT {
+  ?o org:heldBy ?s.
+}
+WHERE {
+  ?s org:holds ?o.
+}

config/delta-consumers/leidinggevenden/mapping/queries/dl2op/pass/address.sparql

@@ -0,0 +1,10 @@
+CONSTRUCT {
+  ?s ?p ?o.
+} WHERE {
+  ?s a <http://www.w3.org/ns/locn#Address>;
+    ?p ?o.
+
+  FILTER NOT EXISTS {
+    ?s ?p "" .
+  }
+}

config/delta-consumers/leidinggevenden/mapping/queries/dl2op/pass/bestuursfunctie.sparql

@@ -0,0 +1,7 @@
+CONSTRUCT {
+  ?s <http://schema.org/contactPoint> ?contactpoint.
+}
+WHERE {
+  ?s a <http://data.lblod.info/vocabularies/leidinggevenden/Bestuursfunctie>.
+  ?s <http://schema.org/contactPoint> ?contactpoint.
+}

config/delta-consumers/leidinggevenden/mapping/queries/dl2op/pass/contactpoint.sparql

@@ -0,0 +1,11 @@
+CONSTRUCT {
+  ?s ?p ?o.
+} WHERE {
+  ?s a <http://schema.org/ContactPoint>;
+    ?p ?o.
+ FILTER (?p NOT IN (<http://schema.org/telephone>))
+
+ FILTER NOT EXISTS {
+   ?s ?p "" .
+ }
+}

config/delta-consumers/leidinggevenden/mapping/queries/dl2op/pass/functionaris.sparql

@@ -0,0 +1,14 @@
+PREFIX mandaat: <http://data.vlaanderen.be/ns/mandaat#>
+
+CONSTRUCT {
+  ?s ?p ?o.
+} WHERE {
+  ?s a <http://data.lblod.info/vocabularies/leidinggevenden/Functionaris>;
+    ?p ?o.
+
+  FILTER (?p NOT IN (mandaat:start, mandaat:einde))
+
+  FILTER NOT EXISTS {
+    ?s ?p "" .
+  }
+}

config/delta-consumers/leidinggevenden/mapping/queries/dl2op/pass/person.sparql

@@ -0,0 +1,16 @@
+CONSTRUCT {
+  ?s ?p ?o.
+} WHERE {
+  ?s a <http://www.w3.org/ns/person#Person>;
+    ?p ?o.
+
+  FILTER (?p IN (
+    <http://www.w3.org/1999/02/22-rdf-syntax-ns#type>,
+    <http://xmlns.com/foaf/0.1/familyName>,
+    <http://mu.semte.ch/vocabularies/core/uuid>
+    ))
+
+  FILTER NOT EXISTS {
+   ?s ?p "" .
+  }
+}

config/delta-consumers/leidinggevenden/mapping/queries/dl2op/predicate/person-foaf-name.sparql

@@ -0,0 +1,21 @@
+PREFIX foaf: <http://xmlns.com/foaf/0.1/>
+PREFIX persoon: <https://data.vlaanderen.be/ns/persoon#>
+PREFIX persoon_loket: <http://data.vlaanderen.be/ns/persoon#>
+PREFIX person: <http://www.w3.org/ns/person#>
+
+# DL and OP both use persoon:gebruikteVoornaam but with different semantics (!)
+
+# First name: persoon:gebruikteVoornaam => foaf:givenName
+# Alternatieve voornaam: foaf:name => persoon:gebruikteVoornaam
+
+# We do both in one query with an OPTIONAL because they need to happen simultaneously
+# not to end up in a loop
+
+# Part 1
+
+CONSTRUCT {
+  ?s persoon:gebruikteVoornaam ?name .
+} WHERE {
+  ?s a person:Person;
+    foaf:name ?name .
+}

config/delta-consumers/leidinggevenden/mapping/queries/dl2op/predicate/person-gebruikte-voornaam.sparql

@@ -0,0 +1,21 @@
+PREFIX foaf: <http://xmlns.com/foaf/0.1/>
+PREFIX persoon: <https://data.vlaanderen.be/ns/persoon#>
+PREFIX persoon_loket: <http://data.vlaanderen.be/ns/persoon#>
+PREFIX person: <http://www.w3.org/ns/person#>
+
+# DL and OP both use persoon:gebruikteVoornaam but with different semantics (!)
+
+# First name: persoon:gebruikteVoornaam => foaf:givenName
+# Alternatieve voornaam: foaf:name => persoon:gebruikteVoornaam
+
+# We do both in one query with an OPTIONAL because they need to happen simultaneously
+# not to end up in a loop
+
+# Part 2
+
+CONSTRUCT {
+  ?s foaf:givenName ?gebruikteVoornaam .
+} WHERE {
+  ?s a person:Person ;
+    persoon_loket:gebruikteVoornaam ?gebruikteVoornaam .
+}

config/delta-consumers/mandatarissen/mapping/queries/dl2op/datetime-format/mandaat-einde.rq

@@ -0,0 +1,11 @@
+PREFIX mandaat: <http://data.vlaanderen.be/ns/mandaat#>
+PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
+
+CONSTRUCT {
+  ?s mandaat:einde ?endDateTime .
+} WHERE {
+  ?s mandaat:einde ?endDate .
+
+  BIND(SUBSTR(STR(?endDate), 1, 10) as ?endDateWithoutTimezone)
+  BIND(STRDT(CONCAT(?endDateWithoutTimezone, 'T02:00:00'), xsd:dateTime) as ?endDateTime)
+}

config/delta-consumers/mandatarissen/mapping/queries/dl2op/datetime-format/mandaat-start.rq

@@ -0,0 +1,11 @@
+PREFIX mandaat: <http://data.vlaanderen.be/ns/mandaat#>
+PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
+
+CONSTRUCT {
+  ?s mandaat:start ?startDateTime .
+} WHERE {
+  ?s mandaat:start ?startDate .
+
+  BIND(SUBSTR(STR(?startDate), 1, 10) as ?startDateWithoutTimezone)
+  BIND(STRDT(CONCAT(?startDateWithoutTimezone, 'T02:00:00'), xsd:dateTime) as ?startDateTime)
+}

config/delta-consumers/mandatarissen/mapping/queries/dl2op/inheritance/Agent.rq

@@ -0,0 +1,14 @@
+PREFIX dc_terms: <http://purl.org/dc/terms/>
+PREFIX mandaat: <http://data.vlaanderen.be/ns/mandaat#>
+PREFIX person: <http://www.w3.org/ns/person#>
+
+CONSTRUCT {
+  ?s a dc_terms:Agent .
+} WHERE {
+  ?s a ?type .
+
+  VALUES ?type {
+    mandaat:Mandataris
+    person:Person
+  }
+}

config/delta-consumers/mandatarissen/mapping/queries/dl2op/inheritance/AgentInPositie.rq

@@ -0,0 +1,11 @@
+PREFIX ch: <http://data.lblod.info/vocabularies/contacthub/>
+PREFIX mandaat: <http://data.vlaanderen.be/ns/mandaat#>
+
+CONSTRUCT {
+  ?s a ch:AgentInPositie .
+} WHERE {
+  ?s a ?type .
+  VALUES ?type {
+    mandaat:Mandataris
+  }
+}

config/delta-consumers/mandatarissen/mapping/queries/dl2op/inverse/holds-to-held-by.rq

@@ -0,0 +1,8 @@
+PREFIX org: <http://www.w3.org/ns/org#>
+
+CONSTRUCT {
+  ?o org:heldBy ?s.
+}
+WHERE {
+  ?s org:holds ?o.
+}

config/delta-consumers/mandatarissen/mapping/queries/dl2op/pass/mandataris.sparql

@@ -0,0 +1,29 @@
+PREFIX mandaat: <http://data.vlaanderen.be/ns/mandaat#>
+
+CONSTRUCT {
+  ?s ?p ?o.
+} WHERE {
+  ?s a <http://data.vlaanderen.be/ns/mandaat#Mandataris>;
+    <http://www.w3.org/ns/org#holds> ?mandate;
+    ?p ?o.
+
+  ?orgaanInTime <http://www.w3.org/ns/org#hasPost> ?mandate ;
+    <https://data.vlaanderen.be/ns/generiek#isTijdspecialisatieVan> ?orgaan .
+
+  ?orgaan <http://data.vlaanderen.be/ns/besluit#bestuurt> ?bestuur .
+  ?bestuur <http://www.w3.org/ns/org#classification> ?classification .
+
+
+  FILTER(?classification IN (
+    <http://data.vlaanderen.be/id/concept/BestuurseenheidClassificatieCode/5ab0e9b8a3b2ca7c5e000000>,
+    <http://data.vlaanderen.be/id/concept/BestuurseenheidClassificatieCode/5ab0e9b8a3b2ca7c5e000001>,
+    <http://data.vlaanderen.be/id/concept/BestuurseenheidClassificatieCode/5ab0e9b8a3b2ca7c5e000002>,
+    <http://data.vlaanderen.be/id/concept/BestuurseenheidClassificatieCode/5ab0e9b8a3b2ca7c5e000003>
+  ))
+
+  FILTER (?p NOT IN (mandaat:start, mandaat:einde))
+
+  FILTER NOT EXISTS {
+    ?s ?p "" .
+  }
+}

config/delta-consumers/mandatarissen/mapping/queries/dl2op/pass/person.sparql

@@ -0,0 +1,16 @@
+CONSTRUCT {
+  ?s ?p ?o.
+} WHERE {
+  ?s a <http://www.w3.org/ns/person#Person>;
+    ?p ?o.
+
+  FILTER (?p IN (
+    <http://www.w3.org/1999/02/22-rdf-syntax-ns#type>,
+    <http://xmlns.com/foaf/0.1/familyName>,
+    <http://mu.semte.ch/vocabularies/core/uuid>
+    ))
+
+  FILTER NOT EXISTS {
+   ?s ?p "" .
+  }
+}

config/delta-consumers/mandatarissen/mapping/queries/dl2op/predicate/person-foaf-name.rq

@@ -0,0 +1,21 @@
+PREFIX foaf: <http://xmlns.com/foaf/0.1/>
+PREFIX persoon: <https://data.vlaanderen.be/ns/persoon#>
+PREFIX persoon_loket: <http://data.vlaanderen.be/ns/persoon#>
+PREFIX person: <http://www.w3.org/ns/person#>
+
+# DL and OP both use persoon:gebruikteVoornaam but with different semantics (!)
+
+# First name: persoon:gebruikteVoornaam => foaf:givenName
+# Alternatieve voornaam: foaf:name => persoon:gebruikteVoornaam
+
+# We do both in one query with an OPTIONAL because they need to happen simultaneously
+# not to end up in a loop
+
+# Part 1
+
+CONSTRUCT {
+  ?s persoon:gebruikteVoornaam ?name .
+} WHERE {
+  ?s a person:Person;
+    foaf:name ?name .
+}

config/delta-consumers/mandatarissen/mapping/queries/dl2op/predicate/person-gebruikte-voornaam.rq

@@ -0,0 +1,22 @@
+
+PREFIX foaf: <http://xmlns.com/foaf/0.1/>
+PREFIX persoon: <https://data.vlaanderen.be/ns/persoon#>
+PREFIX persoon_loket: <http://data.vlaanderen.be/ns/persoon#>
+PREFIX person: <http://www.w3.org/ns/person#>
+
+# DL and OP both use persoon:gebruikteVoornaam but with different semantics (!)
+
+# First name: persoon:gebruikteVoornaam => foaf:givenName
+# Alternatieve voornaam: foaf:name => persoon:gebruikteVoornaam
+
+# We do both in one query with an OPTIONAL because they need to happen simultaneously
+# not to end up in a loop
+
+# Part 2
+
+CONSTRUCT {
+  ?s foaf:givenName ?gebruikteVoornaam .
+} WHERE {
+  ?s a person:Person ;
+    persoon_loket:gebruikteVoornaam ?gebruikteVoornaam .
+}

config/delta-consumers/worship-services-main-info/mapping/queries/dl2op/data-cleanup/phone.rq

@@ -0,0 +1,21 @@
+PREFIX schema: <http://schema.org/>
+
+CONSTRUCT {
+  ?s schema:telephone ?cleanedPhoneNumber .
+} WHERE {
+  ?s schema:telephone ?phoneNumber .
+
+  # Remove all non digits
+  BIND(REPLACE(?phoneNumber, """[^+\\d]""", "") AS ?scrapeResult)
+
+  # Replace leading 00 with +
+  BIND(REPLACE(?scrapeResult, """^00""", "+") AS ?replacedZeroZero)
+
+  BIND(REPLACE(?replacedZeroZero, """^0""", "+32") AS ?defaultBe)
+
+  BIND(REPLACE(?defaultBe, """\\+320""", "+32") AS ?extendedDefaultBe)
+
+  BIND(REPLACE(?extendedDefaultBe, """^320""", "+32") AS ?finalResult)
+
+  BIND(IF(REGEX(?finalResult, """^\\+((32\\d{8})|(324\\d{8})|((?!32)\\d+))$"""), CONCAT("tel:", ?finalResult), ?phoneNumber) AS ?cleanedPhoneNumber)
+}

config/delta-consumers/worship-services-main-info/mapping/queries/dl2op/date-to-datetime/contacthub-eindedatum.rq

@@ -0,0 +1,10 @@
+PREFIX ch: <http://data.lblod.info/vocabularies/contacthub/>
+PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
+
+CONSTRUCT {
+  ?s ch:eindedatum ?endDateTime .
+} WHERE {
+  ?s ch:eindedatum ?endDate .
+
+  BIND(STRDT(CONCAT(STR(?endDate), 'T02:00:00'), xsd:dateTime) as ?endDateTime)
+}

config/delta-consumers/worship-services-main-info/mapping/queries/dl2op/date-to-datetime/contacthub-startdatum.rq

@@ -0,0 +1,10 @@
+PREFIX ch: <http://data.lblod.info/vocabularies/contacthub/>
+PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
+
+CONSTRUCT {
+  ?s ch:startdatum ?startDateTime .
+} WHERE {
+  ?s ch:startdatum ?startDate .
+
+  BIND(STRDT(CONCAT(STR(?startDate), 'T02:00:00'), xsd:dateTime) as ?startDateTime)
+}

config/delta-consumers/worship-services-main-info/mapping/queries/dl2op/date-to-datetime/ere-geplande-einddatum-aanstelling.rq

@@ -0,0 +1,10 @@
+PREFIX ere: <http://data.lblod.info/vocabularies/erediensten/>
+PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
+
+CONSTRUCT {
+  ?s ere:geplandeEinddatumAanstelling ?dateTime .
+} WHERE {
+  ?s ere:geplandeEinddatumAanstelling ?date .
+
+  BIND(STRDT(CONCAT(STR(?date), 'T02:00:00'), xsd:dateTime) as ?dateTime)
+}

config/delta-consumers/worship-services-main-info/mapping/queries/dl2op/inheritance/Agent.rq

@@ -0,0 +1,20 @@
+PREFIX ch: <http://data.lblod.info/vocabularies/contacthub/>
+PREFIX dc_terms: <http://purl.org/dc/terms/>
+PREFIX ere: <http://data.lblod.info/vocabularies/erediensten/>
+PREFIX lblodlg: <http://data.lblod.info/vocabularies/leidinggevenden/>
+PREFIX mandaat: <http://data.vlaanderen.be/ns/mandaat#>
+PREFIX person: <http://www.w3.org/ns/person#>
+
+CONSTRUCT {
+  ?s a dc_terms:Agent .
+} WHERE {
+  ?s a ?type .
+
+  VALUES ?type {
+    lblodlg:Functionaris
+    mandaat:Mandataris
+    ere:EredienstMandataris
+    ere:RolBedienaar
+    person:Person
+  }
+}

config/delta-consumers/worship-services-main-info/mapping/queries/dl2op/inheritance/AgentInPositie.rq

@@ -0,0 +1,17 @@
+PREFIX ch: <http://data.lblod.info/vocabularies/contacthub/>
+PREFIX ere: <http://data.lblod.info/vocabularies/erediensten/>
+PREFIX lblodlg: <http://data.lblod.info/vocabularies/leidinggevenden/>
+PREFIX mandaat: <http://data.vlaanderen.be/ns/mandaat#>
+
+CONSTRUCT {
+  ?s a ch:AgentInPositie .
+} WHERE {
+  ?s a ?type .
+
+  VALUES ?type {
+    lblodlg:Functionaris
+    mandaat:Mandataris
+    ere:EredienstMandataris
+    ere:RolBedienaar
+  }
+}

config/delta-consumers/worship-services-main-info/mapping/queries/dl2op/inverse/held-by-to-holds.rq

@@ -0,0 +1,8 @@
+PREFIX org: <http://www.w3.org/ns/org#>
+
+CONSTRUCT {
+  ?o org:holds ?s.
+}
+WHERE {
+  ?s org:heldBy ?o.
+}

config/delta-consumers/worship-services-main-info/mapping/queries/dl2op/inverse/holds-to-held-by.rq

@@ -0,0 +1,8 @@
+PREFIX org: <http://www.w3.org/ns/org#>
+
+CONSTRUCT {
+  ?o org:heldBy ?s.
+}
+WHERE {
+  ?s org:holds ?o.
+}

config/delta-consumers/worship-services-main-info/mapping/queries/dl2op/pass/address.sparql

@@ -0,0 +1,11 @@
+
+CONSTRUCT {
+  ?s ?p ?o.
+} WHERE {
+  ?s a <http://www.w3.org/ns/locn#Address>;
+    ?p ?o.
+
+  FILTER NOT EXISTS {
+    ?s ?p "" .
+  }
+}

config/delta-consumers/worship-services-main-info/mapping/queries/dl2op/pass/bestuursfunctie.sparql

@@ -0,0 +1,7 @@
+CONSTRUCT {
+  ?s <http://schema.org/contactPoint> ?contactpoint.
+}
+WHERE {
+  ?s a <http://data.lblod.info/vocabularies/leidinggevenden/Bestuursfunctie>.
+  ?s <http://schema.org/contactPoint> ?contactpoint.
+}

config/delta-consumers/worship-services-main-info/mapping/queries/dl2op/pass/contactpoint.sparql

@@ -0,0 +1,12 @@
+CONSTRUCT {
+  ?s ?p ?o.
+} WHERE {
+  ?s a <http://schema.org/ContactPoint>;
+    ?p ?o.
+
+ FILTER (?p NOT IN (<http://schema.org/telephone>))
+
+ FILTER NOT EXISTS {
+   ?s ?p "" .
+ }
+}

config/delta-consumers/worship-services-main-info/mapping/queries/dl2op/pass/eredienst-mandataris.sparql

@@ -0,0 +1,15 @@
+PREFIX ch: <http://data.lblod.info/vocabularies/contacthub/>
+PREFIX ere: <http://data.lblod.info/vocabularies/erediensten/>
+
+CONSTRUCT {
+  ?s ?p ?o.
+} WHERE {
+  ?s a <http://data.lblod.info/vocabularies/erediensten/EredienstMandataris>;
+    ?p ?o.
+
+ FILTER (?p NOT IN (ch:startdatum, ch:eindedatum, ere:geplandeEinddatumAanstelling))
+
+ FILTER NOT EXISTS {
+   ?s ?p "" .
+ }
+}

config/delta-consumers/worship-services-main-info/mapping/queries/dl2op/pass/person.sparql

@@ -0,0 +1,16 @@
+CONSTRUCT {
+  ?s ?p ?o.
+} WHERE {
+  ?s a <http://www.w3.org/ns/person#Person>;
+    ?p ?o.
+
+  FILTER (?p IN (
+    <http://www.w3.org/1999/02/22-rdf-syntax-ns#type>,
+    <http://mu.semte.ch/vocabularies/core/uuid>,
+    <http://xmlns.com/foaf/0.1/familyName>
+  ))
+
+  FILTER NOT EXISTS {
+   ?s ?p "" .
+  }
+}

config/delta-consumers/worship-services-main-info/mapping/queries/dl2op/pass/rol-bedienaar.sparql

@@ -0,0 +1,10 @@
+CONSTRUCT {
+  ?s ?p ?o.
+} WHERE {
+  ?s a <http://data.lblod.info/vocabularies/erediensten/RolBedienaar>;
+    ?p ?o.
+
+ FILTER NOT EXISTS {
+   ?s ?p "" .
+ }
+}

config/delta-consumers/worship-services-main-info/mapping/queries/dl2op/predicate/gebruikte-voornaam.rq

@@ -0,0 +1,14 @@
+PREFIX foaf: <http://xmlns.com/foaf/0.1/>
+PREFIX persoon: <https://data.vlaanderen.be/ns/persoon#>
+PREFIX person: <http://www.w3.org/ns/person#>
+
+# DL and OP both use persoon:gebruikteVoornaam but with different semantics (!)
+
+# Alternatieve voornaam: foaf:name => persoon:gebruikteVoornaam
+
+CONSTRUCT {
+  ?s persoon:gebruikteVoornaam ?name .
+} WHERE {
+  ?s a person:Person ;
+    foaf:name ?name .
+}

config/delta-consumers/worship-services-main-info/mapping/queries/dl2op/predicate/given-name.rq

@@ -0,0 +1,14 @@
+PREFIX foaf: <http://xmlns.com/foaf/0.1/>
+PREFIX persoon_loket: <http://data.vlaanderen.be/ns/persoon#>
+PREFIX person: <http://www.w3.org/ns/person#>
+
+# DL and OP both use persoon:gebruikteVoornaam but with different semantics (!)
+
+# First name: persoon:gebruikteVoornaam => foaf:givenName
+
+CONSTRUCT {
+  ?s foaf:givenName ?gebruikteVoornaam .
+} WHERE {
+  ?s a person:Person ;
+    persoon_loket:gebruikteVoornaam ?gebruikteVoornaam .
+}

config/delta-consumers/worship-services-private-info/mapping/queries/dl2op/pass/geboorte.sparql

@@ -0,0 +1,10 @@
+CONSTRUCT {
+  ?s ?p ?o.
+} WHERE {
+  ?s a <http://data.vlaanderen.be/ns/persoon#Geboorte>;
+    ?p ?o.
+
+ FILTER NOT EXISTS {
+   ?s ?p "" .
+ }
+}

config/delta-consumers/worship-services-private-info/mapping/queries/dl2op/persoon-http-to-https/Geboorte.rq

@@ -0,0 +1,10 @@
+PREFIX persoon: <https://data.vlaanderen.be/ns/persoon#>
+PREFIX persoon_loket: <http://data.vlaanderen.be/ns/persoon#>
+
+# Map from http persoon to https persoon namespace
+
+CONSTRUCT {
+  ?s a persoon:Geboorte
+} WHERE {
+  ?s a persoon_loket:Geboorte .
+}

config/delta-consumers/worship-services-private-info/mapping/queries/dl2op/persoon-http-to-https/datum.rq

@@ -0,0 +1,10 @@
+PREFIX persoon: <https://data.vlaanderen.be/ns/persoon#>
+PREFIX persoon_loket: <http://data.vlaanderen.be/ns/persoon#>
+
+# Map from http persoon to https persoon namespace
+
+CONSTRUCT {
+  ?s persoon:datum ?o .
+} WHERE {
+  ?s persoon_loket:datum ?o .
+}

config/delta-consumers/worship-services-private-info/mapping/queries/dl2op/persoon-http-to-https/geslacht.rq

@@ -0,0 +1,10 @@
+PREFIX persoon: <https://data.vlaanderen.be/ns/persoon#>
+PREFIX persoon_loket: <http://data.vlaanderen.be/ns/persoon#>
+
+# Map from http persoon to https persoon namespace
+
+CONSTRUCT {
+  ?s persoon:geslacht ?o .
+} WHERE {
+  ?s persoon_loket:geslacht ?o .
+}

config/delta-consumers/worship-services-private-info/mapping/queries/dl2op/persoon-http-to-https/heeftGeboorte.rq

@@ -0,0 +1,10 @@
+PREFIX persoon: <https://data.vlaanderen.be/ns/persoon#>
+PREFIX persoon_loket: <http://data.vlaanderen.be/ns/persoon#>
+
+# Map from http persoon to https persoon namespace
+
+CONSTRUCT {
+  ?s persoon:heeftGeboorte ?o .
+} WHERE {
+  ?s persoon_loket:heeftGeboorte ?o .
+}

config/delta-consumers/worship-services-private-info/mapping/queries/dl2op/persoon-http-to-https/heeftNationaliteit.rq

@@ -0,0 +1,10 @@
+PREFIX persoon: <https://data.vlaanderen.be/ns/persoon#>
+PREFIX persoon_loket: <http://data.vlaanderen.be/ns/persoon#>
+
+# Map from http persoon to https persoon namespace
+
+CONSTRUCT {
+  ?s persoon:heeftNationaliteit ?o .
+} WHERE {
+  ?s persoon_loket:heeftNationaliteit ?o .
+}

config/delta-consumers/worship-services-private-info/mapping/queries/dl2op/persoon-http-to-https/registratie.rq

@@ -0,0 +1,10 @@
+PREFIX persoon: <https://data.vlaanderen.be/ns/persoon#>
+PREFIX persoon_loket: <http://data.vlaanderen.be/ns/persoon#>
+
+# Map from http persoon to https persoon namespace
+
+CONSTRUCT {
+  ?s persoon:registratie ?o .
+} WHERE {
+  ?s persoon_loket:registratie ?o .
+}

config/delta-consumers/worship-services-private-info/mapping/queries/dl2op/rrn/identifier.rq

@@ -0,0 +1,19 @@
+PREFIX generiek: <https://data.vlaanderen.be/ns/generiek#>
+PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
+PREFIX mu: <http://mu.semte.ch/vocabularies/core/>
+
+CONSTRUCT {
+  ?identifier
+    generiek:gestructureerdeIdentificator ?structuredIdentifier ;
+    skos:notation "Rijksregisternummer" .
+
+  ?structuredIdentifier a generiek:GestructureerdeIdentificator ;
+    mu:uuid ?uuid ;
+    generiek:lokaleIdentificator ?notation .
+}
+WHERE {
+    ?identifier skos:notation ?notation.
+
+    BIND(MD5(CONCAT(STR(?identifier),"STRUCTURED_IDENTIFIER")) as ?uuid)
+    BIND(IRI(CONCAT("http://data.lblod.info/id/gestructureerdeIdentificatoren/", ?uuid)) AS ?structuredIdentifier)
+}

config/delta-consumers/worship-services-private-info/mapping/queries/dl2op/rrn/registratie.rq

@@ -0,0 +1,10 @@
+PREFIX adms: <http://www.w3.org/ns/adms#>
+PREFIX persoon: <https://data.vlaanderen.be/ns/persoon#>
+
+# identifier property in the export is only used for RRN.
+CONSTRUCT {
+  ?person persoon:registratie ?rrn .
+}
+WHERE {
+  ?person adms:identifier ?rrn .
+}

config/delta-producer/background-job-initiator/config.json

@@ -1,12 +1,32 @@
 [
-    {
-        "name": "organizations",
-        "jobsGraph": "http://mu.semte.ch/graphs/jobs",
-        "dumpFileCreationJobOperation": "http://redpencil.data.gift/id/jobs/concept/JobOperation/deltas/deltaDumpFileCreation/organizations",
-        "initialPublicationGraphSyncJobOperation": "http://redpencil.data.gift/id/jobs/concept/JobOperation/deltas/initialPublicationGraphSyncing/organizations",
-        "healingJobOperation": "http://redpencil.data.gift/id/jobs/concept/JobOperation/deltas/healingOperation/organizations",
-        "cronPatternDumpJob": "42 * * * *",
-        "cronPatternHealingJob": "0 0 4 * * *",
-        "startInitialSync": true
-    }
-]
+  {
+    "name": "organizations",
+    "jobsGraph": "http://mu.semte.ch/graphs/jobs",
+    "dumpFileCreationJobOperation": "http://redpencil.data.gift/id/jobs/concept/JobOperation/deltas/deltaDumpFileCreation/organizations",
+    "initialPublicationGraphSyncJobOperation": "http://redpencil.data.gift/id/jobs/concept/JobOperation/deltas/initialPublicationGraphSyncing/organizations",
+    "healingJobOperation": "http://redpencil.data.gift/id/jobs/concept/JobOperation/deltas/healingOperation/organizations",
+    "cronPatternDumpJob": "42 * * * *",
+    "cronPatternHealingJob": "0 0 4 * * *",
+    "startInitialSync": true
+  },
+  {
+    "name": "public",
+    "jobsGraph": "http://mu.semte.ch/graphs/jobs",
+    "dumpFileCreationJobOperation": "http://redpencil.data.gift/id/jobs/concept/JobOperation/deltas/deltaDumpFileCreation/public",
+    "initialPublicationGraphSyncJobOperation": "http://redpencil.data.gift/id/jobs/concept/JobOperation/deltas/initialPublicationGraphSyncing/public",
+    "healingJobOperation": "http://redpencil.data.gift/id/jobs/concept/JobOperation/deltas/healingOperation/public",
+    "cronPatternDumpJob": "11 */4 * * *",
+    "cronPatternHealingJob": "0 0 4 * * *",
+    "startInitialSync": true
+  },
+  {
+    "name": "worship-posts",
+    "jobsGraph": "http://mu.semte.ch/graphs/jobs",
+    "dumpFileCreationJobOperation": "http://redpencil.data.gift/id/jobs/concept/JobOperation/deltas/deltaDumpFileCreation/worship-posts",
+    "initialPublicationGraphSyncJobOperation": "http://redpencil.data.gift/id/jobs/concept/JobOperation/deltas/initialPublicationGraphSyncing/worship-posts",
+    "healingJobOperation": "http://redpencil.data.gift/id/jobs/concept/JobOperation/deltas/healingOperation/worship-posts",
+    "cronPatternDumpJob": "17 */4 * * *",
+    "cronPatternHealingJob": "0 0 3 * * *",
+    "startInitialSync": true
+  }
+]

config/delta-producer/dump-file-publisher/config.json

@@ -17,29 +17,11 @@
     "targetFilesGraph": "http://redpencil.data.gift/id/deltas/producer/organizations-deltas",
     "cleanupOldDumps": true
   },
-  "http://redpencil.data.gift/id/jobs/concept/JobOperation/deltas/deltaDumpFileCreation/organizations-public-info": {
-    "dcatDataSetSubject": "http://data.lblod.info/datasets/delta-producer/dumps/OrganizationsPublicInfoCacheGraphDump",
-    "targetGraph": "http://redpencil.data.gift/id/deltas/producer/organizations-public-info",
-    "fileBaseName": "dump-organizations-public-info",
-    "publicationGraphEndpoint": "http://publication-triplestore:8890/sparql",
-    "targetDcatGraph": "http://mu.semte.ch/graphs/public",
-    "targetFilesGraph": "http://mu.semte.ch/graphs/public",
-    "cleanupOldDumps": true
-  },
-  "http://redpencil.data.gift/id/jobs/concept/JobOperation/deltas/initialPublicationGraphSyncing/organizations-public-info": {
-    "dcatDataSetSubject": "http://data.lblod.info/datasets/delta-producer/dumps/OrganizationsPublicInfoCacheGraphDump",
-    "targetGraph": "http://redpencil.data.gift/id/deltas/producer/organizations-public-info",
-    "fileBaseName": "dump-organizations-public-info",
-    "publicationGraphEndpoint": "http://publication-triplestore:8890/sparql",
-    "targetDcatGraph": "http://mu.semte.ch/graphs/public",
-    "targetFilesGraph": "http://mu.semte.ch/graphs/public",
-    "cleanupOldDumps": true
-  },
   "http://redpencil.data.gift/id/jobs/concept/JobOperation/deltas/deltaDumpFileCreation/public": {
     "dcatDataSetSubject": "http://data.lblod.info/datasets/delta-producer/dumps/PublicCacheGraphDump",
     "targetGraph": "http://redpencil.data.gift/id/deltas/producer/public",
     "fileBaseName": "dump-public",
-    "publicationGraphEndpoint": "http://triplestore:8890/sparql",
+    "publicationGraphEndpoint": "http://publication-triplestore:8890/sparql",
     "targetDcatGraph": "http://mu.semte.ch/graphs/public",
     "targetFilesGraph": "http://mu.semte.ch/graphs/public",
     "cleanupOldDumps": true
@@ -48,7 +30,7 @@
     "dcatDataSetSubject": "http://data.lblod.info/datasets/delta-producer/dumps/PublicCacheGraphDump",
     "targetGraph": "http://redpencil.data.gift/id/deltas/producer/public",
     "fileBaseName": "dump-public",
-    "publicationGraphEndpoint": "http://triplestore:8890/sparql",
+    "publicationGraphEndpoint": "http://publication-triplestore:8890/sparql",
     "targetDcatGraph": "http://mu.semte.ch/graphs/public",
     "targetFilesGraph": "http://mu.semte.ch/graphs/public",
     "cleanupOldDumps": true
@@ -57,7 +39,7 @@
     "dcatDataSetSubject": "http://data.lblod.info/datasets/delta-producer/dumps/WorshipPostsCacheGraphDump",
     "targetGraph": "http://redpencil.data.gift/id/deltas/producer/worship-posts",
     "fileBaseName": "dump-worship-posts",
-    "publicationGraphEndpoint": "http://triplestore:8890/sparql",
+    "publicationGraphEndpoint": "http://publication-triplestore:8890/sparql",
     "targetDcatGraph": "http://mu.semte.ch/graphs/public",
     "targetFilesGraph": "http://mu.semte.ch/graphs/public",
     "cleanupOldDumps": true
@@ -66,7 +48,7 @@
     "dcatDataSetSubject": "http://data.lblod.info/datasets/delta-producer/dumps/WorshipPostsCacheGraphDump",
     "targetGraph": "http://redpencil.data.gift/id/deltas/producer/worship-posts",
     "fileBaseName": "dump-worship-posts",
-    "publicationGraphEndpoint": "http://triplestore:8890/sparql",
+    "publicationGraphEndpoint": "http://publication-triplestore:8890/sparql",
     "targetDcatGraph": "http://mu.semte.ch/graphs/public",
     "targetFilesGraph": "http://mu.semte.ch/graphs/public",
     "cleanupOldDumps": true

config/delta-producer/organizations-public-info/export.json

@@ -205,6 +205,16 @@
         "http://www.w3.org/2004/02/skos/core#prefLabel"
       ]
     },
+    {
+      "type": "http://mu.semte.ch/vocabularies/ext/RepresentatiefOrgaanClassificatieCode",
+      "graphsFilter": [
+        "http://mu.semte.ch/graphs/public"
+      ],
+      "properties": [
+        "http://mu.semte.ch/vocabularies/core/uuid",
+        "http://www.w3.org/2004/02/skos/core#prefLabel"
+      ]
+    },
     {
       "type": "http://data.vlaanderen.be/ns/besluit#Bestuursorgaan",
       "graphsFilter": [

config/delta-producer/organizations/export.json

@@ -228,7 +228,8 @@
         "http://www.w3.org/ns/org#hasPost",
         "http://www.w3.org/ns/org#linkedTo",
         "http://www.w3.org/ns/org#hasSubOrganization",
-        "http://data.lblod.info/vocabularies/erediensten/typeEredienst"
+        "http://data.lblod.info/vocabularies/erediensten/typeEredienst",
+        "http://www.w3.org/ns/org#classification"
       ]
     },
     {
@@ -273,7 +274,8 @@
         "http://data.vlaanderen.be/ns/besluit#bestuurt",
         "http://www.w3.org/ns/org#classification",
         "https://data.vlaanderen.be/ns/generiek#isTijdspecialisatieVan",
-        "http://www.w3.org/ns/org#hasPost"
+        "http://www.w3.org/ns/org#hasPost",
+        "http://data.lblod.info/vocabularies/leidinggevenden/heeftBestuursfunctie"
       ]
     },
     {
@@ -334,6 +336,7 @@
         "http://www.w3.org/ns/locn#postCode",
         "https://data.vlaanderen.be/ns/adres#gemeentenaam",
         "http://www.w3.org/ns/locn#adminUnitL2",
+        "https://data.vlaanderen.be/ns/adres#adresIsGelegenIn",
         "https://data.vlaanderen.be/ns/adres#land",
         "http://www.w3.org/ns/locn#fullAddress",
         "https://data.vlaanderen.be/ns/adres#verwijstNaar",
@@ -387,7 +390,7 @@
     {
       "type": "http://www.w3.org/ns/prov#Location",
       "graphsFilter": [
-        "http://mu.semte.ch/graphs/shared"
+        "http://mu.semte.ch/graphs/public"
       ],
       "properties": [
         "http://www.w3.org/2000/01/rdf-schema#label",
@@ -509,6 +512,32 @@
       "properties": [
         "http://www.w3.org/2004/02/skos/core#prefLabel"
       ]
+    },
+    {
+      "type": "http://www.w3.org/ns/org#Membership",
+      "graphsFilter": [
+        "http://mu.semte.ch/graphs/worship-service",
+        "http://mu.semte.ch/graphs/shared",
+        "http://mu.semte.ch/graphs/administrative-unit"
+      ],
+      "properties": [
+        "http://mu.semte.ch/vocabularies/core/uuid",
+        "http://www.w3.org/ns/org#organization",
+        "http://www.w3.org/ns/org#member",
+        "http://www.w3.org/ns/org#role"
+      ]
+    },
+    {
+      "type": "http://www.w3.org/ns/org#Role",
+      "graphsFilter": [
+        "http://mu.semte.ch/graphs/public"
+      ],
+      "properties": [
+        "http://mu.semte.ch/vocabularies/core/uuid",
+        "http://www.w3.org/2004/02/skos/core#prefLabel",
+        "http://www.w3.org/2004/02/skos/core#broader",
+        "http://www.w3.org/2004/02/skos/core#topConceptOf"
+      ]
     }
   ]
 }

config/delta-producer/public/export.json

@@ -99,6 +99,21 @@
         "http://www.w3.org/ns/regorg#orgStatus"
       ]
     },
+    {
+      "type": "http://data.vlaanderen.be/ns/besluit#Bestuurseenheid",
+      "strictTypeExport": "true",
+      "graphsFilter": [
+        "http://mu.semte.ch/graphs/public"
+      ],
+      "comment": "Bound to ABB only.",
+      "properties": [
+        "http://mu.semte.ch/vocabularies/core/uuid",
+        "http://www.w3.org/2004/02/skos/core#prefLabel",
+        "http://purl.org/dc/terms/identifier",
+        "http://www.w3.org/ns/org#classification"
+      ],
+      "additionalFilter": "BIND(<http://data.lblod.info/id/bestuurseenheden/141d9d6b-54af-4d17-b313-8d1c30bc3f5b> as ?subject)"
+    },
     {
       "type": "http://www.w3.org/ns/regorg#RegisteredOrganization",
       "strictTypeExport": "true",
@@ -142,8 +157,12 @@
         "http://www.w3.org/2004/02/skos/core#prefLabel",
         "http://www.w3.org/ns/regorg#legalName",
         "http://www.w3.org/2004/02/skos/core#altLabel",
+        "http://www.w3.org/ns/regorg#orgStatus",
+        "http://www.w3.org/ns/org#hasPrimarySite",
+        "http://www.w3.org/ns/org#hasSite",
         "http://www.w3.org/ns/org#linkedTo",
         "http://data.lblod.info/vocabularies/erediensten/typeEredienst",
+        "http://www.w3.org/ns/org#classification",
         "http://www.w3.org/ns/adms#identifier"
       ]
     },
@@ -188,7 +207,28 @@
         "http://mu.semte.ch/vocabularies/core/uuid",
         "http://www.w3.org/ns/org#siteAddress",
         "https://data.vlaanderen.be/ns/organisatie#bestaatUit",
-        "http://data.lblod.info/vocabularies/erediensten/vestigingstype"
+        "http://data.lblod.info/vocabularies/erediensten/vestigingstype",
+        "http://purl.org/dc/terms/description"
+      ]
+    },
+    {
+      "type": "http://www.w3.org/ns/org#Site",
+      "graphsFilter": [
+        "http://mu.semte.ch/graphs/shared",
+        "http://mu.semte.ch/graphs/worship-service",
+        "http://mu.semte.ch/graphs/administrative-unit"
+      ],
+      "pathToConceptScheme": [
+        "^http://www.w3.org/ns/org#hasSite",
+        "http://www.w3.org/ns/org#classification",
+        "http://www.w3.org/2004/02/skos/core#inScheme"
+      ],
+      "properties": [
+        "http://mu.semte.ch/vocabularies/core/uuid",
+        "http://www.w3.org/ns/org#siteAddress",
+        "https://data.vlaanderen.be/ns/organisatie#bestaatUit",
+        "http://data.lblod.info/vocabularies/erediensten/vestigingstype",
+        "http://purl.org/dc/terms/description"
       ]
     },
     {
@@ -212,6 +252,35 @@
         "http://www.w3.org/ns/locn#postCode",
         "https://data.vlaanderen.be/ns/adres#gemeentenaam",
         "http://www.w3.org/ns/locn#adminUnitL2",
+        "https://data.vlaanderen.be/ns/adres#adresIsGelegenIn",
+        "https://data.vlaanderen.be/ns/adres#land",
+        "http://www.w3.org/ns/locn#fullAddress",
+        "https://data.vlaanderen.be/ns/adres#verwijstNaar",
+        "http://purl.org/dc/terms/source"
+      ]
+    },
+    {
+      "type": "http://www.w3.org/ns/locn#Address",
+      "graphsFilter": [
+        "http://mu.semte.ch/graphs/shared",
+        "http://mu.semte.ch/graphs/worship-service",
+        "http://mu.semte.ch/graphs/administrative-unit"
+      ],
+      "pathToConceptScheme": [
+        "^https://data.vlaanderen.be/ns/organisatie#bestaatUit",
+        "^http://www.w3.org/ns/org#hasSite",
+        "http://www.w3.org/ns/org#classification",
+        "http://www.w3.org/2004/02/skos/core#inScheme"
+      ],
+      "properties": [
+        "http://mu.semte.ch/vocabularies/core/uuid",
+        "https://data.vlaanderen.be/ns/adres#Adresvoorstelling.huisnummer",
+        "https://data.vlaanderen.be/ns/adres#Adresvoorstelling.busnummer",
+        "http://www.w3.org/ns/locn#thoroughfare",
+        "http://www.w3.org/ns/locn#postCode",
+        "https://data.vlaanderen.be/ns/adres#gemeentenaam",
+        "http://www.w3.org/ns/locn#adminUnitL2",
+        "https://data.vlaanderen.be/ns/adres#adresIsGelegenIn",
         "https://data.vlaanderen.be/ns/adres#land",
         "http://www.w3.org/ns/locn#fullAddress",
         "https://data.vlaanderen.be/ns/adres#verwijstNaar",
@@ -242,6 +311,30 @@
         "http://schema.org/telephone"
       ]
     },
+    {
+      "type": "http://schema.org/ContactPoint",
+      "graphsFilter": [
+        "http://mu.semte.ch/graphs/shared",
+        "http://mu.semte.ch/graphs/worship-service",
+        "http://mu.semte.ch/graphs/administrative-unit"
+      ],
+      "pathToConceptScheme": [
+        "^http://www.w3.org/ns/org#siteAddress",
+        "^http://www.w3.org/ns/org#hasSite",
+        "http://www.w3.org/ns/org#classification",
+        "http://www.w3.org/2004/02/skos/core#inScheme"
+      ],
+      "properties": [
+        "http://mu.semte.ch/vocabularies/core/uuid",
+        "http://www.w3.org/ns/locn#address",
+        "http://xmlns.com/foaf/0.1/page",
+        "http://schema.org/faxNumber",
+        "http://schema.org/contactType",
+        "http://xmlns.com/foaf/0.1/page",
+        "http://schema.org/email",
+        "http://schema.org/telephone"
+      ]
+    },
     {
       "type": "http://www.w3.org/ns/adms#Identifier",
       "graphsFilter": [
@@ -314,6 +407,16 @@
         "http://www.w3.org/2004/02/skos/core#prefLabel"
       ]
     },
+    {
+      "type": "http://mu.semte.ch/vocabularies/ext/RepresentatiefOrgaanClassificatieCode",
+      "graphsFilter": [
+        "http://mu.semte.ch/graphs/public"
+      ],
+      "properties": [
+        "http://mu.semte.ch/vocabularies/core/uuid",
+        "http://www.w3.org/2004/02/skos/core#prefLabel"
+      ]
+    },
     {
       "type": "http://data.vlaanderen.be/ns/besluit#Bestuursorgaan",
       "graphsFilter": [
@@ -328,7 +431,8 @@
         "http://data.vlaanderen.be/ns/mandaat#bindingEinde",
         "http://data.vlaanderen.be/ns/mandaat#bindingStart",
         "https://data.vlaanderen.be/ns/generiek#isTijdspecialisatieVan",
-        "http://www.w3.org/ns/org#hasPost"
+        "http://www.w3.org/ns/org#hasPost",
+        "http://data.lblod.info/vocabularies/leidinggevenden/heeftBestuursfunctie"
       ],
       "additionalFilter": "?subject (<https://data.vlaanderen.be/ns/generiek#isTijdspecialisatieVan>/<http://www.w3.org/ns/org#classification>/<http://www.w3.org/2004/02/skos/core#inScheme>)|(<http://www.w3.org/ns/org#classification>/<http://www.w3.org/2004/02/skos/core#inScheme>) <http://lblod.data.gift/concept-schemes/10caf4a4-1ff5-4c05-917e-56489910d68b> ."
     },
@@ -421,6 +525,32 @@
         "http://www.w3.org/2004/02/skos/core#prefLabel"
       ]
     },
+    {
+      "type": "http://lblod.data.gift/vocabularies/organisatie/EredienstBeroepen",
+      "graphsFilter": [
+        "http://mu.semte.ch/graphs/public"
+      ],
+      "pathToConceptScheme": [
+        "http://www.w3.org/2004/02/skos/core#inScheme"
+      ],
+      "properties": [
+        "http://mu.semte.ch/vocabularies/core/uuid",
+        "http://www.w3.org/2004/02/skos/core#prefLabel"
+      ]
+    },
+    {
+      "type": "http://lblod.data.gift/vocabularies/organisatie/TypeBetrokkenheid",
+      "graphsFilter": [
+        "http://mu.semte.ch/graphs/public"
+      ],
+      "pathToConceptScheme": [
+        "http://www.w3.org/2004/02/skos/core#inScheme"
+      ],
+      "properties": [
+        "http://mu.semte.ch/vocabularies/core/uuid",
+        "http://www.w3.org/2004/02/skos/core#prefLabel"
+      ]
+    },
     {
       "type": "http://data.vlaanderen.be/ns/mandaat#Mandaat",
       "graphsFilter": [
@@ -436,7 +566,8 @@
       ],
       "properties": [
         "http://mu.semte.ch/vocabularies/core/uuid",
-        "http://www.w3.org/ns/org#role"
+        "http://www.w3.org/ns/org#role",
+        "http://data.vlaanderen.be/ns/mandaat#aantalHouders"
       ]
     },
     {
@@ -457,6 +588,146 @@
         "http://www.w3.org/ns/org#role",
         "http://www.w3.org/2004/02/skos/core#prefLabel"
       ]
+    },
+    {
+      "type": "http://mu.semte.ch/vocabularies/ext/GeregistreerdeOrganisatieClassificatieCode",
+      "graphsFilter": [
+        "http://mu.semte.ch/graphs/public"
+      ],
+      "pathToConceptScheme": [
+        "http://www.w3.org/2004/02/skos/core#inScheme"
+      ],
+      "properties": [
+        "http://mu.semte.ch/vocabularies/core/uuid",
+        "http://www.w3.org/2004/02/skos/core#prefLabel"
+      ]
+    },
+    {
+      "type": "http://lblod.data.gift/vocabularies/organisatie/MandatarisStatusCode",
+      "graphsFilter": [
+        "http://mu.semte.ch/graphs/public"
+      ],
+      "pathToConceptScheme": [
+        "http://www.w3.org/2004/02/skos/core#inScheme"
+      ],
+      "properties": [
+        "http://mu.semte.ch/vocabularies/core/uuid",
+        "http://www.w3.org/2004/02/skos/core#prefLabel"
+      ]
+    },
+    {
+      "type": "http://lblod.data.gift/vocabularies/organisatie/BestuursfunctieCode",
+      "graphsFilter": [
+        "http://mu.semte.ch/graphs/public"
+      ],
+      "pathToConceptScheme": [
+        "http://www.w3.org/2004/02/skos/core#inScheme"
+      ],
+      "properties": [
+        "http://mu.semte.ch/vocabularies/core/uuid",
+        "http://www.w3.org/2004/02/skos/core#prefLabel"
+      ]
+    },
+    {
+      "type": "http://data.lblod.info/vocabularies/leidinggevenden/FunctionarisStatusCode",
+      "graphsFilter": [
+        "http://mu.semte.ch/graphs/public"
+      ],
+      "pathToConceptScheme": [
+        "http://www.w3.org/2004/02/skos/core#inScheme"
+      ],
+      "properties": [
+        "http://mu.semte.ch/vocabularies/core/uuid",
+        "http://www.w3.org/2004/02/skos/core#prefLabel"
+      ]
+    },
+    {
+      "type": "http://mu.semte.ch/vocabularies/ext/OrganizationClassificationCode",
+      "graphsFilter": [
+        "http://mu.semte.ch/graphs/public"
+      ],
+      "pathToConceptScheme": [
+        "http://www.w3.org/2004/02/skos/core#inScheme"
+      ],
+      "properties": [
+        "http://mu.semte.ch/vocabularies/core/uuid",
+        "http://www.w3.org/2004/02/skos/core#prefLabel"
+      ]
+    },
+    {
+      "type": "http://www.w3.org/ns/prov#Location",
+      "graphsFilter": [
+        "http://mu.semte.ch/graphs/public"
+      ],
+      "hasRegexGraphsFilter": false,
+      "properties": [
+        "http://www.w3.org/2000/01/rdf-schema#label",
+        "http://mu.semte.ch/vocabularies/core/uuid",
+        "http://mu.semte.ch/vocabularies/ext/werkingsgebiedNiveau",
+        "http://www.opengis.net/ont/geosparql#sfWithin",
+        "http://www.w3.org/2004/02/skos/core#exactMatch"
+      ]
+    },
+    {
+      "type": "http://www.w3.org/2004/02/skos/core#Concept",
+      "graphsFilter": [
+        "http://mu.semte.ch/graphs/public"
+      ],
+      "pathToConceptScheme": [
+        "http://www.w3.org/2004/02/skos/core#inScheme"
+      ],
+      "properties": [
+        "http://mu.semte.ch/vocabularies/core/uuid",
+        "http://www.w3.org/2004/02/skos/core#altLabel",
+        "http://www.w3.org/2004/02/skos/core#broadMatch",
+        "http://www.w3.org/2004/02/skos/core#broader",
+        "http://www.w3.org/2004/02/skos/core#exactMatch",
+        "http://www.w3.org/2004/02/skos/core#inScheme",
+        "http://www.w3.org/2004/02/skos/core#narrowMatch",
+        "http://www.w3.org/2004/02/skos/core#narrower",
+        "http://www.w3.org/2004/02/skos/core#notation",
+        "http://www.w3.org/2004/02/skos/core#prefLabel",
+        "http://www.w3.org/2004/02/skos/core#relatedMatch",
+        "http://www.w3.org/2004/02/skos/core#topConceptOf",
+        "https://schema.org/endDate",
+        "https://schema.org/startDate"
+      ]
+    },
+    {
+      "type": "http://publications.europa.eu/ontology/euvoc#Country",
+      "graphsFilter": [
+        "http://mu.semte.ch/graphs/public"
+      ],
+      "properties": [
+        "http://www.w3.org/2004/02/skos/core#prefLabel",
+        "http://www.w3.org/2000/01/rdf-schema#label"
+      ]
+    },
+    {
+      "type": "http://www.w3.org/ns/org#Membership",
+      "graphsFilter": [
+        "http://mu.semte.ch/graphs/worship-service",
+        "http://mu.semte.ch/graphs/shared",
+        "http://mu.semte.ch/graphs/administrative-unit"
+      ],
+      "properties": [
+        "http://mu.semte.ch/vocabularies/core/uuid",
+        "http://www.w3.org/ns/org#organization",
+        "http://www.w3.org/ns/org#member",
+        "http://www.w3.org/ns/org#role"
+      ]
+    },
+    {
+      "type": "http://www.w3.org/ns/org#Role",
+      "graphsFilter": [
+        "http://mu.semte.ch/graphs/public"
+      ],
+      "properties": [
+        "http://mu.semte.ch/vocabularies/core/uuid",
+        "http://www.w3.org/2004/02/skos/core#prefLabel",
+        "http://www.w3.org/2004/02/skos/core#broader",
+        "http://www.w3.org/2004/02/skos/core#topConceptOf"
+      ]
     }
   ]
 }

config/delta-producer/publication-graph-maintainer/config.json

@@ -1,25 +1,70 @@
 {
-    "organizations": {
-        "deltaInterval": 10000,
-        "deltaPath": "/organizations/delta",
-        "errorCreatorUri": "http://lblod.data.gift/services/delta-producer-publication-graph-maintainer-organizations",
-        "exportConfigPath": "/config/organizations/export.json",
-        "filesPath": "/organizations/files",
-        "healingJobOperation": "http://redpencil.data.gift/id/jobs/concept/JobOperation/deltas/healingOperation/organizations",
-        "initialPublicationGraphSyncJobOperation": "http://redpencil.data.gift/id/jobs/concept/JobOperation/deltas/initialPublicationGraphSyncing/organizations",
-        "jobsGraph": "http://mu.semte.ch/graphs/jobs",
-        "publicationGraph": "http://redpencil.data.gift/id/deltas/producer/organizations",
-        "publisherUri": "http://data.lblod.info/services/delta-production-json-diff-file-manager/organizations",
-        "queuePollInterval": 3000,
-        "relativeFilePath": "/deltas/organizations",
-        "serveDeltaFiles": true,
-        "skipMuAuthInitialSync": true,
-        "skipMuAuthHealing": true,
-        "useFileDiff": true,
-        "healingInitialBatchSizeInsert": 1000,
-        "useVirtuosoForExpensiveSelects": true,
-        "errorGraph": "http://mu.semte.ch/graphs/jobs",
-        "loginPath": "/organizations/login",
-        "logOutgoingDelta": true
-    }
-}
+  "organizations": {
+    "deltaInterval": 10000,
+    "errorCreatorUri": "http://lblod.data.gift/services/delta-producer-publication-graph-maintainer-organizations",
+    "exportConfigPath": "/config/organizations/export.json",
+    "filesPath": "/organizations/files",
+    "healingJobOperation": "http://redpencil.data.gift/id/jobs/concept/JobOperation/deltas/healingOperation/organizations",
+    "initialPublicationGraphSyncJobOperation": "http://redpencil.data.gift/id/jobs/concept/JobOperation/deltas/initialPublicationGraphSyncing/organizations",
+    "jobsGraph": "http://mu.semte.ch/graphs/jobs",
+    "publicationGraph": "http://redpencil.data.gift/id/deltas/producer/organizations",
+    "publisherUri": "http://data.lblod.info/services/delta-production-json-diff-file-manager/organizations",
+    "queuePollInterval": 3000,
+    "relativeFilePath": "/deltas/organizations",
+    "serveDeltaFiles": true,
+    "skipMuAuthInitialSync": true,
+    "skipMuAuthHealing": true,
+    "useFileDiff": true,
+    "healingInitialBatchSizeInsert": 1000,
+    "useVirtuosoForExpensiveSelects": true,
+    "errorGraph": "http://mu.semte.ch/graphs/jobs",
+    "loginPath": "/organizations/login",
+    "logOutgoingDelta": true
+  },
+  "public": {
+    "deltaInterval": 10000,
+    "errorCreatorUri": "http://lblod.data.gift/services/delta-producer-publication-graph-maintainer-public",
+    "exportConfigPath": "/config/public/export.json",
+    "filesPath": "/public/files",
+    "healingJobOperation": "http://redpencil.data.gift/id/jobs/concept/JobOperation/deltas/healingOperation/public",
+    "initialPublicationGraphSyncJobOperation": "http://redpencil.data.gift/id/jobs/concept/JobOperation/deltas/initialPublicationGraphSyncing/public",
+    "jobsGraph": "http://mu.semte.ch/graphs/jobs",
+    "publicationGraph": "http://redpencil.data.gift/id/deltas/producer/public",
+    "publisherUri": "http://data.lblod.info/services/delta-production-json-diff-file-manager/public",
+    "queuePollInterval": 3000,
+    "relativeFilePath": "/deltas/public",
+    "serveDeltaFiles": true,
+    "skipMuAuthInitialSync": true,
+    "skipMuAuthHealing": true,
+    "useFileDiff": true,
+    "healingInitialBatchSizeInsert": 1000,
+    "healingPatchPublicationGraphTaskOperation": 500,
+    "useVirtuosoForExpensiveSelects": true,
+    "errorGraph": "http://mu.semte.ch/graphs/jobs",
+    "loginPath": "/public/login",
+    "logOutgoingDelta": true
+  },
+  "worship-posts": {
+    "deltaInterval": 10000,
+    "errorCreatorUri": "http://lblod.data.gift/services/delta-producer-publication-graph-maintainer-worship-posts",
+    "exportConfigPath": "/config/worship-posts/export.json",
+    "filesPath": "/worship-posts/files",
+    "healingJobOperation": "http://redpencil.data.gift/id/jobs/concept/JobOperation/deltas/healingOperation/worship-posts",
+    "initialPublicationGraphSyncJobOperation": "http://redpencil.data.gift/id/jobs/concept/JobOperation/deltas/initialPublicationGraphSyncing/worship-posts",
+    "jobsGraph": "http://mu.semte.ch/graphs/jobs",
+    "publicationGraph": "http://redpencil.data.gift/id/deltas/producer/worship-posts",
+    "publisherUri": "http://data.lblod.info/services/delta-production-json-diff-file-manager/worship-posts",
+    "queuePollInterval": 3000,
+    "relativeFilePath": "/deltas/worship-posts",
+    "serveDeltaFiles": true,
+    "skipMuAuthInitialSync": true,
+    "skipMuAuthHealing": true,
+    "useFileDiff": true,
+    "healingInitialBatchSizeInsert": 1000,
+    "healingPatchPublicationGraphTaskOperation": 500,
+    "useVirtuosoForExpensiveSelects": true,
+    "errorGraph": "http://mu.semte.ch/graphs/jobs",
+    "loginPath": "/worship-posts/login",
+    "logOutgoingDelta": true
+  }
+}

config/delta-producer/worship-posts/export.json

@@ -131,6 +131,7 @@
         "http://www.w3.org/ns/org#hasSite",
         "http://www.w3.org/ns/org#linkedTo",
         "http://data.lblod.info/vocabularies/erediensten/typeEredienst",
+        "http://www.w3.org/ns/org#classification",
         "http://www.w3.org/ns/adms#identifier"
       ]
     },

config/delta/rules.js

@@ -33,7 +33,6 @@ export default [
       ignoreFromSelf: true
     }
   },
-
   {
     match: {
       graph: {
@@ -51,7 +50,6 @@ export default [
       ignoreFromSelf: true
     }
   },
-
   {
     match: {
       graph: {
@@ -69,7 +67,6 @@ export default [
       ignoreFromSelf: true
     }
   },
-
   {
     match: {
       predicate: {
@@ -92,59 +89,7 @@ export default [
     match: {
     },
     callback: {
-      url: 'http://delta-producer-publication-graph-maintainer/organizations/delta',
-      method: 'POST'
-    },
-    options: {
-      resourceFormat: 'v0.0.1',
-      gracePeriod: 1000,
-      ignoreFromSelf: true,
-      optOutMuScopeIds: [
-        "http://redpencil.data.gift/id/concept/muScope/deltas/initialSync",
-        "http://redpencil.data.gift/id/concept/muScope/deltas/publicationGraphMaintenance"
-      ]
-    }
-  },
-  {
-    match: {
-    },
-    callback: {
-      url: 'http://delta-producer-pub-graph-maintainer-organizations-public-info/delta',
-      method: 'POST'
-    },
-    options: {
-      resourceFormat: 'v0.0.1',
-      gracePeriod: 1000,
-      ignoreFromSelf: true,
-      optOutMuScopeIds: [
-        "http://redpencil.data.gift/id/concept/muScope/deltas/initialSync",
-        "http://redpencil.data.gift/id/concept/muScope/deltas/publicationGraphMaintenance"
-      ]
-    }
-  },
-  {
-    match: {
-    },
-    callback: {
-      url: 'http://delta-producer-pub-graph-maintainer-public/delta',
-      method: 'POST'
-    },
-    options: {
-      resourceFormat: 'v0.0.1',
-      gracePeriod: 1000,
-      ignoreFromSelf: true,
-      optOutMuScopeIds: [
-        "http://redpencil.data.gift/id/concept/muScope/deltas/initialSync",
-        "http://redpencil.data.gift/id/concept/muScope/deltas/publicationGraphMaintenance"
-      ]
-    }
-  },
-  {
-    match: {
-      // anything
-    },
-    callback: {
-      url: 'http://delta-producer-pub-graph-maintainer-worship-posts/delta',
+      url: 'http://delta-producer-publication-graph-maintainer/delta',
       method: 'POST'
     },
     options: {
@@ -233,6 +178,27 @@ export default [
       ignoreFromSelf: true,
       optOutMuScopeIds: [ "http://redpencil.data.gift/id/concept/muScope/deltas/initialSync" ]
     }
+  },
+  {
+    match: {
+      predicate: {
+        type: 'uri',
+        value: 'http://www.w3.org/1999/02/22-rdf-syntax-ns#type'
+      },
+      object: {
+        type: 'uri',
+        value:'http://open-services.net/ns/core#Error'
+      }
+    },
+    callback: {
+      url: 'http://error-alert/delta',
+      method:'POST'
+    },
+    options: {
+      resourceFormat: 'v0.0.1',
+      gracePeriod: 1000,
+      ignoreFromSelf: true
+    }
   }
 /*
   Commenting ldes rules until ready to go to qa/prod. In the meantime, we override the configuration on dev.

config/dev-migrations/20241003152155-remove-invalid-generic-memberships.sparql

@@ -0,0 +1,31 @@
+PREFIX org: <http://www.w3.org/ns/org#>
+
+DELETE {
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?membership ?p ?o .
+  }
+} WHERE {
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?membership a org:Membership ;
+    org:member ?member ;
+    org:organization ?organization ;
+    org:role <http://data.lblod.info/id/rollen/4ec7d5c39bdc4e84b4174379b9e22ad8> ; # has a relation with
+    ?p ?o .
+
+    ?organization org:classification ?organizationClass .
+    FILTER (?organizationClass IN
+            (
+             <http://data.vlaanderen.be/id/concept/BestuurseenheidClassificatieCode/ea446861-2c51-45fa-afd3-4e4a37b71562> , # HVZ
+             <http://data.vlaanderen.be/id/concept/BestuurseenheidClassificatieCode/80310756-ce0a-4a1b-9b8e-7c01b6cc7a2d> , # APB
+             <http://data.vlaanderen.be/id/concept/BestuurseenheidClassificatieCode/d01bb1f6-2439-4e33-9c25-1fc295de2e71> , # Dienstverlenende vereniging
+             <http://data.vlaanderen.be/id/concept/BestuurseenheidClassificatieCode/cd93f147-3ece-4308-acab-5c5ada3ec63d> , # Opdrachthoudende vereniging
+             <http://data.vlaanderen.be/id/concept/BestuurseenheidClassificatieCode/4b8450cf-a326-4c66-9e63-b4ec10acc7f6> , # Opdrachthoudende vereniging met private deelname
+             <http://data.vlaanderen.be/id/concept/BestuurseenheidClassificatieCode/b156b67f-c5f4-4584-9b30-4c090be02fdc> , # Projectvereniging
+             <http://data.vlaanderen.be/id/concept/BestuurseenheidClassificatieCode/a3922c6d-425b-474f-9a02-ffb71a436bfc> , # Politiezone
+             <http://data.vlaanderen.be/id/concept/BestuurseenheidClassificatieCode/34b5af85-dc9f-468f-9e03-ef89b174c267> , # Autonome verzorgingsinstelling
+             <http://data.vlaanderen.be/id/concept/BestuurseenheidClassificatieCode/36a82ba0-7ff1-4697-a9dd-2e94df73b721> , # AGB
+             <http://data.vlaanderen.be/id/concept/BestuurseenheidClassificatieCode/e8294b73-87c9-4fa2-9441-1937350763c9> # Welzijnsvereniging
+             )
+            )
+  }
+}

config/dev-migrations/20251118120821-add-status-to-bestuurseenheden.sparql

@@ -0,0 +1,19 @@
+DELETE {
+  GRAPH ?g {
+    ?s <http://www.w3.org/ns/regorg#orgStatus> ?status.
+  }
+}
+INSERT {
+  GRAPH ?g {
+    ?s <http://www.w3.org/ns/regorg#orgStatus> <http://lblod.data.gift/concepts/63cc561de9188d64ba5840a42ae8f0d6>.
+  }
+}
+WHERE {
+  GRAPH ?g {
+    ?s a <http://www.w3.org/ns/org#Organization>.
+
+    OPTIONAL {
+      ?s <http://www.w3.org/ns/regorg#orgStatus> ?status.
+    }
+  }
+}

config/dev-migrations/20251118121407-add-missing-concept-schemes.sparql

@@ -0,0 +1,33 @@
+INSERT DATA {
+  GRAPH <http://mu.semte.ch/graphs/public> {
+
+
+
+<http://lblod.data.gift/concept-schemes/b8dcb04f0d1feb86ff01cc0677363b20>	<http://www.w3.org/2004/02/skos/core#prefLabel>	"Organisatie status code" .
+<http://lblod.data.gift/concepts/abf4fee82019f88cf122f986830621ab>	<http://www.w3.org/2004/02/skos/core#inScheme>	<http://lblod.data.gift/concept-schemes/b8dcb04f0d1feb86ff01cc0677363b20> .
+<http://lblod.data.gift/concept-schemes/b8dcb04f0d1feb86ff01cc0677363b20>	<http://mu.semte.ch/vocabularies/core/uuid>	"b8dcb04f0d1feb86ff01cc0677363b20" .
+<http://lblod.data.gift/concepts/63cc561de9188d64ba5840a42ae8f0d6>	<http://mu.semte.ch/vocabularies/core/uuid>	"63cc561de9188d64ba5840a42ae8f0d6" .
+<http://lblod.data.gift/concept-schemes/b8dcb04f0d1feb86ff01cc0677363b20>	<http://www.w3.org/1999/02/22-rdf-syntax-ns#type>	<http://www.w3.org/2004/02/skos/core#ConceptScheme> .
+<http://lblod.data.gift/concepts/d02c4e12bf88d2fdf5123b07f29c9311>	<http://www.w3.org/2004/02/skos/core#topConceptOf>	<http://lblod.data.gift/concept-schemes/b8dcb04f0d1feb86ff01cc0677363b20> .
+<http://lblod.data.gift/concepts/63cc561de9188d64ba5840a42ae8f0d6>	<http://www.w3.org/2004/02/skos/core#inScheme>	<http://lblod.data.gift/concept-schemes/b8dcb04f0d1feb86ff01cc0677363b20> .
+<http://lblod.data.gift/concepts/d02c4e12bf88d2fdf5123b07f29c9311>	<http://www.w3.org/1999/02/22-rdf-syntax-ns#type>	<http://lblod.data.gift/vocabularies/organisatie/OrganisatieStatusCode> .
+<http://lblod.data.gift/concepts/63cc561de9188d64ba5840a42ae8f0d6>	<http://www.w3.org/2004/02/skos/core#topConceptOf>	<http://lblod.data.gift/concept-schemes/b8dcb04f0d1feb86ff01cc0677363b20> .
+<http://lblod.data.gift/concepts/d02c4e12bf88d2fdf5123b07f29c9311>	<http://www.w3.org/2004/02/skos/core#inScheme>	<http://lblod.data.gift/concept-schemes/b8dcb04f0d1feb86ff01cc0677363b20> .
+<http://lblod.data.gift/concepts/abf4fee82019f88cf122f986830621ab>	<http://www.w3.org/2004/02/skos/core#prefLabel>	"In oprichting" .
+<http://lblod.data.gift/concepts/abf4fee82019f88cf122f986830621ab>	<http://www.w3.org/1999/02/22-rdf-syntax-ns#type>	<http://lblod.data.gift/vocabularies/organisatie/OrganisatieStatusCode> .
+<http://lblod.data.gift/concepts/63cc561de9188d64ba5840a42ae8f0d6>	<http://www.w3.org/2004/02/skos/core#inScheme>	<http://lblod.data.gift/concept-schemes/10caf4a4-1ff5-4c05-917e-56489910d68b> .
+<http://lblod.data.gift/concepts/abf4fee82019f88cf122f986830621ab>	<http://www.w3.org/2004/02/skos/core#topConceptOf>	<http://lblod.data.gift/concept-schemes/b8dcb04f0d1feb86ff01cc0677363b20> .
+<http://lblod.data.gift/concepts/d02c4e12bf88d2fdf5123b07f29c9311>	<http://www.w3.org/2004/02/skos/core#prefLabel>	"Niet Actief" .
+<http://lblod.data.gift/concepts/abf4fee82019f88cf122f986830621ab>	<http://mu.semte.ch/vocabularies/core/uuid>	"abf4fee82019f88cf122f986830621ab" .
+<http://lblod.data.gift/concepts/d02c4e12bf88d2fdf5123b07f29c9311>	<http://www.w3.org/2004/02/skos/core#inScheme>	<http://lblod.data.gift/concept-schemes/10caf4a4-1ff5-4c05-917e-56489910d68b> .
+<http://lblod.data.gift/concepts/63cc561de9188d64ba5840a42ae8f0d6>	<http://www.w3.org/2004/02/skos/core#prefLabel>	"Actief" .
+<http://lblod.data.gift/concepts/63cc561de9188d64ba5840a42ae8f0d6>	<http://www.w3.org/1999/02/22-rdf-syntax-ns#type>	<http://lblod.data.gift/vocabularies/organisatie/OrganisatieStatusCode> .
+<http://lblod.data.gift/concepts/d02c4e12bf88d2fdf5123b07f29c9311>	<http://mu.semte.ch/vocabularies/core/uuid>	"d02c4e12bf88d2fdf5123b07f29c9311" .
+<http://lblod.data.gift/concepts/abf4fee82019f88cf122f986830621ab>	<http://www.w3.org/2004/02/skos/core#inScheme>	<http://lblod.data.gift/concept-schemes/10caf4a4-1ff5-4c05-917e-56489910d68b> .
+
+
+
+
+  }
+
+}

config/dispatcher/dispatcher.ex

@@ -1,364 +1,436 @@
 defmodule Dispatcher do
   use Matcher
 
-  define_accept_types [
+  define_accept_types(
     html: ["text/html", "application/xhtml+html"],
     json: ["application/json", "application/vnd.api+json"],
     upload: ["multipart/form-data"],
     sparql_json: ["application/sparql-results+json"],
-    any: [ "*/*" ],
-  ]
-
-  define_layers [ :api_services, :api, :frontend, :not_found ]
+    any: ["*/*"]
+  )
 
+  define_layers([:api_services, :api, :frontend, :not_found])
 
   ###############################################################
   # domain.json
   ###############################################################
 
-  match "/people/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/people/"
+  match "/people/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/people/")
   end
 
-  match "/agents-in-position/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/agents-in-position/"
+  match "/agents-in-position/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/agents-in-position/")
   end
 
-  match "/posts/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/posts/"
+  match "/posts/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/posts/")
   end
 
-  match "/mandatories/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/mandatories/"
+  match "/mandatories/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/mandatories/")
   end
 
-  match "/mandatory-status-codes/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/mandatory-status-codes/"
+  match "/mandatory-status-codes/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/mandatory-status-codes/")
   end
 
-  match "/worship-mandatories/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/worship-mandatories/"
+  match "/worship-mandatories/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/worship-mandatories/")
   end
 
-  match "/half-elections/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/half-elections/"
+  match "/half-elections/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/half-elections/")
   end
 
-  match "/contact-points/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/contact-points/"
+  match "/contact-points/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/contact-points/")
   end
 
-  match "/mandates/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/mandates/"
+  match "/mandates/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/mandates/")
   end
 
-  match "/board-positions/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/board-positions/"
+  match "/board-positions/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/board-positions/")
   end
 
-  match "/board-position-codes/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/board-position-codes/"
+  match "/board-position-codes/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/board-position-codes/")
   end
 
-  match "/functionaries/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/functionaries/"
+  match "/functionaries/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/functionaries/")
   end
 
-  match "/agent-status-codes/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/agent-status-codes/"
+  match "/agent-status-codes/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/agent-status-codes/")
   end
 
-  match "/roles/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/roles/"
+  match "/roles/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/roles/")
   end
 
-  match "/organizations/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/organizations/"
+  match "/organizations/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/organizations/")
   end
 
-  match "/registered-organizations/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/registered-organizations/"
+  match "/registered-organizations/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/registered-organizations/")
   end
 
-  match "/administrative-units/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/administrative-units/"
+  match "/administrative-units/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/administrative-units/")
   end
 
-  match "/administrative-unit-classification-codes/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/administrative-unit-classification-codes/"
+  match "/administrative-unit-classification-codes/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/administrative-unit-classification-codes/")
   end
 
-  match "/organization-classification-codes/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/organization-classification-codes/"
+  match "/organization-classification-codes/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/organization-classification-codes/")
   end
 
-  match "/registered-organization-classification-codes/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/registered-organization-classification-codes/"
+  match "/registered-organization-classification-codes/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/registered-organization-classification-codes/")
   end
 
-  match "/worship-administrative-units/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/worship-administrative-units/"
+  match "/worship-administrative-units/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/worship-administrative-units/")
   end
 
-  match "/worship-services/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/worship-services/"
+  match "/worship-services/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/worship-services/")
   end
 
-  match "/recognized-worship-types/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/recognized-worship-types/"
+  match "/recognized-worship-types/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/recognized-worship-types/")
   end
 
-  match "/central-worship-services/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/central-worship-services/"
+  match "/central-worship-services/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/central-worship-services/")
   end
 
-  match "/representative-bodies/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/representative-bodies/"
+  match "/representative-bodies/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/representative-bodies/")
   end
 
-  match "/governing-bodies/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/governing-bodies/"
+  match "/governing-bodies/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/governing-bodies/")
   end
 
-  match "/governing-body-classification-codes/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/governing-body-classification-codes/"
+  match "/governing-body-classification-codes/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/governing-body-classification-codes/")
   end
 
-  match "/local-involvements/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/local-involvements/"
+  match "/local-involvements/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/local-involvements/")
   end
 
-  match "/identifiers/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/identifiers/"
+  match "/identifiers/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/identifiers/")
   end
 
-  match "/structured-identifiers/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/structured-identifiers/"
+  match "/structured-identifiers/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/structured-identifiers/")
   end
 
-  match "/addresses/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/addresses/"
+  match "/addresses/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/addresses/")
   end
 
-  match "/sites/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/sites/"
+  match "/sites/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/sites/")
   end
 
-  match "/change-events/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/change-events/"
+  match "/change-events/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/change-events/")
   end
 
-  match "/change-event-types/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/change-event-types/"
+  match "/change-event-types/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/change-event-types/")
   end
 
-  match "/change-event-results/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/change-event-results/"
+  match "/change-event-results/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/change-event-results/")
   end
 
-  match "/organization-status-codes/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/organization-status-codes/"
+  match "/organization-status-codes/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/organization-status-codes/")
   end
 
-  match "/locations/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/locations/"
+  match "/locations/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/locations/")
   end
 
-  match "/involvement-types/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/involvement-types/"
+  match "/involvement-types/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/involvement-types/")
   end
 
-  match "/ministers/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/ministers/"
+  match "/ministers/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/ministers/")
   end
 
-  match "/minister-conditions/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/minister-conditions/"
+  match "/minister-conditions/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/minister-conditions/")
   end
 
-  match "/dates-of-birth/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/dates-of-birth/"
+  match "/dates-of-birth/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/dates-of-birth/")
   end
 
-  match "/nationalities/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/nationalities/"
+  match "/nationalities/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/nationalities/")
   end
 
-  match "/gender-codes/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/gender-codes/"
-  end
-  match "/concepts/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/concepts/"
+  match "/gender-codes/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/gender-codes/")
   end
 
-  match "/minister-positions/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/minister-positions/"
+  match "/concepts/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/concepts/")
   end
 
-  match "/minister-position-functions/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/minister-position-functions/"
+  match "/minister-positions/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/minister-positions/")
   end
 
-  match "/financing-codes/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/financing-codes/"
+  match "/minister-position-functions/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/minister-position-functions/")
   end
 
-  match "/minister-condition-criterions/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/minister-condition-criterions/"
+  match "/financing-codes/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/financing-codes/")
   end
 
-  match "/document-types-criterions/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/document-types-criterions/"
+  match "/minister-condition-criterions/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/minister-condition-criterions/")
   end
 
-  match "/site-types/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/site-types/"
+  match "/document-types-criterions/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/document-types-criterions/")
   end
 
-  match "/request-reasons/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/request-reasons/"
+  match "/site-types/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/site-types/")
   end
 
-  match "/decisions/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/decisions/"
+  match "/request-reasons/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/request-reasons/")
   end
 
-  match "/decision-activities/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://cache/decision-activities/"
+  match "/decisions/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/decisions/")
+  end
+
+  match "/decision-activities/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/decision-activities/")
+  end
+
+  match "/memberships/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/memberships/")
+  end
+
+  match "/membership-roles/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/membership-roles/")
   end
 
   # There is currently no logic to invalidate the kboOrganization cache when wegwijse updates the content
-  match "/kbo-organizations/*path", %{ accept: [:any], layer: :api} do
-    Proxy.forward conn, path, "http://resource/kbo-organizations/"
+  match "/kbo-organizations/*path", %{accept: [:any], layer: :api} do
+    Proxy.forward(conn, path, "http://resource/kbo-organizations/")
+  end
+
+  # Dashboard
+  match "/jobs/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/jobs/")
+  end
+
+  match "/tasks/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/tasks/")
+  end
+
+  match "/data-containers/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/data-containers/")
+  end
+
+  match "/job-errors/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://cache/job-errors/")
+  end
+
+  match "/reports/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://resource/reports/")
+  end
+
+  match "/log-entries/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://resource/log-entries/")
+  end
+
+  match "/log-levels/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://resource/log-levels/")
+  end
+
+  match "/status-codes/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://resource/status-codes/")
+  end
+
+  match "/log-sources/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://resource/log-sources/")
+  end
+
+  match "/remote-data-objects/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://resource/remote-data-objects/")
   end
 
   ###############
   # LOGIN
   ###############
 
-  match "/accounts", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, [], "http://resource/accounts/"
-  end
-  match "/accounts/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://accountdetail/accounts/"
-  end
-  match "/active-role/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://roles/"
-  end
-  match "/groups/*path", %{ accept: [:json], layer: :api} do
-    Proxy.forward conn, path, "http://resource/groups/"
+  match "/accounts", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, [], "http://resource/accounts/")
   end
 
-  match "/mock/sessions/*path", %{ accept: [:any], layer: :api} do
-    Proxy.forward conn, path, "http://mocklogin/sessions/"
+  match "/accounts/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://accountdetail/accounts/")
   end
 
-  match "/sessions/*path", %{ accept: [:any], layer: :api} do
-    Proxy.forward conn, path, "http://login/sessions/"
+  match "/active-role/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://roles/")
+  end
+
+  match "/groups/*path", %{accept: [:json], layer: :api} do
+    Proxy.forward(conn, path, "http://resource/groups/")
+  end
+
+  match "/mock/sessions/*path", %{accept: [:any], layer: :api} do
+    Proxy.forward(conn, path, "http://mocklogin/sessions/")
+  end
+
+  match "/sessions/*path", %{ reverse_host: ["dashboard" | _rest] } do
+    Proxy.forward conn, path, "http://login-dashboard/sessions/"
+  end
+
+  match "/sessions/*path", %{accept: [:any], layer: :api} do
+    Proxy.forward(conn, path, "http://login/sessions/")
   end
 
   ###############################################################
   # SEARCH
   ###############################################################
 
-  match "/search/*path", %{  accept: %{ json: true }, layer: :api_services} do
-    Proxy.forward conn, path, "http://search/"
+  match "/search/*path", %{accept: %{json: true}, layer: :api_services} do
+    Proxy.forward(conn, path, "http://search/")
   end
 
   ###############
   # API SERVICES
   ###############
 
-  match "/v3/api-docs/*path", %{ layer: :api_services, accept: %{ json: true } } do
-    forward conn, path, "http://kalliope-api/v3/api-docs/"
+  match "/v3/api-docs/*path", %{layer: :api_services, accept: %{json: true}} do
+    forward(conn, path, "http://kalliope-api/v3/api-docs/")
   end
 
-  match "/consolidated/*path", %{ layer: :api_services, accept: %{ json: true } } do
-    forward conn, path, "http://kalliope-api/consolidated/"
+  match "/consolidated/*path", %{layer: :api_services, accept: %{json: true}} do
+    forward(conn, path, "http://kalliope-api/consolidated/")
   end
 
-  match "/adresses-register/*path", %{ layer: :api_services, accept: %{ json: true } } do
-    forward conn, path, "http://adressenregister/"
+  match "/adresses-register/*path", %{layer: :api_services, accept: %{json: true}} do
+    forward(conn, path, "http://adressenregister/")
   end
 
-  match "/worship/person-information-updates/*path", %{ layer: :api_services, accept: %{ json: true } } do
-    forward conn, path, "http://privacy-worship/person-information-updates"
+  match "/worship/person-information-updates/*path", %{
+    layer: :api_services,
+    accept: %{json: true}
+  } do
+    forward(conn, path, "http://privacy-worship/person-information-updates")
   end
 
-  match "/worship/person-information-requests/*path", %{ layer: :api_services, accept: %{ json: true } } do
-    forward conn, path, "http://privacy-worship/person-information-requests"
+  match "/worship/person-information-requests/*path", %{
+    layer: :api_services,
+    accept: %{json: true}
+  } do
+    forward(conn, path, "http://privacy-worship/person-information-requests")
   end
 
-  match "/worship/person-information-ask/*path", %{ layer: :api_services, accept: %{ json: true } } do
-    forward conn, path, "http://privacy-worship/person-information-ask/"
+  match "/worship/person-information-ask/*path", %{layer: :api_services, accept: %{json: true}} do
+    forward(conn, path, "http://privacy-worship/person-information-ask/")
   end
 
-  match "/worship/person-information-validate-ssn/*path", %{ layer: :api_services, accept: %{ json: true } } do
-    forward conn, path, "http://privacy-worship/person-information-validate-ssn/"
+  match "/worship/person-information-validate-ssn/*path", %{
+    layer: :api_services,
+    accept: %{json: true}
+  } do
+    forward(conn, path, "http://privacy-worship/person-information-validate-ssn/")
   end
 
-  match "/person-information-updates/*path", %{ layer: :api_services, accept: %{ json: true } } do
-    forward conn, path, "http://privacy-unit/person-information-updates"
+  match "/person-information-updates/*path", %{layer: :api_services, accept: %{json: true}} do
+    forward(conn, path, "http://privacy-unit/person-information-updates")
   end
 
-  match "/person-information-requests/*path", %{ layer: :api_services, accept: %{ json: true } } do
-    forward conn, path, "http://privacy-unit/person-information-requests"
+  match "/person-information-requests/*path", %{layer: :api_services, accept: %{json: true}} do
+    forward(conn, path, "http://privacy-unit/person-information-requests")
   end
 
-  match "/person-information-ask/*path", %{ layer: :api_services, accept: %{ json: true } } do
-    forward conn, path, "http://privacy-unit/person-information-ask/"
+  match "/person-information-ask/*path", %{layer: :api_services, accept: %{json: true}} do
+    forward(conn, path, "http://privacy-unit/person-information-ask/")
   end
 
-  match "/person-information-validate-ssn/*path", %{ layer: :api_services, accept: %{ json: true } } do
-    forward conn, path, "http://privacy-unit/person-information-validate-ssn/"
+  match "/person-information-validate-ssn/*path", %{layer: :api_services, accept: %{json: true}} do
+    forward(conn, path, "http://privacy-unit/person-information-validate-ssn/")
   end
 
-  get "/uri-info/*path", %{ layer: :api_services, accept: %{ json: true } } do
-    forward conn, path, "http://uri-info/"
+  get "/uri-info/*path", %{layer: :api_services, accept: %{json: true}} do
+    forward(conn, path, "http://uri-info/")
   end
 
-  post "/construct-organization-relationships/*path", %{ layer: :api_services, accept: %{ json: true } } do
-    forward conn, path, "http://construct-organization-relationships/create-relationships/"
+  post "/construct-organization-relationships/*path", %{
+    layer: :api_services,
+    accept: %{json: true}
+  } do
+    forward(conn, path, "http://construct-organization-relationships/")
   end
 
-  post "/kbo-data-sync/*path", %{ layer: :api_services, accept: %{ json: true } } do
-    forward conn, path, "http://kbo-data-sync/sync-kbo-data/"
+  post "/kbo-data-sync/*path", %{layer: :api_services, accept: %{json: true}} do
+    forward(conn, path, "http://kbo-data-sync/sync-kbo-data/")
   end
 
-  get "/ldes/*path", %{ layer: :api_services } do
-    forward conn, path, "http://ldes-backend/"
+  match "/scope-of-operation/*path", %{layer: api_services, accept: %{json: true}} do
+    forward(conn, path, "http://scope-of-operation/")
+  end
+
+  get "/ldes/*path", %{layer: :api_services} do
+    forward(conn, path, "http://ldes-backend/")
   end
 
   #################################################################
   # ERROR REPORT
   #################################################################
 
-  match "/error-reports/*path", %{ layer: :api_services, accept: %{ json: true } } do
-    forward conn, path, "http://error-report-service/error-reports/"
+  match "/error-reports/*path", %{layer: :api_services, accept: %{json: true}} do
+    forward(conn, path, "http://error-report-service/error-reports/")
   end
 
   #################################################################
   # FILES
   #################################################################
 
-  get "/files/:id/download", %{ accept: [:any], } do
-    Proxy.forward conn, [], "http://file/files/" <> id <> "/download"
+  get "/files/:id/download", %{accept: [:any]} do
+    Proxy.forward(conn, [], "http://file/files/" <> id <> "/download")
   end
 
-  get "/files/*path" , %{ layer: :api_services, accept: %{ json: true } } do
-    Proxy.forward conn, path, "http://resource/files/"
+  get "/files/*path", %{layer: :api_services, accept: %{json: true}} do
+    Proxy.forward(conn, path, "http://resource/files/")
   end
 
   #################################################################
   # DCAT
   #################################################################
 
-  get "/datasets/*path", %{ layer: :api_services, accept: %{ json: true } } do
-    Proxy.forward conn, path, "http://resource/datasets/"
+  get "/datasets/*path", %{layer: :api_services, accept: %{json: true}} do
+    Proxy.forward(conn, path, "http://resource/datasets/")
   end
 
-  get "/distributions/*path", %{ layer: :api_services, accept: %{ json: true } } do
-    Proxy.forward conn, path, "http://resource/distributions/"
+  get "/distributions/*path", %{layer: :api_services, accept: %{json: true}} do
+    Proxy.forward(conn, path, "http://resource/distributions/")
   end
 
   #################################################################
@@ -366,15 +438,7 @@ defmodule Dispatcher do
   #################################################################
 
   get "/sync/organizations/files/*path" do
-    Proxy.forward conn, path, "http://delta-producer-publication-graph-maintainer/files/"
-  end
-
-  #################################################################
-  #  DELTA: organizations-public-info
-  #################################################################
-
-  get "/sync/organizations-public-info/files/*path" do
-    Proxy.forward conn, path, "http://delta-producer-pub-graph-maintainer-organizations-public-info/files/"
+    Proxy.forward conn, path, "http://delta-producer-publication-graph-maintainer/organizations/files/"
   end
 
   #################################################################
@@ -382,7 +446,7 @@ defmodule Dispatcher do
   #################################################################
 
   get "/sync/public/files/*path" do
-    Proxy.forward conn, path, "http://delta-producer-pub-graph-maintainer-public/files/"
+    Proxy.forward(conn, path, "http://delta-producer-publication-graph-maintainer/public/files/")
   end
 
   #################################################################
@@ -390,7 +454,7 @@ defmodule Dispatcher do
   #################################################################
 
   get "/sync/worship-posts/files/*path" do
-    Proxy.forward conn, path, "http://delta-producer-pub-graph-maintainer-worship-posts/files/"
+    Proxy.forward(conn, path, "http://delta-producer-publication-graph-maintainer/worship-posts/files/")
   end
 
   #################################################################
@@ -398,44 +462,64 @@ defmodule Dispatcher do
   #################################################################
 
   get "/download-exports/*path" do
-    Proxy.forward conn, path, "http://download-exports/"
+    Proxy.forward(conn, path, "http://download-exports/")
   end
 
   ###############################################################
   # frontend layer
   ###############################################################
 
-  match "/assets/*path", %{ layer: :api } do
-    Proxy.forward conn, path, "http://frontend/assets/"
+  get "/assets/*path",  %{ reverse_host: ["dashboard" | _rest], layer: :api }  do
+    forward conn, path, "http://frontend-dashboard/assets/"
   end
 
-  match "/@appuniversum/*path", %{ layer: :api } do
-    Proxy.forward conn, path, "http://frontend/@appuniversum/"
+  get "/@appuniversum/*path", %{ reverse_host: ["dashboard" | _rest], layer: :api} do
+    forward conn, path, "http://frontend-dashboard/@appuniversum/"
   end
 
-  match "/*path", %{ accept: [:html], layer: :api } do
-    Proxy.forward conn, [], "http://frontend/index.html"
+  match "/*_path", %{ reverse_host: ["dashboard" | _rest], layer: :api } do
+    # *_path allows a path to be supplied, but will not yield
+    # an error that we don't use the path variable.
+    forward conn, [], "http://frontend-dashboard/index.html"
   end
 
-  match "/*_path", %{ layer: :frontend } do
-    Proxy.forward conn, [], "http://frontend/index.html"
+  match "/assets/*path", %{layer: :api} do
+    Proxy.forward(conn, path, "http://frontend/assets/")
+  end
+
+  match "/@appuniversum/*path", %{layer: :api} do
+    Proxy.forward(conn, path, "http://frontend/@appuniversum/")
+  end
+
+  match "/*path", %{accept: [:html], layer: :api} do
+    Proxy.forward(conn, [], "http://frontend/index.html")
+  end
+
+  match "/*_path", %{layer: :frontend} do
+    Proxy.forward(conn, [], "http://frontend/index.html")
   end
 
   ###############################################################
   # sparql endpoint
   ###############################################################
 
-  post "/sparql/*path", %{ layer: :api_services, accept: %{ sparql_json: true } } do
-    forward conn, path, "http://db:8890/sparql/"
+#  post "/sparql/*path", %{layer: :api_services, accept: %{sparql_json: true}} do
+#    forward(conn, path, "http://db:8890/sparql/")
+#  end
+
+  #################################################################
+  # Health checks
+  #################################################################
+
+  get "/health/accounts", %{layer: :api} do
+    Proxy.forward(conn, [], "http://resource/accounts/")
   end
 
   ###############################################################
   # errors
   ###############################################################
 
-  match "/*_path", %{ accept: [:any], layer: :not_found} do
-    send_resp( conn, 404, "{\"error\": {\"code\": 404}")
+  match "/*_path", %{accept: [:any], layer: :not_found} do
+    send_resp(conn, 404, "{\"error\": {\"code\": 404}")
   end
-
-
 end

config/error-alert/config.json

@@ -0,0 +1,8 @@
+{
+  "creators": [
+    "http://lblod.data.gift/services/delta-producer-background-jobs-initiator/organizations",
+    "http://lblod.data.gift/services/delta-producer-background-jobs-initiator/public",
+    "http://lblod.data.gift/services/delta-producer-background-jobs-initiator/organizations-public-info",
+    "http://lblod.data.gift/services/delta-producer-background-jobs-initiator/worship-posts"
+  ]
+}

config/jobs-controller/config.json

@@ -63,38 +63,6 @@
       }
     ]
   },
-  "http://redpencil.data.gift/id/jobs/concept/JobOperation/deltas/initialPublicationGraphSyncing/organizations-public-info": {
-    "tasksConfiguration": [
-      {
-        "currentOperation": null,
-        "nextOperation": "http://redpencil.data.gift/id/jobs/concept/TaskOperation/deltas/initialPublicationGraphSyncing",
-        "nextIndex": "0"
-      },
-      {
-        "currentOperation": "http://redpencil.data.gift/id/jobs/concept/TaskOperation/deltas/initialPublicationGraphSyncing",
-        "nextOperation": "http://redpencil.data.gift/id/jobs/concept/TaskOperation/deltas/deltaDumpFileCreation",
-        "nextIndex": "1"
-      }
-    ]
-  },
-  "http://redpencil.data.gift/id/jobs/concept/JobOperation/deltas/healingOperation/organizations-public-info": {
-    "tasksConfiguration": [
-      {
-        "currentOperation": null,
-        "nextOperation": "http://redpencil.data.gift/id/jobs/concept/TaskOperation/deltas/healing/patchPublicationGraph",
-        "nextIndex": "0"
-      }
-    ]
-  },
-  "http://redpencil.data.gift/id/jobs/concept/JobOperation/deltas/deltaDumpFileCreation/organizations-public-info": {
-    "tasksConfiguration": [
-      {
-        "currentOperation": null,
-        "nextOperation": "http://redpencil.data.gift/id/jobs/concept/TaskOperation/deltas/deltaDumpFileCreation",
-        "nextIndex": "0"
-      }
-    ]
-  },
   "http://redpencil.data.gift/id/jobs/concept/JobOperation/deltas/initialPublicationGraphSyncing/public": {
     "tasksConfiguration": [
       {

config/loket-consumer/leidinggevenden-dispatching/config.js

@@ -1,32 +0,0 @@
-const BATCH_SIZE = parseInt(process.env.BATCH_SIZE) || 100;
-const MU_CALL_SCOPE_ID_INITIAL_SYNC = process.env.MU_CALL_SCOPE_ID_INITIAL_SYNC || 'http://redpencil.data.gift/id/concept/muScope/deltas/consumer/initialSync';
-const BYPASS_MU_AUTH_FOR_EXPENSIVE_QUERIES = process.env.BYPASS_MU_AUTH_FOR_EXPENSIVE_QUERIES == 'true' ? true : false;
-const DIRECT_DATABASE_ENDPOINT = process.env.DIRECT_DATABASE_ENDPOINT || 'http://triplestore:8890/sparql';
-const MAX_DB_RETRY_ATTEMPTS = parseInt(process.env.MAX_DB_RETRY_ATTEMPTS || 5);
-const MAX_REASONING_RETRY_ATTEMPTS = parseInt(process.env.MAX_REASONING_RETRY_ATTEMPTS || 5);
-const SLEEP_BETWEEN_BATCHES = parseInt(process.env.SLEEP_BETWEEN_BATCHES || 1000);
-const SLEEP_TIME_AFTER_FAILED_REASONING_OPERATION = parseInt(process.env.SLEEP_TIME_AFTER_FAILED_REASONING_OPERATION || 10000);
-const SLEEP_TIME_AFTER_FAILED_DB_OPERATION = parseInt(process.env.SLEEP_TIME_AFTER_FAILED_DB_OPERATION || 60000);
-
-// Graphs
-const TARGET_GRAPH = process.env.TARGET_GRAPH || `http://mu.semte.ch/graphs/administrative-unit`;
-const LANDING_ZONE_GRAPH = process.env.DCR_LANDING_ZONE_GRAPH || `http://mu.semte.ch/graphs/landing-zone/leidinggevenden`;
-
-// ENDPOINTS
-const LANDING_ZONE_DATABASE = process.env.DCR_LANDING_ZONE_DATABASE || 'db';
-const LANDING_ZONE_DATABASE_ENDPOINT = process.env.DCR_LANDING_ZONE_DATABASE_ENDPOINT || `http://${LANDING_ZONE_DATABASE}:8890/sparql`;
-
-module.exports = {
-  BATCH_SIZE,
-  MU_CALL_SCOPE_ID_INITIAL_SYNC,
-  BYPASS_MU_AUTH_FOR_EXPENSIVE_QUERIES,
-  DIRECT_DATABASE_ENDPOINT,
-  MAX_DB_RETRY_ATTEMPTS,
-  MAX_REASONING_RETRY_ATTEMPTS,
-  SLEEP_BETWEEN_BATCHES,
-  SLEEP_TIME_AFTER_FAILED_DB_OPERATION,
-  SLEEP_TIME_AFTER_FAILED_REASONING_OPERATION,
-  LANDING_ZONE_GRAPH,
-  TARGET_GRAPH,
-  LANDING_ZONE_DATABASE_ENDPOINT
-};

config/loket-consumer/leidinggevenden-dispatching/delta-context-config.js

@@ -1,72 +0,0 @@
-
-const PREFIXES = `
-PREFIX org: <http://www.w3.org/ns/org#>
-PREFIX mandaat: <http://data.vlaanderen.be/ns/mandaat#>
-PREFIX lblodlg: <http://data.lblod.info/vocabularies/leidinggevenden/>
-PREFIX besluit: <http://data.vlaanderen.be/ns/besluit#>`
-
-const contextConfig = {
-  addTypes: {
-    scope: 'all', // 'inserts, 'deletes', 'all' or none. To add rdf:type to subjects of inserts, deletes or both
-    exhausitive: false, // true or false: find all types for a subject, even if one is already present in delta
-  },
-  contextQueries: [
-    // Case Leidnggevenden
-    // They can be triggered when being inserted or when their link to a bestuursfunctie is added
-    {
-      trigger: { // subjectType or predicateValue
-        subjectType: "lblodlg:Functionaris"
-      },
-      queryTemplate: (subject) => `
-        ${PREFIXES}
-        CONSTRUCT {
-          ?functionaris org:holds ?bestuursfunctie ;
-            ?p ?o .
-          ?orgaanInTime lblodlg:heeftBestuursfunctie ?bestuursfunctie ;
-            mandaat:isTijdspecialisatieVan ?orgaan .
-          ?orgaan besluit:bestuurt ?eenheid .
-          ?eenheid besluit:classificatie ?classification .
-        } WHERE {
-          GRAPH <http://mu.semte.ch/graphs/landing-zone/leidinggevenden> {
-            VALUES ?functionaris { ${subject} }
-            ?functionaris org:holds ?bestuursfunctie ;
-              ?p ?o .
-            ?orgaanInTime lblodlg:heeftBestuursfunctie ?bestuursfunctie ;
-              mandaat:isTijdspecialisatieVan ?orgaan .
-            ?orgaan besluit:bestuurt ?eenheid .
-            ?eenheid besluit:classificatie ?classification .
-          }
-        }`
-    },
-    {
-      trigger: {
-        predicateValue: "org:holds"
-      },
-      queryTemplate: (subject) => `
-        ${PREFIXES}
-        CONSTRUCT {
-          ?functionaris org:holds ?bestuursfunctie ;
-            ?p ?o .
-          ?orgaanInTime lblodlg:heeftBestuursfunctie ?bestuursfunctie ;
-            mandaat:isTijdspecialisatieVan ?orgaan .
-          ?orgaan besluit:bestuurt ?eenheid .
-          ?eenheid besluit:classificatie ?classification .
-        } WHERE {
-          GRAPH <http://mu.semte.ch/graphs/landing-zone/leidinggevenden> {
-            VALUES ?functionaris { ${subject} }
-            ?functionaris org:holds ?bestuursfunctie ;
-              ?p ?o .
-            ?orgaanInTime lblodlg:heeftBestuursfunctie ?bestuursfunctie ;
-              mandaat:isTijdspecialisatieVan ?orgaan .
-            ?orgaan besluit:bestuurt ?eenheid .
-            ?eenheid besluit:classificatie ?classification .
-          }
-        }`
-    }
-  ]
-}
-
-module.exports = {
-  contextConfig,
-  PREFIXES
-};

config/loket-consumer/leidinggevenden-dispatching/delta-sync-dispatching.js

@@ -1,85 +0,0 @@
-const {
-  transformStatements,
-  deleteFromTargetGraph,
-  insertIntoTargetGraph,
-} = require('./util');
-
-/**
-* Dispatch the fetched information to a target graph.
-* Note: <share://file/data> will be ADDED to it's own graph.
-*   We take only care of adding them, not updating triples, this is a TODO
-* @param { mu, muAuthSudo } lib - The provided libraries from the host service.
-* @param { termObjectChangeSets: { deletes, inserts } } data - The fetched changes sets, which objects of serialized Terms
-*          [ {
-*              graph: "<http://foo>",
-*              subject: "<http://bar>",
-*              predicate: "<http://baz>",
-*              object: "<http://boom>^^<http://datatype>"
-*            }
-*         ]
-* @return {void} Nothing
-*/
-async function dispatch(lib, data) {
-  const { mu, fetch } = lib;
-  let { termObjectChangeSets, termObjectChangeSetsWithContext } = data;
-
-  // Both arrays are the same length, so we can zip them together for easier processing
-  zippedChangeSets = termObjectChangeSets.map((o, i) => ({
-    original: o,
-    withContext: termObjectChangeSetsWithContext[i]
-  }));
-
-  console.log(`Received ${zippedChangeSets.length} change sets`)
-
-  for (let { original, withContext } of zippedChangeSets) {
-    let originalDeletes, originalInserts, insertsWithContext, deletesWithContext;
-    try {
-      originalInserts = original.inserts.map(o => `${o.subject} ${o.predicate} ${o.object}.`);
-      originalDeletes = original.deletes.map(o => `${o.subject} ${o.predicate} ${o.object}.`);
-    } catch (e) {
-      console.log(`Error while processing change set: ${e}`);
-      console.log(`Original change set: ${JSON.stringify(original)}`);
-      throw e;
-    }
-
-    // Extra context needed for mapping from DL to OP model and filtering based on type.
-    try {
-      insertsWithContext = withContext.inserts.map(o => `${o.subject} ${o.predicate} ${o.object}.`);
-      deletesWithContext = withContext.deletes.map(o => `${o.subject} ${o.predicate} ${o.object}.`);
-    } catch (e) {
-      console.log(`Error while processing change set with context: ${e}`);
-      console.log(`Change set with context: ${JSON.stringify(withContext)}`);
-      throw e;
-    }
-
-    if (originalDeletes.length) {
-      // Map deletes from DL to OP model
-      const transformedDeletes = await transformStatements(fetch, deletesWithContext);
-
-      if (!transformedDeletes.length) {
-        console.log(`Warn: Delete statements mapped to empty result.`);
-        console.log(`Input: ${deletesWithContext}`);
-        console.log(`Output: ${transformedDeletes}`);
-      } else {
-        await deleteFromTargetGraph(lib, transformedDeletes);
-      }
-    }
-
-    if (originalInserts.length) {
-      // Map inserts from DL to OP model
-      const transformedInserts = await transformStatements(fetch, insertsWithContext);
-
-      if (!transformedInserts.length) {
-        console.log(`Warn: Insert statements mapped to empty result.`);
-        console.log(`Input: ${insertsWithContext}`);
-        console.log(`Output: ${transformedInserts}`);
-      } else {
-        await insertIntoTargetGraph(lib, transformedInserts);
-      }
-    }
-  }
-}
-
-module.exports = {
-  dispatch
-};

config/loket-consumer/leidinggevenden-dispatching/initial-sync-dispatching.js

@@ -1,85 +0,0 @@
-const { batchedDbUpdate, transformLandingZoneGraph } = require('./util');
-const {
-  BYPASS_MU_AUTH_FOR_EXPENSIVE_QUERIES,
-  DIRECT_DATABASE_ENDPOINT,
-  MU_CALL_SCOPE_ID_INITIAL_SYNC,
-  BATCH_SIZE,
-  MAX_DB_RETRY_ATTEMPTS,
-  SLEEP_BETWEEN_BATCHES,
-  SLEEP_TIME_AFTER_FAILED_DB_OPERATION,
-  TARGET_GRAPH,
-  LANDING_ZONE_GRAPH,
-  LANDING_ZONE_DATABASE_ENDPOINT
-} = require('./config');
-
-const endpoint = BYPASS_MU_AUTH_FOR_EXPENSIVE_QUERIES ? DIRECT_DATABASE_ENDPOINT : process.env.MU_SPARQL_ENDPOINT;
-
-/**
-* Dispatch the fetched information to a target graph.
-* @param { mu, muAuthSudo } lib - The provided libraries from the host service.
-* @param { termObjects } data - The fetched quad information, which objects of serialized Terms
-*          [ {
-*              graph: "<http://foo>",
-*              subject: "<http://bar>",
-*              predicate: "<http://baz>",
-*              object: "<http://boom>^^<http://datatype>"
-*            }
-*         ]
-* @return {void} Nothing
-*/
-async function dispatch(lib, data) {
-  const { mu, muAuthSudo } = lib;
-  const { termObjects } = data;
-
-  // Steps:
-  // 1. Write the triples to the landing zone graph - withouth any mapping or filtering
-  // 2. One-time reasoning run with the full landing zone graph and write the results to the target graph (on-finish)
-
-  if (BYPASS_MU_AUTH_FOR_EXPENSIVE_QUERIES) {
-    console.warn(`Service configured to skip MU_AUTH!`);
-  }
-  console.log(`Using ${endpoint} to insert triples`);
-
-  if (termObjects.length) {
-    const originalInsertTriples = termObjects.map(o => `${o.subject} ${o.predicate} ${o.object}.`);
-
-    await batchedDbUpdate(
-      muAuthSudo.updateSudo,
-      LANDING_ZONE_GRAPH,
-      originalInsertTriples,
-      { 'mu-call-scope-id': MU_CALL_SCOPE_ID_INITIAL_SYNC },
-      LANDING_ZONE_DATABASE_ENDPOINT,
-      BATCH_SIZE,
-      MAX_DB_RETRY_ATTEMPTS,
-      SLEEP_BETWEEN_BATCHES,
-      SLEEP_TIME_AFTER_FAILED_DB_OPERATION
-    );
-  }
-}
-
-async function onFinishInitialIngest(lib) {
-  const { muAuthSudo, fetch } = lib;
-
-  console.log(`!! On-finish triggered !!`);
-
-  const transformedInsertTriples = await transformLandingZoneGraph(fetch, endpoint);
-
-  console.log(`Transformed ${transformedInsertTriples.length} triples`);
-
-  await batchedDbUpdate(
-    muAuthSudo.updateSudo,
-    TARGET_GRAPH,
-    transformedInsertTriples,
-    { 'mu-call-scope-id': MU_CALL_SCOPE_ID_INITIAL_SYNC },
-    endpoint,
-    BATCH_SIZE,
-    MAX_DB_RETRY_ATTEMPTS,
-    SLEEP_BETWEEN_BATCHES,
-    SLEEP_TIME_AFTER_FAILED_DB_OPERATION
-  );
-}
-
-module.exports = {
-  dispatch,
-  onFinishInitialIngest
-};

config/loket-consumer/leidinggevenden-dispatching/util.js

@@ -1,168 +0,0 @@
-const {
-  MAX_REASONING_RETRY_ATTEMPTS,
-  SLEEP_TIME_AFTER_FAILED_REASONING_OPERATION,
-  MAX_DB_RETRY_ATTEMPTS,
-  SLEEP_BETWEEN_BATCHES,
-  SLEEP_TIME_AFTER_FAILED_DB_OPERATION,
-  TARGET_GRAPH,
-  LANDING_ZONE_GRAPH,
-  BATCH_SIZE,
-} = require('./config');
-
-async function batchedDbUpdate(
-  muUpdate,
-  graph,
-  triples,
-  extraHeaders,
-  endpoint,
-  batchSize,
-  maxAttempts,
-  sleepBetweenBatches = 1000,
-  sleepTimeOnFail = 1000,
-  operation = 'INSERT'
-) {
-
-  for (let i = 0; i < triples.length; i += batchSize) {
-    console.log(`Inserting triples in batch: ${i}-${i + batchSize}`);
-
-    const batch = triples.slice(i, i + batchSize).join('\n');
-
-    const insertCall = async () => {
-      await muUpdate(`
-${operation} DATA {
-GRAPH <${graph}> {
-${batch}
-}
-}
-`, extraHeaders, endpoint);
-    };
-
-    await operationWithRetry(insertCall, 0, maxAttempts, sleepTimeOnFail);
-
-    console.log(`Sleeping before next query execution: ${sleepBetweenBatches}`);
-    await new Promise(r => setTimeout(r, sleepBetweenBatches));
-  }
-}
-
-async function operationWithRetry(callback,
-  attempt,
-  maxAttempts,
-  sleepTimeOnFail) {
-  try {
-    if (typeof callback === "function")
-      return await callback();
-    else // Catch error from promise - not how I would do it normally, but allows re use of existing code.
-      return await callback;
-  }
-  catch (e) {
-    console.log(`Operation failed for ${callback.toString()}, attempt: ${attempt} of ${maxAttempts}`);
-    console.log(`Error: ${e}`);
-    console.log(`Sleeping ${sleepTimeOnFail} ms`);
-
-    if (attempt >= maxAttempts) {
-      console.log(`Max attempts reached for ${callback.toString()}, giving up`);
-      throw e;
-    }
-
-    await new Promise(r => setTimeout(r, sleepTimeOnFail));
-    return operationWithRetry(callback, ++attempt, maxAttempts, sleepTimeOnFail);
-  }
-}
-
-/**
- * Splits an array into two parts, a part that passes and a part that fails a predicate function.
- * Credits: https://github.com/benjay10
- * @public
- * @function partition
- * @param {Array} arr - Array to be partitioned
- * @param {Function} fn - Function that accepts single argument: an element of the array, and should return a truthy or falsy value.
- * @returns {Object} Object that contains keys passes and fails, each representing an array with elemets that pass or fail the predicate respectively
- */
-function partition(arr, fn) {
-  let passes = [], fails = [];
-  arr.forEach((item) => (fn(item) ? passes : fails).push(item));
-  return { passes, fails };
-}
-
-/**
- * Send triples to reasoning service for conversion
- *
- */
-async function transformTriples(fetch, triples, mapping) {
-  return await operationWithRetry(mainConversion(fetch, triples, mapping), 0,
-    MAX_REASONING_RETRY_ATTEMPTS, SLEEP_TIME_AFTER_FAILED_REASONING_OPERATION);
-}
-
-async function mainConversion(fetch, triples, mapping) {
-  let formdata = new URLSearchParams();
-  formdata.append("data", triples);
-
-  let requestOptions = {
-    method: 'POST',
-    body: formdata,
-    redirect: 'follow'
-  };
-
-  const response = await fetch(`http://reasoner/reason/dl2op/${mapping}`, requestOptions)
-  return await response.text();
-}
-
-async function transformStatements(fetch, triples, mapping = 'main') {
-  console.log(`Received ${JSON.stringify(triples)} to transform`);
-  const transformedTriples = await transformTriples(fetch, triples.join('\n'), mapping);
-  statements = transformedTriples ? transformedTriples.replace(/\n{2,}/g, '').split('\n') : [];
-  console.log(`CONVERSION: FROM ${triples.length} triples to ${statements.length}`);
-  return statements;
-}
-
-async function deleteFromTargetGraph(lib, statements) {
-  console.log(`Deleting ${statements.length} statements from target graph`);
-  console.log(`Statements:  ${JSON.stringify(statements)}`)
-  await batchedDbUpdate(
-    lib.muAuthSudo.updateSudo,
-    TARGET_GRAPH,
-    statements,
-    {},
-    process.env.MU_SPARQL_ENDPOINT,
-    BATCH_SIZE,
-    MAX_DB_RETRY_ATTEMPTS,
-    SLEEP_BETWEEN_BATCHES,
-    SLEEP_TIME_AFTER_FAILED_DB_OPERATION,
-    'DELETE');
-}
-
-async function insertIntoTargetGraph(lib, statements) {
-  console.log(`Inserting ${statements.length} statements into target graph`);
-  console.log(`Statements:  ${JSON.stringify(statements)}`)
-
-  await batchedDbUpdate(
-    lib.muAuthSudo.updateSudo,
-    TARGET_GRAPH,
-    statements,
-    {},
-    process.env.MU_SPARQL_ENDPOINT,
-    BATCH_SIZE,
-    MAX_DB_RETRY_ATTEMPTS,
-    SLEEP_BETWEEN_BATCHES,
-    SLEEP_TIME_AFTER_FAILED_DB_OPERATION,
-    'INSERT');
-}
-
-async function transformLandingZoneGraph(fetch, endpoint, mapping = 'main') {
-  console.log(`Transforming landing zone graph: ${LANDING_ZONE_GRAPH}`);
-
-  const response = await fetch(`http://reasoner/reason/dl2op/${mapping}?data=${encodeURIComponent(`${endpoint}?default-graph-uri=&query=CONSTRUCT+%7B%0D%0A%3Fs+%3Fp+%3Fo%0D%0A%7D+WHERE+%7B%0D%0A+GRAPH+<${LANDING_ZONE_GRAPH}>+%7B%0D%0A%3Fs+%3Fp+%3Fo%0D%0A%7D%0D%0A%7D&should-sponge=&format=text%2Fplain&timeout=0&run=+Run+Query`)}`);
-  const text = await response.text();
-  const statements = text.replace(/\n{2,}/g, '').split('\n');
-
-  return statements;
-}
-
-module.exports = {
-  batchedDbUpdate,
-  partition,
-  transformStatements,
-  transformLandingZoneGraph,
-  deleteFromTargetGraph,
-  insertIntoTargetGraph
-};

config/loket-consumer/mandatarissen-dispatching/config.js

@@ -1,33 +0,0 @@
-const BATCH_SIZE = parseInt(process.env.BATCH_SIZE) || 100;
-const MU_CALL_SCOPE_ID_INITIAL_SYNC = process.env.MU_CALL_SCOPE_ID_INITIAL_SYNC || 'http://redpencil.data.gift/id/concept/muScope/deltas/consumer/initialSync';
-const BYPASS_MU_AUTH_FOR_EXPENSIVE_QUERIES = process.env.BYPASS_MU_AUTH_FOR_EXPENSIVE_QUERIES == 'true' ? true : false;
-const DIRECT_DATABASE_ENDPOINT = process.env.DIRECT_DATABASE_ENDPOINT || 'http://triplestore:8890/sparql';
-const MAX_DB_RETRY_ATTEMPTS = parseInt(process.env.MAX_DB_RETRY_ATTEMPTS || 5);
-const MAX_REASONING_RETRY_ATTEMPTS = parseInt(process.env.MAX_REASONING_RETRY_ATTEMPTS || 5);
-const SLEEP_BETWEEN_BATCHES = parseInt(process.env.SLEEP_BETWEEN_BATCHES || 1000);
-const SLEEP_TIME_AFTER_FAILED_REASONING_OPERATION = parseInt(process.env.SLEEP_TIME_AFTER_FAILED_REASONING_OPERATION || 10000);
-const SLEEP_TIME_AFTER_FAILED_DB_OPERATION = parseInt(process.env.SLEEP_TIME_AFTER_FAILED_DB_OPERATION || 60000);
-
-// Graphs
-const TARGET_GRAPH = process.env.TARGET_GRAPH || `http://mu.semte.ch/graphs/administrative-unit`;
-const LANDING_ZONE_GRAPH = process.env.DCR_LANDING_ZONE_GRAPH || `http://mu.semte.ch/graphs/landing-zone/leidinggevenden`;
-
-// ENDPOINTS
-const LANDING_ZONE_DATABASE = process.env.DCR_LANDING_ZONE_DATABASE || 'db';
-const LANDING_ZONE_DATABASE_ENDPOINT = process.env.DCR_LANDING_ZONE_DATABASE_ENDPOINT || `http://${LANDING_ZONE_DATABASE}:8890/sparql`;
-
-module.exports = {
-  BATCH_SIZE,
-  MU_CALL_SCOPE_ID_INITIAL_SYNC,
-  BYPASS_MU_AUTH_FOR_EXPENSIVE_QUERIES,
-  DIRECT_DATABASE_ENDPOINT,
-  MAX_DB_RETRY_ATTEMPTS,
-  MAX_REASONING_RETRY_ATTEMPTS,
-  SLEEP_BETWEEN_BATCHES,
-  SLEEP_TIME_AFTER_FAILED_DB_OPERATION,
-  SLEEP_TIME_AFTER_FAILED_REASONING_OPERATION,
-  LANDING_ZONE_GRAPH,
-  LANDING_ZONE_DATABASE_ENDPOINT,
-  TARGET_GRAPH
-};
-

config/loket-consumer/mandatarissen-dispatching/delta-context-config.js

@@ -1,72 +0,0 @@
-
-const PREFIXES = `
-PREFIX org: <http://www.w3.org/ns/org#>
-PREFIX mandaat: <http://data.vlaanderen.be/ns/mandaat#>
-PREFIX lblodlg: <http://data.lblod.info/vocabularies/leidinggevenden/>
-PREFIX besluit: <http://data.vlaanderen.be/ns/besluit#>`
-
-const contextConfig = {
-  addTypes: {
-    scope: 'all', // 'inserts, 'deletes', 'all' or none. To add rdf:type to subjects of inserts, deletes or both
-    exhausitive: false, // true or false: find all types for a subject, even if one is already present in delta
-  },
-  contextQueries: [
-    // Case Mandatarissen
-    // They can be triggered when being inserted or when their link to a mandate is added
-    {
-      trigger: { // subjectType or predicateValue
-        subjectType: "mandaat:Mandataris"
-      },
-      queryTemplate: (subject) => `
-        ${PREFIXES}
-        CONSTRUCT {
-          ?mandataris org:holds ?mandate ;
-            ?p ?o .
-          ?orgaanInTime org:hasPost ?mandate ;
-            mandaat:isTijdspecialisatieVan ?orgaan .
-          ?orgaan besluit:bestuurt ?eenheid .
-          ?eenheid besluit:classificatie ?classification .
-        } WHERE {
-          GRAPH <http://mu.semte.ch/graphs/landing-zone/mandatarissen> {
-            VALUES ?mandataris { ${subject} }
-            ?mandataris org:holds ?mandate ;
-              ?p ?o .
-            ?orgaanInTime org:hasPost ?mandate ;
-              mandaat:isTijdspecialisatieVan ?orgaan .
-            ?orgaan besluit:bestuurt ?eenheid .
-            ?eenheid besluit:classificatie ?classification .
-          }
-        }`
-    },
-    {
-      trigger: {
-        predicateValue: "org:holds"
-      },
-      queryTemplate: (subject) => `
-        ${PREFIXES}
-        CONSTRUCT {
-          ?mandataris org:holds ?mandate ;
-            ?p ?o .
-          ?orgaanInTime org:hasPost ?mandate ;
-            mandaat:isTijdspecialisatieVan ?orgaan .
-          ?orgaan besluit:bestuurt ?eenheid .
-          ?eenheid besluit:classificatie ?classification .
-        } WHERE {
-          GRAPH <http://mu.semte.ch/graphs/landing-zone/mandatarissen> {
-            VALUES ?mandataris { ${subject} }
-            ?mandataris org:holds ?mandate ;
-              ?p ?o .
-            ?orgaanInTime org:hasPost ?mandate ;
-              mandaat:isTijdspecialisatieVan ?orgaan .
-            ?orgaan besluit:bestuurt ?eenheid .
-            ?eenheid besluit:classificatie ?classification .
-          }
-        }`
-    }
-  ]
-}
-
-module.exports = {
-  contextConfig,
-  PREFIXES
-};

config/loket-consumer/mandatarissen-dispatching/delta-sync-dispatching.js

@@ -1,85 +0,0 @@
-const {
-  transformStatements,
-  deleteFromTargetGraph,
-  insertIntoTargetGraph,
-} = require('./util');
-
-/**
-* Dispatch the fetched information to a target graph.
-* Note: <share://file/data> will be ADDED to it's own graph.
-*   We take only care of adding them, not updating triples, this is a TODO
-* @param { mu, muAuthSudo } lib - The provided libraries from the host service.
-* @param { termObjectChangeSets: { deletes, inserts } } data - The fetched changes sets, which objects of serialized Terms
-*          [ {
-*              graph: "<http://foo>",
-*              subject: "<http://bar>",
-*              predicate: "<http://baz>",
-*              object: "<http://boom>^^<http://datatype>"
-*            }
-*         ]
-* @return {void} Nothing
-*/
-async function dispatch(lib, data) {
-  const { mu, fetch } = lib;
-  let { termObjectChangeSets, termObjectChangeSetsWithContext } = data;
-
-  // Both arrays are the same length, so we can zip them together for easier processing
-  zippedChangeSets = termObjectChangeSets.map((o, i) => ({
-    original: o,
-    withContext: termObjectChangeSetsWithContext[i]
-  }));
-
-  console.log(`Received ${zippedChangeSets.length} change sets`)
-
-  for (let { original, withContext } of zippedChangeSets) {
-    let originalDeletes, originalInserts, insertsWithContext, deletesWithContext;
-    try {
-      originalInserts = original.inserts.map(o => `${o.subject} ${o.predicate} ${o.object}.`);
-      originalDeletes = original.deletes.map(o => `${o.subject} ${o.predicate} ${o.object}.`);
-    } catch (e) {
-      console.log(`Error while processing change set: ${e}`);
-      console.log(`Original change set: ${JSON.stringify(original)}`);
-      throw e;
-    }
-
-    // Extra context needed for mapping from DL to OP model and filtering based on type.
-    try {
-      insertsWithContext = withContext.inserts.map(o => `${o.subject} ${o.predicate} ${o.object}.`);
-      deletesWithContext = withContext.deletes.map(o => `${o.subject} ${o.predicate} ${o.object}.`);
-    } catch (e) {
-      console.log(`Error while processing change set with context: ${e}`);
-      console.log(`Change set with context: ${JSON.stringify(withContext)}`);
-      throw e;
-    }
-
-    if (originalDeletes.length) {
-      // Map deletes from DL to OP model
-      const transformedDeletes = await transformStatements(fetch, deletesWithContext);
-
-      if (!transformedDeletes.length) {
-        console.log(`Warn: Delete statements mapped to empty result.`);
-        console.log(`Input: ${deletesWithContext}`);
-        console.log(`Output: ${transformedDeletes}`);
-      } else {
-        await deleteFromTargetGraph(lib, transformedDeletes);
-      }
-    }
-
-    if (originalInserts.length) {
-      // Map inserts from DL to OP model
-      const transformedInserts = await transformStatements(fetch, insertsWithContext);
-
-      if (!transformedInserts.length) {
-        console.log(`Warn: Insert statements mapped to empty result.`);
-        console.log(`Input: ${insertsWithContext}`);
-        console.log(`Output: ${transformedInserts}`);
-      } else {
-        await insertIntoTargetGraph(lib, transformedInserts);
-      }
-    }
-  }
-}
-
-module.exports = {
-  dispatch
-};

config/loket-consumer/mandatarissen-dispatching/initial-sync-dispatching.js

@@ -1,85 +0,0 @@
-const { batchedDbUpdate, transformLandingZoneGraph } = require('./util');
-const {
-  BYPASS_MU_AUTH_FOR_EXPENSIVE_QUERIES,
-  DIRECT_DATABASE_ENDPOINT,
-  MU_CALL_SCOPE_ID_INITIAL_SYNC,
-  BATCH_SIZE,
-  MAX_DB_RETRY_ATTEMPTS,
-  SLEEP_BETWEEN_BATCHES,
-  SLEEP_TIME_AFTER_FAILED_DB_OPERATION,
-  TARGET_GRAPH,
-  LANDING_ZONE_GRAPH,
-  LANDING_ZONE_DATABASE_ENDPOINT
-} = require('./config');
-
-const endpoint = BYPASS_MU_AUTH_FOR_EXPENSIVE_QUERIES ? DIRECT_DATABASE_ENDPOINT : process.env.MU_SPARQL_ENDPOINT;
-
-/**
-* Dispatch the fetched information to a target graph.
-* @param { mu, muAuthSudo } lib - The provided libraries from the host service.
-* @param { termObjects } data - The fetched quad information, which objects of serialized Terms
-*          [ {
-*              graph: "<http://foo>",
-*              subject: "<http://bar>",
-*              predicate: "<http://baz>",
-*              object: "<http://boom>^^<http://datatype>"
-*            }
-*         ]
-* @return {void} Nothing
-*/
-async function dispatch(lib, data) {
-  const { mu, muAuthSudo } = lib;
-  const { termObjects } = data;
-
-  // Steps:
-  // 1. Write the triples to the landing zone graph - withouth any mapping or filtering
-  // 2. One-time reasoning run with the full landing zone graph and write the results to the target graph (on-finish)
-
-  if (BYPASS_MU_AUTH_FOR_EXPENSIVE_QUERIES) {
-    console.warn(`Service configured to skip MU_AUTH!`);
-  }
-  console.log(`Using ${endpoint} to insert triples`);
-
-  if (termObjects.length) {
-    const originalInsertTriples = termObjects.map(o => `${o.subject} ${o.predicate} ${o.object}.`);
-
-    await batchedDbUpdate(
-      muAuthSudo.updateSudo,
-      LANDING_ZONE_GRAPH,
-      originalInsertTriples,
-      { 'mu-call-scope-id': MU_CALL_SCOPE_ID_INITIAL_SYNC },
-      LANDING_ZONE_DATABASE_ENDPOINT,
-      BATCH_SIZE,
-      MAX_DB_RETRY_ATTEMPTS,
-      SLEEP_BETWEEN_BATCHES,
-      SLEEP_TIME_AFTER_FAILED_DB_OPERATION
-    );
-  }
-}
-
-async function onFinishInitialIngest(lib) {
-  const { muAuthSudo, fetch } = lib;
-
-  console.log(`!! On-finish triggered !!`);
-
-  const transformedInsertTriples = await transformLandingZoneGraph(fetch, endpoint);
-
-  console.log(`Transformed ${transformedInsertTriples.length} triples`);
-
-  await batchedDbUpdate(
-    muAuthSudo.updateSudo,
-    TARGET_GRAPH,
-    transformedInsertTriples,
-    { 'mu-call-scope-id': MU_CALL_SCOPE_ID_INITIAL_SYNC },
-    endpoint,
-    BATCH_SIZE,
-    MAX_DB_RETRY_ATTEMPTS,
-    SLEEP_BETWEEN_BATCHES,
-    SLEEP_TIME_AFTER_FAILED_DB_OPERATION
-  );
-}
-
-module.exports = {
-  dispatch,
-  onFinishInitialIngest
-};

config/loket-consumer/mandatarissen-dispatching/util.js

@@ -1,168 +0,0 @@
-const {
-  MAX_REASONING_RETRY_ATTEMPTS,
-  SLEEP_TIME_AFTER_FAILED_REASONING_OPERATION,
-  MAX_DB_RETRY_ATTEMPTS,
-  SLEEP_BETWEEN_BATCHES,
-  SLEEP_TIME_AFTER_FAILED_DB_OPERATION,
-  TARGET_GRAPH,
-  LANDING_ZONE_GRAPH,
-  BATCH_SIZE
-} = require('./config');
-
-async function batchedDbUpdate(
-  muUpdate,
-  graph,
-  triples,
-  extraHeaders,
-  endpoint,
-  batchSize,
-  maxAttempts,
-  sleepBetweenBatches = 1000,
-  sleepTimeOnFail = 1000,
-  operation = 'INSERT'
-) {
-
-  for (let i = 0; i < triples.length; i += batchSize) {
-    console.log(`Inserting triples in batch: ${i}-${i + batchSize}`);
-
-    const batch = triples.slice(i, i + batchSize).join('\n');
-
-    const insertCall = async () => {
-      await muUpdate(`
-${operation} DATA {
-GRAPH <${graph}> {
-${batch}
-}
-}
-`, extraHeaders, endpoint);
-    };
-
-    await operationWithRetry(insertCall, 0, maxAttempts, sleepTimeOnFail);
-
-    console.log(`Sleeping before next query execution: ${sleepBetweenBatches}`);
-    await new Promise(r => setTimeout(r, sleepBetweenBatches));
-  }
-}
-
-async function operationWithRetry(callback,
-  attempt,
-  maxAttempts,
-  sleepTimeOnFail) {
-  try {
-    if (typeof callback === "function")
-      return await callback();
-    else // Catch error from promise - not how I would do it normally, but allows re use of existing code.
-      return await callback;
-  }
-  catch (e) {
-    console.log(`Operation failed for ${callback.toString()}, attempt: ${attempt} of ${maxAttempts}`);
-    console.log(`Error: ${e}`);
-    console.log(`Sleeping ${sleepTimeOnFail} ms`);
-
-    if (attempt >= maxAttempts) {
-      console.log(`Max attempts reached for ${callback.toString()}, giving up`);
-      throw e;
-    }
-
-    await new Promise(r => setTimeout(r, sleepTimeOnFail));
-    return operationWithRetry(callback, ++attempt, maxAttempts, sleepTimeOnFail);
-  }
-}
-
-/**
- * Splits an array into two parts, a part that passes and a part that fails a predicate function.
- * Credits: https://github.com/benjay10
- * @public
- * @function partition
- * @param {Array} arr - Array to be partitioned
- * @param {Function} fn - Function that accepts single argument: an element of the array, and should return a truthy or falsy value.
- * @returns {Object} Object that contains keys passes and fails, each representing an array with elemets that pass or fail the predicate respectively
- */
-function partition(arr, fn) {
-  let passes = [], fails = [];
-  arr.forEach((item) => (fn(item) ? passes : fails).push(item));
-  return { passes, fails };
-}
-
-/**
- * Send triples to reasoning service for conversion
- *
- */
-async function transformTriples(fetch, triples, mapping) {
-  return await operationWithRetry(mainConversion(fetch, triples, mapping), 0,
-    MAX_REASONING_RETRY_ATTEMPTS, SLEEP_TIME_AFTER_FAILED_REASONING_OPERATION);
-}
-
-async function mainConversion(fetch, triples, mapping) {
-  let formdata = new URLSearchParams();
-  formdata.append("data", triples);
-
-  let requestOptions = {
-    method: 'POST',
-    body: formdata,
-    redirect: 'follow'
-  };
-
-  const response = await fetch(`http://reasoner/reason/dl2op/${mapping}`, requestOptions)
-  return await response.text();
-}
-
-async function transformStatements(fetch, triples, mapping = 'main') {
-  console.log(`Received ${JSON.stringify(triples)} to transform`);
-  const transformedTriples = await transformTriples(fetch, triples.join('\n'), mapping);
-  statements = transformedTriples ? transformedTriples.replace(/\n{2,}/g, '').split('\n') : [];
-  console.log(`CONVERSION: FROM ${triples.length} triples to ${statements.length}`);
-  return statements;
-}
-
-async function deleteFromTargetGraph(lib, statements) {
-  console.log(`Deleting ${statements.length} statements from target graph`);
-  console.log(`Statements:  ${JSON.stringify(statements)}`)
-  await batchedDbUpdate(
-    lib.muAuthSudo.updateSudo,
-    TARGET_GRAPH,
-    statements,
-    {},
-    process.env.MU_SPARQL_ENDPOINT,
-    BATCH_SIZE,
-    MAX_DB_RETRY_ATTEMPTS,
-    SLEEP_BETWEEN_BATCHES,
-    SLEEP_TIME_AFTER_FAILED_DB_OPERATION,
-    'DELETE');
-}
-
-async function insertIntoTargetGraph(lib, statements) {
-  console.log(`Inserting ${statements.length} statements into target graph`);
-  console.log(`Statements:  ${JSON.stringify(statements)}`)
-
-  await batchedDbUpdate(
-    lib.muAuthSudo.updateSudo,
-    TARGET_GRAPH,
-    statements,
-    {},
-    process.env.MU_SPARQL_ENDPOINT,
-    BATCH_SIZE,
-    MAX_DB_RETRY_ATTEMPTS,
-    SLEEP_BETWEEN_BATCHES,
-    SLEEP_TIME_AFTER_FAILED_DB_OPERATION,
-    'INSERT');
-}
-
-async function transformLandingZoneGraph(fetch, endpoint, mapping = 'main') {
-  console.log(`Transforming landing zone graph: ${LANDING_ZONE_GRAPH}`);
-
-  const response = await fetch(`http://reasoner/reason/dl2op/${mapping}?data=${encodeURIComponent(`${endpoint}?default-graph-uri=&query=CONSTRUCT+%7B%0D%0A%3Fs+%3Fp+%3Fo%0D%0A%7D+WHERE+%7B%0D%0A+GRAPH+<${LANDING_ZONE_GRAPH}>+%7B%0D%0A%3Fs+%3Fp+%3Fo%0D%0A%7D%0D%0A%7D&should-sponge=&format=text%2Fplain&timeout=0&run=+Run+Query`)}`);
-  const text = await response.text();
-  const statements = text.replace(/\n{2,}/g, '').split('\n');
-
-  return statements;
-}
-
-module.exports = {
-  batchedDbUpdate,
-  partition,
-  transformStatements,
-  transformLandingZoneGraph,
-  deleteFromTargetGraph,
-  insertIntoTargetGraph
-};

config/loket-consumer/worship-services-sensitive-dispatching/config.js

@@ -1,40 +0,0 @@
-const BATCH_SIZE = parseInt(process.env.BATCH_SIZE) || 100;
-const MU_CALL_SCOPE_ID_INITIAL_SYNC = process.env.MU_CALL_SCOPE_ID_INITIAL_SYNC || 'http://redpencil.data.gift/id/concept/muScope/deltas/consumer/initialSync';
-const BYPASS_MU_AUTH_FOR_EXPENSIVE_QUERIES = process.env.BYPASS_MU_AUTH_FOR_EXPENSIVE_QUERIES == 'true' ? true : false;
-const DIRECT_DATABASE_ENDPOINT = process.env.DIRECT_DATABASE_ENDPOINT || 'http://triplestore:8890/sparql';
-const MAX_DB_RETRY_ATTEMPTS = parseInt(process.env.MAX_DB_RETRY_ATTEMPTS || 5);
-const MAX_REASONING_RETRY_ATTEMPTS = parseInt(process.env.MAX_REASONING_RETRY_ATTEMPTS || 5);
-const SLEEP_BETWEEN_BATCHES = parseInt(process.env.SLEEP_BETWEEN_BATCHES || 1000);
-const SLEEP_TIME_AFTER_FAILED_REASONING_OPERATION = parseInt(process.env.SLEEP_TIME_AFTER_FAILED_REASONING_OPERATION || 10000);
-const SLEEP_TIME_AFTER_FAILED_DB_OPERATION = parseInt(process.env.SLEEP_TIME_AFTER_FAILED_DB_OPERATION || 60000);
-
-// Graphs
-const TARGET_GRAPH = process.env.TARGET_GRAPH || `http://mu.semte.ch/graphs/worship-service`;
-const PRIVACY_SENSITIVE_GRAPH = process.env.PRIVACY_SENSITIVE_GRAPH || `http://mu.semte.ch/graphs/worship-privacy-centric-graph`;
-const LANDING_ZONE_GRAPH = process.env.DCR_LANDING_ZONE_GRAPH || `http://mu.semte.ch/graphs/landing-zone/leidinggevenden`;
-
-// Mappings
-const MAIN_INFO_MAPPING = "main-info";
-const PRIVATE_INFO_MAPPING = "private-info";
-
-// ENDPOINTS
-const LANDING_ZONE_DATABASE = process.env.DCR_LANDING_ZONE_DATABASE || 'db';
-const LANDING_ZONE_DATABASE_ENDPOINT = process.env.DCR_LANDING_ZONE_DATABASE_ENDPOINT || `http://${LANDING_ZONE_DATABASE}:8890/sparql`;
-
-module.exports = {
-  BATCH_SIZE,
-  MU_CALL_SCOPE_ID_INITIAL_SYNC,
-  BYPASS_MU_AUTH_FOR_EXPENSIVE_QUERIES,
-  DIRECT_DATABASE_ENDPOINT,
-  MAX_DB_RETRY_ATTEMPTS,
-  MAX_REASONING_RETRY_ATTEMPTS,
-  SLEEP_BETWEEN_BATCHES,
-  SLEEP_TIME_AFTER_FAILED_DB_OPERATION,
-  SLEEP_TIME_AFTER_FAILED_REASONING_OPERATION,
-  LANDING_ZONE_GRAPH,
-  LANDING_ZONE_DATABASE_ENDPOINT,
-  TARGET_GRAPH,
-  PRIVACY_SENSITIVE_GRAPH,
-  MAIN_INFO_MAPPING,
-  PRIVATE_INFO_MAPPING
-};

config/loket-consumer/worship-services-sensitive-dispatching/delta-context-config.js

@@ -1,11 +0,0 @@
-const contextConfig = {
-  addTypes: {
-    scope: 'all', // 'inserts, 'deletes', 'all' or none. To add rdf:type to subjects of inserts, deletes or both
-    exhausitive: false, // true or false: find all types for a subject, even if one is already present in delta
-  },
-  contextQueries: []
-}
-
-module.exports = {
-  contextConfig
-};

config/loket-consumer/worship-services-sensitive-dispatching/delta-sync-dispatching.js

@@ -1,114 +0,0 @@
-const {
-  transformStatements,
-  deleteFromTargetGraph,
-  insertIntoTargetGraph,
-  deleteFromPrivateGraph,
-  insertIntoPrivateGraph
-} = require('./util');
-
-const {
-  MAIN_INFO_MAPPING,
-  PRIVATE_INFO_MAPPING
-} = require('./config');
-
-/**
-* Dispatch the fetched information to a target graph.
-* Note: <share://file/data> will be ADDED to it's own graph.
-*   We take only care of adding them, not updating triples, this is a TODO
-* @param { mu, muAuthSudo } lib - The provided libraries from the host service.
-* @param { termObjectChangeSets: { deletes, inserts } } data - The fetched changes sets, which objects of serialized Terms
-*          [ {
-*              graph: "<http://foo>",
-*              subject: "<http://bar>",
-*              predicate: "<http://baz>",
-*              object: "<http://boom>^^<http://datatype>"
-*            }
-*         ]
-* @return {void} Nothing
-*/
-async function dispatch(lib, data) {
-  const { mu, fetch } = lib;
-  let { termObjectChangeSets, termObjectChangeSetsWithContext } = data;
-
-  // Both arrays are the same length, so we can zip them together for easier processing
-  zippedChangeSets = termObjectChangeSets.map((o, i) => ({
-    original: o,
-    withContext: termObjectChangeSetsWithContext[i]
-  }));
-
-  console.log(`Received ${zippedChangeSets.length} change sets`)
-
-  for (let { original, withContext } of zippedChangeSets) {
-    let originalDeletes, originalInserts, insertsWithContext, deletesWithContext;
-    try {
-      originalInserts = original.inserts.map(o => `${o.subject} ${o.predicate} ${o.object}.`);
-      originalDeletes = original.deletes.map(o => `${o.subject} ${o.predicate} ${o.object}.`);
-    } catch (e) {
-      console.log(`Error while processing change set: ${e}`);
-      console.log(`Original change set: ${JSON.stringify(original)}`);
-      throw e;
-    }
-
-    // Extra context needed for mapping from DL to OP model and filtering based on type.
-    try {
-      insertsWithContext = withContext.inserts.map(o => `${o.subject} ${o.predicate} ${o.object}.`);
-      deletesWithContext = withContext.deletes.map(o => `${o.subject} ${o.predicate} ${o.object}.`);
-    } catch (e) {
-      console.log(`Error while processing change set with context: ${e}`);
-      console.log(`Change set with context: ${JSON.stringify(withContext)}`);
-      throw e;
-    }
-
-    if (originalDeletes.length) {
-      // Map main deletes from DL to OP model
-      const transformedMainDeletes = await transformStatements(fetch, deletesWithContext, MAIN_INFO_MAPPING);
-
-      if (!transformedMainDeletes.length) {
-        console.log(`Warn: Delete main statements mapped to empty result.`);
-        console.log(`Input: ${deletesWithContext}`);
-        console.log(`Output: ${transformedMainDeletes}`);
-      } else {
-        await deleteFromTargetGraph(lib, transformedMainDeletes);
-      }
-
-      // Map private deletes from DL to OP model
-      const transformedPrivateDeletes = await transformStatements(fetch, deletesWithContext, PRIVATE_INFO_MAPPING);
-
-      if (!transformedPrivateDeletes.length) {
-        console.log(`Warn: Delete private statements mapped to empty result.`);
-        console.log(`Input: ${deletesWithContext}`);
-        console.log(`Output: ${transformedPrivateDeletes}`);
-      } else {
-        await deleteFromPrivateGraph(lib, transformedPrivateDeletes);
-      }
-    }
-
-    if (originalInserts.length) {
-      // Map main inserts from DL to OP model
-      const transformedMainInserts = await transformStatements(fetch, insertsWithContext, MAIN_INFO_MAPPING);
-
-      if (!transformedMainInserts.length) {
-        console.log(`Warn: Insert statements mapped to empty result.`);
-        console.log(`Input: ${insertsWithContext}`);
-        console.log(`Output: ${transformedMainInserts}`);
-      } else {
-        await insertIntoTargetGraph(lib, transformedMainInserts);
-      }
-
-      // Map private inserts from DL to OP model
-      const transformedPrivateInserts = await transformStatements(fetch, insertsWithContext, PRIVATE_INFO_MAPPING);
-
-      if (!transformedPrivateInserts.length) {
-        console.log(`Warn: Insert statements mapped to empty result.`);
-        console.log(`Input: ${insertsWithContext}`);
-        console.log(`Output: ${transformedPrivateInserts}`);
-      } else {
-        await insertIntoPrivateGraph(lib, transformedPrivateInserts);
-      }
-    }
-  }
-}
-
-module.exports = {
-  dispatch
-};

config/loket-consumer/worship-services-sensitive-dispatching/initial-sync-dispatching.js

@@ -1,106 +0,0 @@
-const { batchedDbUpdate, transformLandingZoneGraph } = require('./util');
-const {
-  BYPASS_MU_AUTH_FOR_EXPENSIVE_QUERIES,
-  DIRECT_DATABASE_ENDPOINT,
-  MU_CALL_SCOPE_ID_INITIAL_SYNC,
-  BATCH_SIZE,
-  MAX_DB_RETRY_ATTEMPTS,
-  SLEEP_BETWEEN_BATCHES,
-  SLEEP_TIME_AFTER_FAILED_DB_OPERATION,
-  TARGET_GRAPH,
-  PRIVACY_SENSITIVE_GRAPH,
-  LANDING_ZONE_GRAPH,
-  LANDING_ZONE_DATABASE_ENDPOINT,
-  MAIN_INFO_MAPPING,
-  PRIVATE_INFO_MAPPING
-} = require('./config');
-
-const endpoint = BYPASS_MU_AUTH_FOR_EXPENSIVE_QUERIES ? DIRECT_DATABASE_ENDPOINT : process.env.MU_SPARQL_ENDPOINT;
-
-/**
-* Dispatch the fetched information to a target graph.
-* @param { mu, muAuthSudo } lib - The provided libraries from the host service.
-* @param { termObjects } data - The fetched quad information, which objects of serialized Terms
-*          [ {
-*              graph: "<http://foo>",
-*              subject: "<http://bar>",
-*              predicate: "<http://baz>",
-*              object: "<http://boom>^^<http://datatype>"
-*            }
-*         ]
-* @return {void} Nothing
-*/
-async function dispatch(lib, data) {
-  const { mu, muAuthSudo } = lib;
-  const { termObjects } = data;
-
-  // Steps:
-  // 1. Write the triples to the landing zone graph - withouth any mapping or filtering
-  // 2. One-time reasoning run with the full landing zone graph and write the results to the target graph (on-finish)
-
-  if (BYPASS_MU_AUTH_FOR_EXPENSIVE_QUERIES) {
-    console.warn(`Service configured to skip MU_AUTH!`);
-  }
-  console.log(`Using ${endpoint} to insert triples`);
-
-  if (termObjects.length) {
-    const originalInsertTriples = termObjects.map(o => `${o.subject} ${o.predicate} ${o.object}.`);
-
-    await batchedDbUpdate(
-      muAuthSudo.updateSudo,
-      LANDING_ZONE_GRAPH,
-      originalInsertTriples,
-      { 'mu-call-scope-id': MU_CALL_SCOPE_ID_INITIAL_SYNC },
-      LANDING_ZONE_DATABASE_ENDPOINT,
-      BATCH_SIZE,
-      MAX_DB_RETRY_ATTEMPTS,
-      SLEEP_BETWEEN_BATCHES,
-      SLEEP_TIME_AFTER_FAILED_DB_OPERATION
-    );
-  }
-}
-
-async function onFinishInitialIngest(lib) {
-  const { muAuthSudo, fetch } = lib;
-
-  console.log(`!! On-finish triggered !!`);
-
-  const transformedMainInsertTriples = await transformLandingZoneGraph(fetch, endpoint, MAIN_INFO_MAPPING);
-  console.log(`Transformed ${transformedMainInsertTriples.length} main triples`);
-
-  if (transformedMainInsertTriples.length) {
-    await batchedDbUpdate(
-      muAuthSudo.updateSudo,
-      TARGET_GRAPH,
-      transformedMainInsertTriples,
-      { 'mu-call-scope-id': MU_CALL_SCOPE_ID_INITIAL_SYNC },
-      endpoint,
-      BATCH_SIZE,
-      MAX_DB_RETRY_ATTEMPTS,
-      SLEEP_BETWEEN_BATCHES,
-      SLEEP_TIME_AFTER_FAILED_DB_OPERATION
-    );
-  }
-
-  const transformedPrivateInsertTriples = await transformLandingZoneGraph(fetch, endpoint, PRIVATE_INFO_MAPPING);
-  console.log(`Transformed ${transformedPrivateInsertTriples.length} private triples`);
-
-  if (transformedPrivateInsertTriples.length) {
-    await batchedDbUpdate(
-      muAuthSudo.updateSudo,
-      PRIVACY_SENSITIVE_GRAPH,
-      transformedPrivateInsertTriples,
-      { 'mu-call-scope-id': MU_CALL_SCOPE_ID_INITIAL_SYNC },
-      endpoint,
-      BATCH_SIZE,
-      MAX_DB_RETRY_ATTEMPTS,
-      SLEEP_BETWEEN_BATCHES,
-      SLEEP_TIME_AFTER_FAILED_DB_OPERATION
-    );
-  }
-}
-
-module.exports = {
-  dispatch,
-  onFinishInitialIngest
-};

config/loket-consumer/worship-services-sensitive-dispatching/util.js

@@ -1,204 +0,0 @@
-const {
-  MAX_REASONING_RETRY_ATTEMPTS,
-  SLEEP_TIME_AFTER_FAILED_REASONING_OPERATION,
-  MAX_DB_RETRY_ATTEMPTS,
-  SLEEP_BETWEEN_BATCHES,
-  SLEEP_TIME_AFTER_FAILED_DB_OPERATION,
-  TARGET_GRAPH,
-  PRIVACY_SENSITIVE_GRAPH,
-  LANDING_ZONE_GRAPH,
-  BATCH_SIZE
-} = require('./config');
-
-async function batchedDbUpdate(
-  muUpdate,
-  graph,
-  triples,
-  extraHeaders,
-  endpoint,
-  batchSize,
-  maxAttempts,
-  sleepBetweenBatches = 1000,
-  sleepTimeOnFail = 1000,
-  operation = 'INSERT'
-) {
-
-  for (let i = 0; i < triples.length; i += batchSize) {
-    console.log(`Inserting triples in batch: ${i}-${i + batchSize}`);
-
-    const batch = triples.slice(i, i + batchSize).join('\n');
-
-    const insertCall = async () => {
-      await muUpdate(`
-${operation} DATA {
-GRAPH <${graph}> {
-${batch}
-}
-}
-`, extraHeaders, endpoint);
-    };
-
-    await operationWithRetry(insertCall, 0, maxAttempts, sleepTimeOnFail);
-
-    console.log(`Sleeping before next query execution: ${sleepBetweenBatches}`);
-    await new Promise(r => setTimeout(r, sleepBetweenBatches));
-  }
-}
-
-async function operationWithRetry(callback,
-  attempt,
-  maxAttempts,
-  sleepTimeOnFail) {
-  try {
-    if (typeof callback === "function")
-      return await callback();
-    else // Catch error from promise - not how I would do it normally, but allows re use of existing code.
-      return await callback;
-  }
-  catch (e) {
-    console.log(`Operation failed for ${callback.toString()}, attempt: ${attempt} of ${maxAttempts}`);
-    console.log(`Error: ${e}`);
-    console.log(`Sleeping ${sleepTimeOnFail} ms`);
-
-    if (attempt >= maxAttempts) {
-      console.log(`Max attempts reached for ${callback.toString()}, giving up`);
-      throw e;
-    }
-
-    await new Promise(r => setTimeout(r, sleepTimeOnFail));
-    return operationWithRetry(callback, ++attempt, maxAttempts, sleepTimeOnFail);
-  }
-}
-
-/**
- * Splits an array into two parts, a part that passes and a part that fails a predicate function.
- * Credits: https://github.com/benjay10
- * @public
- * @function partition
- * @param {Array} arr - Array to be partitioned
- * @param {Function} fn - Function that accepts single argument: an element of the array, and should return a truthy or falsy value.
- * @returns {Object} Object that contains keys passes and fails, each representing an array with elemets that pass or fail the predicate respectively
- */
-function partition(arr, fn) {
-  let passes = [], fails = [];
-  arr.forEach((item) => (fn(item) ? passes : fails).push(item));
-  return { passes, fails };
-}
-
-/**
- * Send triples to reasoning service for conversion
- *
- */
-async function transformTriples(fetch, triples, mapping) {
-  return await operationWithRetry(mainConversion(fetch, triples, mapping), 0,
-    MAX_REASONING_RETRY_ATTEMPTS, SLEEP_TIME_AFTER_FAILED_REASONING_OPERATION);
-}
-
-async function mainConversion(fetch, triples, mapping) {
-  let formdata = new URLSearchParams();
-  formdata.append("data", triples);
-
-  let requestOptions = {
-    method: 'POST',
-    body: formdata,
-    redirect: 'follow'
-  };
-
-  const response = await fetch(`http://reasoner/reason/dl2op/${mapping}`, requestOptions)
-  return await response.text();
-}
-
-async function transformStatements(fetch, triples, mapping = 'main') {
-  console.log(`Received ${JSON.stringify(triples)} to transform`);
-  const transformedTriples = await transformTriples(fetch, triples.join('\n'), mapping);
-  statements = transformedTriples ? transformedTriples.replace(/\n{2,}/g, '').split('\n') : [];
-  console.log(`CONVERSION: FROM ${triples.length} triples to ${statements.length}`);
-  return statements;
-}
-
-async function deleteFromTargetGraph(lib, statements) {
-  console.log(`Deleting ${statements.length} statements from target graph`);
-  console.log(`Statements:  ${JSON.stringify(statements)}`)
-  await batchedDbUpdate(
-    lib.muAuthSudo.updateSudo,
-    TARGET_GRAPH,
-    statements,
-    {},
-    process.env.MU_SPARQL_ENDPOINT,
-    BATCH_SIZE,
-    MAX_DB_RETRY_ATTEMPTS,
-    SLEEP_BETWEEN_BATCHES,
-    SLEEP_TIME_AFTER_FAILED_DB_OPERATION,
-    'DELETE');
-}
-
-async function deleteFromPrivateGraph(lib, statements) {
-  console.log(`Deleting ${statements.length} statements from target graph`);
-  console.log(`Statements:  ${JSON.stringify(statements)}`)
-  await batchedDbUpdate(
-    lib.muAuthSudo.updateSudo,
-    PRIVACY_SENSITIVE_GRAPH,
-    statements,
-    {},
-    process.env.MU_SPARQL_ENDPOINT,
-    BATCH_SIZE,
-    MAX_DB_RETRY_ATTEMPTS,
-    SLEEP_BETWEEN_BATCHES,
-    SLEEP_TIME_AFTER_FAILED_DB_OPERATION,
-    'DELETE');
-}
-
-async function insertIntoTargetGraph(lib, statements) {
-  console.log(`Inserting ${statements.length} statements into target graph`);
-  console.log(`Statements:  ${JSON.stringify(statements)}`)
-
-  await batchedDbUpdate(
-    lib.muAuthSudo.updateSudo,
-    TARGET_GRAPH,
-    statements,
-    {},
-    process.env.MU_SPARQL_ENDPOINT,
-    BATCH_SIZE,
-    MAX_DB_RETRY_ATTEMPTS,
-    SLEEP_BETWEEN_BATCHES,
-    SLEEP_TIME_AFTER_FAILED_DB_OPERATION,
-    'INSERT');
-}
-
-async function insertIntoPrivateGraph(lib, statements) {
-  console.log(`Inserting ${statements.length} statements into target graph`);
-  console.log(`Statements:  ${JSON.stringify(statements)}`)
-
-  await batchedDbUpdate(
-    lib.muAuthSudo.updateSudo,
-    PRIVACY_SENSITIVE_GRAPH,
-    statements,
-    {},
-    process.env.MU_SPARQL_ENDPOINT,
-    BATCH_SIZE,
-    MAX_DB_RETRY_ATTEMPTS,
-    SLEEP_BETWEEN_BATCHES,
-    SLEEP_TIME_AFTER_FAILED_DB_OPERATION,
-    'INSERT');
-}
-
-async function transformLandingZoneGraph(fetch, endpoint, mapping = 'main') {
-  console.log(`Transforming landing zone graph: ${LANDING_ZONE_GRAPH}`);
-
-  const response = await fetch(`http://reasoner/reason/dl2op/${mapping}?data=${encodeURIComponent(`${endpoint}?default-graph-uri=&query=CONSTRUCT+%7B%0D%0A%3Fs+%3Fp+%3Fo%0D%0A%7D+WHERE+%7B%0D%0A+GRAPH+<${LANDING_ZONE_GRAPH}>+%7B%0D%0A%3Fs+%3Fp+%3Fo%0D%0A%7D%0D%0A%7D&should-sponge=&format=text%2Fplain&timeout=0&run=+Run+Query`)}`);
-  const text = await response.text();
-  const statements = text.replace(/\n{2,}/g, '').split('\n');
-
-  return statements;
-}
-
-module.exports = {
-  batchedDbUpdate,
-  partition,
-  transformStatements,
-  transformLandingZoneGraph,
-  deleteFromTargetGraph,
-  insertIntoTargetGraph,
-  deleteFromPrivateGraph,
-  insertIntoPrivateGraph
-};

config/migrations-triggering-indexing/2024/20241119101320-fix-classification-Voetbal-in-de-stad-vzw.sparql

@@ -0,0 +1,11 @@
+DELETE DATA {
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    <http://data.lblod.info/id/geregistreerdeOrganisaties/8b2e8749-98a9-450f-b2f1-7a93115ef513> <http://www.w3.org/ns/org#classification> <http://data.vlaanderen.be/id/concept/GeregistreerdeOrganisatieClassificatieCode/35833ba2-7371-400b-8df2-2912f66fb153> .
+  }
+}
+;
+INSERT DATA {
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    <http://data.lblod.info/id/geregistreerdeOrganisaties/8b2e8749-98a9-450f-b2f1-7a93115ef513> <http://www.w3.org/ns/org#classification> <http://data.vlaanderen.be/id/concept/GeregistreerdeOrganisatieClassificatieCode/3e5c8e30-95a7-47b0-b0a4-210d5bd440a7> .
+  }
+}

config/migrations-triggering-indexing/2024/20241126155527-rename-merged-municipalities-and-ocmws.sparql

@@ -0,0 +1,33 @@
+PREFIX besluit: <http://data.vlaanderen.be/ns/besluit#>
+PREFIX regorg: <http://www.w3.org/ns/regorg#>
+PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
+
+DELETE {
+  GRAPH ?g {
+    ?organisation skos:prefLabel ?oldLabel;
+                  regorg:legalName ?oldName.
+  }
+} INSERT {
+  GRAPH ?g {
+    ?organisation skos:prefLabel ?newName;
+                  regorg:legalName ?newName.
+  }
+} WHERE {
+  GRAPH ?g {
+    ?organisation a besluit:Bestuurseenheid;
+                  skos:prefLabel ?oldLabel;
+                  regorg:legalName ?oldName.
+  }
+  VALUES ?g {
+    <http://mu.semte.ch/graphs/administrative-unit>
+    <http://mu.semte.ch/graphs/shared>
+  }
+  VALUES (?organisation ?newName) {
+    (<http://data.lblod.info/id/bestuurseenheden/99da98a7a0087d3429b084ebfc4eb5d488c593790d4d5af7253982a2e21a6a5f> """Bilzen-Hoeselt""")
+    (<http://data.lblod.info/id/bestuurseenheden/9ae900a5447b7d727ca6496910220d4389aba7f1869923f1bbf9729bdeca28e2> """Bilzen-Hoeselt""")
+    (<http://data.lblod.info/id/bestuurseenheden/05099fa1f6524b8b994d86f61549455d2c00b2a956d5308683ac2d1f810fc729> """Tessenderlo-Ham""")
+    (<http://data.lblod.info/id/bestuurseenheden/42a43591e0db1dca9432f480f0f49f9bd4056c2b131e2fc997497130f5e099d0> """Tessenderlo-Ham""")
+    (<http://data.lblod.info/id/bestuurseenheden/104f32d7fb8d4b8b61b71717301656f136fe046eabaf126fb3325896b5c2d625> """Tongeren-Borgloon""")
+    (<http://data.lblod.info/id/bestuurseenheden/ab684633d605d93dbbe6b9ea40667e2bcf03a0856cafe1825e95b7829ed502a3> """Tongeren-Borgloon""")
+  }
+}

config/migrations-triggering-indexing/2024/20241203144827-mergers-2025-update-organisation-status.sparql

@@ -0,0 +1,102 @@
+PREFIX regorg: <http://www.w3.org/ns/regorg#>
+
+DELETE {
+  GRAPH ?g {
+    ?org regorg:orgStatus ?oldStatus.
+  }
+} INSERT {
+  GRAPH ?g {
+    ?org  regorg:orgStatus <http://lblod.data.gift/concepts/d02c4e12bf88d2fdf5123b07f29c9311>.
+  }
+} WHERE {
+  GRAPH ?g {
+    ?org regorg:orgStatus ?oldStatus.
+  }
+  VALUES ?g {
+    <http://mu.semte.ch/graphs/administrative-unit>
+  }
+  VALUES ?org {
+    <http://data.lblod.info/id/bestuurseenheden/e41ffa04f94bb450a79793020e70d55b5fee5033a5280277998608a9d0913117>
+    <http://data.lblod.info/id/bestuurseenheden/7125fcbfb7a08f8c01efbed115cea602db249c04afe9b5a2cc298cc7949cd040>
+
+    <http://data.lblod.info/id/bestuurseenheden/4f0eb4436c88cf831c35f84e7c34ce32f9ee4e99c5139aff62990e5e531aa1e7>
+    <http://data.lblod.info/id/bestuurseenheden/6af55cecaea3621f53bb32417a36ed6e3d41b2aa5b9f6d62ab3d80cc8ec11539>
+    <http://data.lblod.info/id/bestuurseenheden/c362abc58ac4579ff417824ce620962ac57bc344b34fe8f51d21b35ef54da36d>
+    <http://data.lblod.info/id/bestuurseenheden/72bc672ceb6894d8dd31dc2dff2faad2fd93f4b9f5347d556d092516eaa30aa3>
+    <http://data.lblod.info/id/bestuurseenheden/d3c76fd84794dad5e12c9eed6d6332dc5e5c99daf0e4ddd7ceebf13d5d6c0ccd>
+    <http://data.lblod.info/id/bestuurseenheden/bbc23313f4c8585a0c51f15f920d77123356535cd1cad9af38f5707892c7088d>
+
+    <http://data.lblod.info/id/bestuurseenheden/a3bd0845853278f478f90b14436d3efa99e73249a84d462f0ddc2e5b6e37a156>
+    <http://data.lblod.info/id/bestuurseenheden/785e8d5516378c7382ad0e7e356d2301496d810c6cd46d72ccebb455d3ac525e>
+
+    <http://data.lblod.info/id/bestuurseenheden/05441122597e0b20b61a8968ea1247c07f9014aad1f1f0709d0b1234e3dfbc2f>
+    <http://data.lblod.info/id/bestuurseenheden/3834647b5e95ea20f5ffe5a12a09fbd3d7fdf0d187fab385cb7b841422176ad6>
+
+    <http://data.lblod.info/id/bestuurseenheden/0327a51548f73607f8a5ec11b36711a3c96703686ad93a3d632718c135c295db>
+    <http://data.lblod.info/id/bestuurseenheden/e39bc997aa6dd9f240277735efd745b6a0422614d2b36cf01825c86b1b91a9ee>
+    <http://data.lblod.info/id/bestuurseenheden/0a2d980559eb9f3e0820386541274a72692f495204cd6d951ea1907ecc65829b>
+    <http://data.lblod.info/id/bestuurseenheden/144c75e403c5084af57f5c0c86252b8759b56baf9694326abbcb5ea40ab69f1a>
+
+    <http://data.lblod.info/id/bestuurseenheden/00eb231f51bd6b4f5dcc6536b2d09a174ea8583f5bf28b10bc4fc769a07e511d>
+    <http://data.lblod.info/id/bestuurseenheden/c69192b973a3ca11531b9657b3ee20aa6688fa33ea1ef1392310fec751980ab2>
+    <http://data.lblod.info/id/bestuurseenheden/3eb8c9fae32d02359dcd7b22e2a74e67a5b48388df31ad819c27c688fedd10b0>
+    <http://data.lblod.info/id/bestuurseenheden/ba8a253d98ea7d2ab1afc43e5e2cd145a12ccd86b68e71d023e3c13a83d7498f>
+    <http://data.lblod.info/id/bestuurseenheden/76c45d60f05bf8afc8c71859342b69cac6a57ee2b320b900a60afd6ba9b8eb9d>
+    <http://data.lblod.info/id/bestuurseenheden/5d21ccef2f5aa3ccbf3a74c6f25ebe9f0b1ef5a97b87f3d79c86ae9d6d857df8>
+
+    <http://data.lblod.info/id/bestuurseenheden/af8969752f6b28c66b6bc402d7987fa38774901ac72b95c5cb7976570487c3c9>
+    <http://data.lblod.info/id/bestuurseenheden/fb1be873c4b31e391613dfae8e68edd694b1fdf126eeecb502b1e5cad6f2f682>
+
+    <http://data.lblod.info/id/bestuurseenheden/f6b131de5e40a0dfd4fc93fedf3b95c9bf156ece718b87fe00469dea2564b3fb>
+    <http://data.lblod.info/id/bestuurseenheden/3e077115eeb70b649d44e2ebbb218f2cfcac5d84e130c88584f0ec23f7cebbf9>
+
+    <http://data.lblod.info/id/bestuurseenheden/1313d4a58f9ecf52cc7e274e3549a759b35e731973cc9f5e07562e5650f594dd>
+    <http://data.lblod.info/id/bestuurseenheden/7c6b38fa6fc65879cb674a654617de171adfcfacba086fb918da3749dbaa43b5>
+
+    <http://data.lblod.info/id/bestuurseenheden/c5dbb08e35e2d090a05d119fef4cc161b5ee1f322698cb8ea3c8c6643a521cf2>
+    <http://data.lblod.info/id/bestuurseenheden/f2a9a2392d5e4c0f992a84a0871d048a7b91b14681e9980a63aa6ba4b1f7eb8c>
+
+    <http://data.lblod.info/id/bestuurseenheden/c0b6b5cf198cd939251dff8ed052177cfff245074c6b8d43394c8c97f7b6e945>
+    <http://data.lblod.info/id/bestuurseenheden/8ef1e4a43efd913e6b09b0ddea344b7b5d723a344ad559389a55ae1ff0bebc8f>
+    <http://data.lblod.info/id/bestuurseenheden/04895013fe6301f32aa46deae98abfb833a717ece5a33b6c453674c6d0f4cc5e>
+    <http://data.lblod.info/id/bestuurseenheden/f56f068943a99059f5bfc3b905fe8848a25a47cfa30fb3ed3d294b47815b2255>
+
+    <http://data.lblod.info/id/bestuurseenheden/5a4b2b4f1de1f3b91b0348a7eb6d6aa0ef9420b8ec31374970c9ffe933a79515>
+    <http://data.lblod.info/id/bestuurseenheden/3f57a17fbcdfc13def910c54948bec11d71d3f75946b0336fffc0587bf18d783>
+
+    <http://data.lblod.info/id/bestuurseenheden/2564e21e3650f91625189ccb7eb055e47754a0633c54c7582a899171fef60c52>
+    <http://data.lblod.info/id/bestuurseenheden/60daf27c2cf12dd123447aa300dd7320890391dbd8dd6b4dd7ec02258f8281b3>
+  }
+}
+;
+DELETE {
+  GRAPH ?g {
+    ?org regorg:orgStatus ?oldStatus.
+  }
+} INSERT {
+  GRAPH ?g {
+    ?org  regorg:orgStatus <http://lblod.data.gift/concepts/63cc561de9188d64ba5840a42ae8f0d6>.
+  }
+} WHERE {
+  GRAPH ?g {
+    ?org regorg:orgStatus ?oldStatus.
+  }
+  VALUES ?g {
+    <http://mu.semte.ch/graphs/administrative-unit>
+  }
+  VALUES ?org {
+    <http://data.lblod.info/id/bestuurseenheden/4b44e6f1-113b-4692-b149-44a889b215f2>
+
+    <http://data.lblod.info/id/bestuurseenheden/1ca65d65-54ff-4b44-b750-bd70c91191af>
+    <http://data.lblod.info/id/bestuurseenheden/7d53f659-3a3b-44b1-9e77-3ea067678c0e>
+
+    <http://data.lblod.info/id/bestuurseenheden/b8bb293d-aa22-4b43-bda4-0b6af76e9493>
+    <http://data.lblod.info/id/bestuurseenheden/5d94b2fd-60ee-4e56-a1f0-a586d596adf6>
+
+    <http://data.lblod.info/id/bestuurseenheden/19483103-318e-435a-aa37-45e485406ee9>
+    <http://data.lblod.info/id/bestuurseenheden/fd41f573-7d9a-4d9f-b7d0-d5b6114d1622>
+
+    <http://data.lblod.info/id/bestuurseenheden/add1e4eb-9ec7-4ea6-af82-335aa76b7d48>
+    <http://data.lblod.info/id/bestuurseenheden/650378e7-1bee-4737-91ff-5b20ac4623cf>
+  }
+}

config/migrations-triggering-indexing/2024/20241212174128-archive-agb-stekene.sparql

@@ -0,0 +1,15 @@
+DELETE {
+  GRAPH ?g {
+    <http://data.lblod.info/id/bestuurseenheden/8d6609b4-1a20-46b8-97ed-150c6d53acd6> <http://www.w3.org/ns/regorg#orgStatus> ?orgStatus .
+  }
+}
+INSERT {
+  GRAPH ?g {
+    <http://data.lblod.info/id/bestuurseenheden/8d6609b4-1a20-46b8-97ed-150c6d53acd6> <http://www.w3.org/ns/regorg#orgStatus> <http://lblod.data.gift/concepts/d02c4e12bf88d2fdf5123b07f29c9311> .
+  }
+}
+WHERE {
+  GRAPH ?g {
+    <http://data.lblod.info/id/bestuurseenheden/8d6609b4-1a20-46b8-97ed-150c6d53acd6> <http://www.w3.org/ns/regorg#orgStatus> ?orgStatus .
+  }
+}

config/migrations-triggering-indexing/2024/20241217082404-mergers-2025--update-addresses/20241217082404-change-zwijndrecht-to-province-east-flanders.sparql

@@ -0,0 +1,27 @@
+PREFIX locn: <http://www.w3.org/ns/locn#>
+PREFIX adres: <https://data.vlaanderen.be/ns/adres#>
+PREFIX organisatie: <https://data.vlaanderen.be/ns/organisatie#>
+PREFIX org: <http://www.w3.org/ns/org#>
+
+DELETE {
+  GRAPH ?g {
+    ?address locn:adminUnitL2 "Antwerpen".
+  }
+} INSERT {
+  GRAPH ?g {
+    ?address locn:adminUnitL2 "Oost-Vlaanderen".
+  }
+} WHERE {
+  GRAPH ?g {
+    ?address a locn:Address;
+             adres:gemeentenaam "Zwijndrecht";
+             locn:adminUnitL2 "Antwerpen"
+    FILTER (EXISTS { ?site organisatie:bestaatUit ?address. } ||
+            EXISTS { ?site org:siteAddress/locn:address ?address. })
+
+  }
+  VALUES ?g {
+    <http://mu.semte.ch/graphs/worship-service>
+    <http://mu.semte.ch/graphs/administrative-unit>
+  }
+}

config/migrations-triggering-indexing/2024/20241217082404-mergers-2025--update-addresses/20241217082944-update-address-municipality-to-merged-municipality-name.sparql

@@ -0,0 +1,51 @@
+PREFIX adres: <https://data.vlaanderen.be/ns/adres#>
+PREFIX locn: <http://www.w3.org/ns/locn#>
+PREFIX organisatie: <https://data.vlaanderen.be/ns/organisatie#>
+PREFIX org: <http://www.w3.org/ns/org#>
+
+DELETE {
+  GRAPH ?g {
+    ?address adres:gemeentenaam ?oldMunicipality.
+  }
+}
+INSERT {
+  GRAPH ?g {
+    ?address adres:gemeentenaam ?newMunicipality.
+  }
+}
+WHERE {
+  GRAPH ?g {
+    ?address a locn:Address;
+             adres:gemeentenaam ?oldMunicipality.
+    FILTER (EXISTS { ?site organisatie:bestaatUit ?address. } ||
+            EXISTS { ?site org:siteAddress/locn:address ?address. })
+  }
+  VALUES ?g {
+    <http://mu.semte.ch/graphs/administrative-unit>
+    <http://mu.semte.ch/graphs/worship-service>
+  }
+  VALUES (?oldMunicipality ?newMunicipality) {
+    ("Borsbeek" "Antwerpen")
+    ("Galmaarden" "Pajottegem")
+    ("Gooik" "Pajottegem")
+    ("Herne" "Pajottegem")
+    ("Ruiselede" "Wingene")
+    ("Meulebeke" "Tielt")
+    ("De Pinte" "Nazareth-De Pinte")
+    ("Nazareth" "Nazareth-De Pinte")
+    ("Wachtebeke" "Lochristi")
+    ("Moerbeke" "Lokeren")
+    ("Melle" "Merelbeke-Melle")
+    ("Merelbeke" "Merelbeke-Melle")
+    ("Beveren" "Beveren-Kruibeke-Zwijndrecht")
+    ("Kruibeke" "Beveren-Kruibeke-Zwijndrecht")
+    ("Zwijndrecht" "Beveren-Kruibeke-Zwijndrecht")
+    ("Bilzen" "Bilzen-Hoeselt")
+    ("Hoeselt" "Bilzen-Hoeselt")
+    ("Borgloon" "Tongeren-Borgloon")
+    ("Tongeren" "Tongeren-Borgloon")
+    ("Ham" "Tessenderlo-Ham")
+    ("Tessenderlo" "Tessenderlo-Ham")
+    ("Kortessem" "Hasselt")
+  }
+}

config/migrations-triggering-indexing/2024/20241217082404-mergers-2025--update-addresses/20241217083314-regenerate-full-address-triple-for-changed-addresses.sparql

@@ -0,0 +1,92 @@
+PREFIX locn: <http://www.w3.org/ns/locn#>
+PREFIX adres: <https://data.vlaanderen.be/ns/adres#>
+PREFIX organisatie: <https://data.vlaanderen.be/ns/organisatie#>
+PREFIX org: <http://www.w3.org/ns/org#>
+
+DELETE {
+  GRAPH ?g {
+    ?address locn:fullAddress ?fullAddress.
+  }
+} INSERT {
+  GRAPH ?g {
+    ?address locn:fullAddress ?newFullAddress.
+  }
+} WHERE {
+  GRAPH ?g {
+    ?address locn:fullAddress ?fullAddress;
+             locn:thoroughfare ?street;
+             adres:Adresvoorstelling.huisnummer ?buildingNo;
+             locn:postCode ?postalCode;
+             adres:gemeentenaam ?municipality;
+             adres:land ?country.
+    FILTER NOT EXISTS { ?address adres:Adresvoorstelling.busnummer ?boxNo. }
+    FILTER (EXISTS { ?site organisatie:bestaatUit ?address. } ||
+            EXISTS { ?site org:siteAddress/locn:address ?address. })
+    FILTER (?municipality IN (
+                              "Antwerpen",
+                              "Pajottegem",
+                              "Wingene",
+                              "Tielt",
+                              "Nazareth-De Pinte",
+                              "Lochristi",
+                              "Lokeren",
+                              "Merelbeke-Melle",
+                              "Beveren-Kruibeke-Zwijndrecht",
+                              "Bilzen-Hoeselt",
+                              "Tongeren-Borgloon",
+                              "Tessenderlo-Ham",
+                              "Hasselt"
+                              ))
+    BIND (CONCAT(?street, " ", ?buildingNo, ", ", ?postalCode, " ", ?municipality, ", ", ?country) AS ?newFullAddress)
+    FILTER (?fullAddress != ?newFullAddress)
+  }
+
+  VALUES ?g {
+    <http://mu.semte.ch/graphs/administrative-unit>
+    <http://mu.semte.ch/graphs/worship-service>
+  }
+}
+;
+DELETE {
+  GRAPH ?g {
+    ?address locn:fullAddress ?fullAddress.
+  }
+} INSERT {
+  GRAPH ?g {
+    ?address locn:fullAddress ?newFullAddress.
+  }
+} WHERE {
+  GRAPH ?g {
+    ?address locn:fullAddress ?fullAddress;
+             locn:thoroughfare ?street;
+             adres:Adresvoorstelling.huisnummer ?buildingNo;
+             adres:Adresvoorstelling.busnummer ?boxNo;
+             locn:postCode ?postalCode;
+             adres:gemeentenaam ?municipality;
+             adres:land ?country.
+    FILTER (EXISTS { ?site organisatie:bestaatUit ?address. } ||
+            EXISTS { ?site org:siteAddress/locn:address ?address. })
+    FILTER (?municipality IN (
+                              "Antwerpen",
+                              "Pajottegem",
+                              "Wingene",
+                              "Tielt",
+                              "Nazareth-De Pinte",
+                              "Lochristi",
+                              "Lokeren",
+                              "Merelbeke-Melle",
+                              "Beveren-Kruibeke-Zwijndrecht",
+                              "Bilzen-Hoeselt",
+                              "Tongeren-Borgloon",
+                              "Tessenderlo-Ham",
+                              "Hasselt"
+                              ))
+    BIND (CONCAT(?street, " ", ?buildingNo, " ", ?boxNo, ", ", ?postalCode, " ", ?municipality, ", ", ?country) AS ?newFullAddress)
+    FILTER (?fullAddress != ?newFullAddress)
+  }
+
+  VALUES ?g {
+    <http://mu.semte.ch/graphs/administrative-unit>
+    <http://mu.semte.ch/graphs/worship-service>
+  }
+}

config/migrations-triggering-indexing/2025/20250108194417-mergers-2025-police-zones--deactivate-merged-organisations.sparql

@@ -0,0 +1,25 @@
+PREFIX regorg: <http://www.w3.org/ns/regorg#>
+
+DELETE {
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?s regorg:orgStatus ?activeStatus.
+  }
+} INSERT {
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?s regorg:orgStatus ?notActiveStatus.
+  }
+} WHERE {
+  VALUES ?s {
+    <http://data.lblod.info/id/bestuurseenheden/150440a8bb763d6f53d24ca1c6f1460cb5817c8a261fd90e74aadac3292c3efe>
+    <http://data.lblod.info/id/bestuurseenheden/41d31a2eb697af977aadf64c73e0bcffb094e0f20e4017fe3b49c507079c0976>
+
+    <http://data.lblod.info/id/bestuurseenheden/330516b323c00851cf4b93e6d5d7a2ef4591f68ae6e71ad9c1f432ee70521bb2>
+    <http://data.lblod.info/id/bestuurseenheden/7dd1250996188607bb1e28df1ebea0b943e8db59c08ea01e3c57672531c43ec6>
+    <http://data.lblod.info/id/bestuurseenheden/541c1603eb8c01968f95d1f262854134499e00d135cd7c9147d307ccbd1e20fd>
+
+    <http://data.lblod.info/id/bestuurseenheden/4deae42b2b25bff407f5abc7cbb6d62eb02afaf2892086341c324fdf2c0e6e92>
+    <http://data.lblod.info/id/bestuurseenheden/b7af3483cb5d0021c6f5864a433b211fce30246ef685621d104b1951915c21c4>
+  }
+  BIND (<http://lblod.data.gift/concepts/d02c4e12bf88d2fdf5123b07f29c9311> AS ?notActiveStatus)
+  BIND (<http://lblod.data.gift/concepts/63cc561de9188d64ba5840a42ae8f0d6> AS ?activeStatus)
+}

config/migrations-triggering-indexing/2025/20250109200145-archive-agb-stekene.sparql

@@ -0,0 +1,196 @@
+DELETE {
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?bestuursorganen ?bestuursorganenP ?bestuursorganenO .
+  }
+}
+WHERE {
+  VALUES ?bestuursorganen {
+    <http://data.lblod.info/id/bestuursorganen/24d8e602a8ddbe0adf3dea8c40277706>
+    <http://data.lblod.info/id/bestuursorganen/dad1789d9b9f8f6cb3c81cc2ca88154f>
+    <http://data.lblod.info/id/bestuursorganen/b90d16f2c8043bc20b88da232c0389fc>
+    <http://data.lblod.info/id/bestuursorganen/2f27a688e1728c1906a4bbd8670d7810>
+  }
+
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?bestuursorganen ?bestuursorganenP ?bestuursorganenO .
+  }
+}
+
+;
+
+DELETE {
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?bestuursorganenInTijd ?bestuursorganenInTijdP ?bestuursorganenInTijdO .
+  }
+}
+WHERE {
+  VALUES ?bestuursorganenInTijd {
+    <http://data.lblod.info/id/bestuursorganen/c5bb7bca7363109b94b58b1aef6ed59c>
+    <http://data.lblod.info/id/bestuursorganen/8d43a29c6ed4a8af0cd43d4aba2c7014>
+    <http://data.lblod.info/id/bestuursorganen/f54979f47464bae1cc001788aedb8b86>
+    <http://data.lblod.info/id/bestuursorganen/c9a7d9130e1dd8f26ddb8c96fdd94c64>
+  }
+
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?bestuursorganenInTijd ?bestuursorganenInTijdP ?bestuursorganenInTijdO .
+  }
+}
+
+;
+
+DELETE {
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?identifier ?identifierP ?identifierO .
+  }
+}
+WHERE {
+  VALUES ?identifier {
+    <http://data.lblod.info/id/identificatoren/99554be7-29d2-403a-a7d5-bda842b4bd86>
+    <http://data.lblod.info/id/identificatoren/19847e1b-a2bf-48a1-872e-baf8d4efaa12>
+    <http://data.lblod.info/id/identificatoren/b2123d27-b6bc-4e70-b594-27d35c696713>
+  }
+
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?identifier ?identifierP ?identifierO .
+  }
+}
+
+;
+
+DELETE {
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?structuredIdentifier ?structuredIdentifierP ?structuredIdentifierO .
+  }
+}
+WHERE {
+  VALUES ?structuredIdentifier {
+    <http://data.lblod.info/id/gestructureerdeIdentificatoren/920f98a2-1785-4de1-81ed-5859c224dcb8>
+    <http://data.lblod.info/id/gestructureerdeIdentificatoren/5e1082d8-7751-4bed-a53f-457b347f7f28>
+    <http://data.lblod.info/id/gestructureerdeIdentificatoren/2e978403-d3ef-40ab-a9f9-3a4ae6a35609>
+  }
+
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?structuredIdentifier ?structuredIdentifierP ?structuredIdentifierO .
+  }
+}
+
+;
+
+DELETE {
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?site ?siteP ?siteO .
+  }
+}
+WHERE {
+  VALUES ?site {
+    <http://data.lblod.info/id/vestigingen/daa79855-1de7-4479-8a2e-a8d89db55ee3>
+  }
+
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?site ?siteP ?siteO .
+  }
+}
+
+;
+
+DELETE {
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?address ?addressP ?addressO .
+  }
+}
+WHERE {
+  VALUES ?address {
+    <http://data.lblod.info/id/adressen/df102cb4-8abb-4588-a199-1ce22c211812>
+  }
+
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?address ?addressP ?addressO .
+  }
+}
+
+;
+
+DELETE {
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?contactPoint ?contactPointP ?contactPointO .
+  }
+}
+WHERE {
+  VALUES ?contactPoint {
+    <http://data.lblod.info/id/contact-punten/042e8816-2e9d-45a1-89f6-dcf805882af7>
+    <http://data.lblod.info/id/contact-punten/364d8e1a-cb18-4970-aeaf-5f6f307370ad>
+  }
+
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?contactPoint ?contactPointP ?contactPointO .
+  }
+}
+
+;
+
+DELETE {
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?parentOrganization <http://www.w3.org/ns/org#hasSubOrganization> ?organization .
+  }
+}
+WHERE {
+  BIND(<http://data.lblod.info/id/bestuurseenheden/8d6609b4-1a20-46b8-97ed-150c6d53acd6> AS ?organization)
+  BIND(<http://data.lblod.info/id/bestuurseenheden/6025a5d1ca2262a784f002edd8ce9ca9073ae3d5ebc6b6b5531f05a29e9250af> AS ?parentOrganization)
+
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?parentOrganization <http://www.w3.org/ns/org#hasSubOrganization> ?organization .
+  }
+}
+
+;
+
+DELETE {
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?parentOrganization <http://mu.semte.ch/vocabularies/ext/isFounderOfOrganization> ?organization .
+  }
+}
+WHERE {
+  BIND(<http://data.lblod.info/id/bestuurseenheden/8d6609b4-1a20-46b8-97ed-150c6d53acd6> AS ?organization)
+  BIND(<http://data.lblod.info/id/bestuurseenheden/6025a5d1ca2262a784f002edd8ce9ca9073ae3d5ebc6b6b5531f05a29e9250af> AS ?parentOrganization)
+
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?parentOrganization <http://mu.semte.ch/vocabularies/ext/isFounderOfOrganization> ?organization .
+  }
+}
+
+;
+
+DELETE {
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?membership ?p ?o.
+  }
+}
+WHERE {
+  BIND(<http://data.lblod.info/id/bestuurseenheden/8d6609b4-1a20-46b8-97ed-150c6d53acd6> AS ?organization)
+
+  VALUES ?membershipP {
+    <http://www.w3.org/ns/org#organization>
+    <http://www.w3.org/ns/org#member>
+  }
+
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?membership a <http://www.w3.org/ns/org#Membership> ;
+      ?membershipP ?organization ;
+      ?p ?o .
+  }
+}
+
+;
+
+DELETE {
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?organization ?organizationP ?organizationO .
+  }
+}
+WHERE {
+  BIND(<http://data.lblod.info/id/bestuurseenheden/8d6609b4-1a20-46b8-97ed-150c6d53acd6> AS ?organization)
+
+  GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
+    ?organization ?organizationP ?organizationO .
+  }
+}

config/migrations-triggering-indexing/2025/20250321162045-change-ocmwv-to-vvmw.sparql

@@ -0,0 +1,18 @@
+DELETE {
+  GRAPH <http://mu.semte.ch/graphs/public> {
+    ?s <http://www.w3.org/2004/02/skos/core#prefLabel> ?label
+  }
+}
+INSERT {
+  GRAPH <http://mu.semte.ch/graphs/public> {
+    ?s <http://www.w3.org/2004/02/skos/core#prefLabel> "VVMW".
+  }
+}
+WHERE {
+  VALUES ?s {
+    <http://data.vlaanderen.be/id/concept/BestuurseenheidClassificatieCode/cc4e2d67-603b-4784-9b61-e50bac1ec089>
+  }
+  GRAPH <http://mu.semte.ch/graphs/public> {
+    ?s <http://www.w3.org/2004/02/skos/core#prefLabel> ?label.
+  }
+}