Compare commits
64 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
8e2dc898f5 | ||
|
|
de5bb3e392 | ||
|
|
6ef9690647 | ||
|
|
b6bdd5b067 | ||
|
|
97263fe7ae | ||
|
|
2ef1d8ab7b | ||
|
|
3e7d92624c | ||
|
|
9dffe4b223 | ||
|
|
6f2db80f50 | ||
|
|
8308644e82 | ||
|
|
7ac0394d2f | ||
|
|
215deda183 | ||
|
|
729bee7f14 | ||
|
|
8146686b48 | ||
|
|
8d133fd1c9 | ||
|
|
101618c156 | ||
|
|
b51293ca3a | ||
|
|
e1eca4b185 | ||
|
|
1c46d49976 | ||
|
|
68cbd39c9d | ||
|
|
4982224a78 | ||
|
|
cbf320711d | ||
|
|
6dfb0a656f | ||
|
|
387ef1eeea | ||
|
|
5a3ae13eb0 | ||
|
|
57dca52171 | ||
|
|
e7e76c187b | ||
|
|
6a85ec7c96 | ||
|
|
b272e1500c | ||
|
|
7c2a54264a | ||
|
|
74327cc371 | ||
|
|
bfb28bb698 | ||
|
|
cc8125cb14 | ||
|
|
329a63a0b9 | ||
|
|
4891b6829d | ||
|
|
ea2d25d177 | ||
|
|
8c49134011 | ||
|
|
d203077364 | ||
|
|
eee9f18561 | ||
|
|
a8681a02f3 | ||
|
|
edbba1eeaa | ||
|
|
0c9355c94f | ||
|
|
b6a42c66cf | ||
|
|
96b36d2cf6 | ||
|
|
19a0804b56 | ||
|
|
a4684811df | ||
|
|
dc18662517 | ||
|
|
c6ba52a6bc | ||
|
|
ee1c92b5fe | ||
|
|
32b6520bfa | ||
|
|
0e9483b4f8 | ||
|
|
b65e0cbeaa | ||
|
|
fbf8a4e2dc | ||
|
|
1bff031596 | ||
|
|
b2deb421bc | ||
|
|
e078ae48a7 | ||
|
|
e2479f626a | ||
|
|
df8512d8f9 | ||
|
|
ed0863c0bf | ||
|
|
4131395159 | ||
|
|
325ebe991c | ||
|
|
179ba7b315 | ||
|
|
714fda3dc8 | ||
|
|
0be27e1ef2 |
51
CHANGELOG.md
51
CHANGELOG.md
@@ -1,6 +1,55 @@
|
||||
# Changelog
|
||||
## Unreleased
|
||||
|
||||
## v1.35.0 (2025-08-04)
|
||||
### Frontend
|
||||
- Bump to [v1.34.0](https://github.com/lblod/frontend-organization-portal/releases/tag/v1.34.0)
|
||||
### Backend
|
||||
- Add a service to manage organisation's their scope of operation [OP-3205]
|
||||
- datamodel: Allow a location to be within multiple other locations [OP-3205]
|
||||
- datafix: set scope of operation for worship administrative units [OP-3626]
|
||||
- Clean addresses linked to both sites and contact points of mandatarissen and addresses that are in the wrong graph (admin unit graph but linked to a contact point in the worship graph) [DL-6701]
|
||||
- datafix: cleanup of general memberships if specific ones exist [OP-3640]
|
||||
- datafix: cleanup of duplicate memberships [OP-3634]
|
||||
- Fix KBO organizations' date formats [OP-3560]
|
||||
|
||||
### Deploy notes
|
||||
```
|
||||
drc pull scope-of-operation; drc up -d scope-of-operation
|
||||
drc restart db resource dispatcher
|
||||
drc pull frontend; drc up -d frontend
|
||||
drc restart migrations; drc logs -ft --tail=200 migrations
|
||||
drc up -d kbo-data-sync
|
||||
```
|
||||
|
||||
The labels for new location resources created when setting the scope of operation for worship services are not necessarily alphabetically sorted. To correct this run the `correct-location-labels` project script using [mu-cli](https://github.com/mu-semtech/mu-cli) and restart the migrations service once more to execute the generated local migrations.
|
||||
|
||||
```
|
||||
mu script project-scripts correct-location-labels
|
||||
drc restart migrations; drc logs -ft --tail=200 migrations
|
||||
```
|
||||
|
||||
## v1.34.0 (2025-07-11)
|
||||
### Frontend
|
||||
- Bump to version [v1.33.0](https://github.com/lblod/frontend-organization-portal/releases/tag/v1.33.0)
|
||||
### Backend
|
||||
- Add Brussels Hoofdstedelijk Gewest as province and location, update province of addresses in Brussel's area, update resource config to add `skos:note` on organizations [OP-3624]
|
||||
- Add Brussel as a municipality [OP-3628]
|
||||
- Revert Brussel as province and municipality and update back province of addresses in Brussel's area [OP-3637]
|
||||
- Bump delta-notifier service to most recent version [OP-3190]
|
||||
- Add URI to resource's organization responses
|
||||
- Fix query for organizations report [OP-3636]
|
||||
- Bump report-generation-service to most recent version
|
||||
- Write migration to harmonize date formats of change events [OP-3559]
|
||||
### Deploy notes
|
||||
```
|
||||
drc pull deltanotifier; drc up -d deltanotifier
|
||||
drc restart migrations-triggering-indexing resource
|
||||
drc pull frontend; drc up -d frontend
|
||||
drc pull report-generation; drc up -d report-generation
|
||||
drc restart migrations; drc logs -ft --tail=200 migrations
|
||||
```
|
||||
|
||||
## v1.33.3 (2025-07-04)
|
||||
### Backend
|
||||
- datafix: re-add Havenbedrijf Antwerpen as an AGB [OP-3639]
|
||||
@@ -12,7 +61,7 @@ drc exec delta-producer-background-jobs-initiator curl -X POST http://localhost/
|
||||
|
||||
## v1.33.2 (2025-06-20)
|
||||
### Backend
|
||||
- datafix: correct name for locations whose municipalities kept their URI in the 2025 mergers
|
||||
- datafix: correct name for locations whose municipalities kept their URI in the 2025 mergers [OP-3627]
|
||||
### Deploy notes
|
||||
```
|
||||
drc restart migrations-triggering-indexing && drc logs -ft --tail=200
|
||||
|
||||
@@ -53,7 +53,8 @@ defmodule Acl.UserGroups.Config do
|
||||
"http://lblod.data.gift/vocabularies/organisatie/VoorwaardenBedienaarCriterium",
|
||||
"http://lblod.data.gift/vocabularies/organisatie/BedienaarCriteriumBewijsstuk",
|
||||
"http://lblod.data.gift/vocabularies/organisatie/EredienstBeroepen",
|
||||
"http://lblod.data.gift/vocabularies/organisatie/Rechtsvormtype"
|
||||
"http://lblod.data.gift/vocabularies/organisatie/Rechtsvormtype",
|
||||
"http://www.w3.org/ns/prov#Location"
|
||||
]
|
||||
|
||||
@org_type [
|
||||
@@ -80,11 +81,10 @@ defmodule Acl.UserGroups.Config do
|
||||
"http://data.lblod.info/vocabularies/contacthub/AgentInPositie",
|
||||
"http://mu.semte.ch/vocabularies/ext/KboOrganisatie",
|
||||
"http://www.w3.org/ns/org#Membership",
|
||||
"http://www.w3.org/2006/time#ProperInterval"
|
||||
"http://www.w3.org/2006/time#ProperInterval",
|
||||
]
|
||||
|
||||
@shared_protected_type [
|
||||
"http://www.w3.org/ns/prov#Location",
|
||||
"http://xmlns.com/foaf/0.1/Image",
|
||||
"http://www.semanticdesktop.org/ontologies/2007/03/22/nfo#FileDataObject"
|
||||
]
|
||||
@@ -117,7 +117,7 @@ defmodule Acl.UserGroups.Config do
|
||||
?account <http://mu.semte.ch/vocabularies/ext/sessionRole> ?session_role.
|
||||
FILTER( ?session_role = \"dashboard-user\" )
|
||||
}"
|
||||
}
|
||||
}
|
||||
end
|
||||
|
||||
def user_groups do
|
||||
@@ -256,7 +256,7 @@ defmodule Acl.UserGroups.Config do
|
||||
"http://www.semanticdesktop.org/ontologies/2007/03/22/nfo#FileDataObject"
|
||||
]
|
||||
}
|
||||
},
|
||||
}
|
||||
]
|
||||
},
|
||||
|
||||
|
||||
@@ -393,6 +393,10 @@ defmodule Dispatcher do
|
||||
forward(conn, path, "http://kbo-data-sync/sync-kbo-data/")
|
||||
end
|
||||
|
||||
match "/scope-of-operation/*path", %{layer: api_services, accept: %{json: true}} do
|
||||
forward(conn, path, "http://scope-of-operation/")
|
||||
end
|
||||
|
||||
get "/ldes/*path", %{layer: :api_services} do
|
||||
forward(conn, path, "http://ldes-backend/")
|
||||
end
|
||||
|
||||
@@ -0,0 +1,112 @@
|
||||
PREFIX ere: <http://data.lblod.info/vocabularies/erediensten/>
|
||||
PREFIX org: <http://www.w3.org/ns/org#>
|
||||
PREFIX besluit: <http://data.vlaanderen.be/ns/besluit#>
|
||||
|
||||
# This migration extracts the relations between (central) worship services and
|
||||
# their locations according to the rules provided by business. These links are
|
||||
# inserted into a temporary graph for further processing.
|
||||
|
||||
# Worship services
|
||||
## Municipality organised
|
||||
INSERT {
|
||||
GRAPH <http://mu.semte.ch/graphs/temp-for-worship-organizations-scope> {
|
||||
?ws besluit:werkingsgebied ?scope .
|
||||
}
|
||||
} WHERE {
|
||||
GRAPH <http://mu.semte.ch/graphs/worship-service> {
|
||||
?ws a ere:BestuurVanDeEredienst ;
|
||||
ere:typeEredienst ?worshipType .
|
||||
VALUES ?worshipType {
|
||||
<http://lblod.data.gift/concepts/99536dd6eb0d2ef38a89efafb17e7389> # Anglicaans
|
||||
<http://lblod.data.gift/concepts/1a1abeafc973d27cebcb2d7a15b2d823> # Israëlitisch
|
||||
<http://lblod.data.gift/concepts/e8cba1540b35a32e9cb45126c38c03c6> # Protestants
|
||||
<http://lblod.data.gift/concepts/b13d1d623626bc1ee75c7d20bc60e3c0> # Rooms-Katholiek
|
||||
}
|
||||
|
||||
?involvement a ere:BetrokkenLokaleBesturen ;
|
||||
org:organization ?ws ;
|
||||
ere:typebetrokkenheid ?involvementType .
|
||||
VALUES ?involvementType {
|
||||
<http://lblod.data.gift/concepts/86fcbbbff764f1cba4c7e10dbbae578e> # Mede-financierend
|
||||
<http://lblod.data.gift/concepts/ac400cc9f135ac7873fb3e551ec738c1> # Toezichthoudend
|
||||
}
|
||||
?org ere:betrokkenBestuur ?involvement .
|
||||
}
|
||||
GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
|
||||
?org besluit:werkingsgebied ?scope .
|
||||
}
|
||||
}
|
||||
;
|
||||
## Province organised
|
||||
INSERT {
|
||||
GRAPH <http://mu.semte.ch/graphs/temp-for-worship-organizations-scope> {
|
||||
?ws besluit:werkingsgebied ?scope .
|
||||
}
|
||||
} WHERE {
|
||||
GRAPH <http://mu.semte.ch/graphs/worship-service> {
|
||||
?ws a ere:BestuurVanDeEredienst ;
|
||||
ere:typeEredienst ?worshipType .
|
||||
VALUES ?worshipType {
|
||||
<http://lblod.data.gift/concepts/9d8bd472a00bf0a5c7b7186606365a52> # Islamitisch
|
||||
<http://lblod.data.gift/concepts/84bcd6896f575bae4857ff8d2764bed8> # Orthodox
|
||||
}
|
||||
|
||||
?involvement a ere:BetrokkenLokaleBesturen ;
|
||||
org:organization ?ws ;
|
||||
ere:typebetrokkenheid ?involvementType .
|
||||
VALUES ?involvementType {
|
||||
<http://lblod.data.gift/concepts/0f845f00ee76099c89518cbaf6a7b77f> # Adviserend
|
||||
}
|
||||
?org ere:betrokkenBestuur ?involvement .
|
||||
}
|
||||
GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
|
||||
?org besluit:werkingsgebied ?scope .
|
||||
}
|
||||
}
|
||||
;
|
||||
# Central worship services
|
||||
## Municipality organised
|
||||
INSERT {
|
||||
GRAPH <http://mu.semte.ch/graphs/temp-for-worship-organizations-scope> {
|
||||
?ws besluit:werkingsgebied ?scope .
|
||||
}
|
||||
} WHERE {
|
||||
GRAPH <http://mu.semte.ch/graphs/worship-service> {
|
||||
?ws a ere:CentraalBestuurVanDeEredienst ;
|
||||
ere:typeEredienst ?worshipType .
|
||||
VALUES ?worshipType {
|
||||
<http://lblod.data.gift/concepts/b13d1d623626bc1ee75c7d20bc60e3c0> # Rooms-Katholiek
|
||||
}
|
||||
|
||||
?involvement a ere:BetrokkenLokaleBesturen ;
|
||||
org:organization ?ws ;
|
||||
ere:typebetrokkenheid ?involvementType .
|
||||
VALUES ?involvementType {
|
||||
<http://lblod.data.gift/concepts/ac400cc9f135ac7873fb3e551ec738c1> # Toezichthoudend
|
||||
}
|
||||
?org ere:betrokkenBestuur ?involvement .
|
||||
}
|
||||
GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
|
||||
?org besluit:werkingsgebied ?scope .
|
||||
}
|
||||
}
|
||||
;
|
||||
## Province organised
|
||||
## Note: Business requested to use the location for the municipality of Ghent as
|
||||
## default value. They will update it manually afterwards.
|
||||
INSERT {
|
||||
GRAPH <http://mu.semte.ch/graphs/temp-for-worship-organizations-scope> {
|
||||
?ws besluit:werkingsgebied ?locationGhent .
|
||||
}
|
||||
} WHERE {
|
||||
GRAPH <http://mu.semte.ch/graphs/worship-service> {
|
||||
?ws a ere:CentraalBestuurVanDeEredienst ;
|
||||
ere:typeEredienst ?worshipType .
|
||||
VALUES ?worshipType {
|
||||
<http://lblod.data.gift/concepts/9d8bd472a00bf0a5c7b7186606365a52> # Islamitisch
|
||||
<http://lblod.data.gift/concepts/84bcd6896f575bae4857ff8d2764bed8> # Orthodox
|
||||
}
|
||||
|
||||
BIND(URI("http://data.lblod.info/id/werkingsgebieden/ce94eeae827cdc5c00f3f2f4276ff628580edefb3aa5861e4669b0c5d93adc57") AS ?locationGhent)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,24 @@
|
||||
PREFIX besluit: <http://data.vlaanderen.be/ns/besluit#>
|
||||
|
||||
# This migration sets the scope of operation for worship organisations that have
|
||||
# single, already existing, location as scope. This migration depends on
|
||||
# `20250707132349-chore--extract-scopes-for-worship-organisations` being
|
||||
# executed first, as it uses the data created by the migration.
|
||||
INSERT {
|
||||
GRAPH <http://mu.semte.ch/graphs/worship-service> {
|
||||
?ws besluit:werkingsgebied ?scope .
|
||||
}
|
||||
} WHERE {
|
||||
{
|
||||
SELECT DISTINCT ?ws (count(DISTINCT ?scope) as ?count)
|
||||
WHERE {
|
||||
GRAPH <http://mu.semte.ch/graphs/temp-for-worship-organizations-scope> {
|
||||
?ws besluit:werkingsgebied ?scope .
|
||||
}
|
||||
}
|
||||
}
|
||||
FILTER (?count = 1)
|
||||
GRAPH <http://mu.semte.ch/graphs/temp-for-worship-organizations-scope> {
|
||||
?ws besluit:werkingsgebied ?scope .
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,49 @@
|
||||
PREFIX prov: <http://www.w3.org/ns/prov#>
|
||||
PREFIX mu: <http://mu.semte.ch/vocabularies/core/>
|
||||
PREFIX ext: <http://mu.semte.ch/vocabularies/ext/>
|
||||
PREFIX geo: <http://www.opengis.net/ont/geosparql#>
|
||||
PREFIX besluit: <http://data.vlaanderen.be/ns/besluit#>
|
||||
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
|
||||
|
||||
# This migration sets the scope of operation for worship organisations that have
|
||||
# multiple location as scope. It creates new aggregate locations to contain the
|
||||
# already existing municipality/province level ones. This migration depends on
|
||||
# `20250707132349-chore--extract-scopes-for-worship-organisations` being
|
||||
# executed first, as it uses the data created by the migration.
|
||||
INSERT {
|
||||
GRAPH <http://mu.semte.ch/graphs/public> {
|
||||
?location a prov:Location ;
|
||||
mu:uuid ?locationUuid ;
|
||||
rdfs:label ?scopeLabel ;
|
||||
ext:werkingsgebiedNiveau """Samengesteld werkingsgebied""" .
|
||||
?scope geo:sfWithin ?location .
|
||||
}
|
||||
GRAPH <http://mu.semte.ch/graphs/worship-service> {
|
||||
?ws besluit:werkingsgebied ?location .
|
||||
}
|
||||
} WHERE {
|
||||
{
|
||||
SELECT DISTINCT ?ws (count(DISTINCT ?scope) AS ?count) (GROUP_CONCAT(?uuid," ") AS ?scopeUuids) (GROUP_CONCAT(?label,", ") AS ?scopeLabel)
|
||||
WHERE {
|
||||
GRAPH <http://mu.semte.ch/graphs/temp-for-worship-organizations-scope> {
|
||||
?ws besluit:werkingsgebied ?scope .
|
||||
}
|
||||
GRAPH <http://mu.semte.ch/graphs/public> {
|
||||
?scope mu:uuid ?uuid ;
|
||||
rdfs:label ?label .
|
||||
}
|
||||
}
|
||||
}
|
||||
FILTER (?count > 1)
|
||||
GRAPH <http://mu.semte.ch/graphs/temp-for-worship-organizations-scope> {
|
||||
?ws besluit:werkingsgebied ?scope .
|
||||
}
|
||||
# NOTE (10/07/2025): The SPARQL specification does not specify any order for
|
||||
# `GROUP_CONCAT`. But virtuoso does seem to concatenate elements always in the
|
||||
# same order. Therefore, this way of generating a UUID is a bit of a hack to
|
||||
# ensure that new locations that contain the same locations will receive the
|
||||
# same UUUID, which results in the same location resource being inserted
|
||||
# twice.
|
||||
BIND (STR(SHA256(?scopeUuids)) AS ?locationUuid)
|
||||
BIND (URI(CONCAT("http://data.lblod.info/id/werkingsgebieden/", ?locationUuid)) AS ?location)
|
||||
}
|
||||
@@ -0,0 +1 @@
|
||||
CLEAR GRAPH <http://mu.semte.ch/graphs/temp-for-worship-organizations-scope>
|
||||
@@ -0,0 +1,26 @@
|
||||
PREFIX org: <http://www.w3.org/ns/org#>
|
||||
PREFIX dcterms: <http://purl.org/dc/terms/>
|
||||
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
|
||||
|
||||
DELETE {
|
||||
GRAPH ?g {
|
||||
?s dcterms:date ?oldDate .
|
||||
}
|
||||
}
|
||||
INSERT {
|
||||
GRAPH ?g {
|
||||
?s dcterms:date ?newDate .
|
||||
}
|
||||
}
|
||||
WHERE {
|
||||
GRAPH ?g {
|
||||
?s a org:ChangeEvent ;
|
||||
dcterms:date ?oldDate .
|
||||
FILTER(DATATYPE(?oldDate) != xsd:date)
|
||||
FILTER(?g IN (
|
||||
<http://mu.semte.ch/graphs/administrative-unit>,
|
||||
<http://mu.semte.ch/graphs/worship-service>
|
||||
))
|
||||
BIND(STRDT(SUBSTR(STR(?oldDate), 1, 10), xsd:date) AS ?newDate)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,30 @@
|
||||
DELETE {
|
||||
GRAPH ?h {
|
||||
?site <https://data.vlaanderen.be/ns/organisatie#bestaatUit> ?oldAddressUri .
|
||||
}
|
||||
} INSERT {
|
||||
GRAPH ?g {
|
||||
?newAddressUri a <http://www.w3.org/ns/locn#Address> ;
|
||||
<http://mu.semte.ch/vocabularies/core/uuid> ?newAddressUuid ;
|
||||
?p ?o .
|
||||
|
||||
?site <https://data.vlaanderen.be/ns/organisatie#bestaatUit> ?newAddressUri .
|
||||
}
|
||||
} WHERE {
|
||||
GRAPH ?g {
|
||||
?oldAddressUri a <http://www.w3.org/ns/locn#Address> ;
|
||||
?p ?o .
|
||||
|
||||
FILTER (?p NOT IN (<http://mu.semte.ch/vocabularies/core/uuid>))
|
||||
}
|
||||
|
||||
?mandatarisContact <http://www.w3.org/ns/locn#address> ?oldAddressUri .
|
||||
|
||||
GRAPH ?h {
|
||||
?site a <http://www.w3.org/ns/org#Site>;
|
||||
<https://data.vlaanderen.be/ns/organisatie#bestaatUit> ?oldAddressUri .
|
||||
}
|
||||
|
||||
BIND(MD5(CONCAT(?oldAddressUri,"NEW_ADDRESS")) AS ?newAddressUuid) .
|
||||
BIND(IRI(CONCAT("http://data.lblod.info/id/adressen/", STR(?newAddressUuid))) AS ?newAddressUri) .
|
||||
}
|
||||
@@ -0,0 +1,33 @@
|
||||
PREFIX org: <http://www.w3.org/ns/org#>
|
||||
PREFIX mu: <http://mu.semte.ch/vocabularies/core/>
|
||||
PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
|
||||
|
||||
DELETE {
|
||||
GRAPH ?g {
|
||||
?membership ?p ?o .
|
||||
}
|
||||
}
|
||||
WHERE {
|
||||
GRAPH ?g {
|
||||
# Find memberships that use the general 'gerelateerd' role
|
||||
?membership a org:Membership ;
|
||||
org:role <http://data.lblod.info/id/rollen/4ec7d5c39bdc4e84b4174379b9e22ad8> ;
|
||||
org:member ?orgMember ;
|
||||
org:organization ?organization .
|
||||
|
||||
# Only delete if there's a specific role for the inverse membership
|
||||
FILTER EXISTS {
|
||||
?specific_membership a org:Membership ;
|
||||
org:member ?organization ;
|
||||
org:organization ?orgMember ;
|
||||
org:role ?specificRole .
|
||||
|
||||
VALUES ?specificRole {
|
||||
<http://data.lblod.info/id/rollen/2152eb830b1143bfb97a7dd9596d6c63> # participant
|
||||
<http://data.lblod.info/id/rollen/73d5e1cf250d42fab15926771f07505a> # stichtend lid
|
||||
}
|
||||
}
|
||||
|
||||
?membership ?p ?o .
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,18 @@
|
||||
DELETE {
|
||||
GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
|
||||
?address ?p ?o .
|
||||
}
|
||||
} INSERT {
|
||||
GRAPH <http://mu.semte.ch/graphs/worship-service> {
|
||||
?address ?p ?o .
|
||||
}
|
||||
} WHERE {
|
||||
GRAPH <http://mu.semte.ch/graphs/worship-service> {
|
||||
?contactPoint a <http://schema.org/ContactPoint> ;
|
||||
<http://www.w3.org/ns/locn#address> ?address .
|
||||
}
|
||||
|
||||
GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
|
||||
?address ?p ?o .
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,40 @@
|
||||
PREFIX org: <http://www.w3.org/ns/org#>
|
||||
PREFIX mu: <http://mu.semte.ch/vocabularies/core/>
|
||||
|
||||
DELETE {
|
||||
GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
|
||||
?membership org:organization ?ocmw ;
|
||||
org:member ?municipality .
|
||||
}
|
||||
}
|
||||
INSERT {
|
||||
GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
|
||||
?membership org:organization ?municipality ;
|
||||
org:member ?ocmw .
|
||||
}
|
||||
}
|
||||
WHERE {
|
||||
GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
|
||||
?membership a org:Membership ;
|
||||
org:organization ?ocmw ;
|
||||
org:member ?municipality ;
|
||||
org:role <http://data.lblod.info/id/rollen/4ec7d5c39bdc4e84b4174379b9e22ad8> .
|
||||
}
|
||||
|
||||
GRAPH ?graphMunicipality {
|
||||
?municipality org:classification <http://data.vlaanderen.be/id/concept/BestuurseenheidClassificatieCode/5ab0e9b8a3b2ca7c5e000001> .
|
||||
}
|
||||
|
||||
GRAPH ?graphOCMW {
|
||||
?ocmw org:classification <http://data.vlaanderen.be/id/concept/BestuurseenheidClassificatieCode/5ab0e9b8a3b2ca7c5e000002> .
|
||||
}
|
||||
|
||||
FILTER NOT EXISTS {
|
||||
GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
|
||||
?correctMembership a org:Membership ;
|
||||
org:organization ?municipality ;
|
||||
org:member ?ocmw ;
|
||||
org:role <http://data.lblod.info/id/rollen/4ec7d5c39bdc4e84b4174379b9e22ad8> .
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,33 @@
|
||||
PREFIX org: <http://www.w3.org/ns/org#>
|
||||
PREFIX mu: <http://mu.semte.ch/vocabularies/core/>
|
||||
|
||||
DELETE {
|
||||
GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
|
||||
?duplicateMembership ?p ?o .
|
||||
}
|
||||
} WHERE {
|
||||
GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
|
||||
?duplicateMembership a org:Membership ;
|
||||
org:organization ?ocmw ;
|
||||
org:member ?municipality ;
|
||||
org:role <http://data.lblod.info/id/rollen/4ec7d5c39bdc4e84b4174379b9e22ad8> ;
|
||||
?p ?o .
|
||||
|
||||
# Find the other memberships with swapped organizations
|
||||
?correctMembership a org:Membership ;
|
||||
org:organization ?municipality ;
|
||||
org:member ?ocmw ;
|
||||
org:role <http://data.lblod.info/id/rollen/4ec7d5c39bdc4e84b4174379b9e22ad8> .
|
||||
}
|
||||
|
||||
GRAPH <http://mu.semte.ch/graphs/shared> {
|
||||
?municipality org:classification ?municipalityClassification .
|
||||
}
|
||||
|
||||
GRAPH <http://mu.semte.ch/graphs/administrative-unit> {
|
||||
?ocmw org:classification ?ocmwClassification .
|
||||
}
|
||||
|
||||
FILTER(?municipalityClassification = <http://data.vlaanderen.be/id/concept/BestuurseenheidClassificatieCode/5ab0e9b8a3b2ca7c5e000001>)
|
||||
FILTER(?ocmwClassification = <http://data.vlaanderen.be/id/concept/BestuurseenheidClassificatieCode/5ab0e9b8a3b2ca7c5e000002>)
|
||||
}
|
||||
@@ -0,0 +1,28 @@
|
||||
DELETE {
|
||||
GRAPH ?g {
|
||||
?s ?p ?date .
|
||||
}
|
||||
}
|
||||
INSERT {
|
||||
GRAPH ?g {
|
||||
?s ?p ?dateTime .
|
||||
}
|
||||
}
|
||||
WHERE {
|
||||
VALUES ?g {
|
||||
<http://mu.semte.ch/graphs/administrative-unit>
|
||||
<http://mu.semte.ch/graphs/worship-service>
|
||||
}
|
||||
|
||||
GRAPH ?g {
|
||||
?s a <http://mu.semte.ch/vocabularies/ext/KboOrganisatie> ;
|
||||
?p ?date .
|
||||
|
||||
VALUES ?p {
|
||||
<http://purl.org/dc/terms/modified>
|
||||
<http://mu.semte.ch/vocabularies/ext/startDate>
|
||||
}
|
||||
|
||||
BIND(STRDT(CONCAT(STR(?date), "T02:00:00"), xsd:dateTime) AS ?dateTime)
|
||||
}
|
||||
}
|
||||
@@ -1,30 +1,25 @@
|
||||
import { generateReportFromData, batchedQuery } from '../helpers.js';
|
||||
import { PREFIXES } from './utils'
|
||||
import { generateReportFromData, batchedQuery } from "../helpers.js";
|
||||
import { PREFIXES } from "./utils";
|
||||
|
||||
export default {
|
||||
cronPattern: '0 00 23 * * *',
|
||||
name: 'organizations',
|
||||
cronPattern: "0 00 23 * * *",
|
||||
name: "organizations",
|
||||
execute: async () => {
|
||||
const reportData = {
|
||||
title: 'organizations',
|
||||
description: 'List of the Organizations',
|
||||
filePrefix: 'exports/organizations'
|
||||
title: "organizations",
|
||||
description: "List of the Organizations",
|
||||
filePrefix: "exports/organizations",
|
||||
};
|
||||
|
||||
console.log('Generating organizations report');
|
||||
console.log("Generating organizations report");
|
||||
|
||||
const queryStringPart1 = `
|
||||
${PREFIXES}
|
||||
|
||||
SELECT DISTINCT ?bestuur ?label ?classification ?typeEredienst
|
||||
WHERE {
|
||||
VALUES ?type {
|
||||
org:Organization
|
||||
besluit:Bestuurseenheid
|
||||
}
|
||||
|
||||
GRAPH ?g {
|
||||
?bestuur a ?type .
|
||||
?bestuur a org:Organization .
|
||||
OPTIONAL { ?bestuur skos:prefLabel ?label. }
|
||||
}
|
||||
|
||||
@@ -44,12 +39,12 @@ export default {
|
||||
|
||||
const queryResponsePart1 = await batchedQuery(queryStringPart1);
|
||||
|
||||
const dataPart1 = queryResponsePart1.results.bindings.reduce( (acc, row) => {
|
||||
acc[getSafeValue(row, 'bestuur')] = {
|
||||
bestuur: getSafeValue(row, 'bestuur'),
|
||||
label: getSafeValue(row, 'label'),
|
||||
classification: getSafeValue(row, 'classification'),
|
||||
typeEredienst: getSafeValue(row, 'typeEredienst'),
|
||||
const dataPart1 = queryResponsePart1.results.bindings.reduce((acc, row) => {
|
||||
acc[getSafeValue(row, "bestuur")] = {
|
||||
bestuur: getSafeValue(row, "bestuur"),
|
||||
label: getSafeValue(row, "label"),
|
||||
classification: getSafeValue(row, "classification"),
|
||||
typeEredienst: getSafeValue(row, "typeEredienst"),
|
||||
};
|
||||
return acc;
|
||||
}, {});
|
||||
@@ -59,17 +54,12 @@ export default {
|
||||
|
||||
SELECT DISTINCT ?bestuur ?provincie
|
||||
WHERE {
|
||||
VALUES ?type {
|
||||
org:Organization
|
||||
besluit:Bestuurseenheid
|
||||
}
|
||||
|
||||
GRAPH ?g {
|
||||
?bestuur a ?type .
|
||||
?bestuur a org:Organization .
|
||||
}
|
||||
|
||||
GRAPH ?h {
|
||||
OPTIONAL {
|
||||
OPTIONAL {
|
||||
GRAPH ?h {
|
||||
?bestuur org:hasPrimarySite ?primarySite.
|
||||
|
||||
OPTIONAL {
|
||||
@@ -77,35 +67,37 @@ export default {
|
||||
OPTIONAL { ?sa locn:adminUnitL2 ?provincie. }
|
||||
}
|
||||
}
|
||||
FILTER (?h IN ( <http://mu.semte.ch/graphs/administrative-unit>, <http://mu.semte.ch/graphs/worship-service> ))
|
||||
}
|
||||
|
||||
FILTER (?g IN ( <http://mu.semte.ch/graphs/administrative-unit>, <http://mu.semte.ch/graphs/worship-service>, <http://mu.semte.ch/graphs/shared>))
|
||||
FILTER (?h IN ( <http://mu.semte.ch/graphs/administrative-unit>, <http://mu.semte.ch/graphs/worship-service>, <http://mu.semte.ch/graphs/shared>))
|
||||
|
||||
}
|
||||
`;
|
||||
|
||||
const queryResponsePart2 = await batchedQuery(queryStringPart2);
|
||||
const dataPart2 = queryResponsePart2.results.bindings.reduce( (acc, row) => {
|
||||
const dataPart2 = queryResponsePart2.results.bindings.reduce((acc, row) => {
|
||||
let dataPart = {
|
||||
bestuur: getSafeValue(row, 'bestuur'),
|
||||
provincie: getSafeValue(row, 'provincie')
|
||||
bestuur: getSafeValue(row, "bestuur"),
|
||||
provincie: getSafeValue(row, "provincie"),
|
||||
};
|
||||
|
||||
acc[getSafeValue(row, 'bestuur')] = Object.assign(dataPart, dataPart1[getSafeValue(row, 'bestuur')]);
|
||||
acc[getSafeValue(row, "bestuur")] = Object.assign(
|
||||
dataPart,
|
||||
dataPart1[getSafeValue(row, "bestuur")],
|
||||
);
|
||||
return acc;
|
||||
}, {});
|
||||
|
||||
await generateReportFromData(Object.values(dataPart2), [
|
||||
'bestuur',
|
||||
'label',
|
||||
'classification',
|
||||
'typeEredienst',
|
||||
'provincie'
|
||||
], reportData);
|
||||
}
|
||||
await generateReportFromData(
|
||||
Object.values(dataPart2),
|
||||
["bestuur", "label", "classification", "typeEredienst", "provincie"],
|
||||
reportData,
|
||||
);
|
||||
},
|
||||
};
|
||||
|
||||
function getSafeValue(entry, property){
|
||||
function getSafeValue(entry, property) {
|
||||
return entry[property] ? wrapInQuote(entry[property].value) : null;
|
||||
}
|
||||
|
||||
|
||||
@@ -469,6 +469,7 @@
|
||||
"inverse": true
|
||||
}
|
||||
},
|
||||
"features": ["include-uri"],
|
||||
"new-resource-base": "http://data.lblod.info/id/organisaties/"
|
||||
},
|
||||
"administrative-units": {
|
||||
@@ -944,7 +945,7 @@
|
||||
"located-within": {
|
||||
"predicate": "geo:sfWithin",
|
||||
"target": "locations",
|
||||
"cardinality": "one"
|
||||
"cardinality": "many"
|
||||
},
|
||||
"locations": {
|
||||
"predicate": "geo:sfWithin",
|
||||
|
||||
@@ -74,6 +74,8 @@ services:
|
||||
restart: "no"
|
||||
construct-organization-relationships:
|
||||
restart: "no"
|
||||
scope-of-operation:
|
||||
restart: "no"
|
||||
mandatarissen-consumer:
|
||||
restart: "no"
|
||||
leidinggevenden-consumer:
|
||||
|
||||
@@ -114,7 +114,7 @@ services:
|
||||
restart: always
|
||||
logging: *default-logging
|
||||
deltanotifier:
|
||||
image: cecemel/delta-notifier:0.2.0-beta.2
|
||||
image: semtech/mu-delta-notifier:0.4.0
|
||||
volumes:
|
||||
- ./config/delta:/config
|
||||
labels:
|
||||
@@ -194,7 +194,7 @@ services:
|
||||
restart: always
|
||||
logging: *default-logging
|
||||
frontend:
|
||||
image: lblod/frontend-organization-portal:1.32.0
|
||||
image: lblod/frontend-organization-portal:1.34.0
|
||||
volumes:
|
||||
- ./config/frontend/add-x-frame-options-header.conf:/config/add-x-frame-options.conf
|
||||
environment:
|
||||
@@ -285,8 +285,16 @@ services:
|
||||
- "logging=true"
|
||||
restart: always
|
||||
logging: *default-logging
|
||||
scope-of-operation:
|
||||
image: lblod/scope-of-operation-service:0.1.1
|
||||
links:
|
||||
- db:database
|
||||
labels:
|
||||
- "logging=true"
|
||||
restart: always
|
||||
logging: *default-logging
|
||||
report-generation:
|
||||
image: lblod/loket-report-generation-service:0.7.0
|
||||
image: lblod/loket-report-generation-service:0.8.3
|
||||
environment:
|
||||
DEFAULT_GRAPH: "http://mu.semte.ch/graphs/shared"
|
||||
ONLY_KEEP_LATEST_REPORT: "true"
|
||||
@@ -310,7 +318,7 @@ services:
|
||||
restart: always
|
||||
logging: *default-logging
|
||||
kbo-data-sync:
|
||||
image: lblod/sync-wegwijs-organization-service:2.0.3
|
||||
image: lblod/sync-wegwijs-organization-service:2.0.4
|
||||
links:
|
||||
- db:database
|
||||
labels:
|
||||
|
||||
@@ -1,6 +1,38 @@
|
||||
{
|
||||
"version": "0.1",
|
||||
"scripts": [
|
||||
{
|
||||
"documentation": {
|
||||
"command": "create-export",
|
||||
"description": "create export of administrative units and their bodies",
|
||||
"arguments": []
|
||||
},
|
||||
"environment": {
|
||||
"image": "nvdk/ruby-semantic-works-cli",
|
||||
"interactive": true,
|
||||
"script": "export/run.rb",
|
||||
"join_networks": true
|
||||
},
|
||||
"mounts": {
|
||||
"app": "/app/"
|
||||
}
|
||||
},
|
||||
{
|
||||
"documentation": {
|
||||
"command": "correct-location-labels",
|
||||
"description": "generate migrations to correct the labels of aggregated locations",
|
||||
"arguments": []
|
||||
},
|
||||
"environment": {
|
||||
"image": "nvdk/ruby-semantic-works-cli",
|
||||
"interactive": true,
|
||||
"script": "correct-location-labels/run.rb",
|
||||
"join_networks": true
|
||||
},
|
||||
"mounts": {
|
||||
"app": "/app/"
|
||||
}
|
||||
},
|
||||
{
|
||||
"documentation": {
|
||||
"command": "mu-search-delta",
|
||||
|
||||
106
scripts/project/correct-location-labels/run.rb
Executable file
106
scripts/project/correct-location-labels/run.rb
Executable file
@@ -0,0 +1,106 @@
|
||||
#!/usr/bin/env ruby
|
||||
require 'date'
|
||||
require 'rdf/turtle'
|
||||
require 'fileutils'
|
||||
|
||||
$stdout.sync = true
|
||||
ENV['LOG_SPARQL_ALL']='false'
|
||||
ENV['MU_SPARQL_ENDPOINT']='http://triplestore:8890/sparql'
|
||||
ENV['MU_SPARQL_TIMEOUT']='180'
|
||||
require 'mu/auth-sudo'
|
||||
|
||||
def ensure_dir(path)
|
||||
unless Dir.exist?(path)
|
||||
FileUtils.mkdir_p(path)
|
||||
end
|
||||
end
|
||||
|
||||
def sparql_escape_uri(value)
|
||||
'<' + value.to_s.gsub(/[\\"<>]/) { |s| '\\' + s } + '>'
|
||||
end
|
||||
|
||||
def sparql_escape_string(str)
|
||||
'"""' + str.gsub(/[\\"]/) { |s| '\\' + s } + '"""'
|
||||
end
|
||||
|
||||
def get_locations()
|
||||
query = <<~EOF
|
||||
PREFIX prov: <http://www.w3.org/ns/prov#>
|
||||
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
|
||||
PREFIX ext: <http://mu.semte.ch/vocabularies/ext/>
|
||||
|
||||
SELECT DISTINCT ?location ?label
|
||||
WHERE {
|
||||
GRAPH <http://mu.semte.ch/graphs/public> {
|
||||
?location a prov:Location ;
|
||||
rdfs:label ?label ;
|
||||
ext:werkingsgebiedNiveau """Samengesteld werkingsgebied""" .
|
||||
}
|
||||
}
|
||||
EOF
|
||||
Mu::AuthSudo.query(query)
|
||||
end
|
||||
|
||||
def sort_literal(literal)
|
||||
literal
|
||||
.humanize
|
||||
.split(", ")
|
||||
.sort
|
||||
.join(", ")
|
||||
end
|
||||
|
||||
# Create output directory
|
||||
timestamp=`date +%Y%0m%0d%0H%0M%0S`.strip.to_i
|
||||
output_dir = "/app/config/migrations/local/2025/#{timestamp}-correct-labels-aggregate-locations/"
|
||||
ensure_dir(output_dir)
|
||||
|
||||
|
||||
# Generate SPARQL migration to remove existing labels
|
||||
path = File.join(output_dir, "#{timestamp}-remove-labels-aggregate-locations.sparql")
|
||||
File.open(path, 'w') do |file|
|
||||
file.write <<EOF
|
||||
PREFIX prov: <http://www.w3.org/ns/prov#>
|
||||
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
|
||||
PREFIX ext: <http://mu.semte.ch/vocabularies/ext/>
|
||||
|
||||
DELETE {
|
||||
GRAPH <http://mu.semte.ch/graphs/public> {
|
||||
?location rdfs:label ?label
|
||||
}
|
||||
} WHERE {
|
||||
GRAPH <http://mu.semte.ch/graphs/public> {
|
||||
?location a prov:Location ;
|
||||
rdfs:label ?label ;
|
||||
ext:werkingsgebiedNiveau """Samengesteld werkingsgebied""" .
|
||||
}
|
||||
}
|
||||
EOF
|
||||
end
|
||||
|
||||
|
||||
# Generate TTL migration to insert alphabetically sorted labels
|
||||
timestamp += 1
|
||||
filename = "#{timestamp}-insert-labels-aggregate-locations"
|
||||
|
||||
path = File.join(output_dir, "#{filename}.graph")
|
||||
File.open(path, 'w') do |file|
|
||||
file.write <<EOF
|
||||
http://mu.semte.ch/graphs/public
|
||||
EOF
|
||||
end
|
||||
|
||||
path = File.join(output_dir, "#{filename}.ttl")
|
||||
File.open(path, 'w') do |file|
|
||||
file.write <<EOF
|
||||
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
|
||||
EOF
|
||||
end
|
||||
|
||||
locations = get_locations()
|
||||
locations.each do |location|
|
||||
File.open(path, 'a') do |file|
|
||||
file.write <<EOF
|
||||
#{sparql_escape_uri(location[:location])} rdfs:label "#{(sort_literal(location[:label]))}" .
|
||||
EOF
|
||||
end
|
||||
end
|
||||
22
scripts/project/export/enrichment.owl
Normal file
22
scripts/project/export/enrichment.owl
Normal file
@@ -0,0 +1,22 @@
|
||||
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
|
||||
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#>.
|
||||
@prefix owl: <http://www.w3.org/2002/07/owl#> .
|
||||
@prefix generiek: <https://data.vlaanderen.be/ns/generiek#> .
|
||||
@prefix mandaat: <http://data.vlaanderen.be/ns/mandaat#> .
|
||||
@prefix org: <http://www.w3.org/ns/org#> .
|
||||
@prefix besluit: <http://data.vlaanderen.be/ns/besluit#> .
|
||||
@prefix ere: <http://data.lblod.info/vocabularies/erediensten/> .
|
||||
|
||||
<http://lblod.data.gift/mappings.owl> a owl:Ontology .
|
||||
|
||||
# Equivalent Properties
|
||||
generiek:isTijdspecialisatieVan owl:equivalentProperty
|
||||
<http://data.vlaanderen.be/ns/generiek#isTijdspecialisatieVan>,
|
||||
mandaat:isTijdspecialisatieVan .
|
||||
org:classification owl:equivalentProperty besluit:classificatie .
|
||||
generiek:gestructureerdeIdentificator owl:equivalentProperty <http://data.vlaanderen.be/ns/generiek#gestructureerdeIdentificator>.
|
||||
|
||||
# Subclass mappings
|
||||
ere:BestuurVanDeEredienst rdfs:subClassOf besluit:Bestuurseenheid .
|
||||
ere:CentraalBestuurVanDeEredienst rdfs:subClassOf besluit:Bestuurseenheid .
|
||||
ere:RepresentatiefOrgaan rdfs:subClassOf besluit:Bestuurseenheid.
|
||||
358
scripts/project/export/run.rb
Executable file
358
scripts/project/export/run.rb
Executable file
@@ -0,0 +1,358 @@
|
||||
#!/usr/bin/env ruby
|
||||
|
||||
require 'bundler/inline'
|
||||
require 'yaml'
|
||||
require 'securerandom'
|
||||
require 'date'
|
||||
require 'rdf/vocab'
|
||||
require 'tty-prompt'
|
||||
require 'rdf/turtle'
|
||||
require 'rdf/reasoner'
|
||||
|
||||
$stdout.sync = true
|
||||
ENV['LOG_SPARQL_ALL']='false'
|
||||
ENV['MU_SPARQL_ENDPOINT']='http://triplestore:8890/sparql'
|
||||
ENV['MU_SPARQL_TIMEOUT']='180'
|
||||
require 'mu/auth-sudo'
|
||||
|
||||
|
||||
|
||||
def sparql_escape_uri(value)
|
||||
'<' + value.to_s.gsub(/[\\"<>]/) { |s| '\\' + s } + '>'
|
||||
end
|
||||
|
||||
def sparql_escape_string(str)
|
||||
'"""' + str.gsub(/[\\"]/) { |s| '\\' + s } + '"""'
|
||||
end
|
||||
|
||||
def search_classifications
|
||||
query = <<~EOF
|
||||
PREFIX org: <http://www.w3.org/ns/org#>
|
||||
PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
|
||||
PREFIX besluit: <http://data.vlaanderen.be/ns/besluit#>
|
||||
SELECT DISTINCT ?type ?name ?classification WHERE {
|
||||
?org a ?type;
|
||||
org:classification ?classification.
|
||||
?classification skos:prefLabel ?name.
|
||||
VALUES ?type { besluit:Bestuurseenheid }
|
||||
} ORDER BY ?name
|
||||
EOF
|
||||
Mu::AuthSudo.query(query)
|
||||
end
|
||||
|
||||
def get_admin_units(classifications)
|
||||
query = <<~EOF
|
||||
PREFIX org: <http://www.w3.org/ns/org#>
|
||||
PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
|
||||
PREFIX besluit: <http://data.vlaanderen.be/ns/besluit#>
|
||||
SELECT DISTINCT ?org ?name WHERE {
|
||||
?org a ?type;
|
||||
skos:prefLabel ?name;
|
||||
org:classification ?classification.
|
||||
VALUES ?type { besluit:Bestuurseenheid }
|
||||
VALUES ?classification { #{classifications.map{ |c| sparql_escape_uri(c) }.join(" ")}}
|
||||
}
|
||||
EOF
|
||||
Mu::AuthSudo.query(query)
|
||||
end
|
||||
|
||||
def export_admin_unit(unit)
|
||||
repository = RDF::Repository.new
|
||||
query = <<~EOF
|
||||
PREFIX org: <http://www.w3.org/ns/org#>
|
||||
PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
|
||||
PREFIX besluit: <http://data.vlaanderen.be/ns/besluit#>
|
||||
PREFIX generiek: <https://data.vlaanderen.be/ns/generiek#>
|
||||
CONSTRUCT {
|
||||
?org ?p ?o
|
||||
}
|
||||
WHERE {
|
||||
VALUES ?org { #{sparql_escape_uri(unit)}}
|
||||
?org ?p ?o.
|
||||
VALUES ?p {
|
||||
rdf:type
|
||||
org:classification
|
||||
besluit:classificatie
|
||||
skos:prefLabel
|
||||
<http://mu.semte.ch/vocabularies/core/uuid>
|
||||
<http://www.w3.org/ns/regorg#orgStatus>
|
||||
<http://data.lblod.info/vocabularies/erediensten/betrokkenBestuur>
|
||||
}
|
||||
}
|
||||
EOF
|
||||
repository << Mu::AuthSudo.query(query)
|
||||
query = <<~EOF
|
||||
PREFIX org: <http://www.w3.org/ns/org#>
|
||||
PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
|
||||
PREFIX besluit: <http://data.vlaanderen.be/ns/besluit#>
|
||||
PREFIX generiek: <https://data.vlaanderen.be/ns/generiek#>
|
||||
CONSTRUCT {
|
||||
?org <http://www.w3.org/ns/adms#identifier> ?id.
|
||||
?id ?p ?o.
|
||||
?id skos:notation ?localId.
|
||||
?id <http://www.w3.org/ns/adms#schemaAgency> ?source.
|
||||
}
|
||||
WHERE {
|
||||
VALUES ?org { #{sparql_escape_uri(unit)}}
|
||||
?org <http://www.w3.org/ns/adms#identifier> ?id.
|
||||
?id ?p ?o.
|
||||
FILTER (?p NOT IN(<http://www.w3.org/2004/02/skos/core#notation>, <https://data.vlaanderen.be/ns/generiek#gestructureerdeIdentificator>))
|
||||
?id <http://www.w3.org/2004/02/skos/core#notation> ?source.
|
||||
FILTER(?source != "SharePoint identificator")
|
||||
?id <https://data.vlaanderen.be/ns/generiek#gestructureerdeIdentificator> ?gestructureerdeIdentificator.
|
||||
?gestructureerdeIdentificator <https://data.vlaanderen.be/ns/generiek#lokaleIdentificator> ?localId
|
||||
}
|
||||
EOF
|
||||
repository << Mu::AuthSudo.query(query)
|
||||
query = <<~EOF
|
||||
PREFIX org: <http://www.w3.org/ns/org#>
|
||||
PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
|
||||
PREFIX besluit: <http://data.vlaanderen.be/ns/besluit#>
|
||||
PREFIX generiek: <https://data.vlaanderen.be/ns/generiek#>
|
||||
CONSTRUCT {
|
||||
?org besluit:werkingsgebied ?gebied.
|
||||
?gebied ?p ?o.
|
||||
}
|
||||
WHERE {
|
||||
VALUES ?org { #{sparql_escape_uri(unit)}}
|
||||
?org besluit:werkingsgebied ?gebied.
|
||||
?gebied ?p ?o
|
||||
}
|
||||
EOF
|
||||
repository << Mu::AuthSudo.query(query)
|
||||
|
||||
query = <<~EOF
|
||||
PREFIX org: <http://www.w3.org/ns/org#>
|
||||
PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
|
||||
PREFIX besluit: <http://data.vlaanderen.be/ns/besluit#>
|
||||
PREFIX generiek: <https://data.vlaanderen.be/ns/generiek#>
|
||||
PREFIX mandaat: <https://data.vlaanderen.be/ns/mandaat#>
|
||||
CONSTRUCT {
|
||||
?bestuursorgaan ?bp ?bo.
|
||||
?bestuursorgaanIntijd generiek:isTijdspecialisatieVan ?bestuursorgaan.
|
||||
?bestuursorgaanIntijd ?bop ?boo.
|
||||
}
|
||||
WHERE {
|
||||
VALUES ?org { #{sparql_escape_uri(unit)}}
|
||||
?bestuursorgaan besluit:bestuurt ?org.
|
||||
?bestuursorgaan ?bp ?bo.
|
||||
values ?tijdspecialisatie {
|
||||
generiek:isTijdspecialisatieVan
|
||||
mandaat:isTijdspecialisatieVan
|
||||
}
|
||||
?bestuursorgaanIntijd ?tijdspecialisatie ?bestuursorgaan.
|
||||
?bestuursorgaanIntijd ?bop ?boo.
|
||||
}
|
||||
|
||||
EOF
|
||||
repository << Mu::AuthSudo.query(query)
|
||||
query = <<~EOF
|
||||
PREFIX org: <http://www.w3.org/ns/org#>
|
||||
PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
|
||||
PREFIX besluit: <http://data.vlaanderen.be/ns/besluit#>
|
||||
PREFIX generiek: <https://data.vlaanderen.be/ns/generiek#>
|
||||
CONSTRUCT {
|
||||
?mandaat ?p ?o.
|
||||
?role ?roleP ?roleO.
|
||||
}
|
||||
WHERE {
|
||||
VALUES ?org { #{sparql_escape_uri(unit)}}
|
||||
?bestuursorgaan besluit:bestuurt ?org.
|
||||
?bestuursorgaanIntijd generiek:isTijdspecialisatieVan ?bestuursorgaan.
|
||||
VALUES ?post { org:hasPost <http://data.lblod.info/vocabularies/leidinggevenden/heeftBestuursfunctie>}
|
||||
?bestuursorgaanIntijd ?post ?mandaat.
|
||||
?mandaat ?p ?o.
|
||||
?mandaat org:role ?role.
|
||||
OPTIONAL {
|
||||
?role ?roleP ?roleO.
|
||||
FILTER(!STRSTARTS(STR(?role), "http://data.vlaanderen.be"))
|
||||
}
|
||||
}
|
||||
EOF
|
||||
repository << Mu::AuthSudo.query(query)
|
||||
repository
|
||||
end
|
||||
|
||||
def ensure_dir(path)
|
||||
unless Dir.exist?(path)
|
||||
Dir.mkdir(path)
|
||||
end
|
||||
end
|
||||
|
||||
def sanitize_unix_path(path)
|
||||
# Regular expression to match characters not allowed in Unix paths
|
||||
sanitized_path = path.gsub(/[^a-zA-Z0-9._\-\/]/, '-')
|
||||
sanitized_path
|
||||
end
|
||||
|
||||
|
||||
def enrich_graph(data)
|
||||
RDF::Reasoner.apply(:owl,:rdfs)
|
||||
graph = RDF::Graph.load("./enrichment.owl")
|
||||
graph << clean_up_dates(data)
|
||||
graph.entail!(:subClassOf)
|
||||
graph.entail!(:equivalentProperty)
|
||||
graph
|
||||
end
|
||||
|
||||
DATE_PREDICATES=[
|
||||
RDF::URI("http://data.vlaanderen.be/ns/mandaat#bindingStart"),
|
||||
RDF::URI("http://data.vlaanderen.be/ns/mandaat#bindingEinde"),
|
||||
RDF::URI("https://data.vlaanderen.be/ns/generiek#bindingStart"),
|
||||
RDF::URI("https://data.vlaanderen.be/ns/generiek#bindingEinde")
|
||||
]
|
||||
|
||||
def convert_date_to_datetime(date_literal)
|
||||
begin
|
||||
date = Date.parse(date_literal.to_s)
|
||||
datetime = DateTime.new(date.year, date.month, date.day)
|
||||
RDF::Literal.new(datetime.strftime("%Y-%m-%dT%T"), datatype: RDF::XSD.dateTime)
|
||||
rescue ArgumentError => e
|
||||
puts "Invalid date format for #{date_literal}: #{e.message}"
|
||||
nil
|
||||
end
|
||||
end
|
||||
|
||||
def clean_up_dates(data)
|
||||
clean_data = RDF::Graph.new
|
||||
data.each_statement do |statement|
|
||||
if DATE_PREDICATES.include?(statement.predicate)
|
||||
obj = statement.object
|
||||
if obj.is_a?(RDF::Literal) && obj.datatype == RDF::XSD.date
|
||||
new_object = convert_date_to_datetime(obj)
|
||||
if new_object
|
||||
clean_data << [statement.subject, statement.predicate, new_object] # Add the new statement
|
||||
end
|
||||
elsif obj.is_a?(RDF::Literal) && obj.datatype == RDF::XSD.dateTime
|
||||
clean_data << statement
|
||||
else
|
||||
puts "unexpected value #{obj.inspect}for statement:\n #{statement.inspect}"
|
||||
end
|
||||
else
|
||||
clean_data << statement
|
||||
end
|
||||
end
|
||||
clean_data
|
||||
end
|
||||
|
||||
prompt = TTY::Prompt.new
|
||||
prompt.say("\n\n")
|
||||
prompt.say("This script will help you export administrative units and their respective bodies")
|
||||
options = search_classifications.map do |binding|
|
||||
{name: "#{binding[:name]}", value: binding[:classification]}
|
||||
end
|
||||
|
||||
if options.size > 0
|
||||
classifications = prompt.multi_select("Please select the matching admin unit classifications", options)
|
||||
admin_units = get_admin_units(classifications)
|
||||
prompt.say("found #{admin_units.size} admin units")
|
||||
output_dir = prompt.ask("Provide the directory the exports should be written to?", default: "/app/exports/")
|
||||
ensure_dir(output_dir)
|
||||
admin_units.each do |unit|
|
||||
repo = export_admin_unit(unit[:org])
|
||||
repo = enrich_graph(repo)
|
||||
file_name = sanitize_unix_path(unit[:name].to_s)
|
||||
timestamp=`date +%Y%0m%0d%0H%0M%0S`.strip.to_i
|
||||
path = File.join(output_dir, "#{timestamp}-remove-data-#{file_name}-bestuurseenheid.sparql")
|
||||
File.open(path, 'w') do |file|
|
||||
file.write <<EOF
|
||||
PREFIX besluit: <http://data.vlaanderen.be/ns/besluit#>
|
||||
PREFIX mandaat: <http://data.vlaanderen.be/ns/mandaat#>
|
||||
PREFIX generiek: <https://data.vlaanderen.be/ns/generiek#>
|
||||
PREFIX org: <http://www.w3.org/ns/org#>
|
||||
PREFIX prov: <http://www.w3.org/ns/prov#>
|
||||
DELETE {
|
||||
GRAPH <http://mu.semte.ch/graphs/public> {
|
||||
?id ?p ?o.
|
||||
}
|
||||
} WHERE {
|
||||
GRAPH <http://mu.semte.ch/graphs/public> {
|
||||
<#{unit[:org]}> <http://www.w3.org/ns/adms#identifier> ?id.
|
||||
?id ?p ?o.
|
||||
}
|
||||
};
|
||||
DELETE {
|
||||
GRAPH <http://mu.semte.ch/graphs/public> {
|
||||
?gebied ?p ?o.
|
||||
}
|
||||
} WHERE {
|
||||
GRAPH <http://mu.semte.ch/graphs/public> {
|
||||
<#{unit[:org]}> besluit:werkingsgebied ?gebied.
|
||||
?gebied ?p ?o.
|
||||
}
|
||||
};
|
||||
DELETE WHERE {
|
||||
GRAPH <http://mu.semte.ch/graphs/public> {
|
||||
<#{unit[:org]}> ?p ?o
|
||||
}
|
||||
};
|
||||
EOF
|
||||
end
|
||||
path = File.join(output_dir, "#{timestamp}-remove-data-#{file_name}-bestuursorganen.sparql")
|
||||
File.open(path, 'w') do |file|
|
||||
file.write <<EOF
|
||||
PREFIX besluit: <http://data.vlaanderen.be/ns/besluit#>
|
||||
PREFIX mandaat: <http://data.vlaanderen.be/ns/mandaat#>
|
||||
PREFIX generiek: <https://data.vlaanderen.be/ns/generiek#>
|
||||
PREFIX org: <http://www.w3.org/ns/org#>
|
||||
PREFIX prov: <http://www.w3.org/ns/prov#>
|
||||
DELETE {
|
||||
GRAPH <http://mu.semte.ch/graphs/public> {
|
||||
?mandaat ?p ?o
|
||||
}
|
||||
} WHERE {
|
||||
GRAPH <http://mu.semte.ch/graphs/public> {
|
||||
?bestuursorgaan besluit:bestuurt <#{unit[:org]}> .
|
||||
?bestuursorgaanIntijd mandaat:isTijdspecialisatieVan ?bestuursorgaan.
|
||||
VALUES ?post { org:hasPost <http://data.lblod.info/vocabularies/leidinggevenden/heeftBestuursfunctie>}
|
||||
?bestuursorgaanIntijd ?post ?mandaat.
|
||||
?mandaat ?p ?o.
|
||||
}
|
||||
};
|
||||
DELETE {
|
||||
GRAPH <http://mu.semte.ch/graphs/public> {
|
||||
?mandaat ?p ?o
|
||||
}
|
||||
} WHERE {
|
||||
GRAPH <http://mu.semte.ch/graphs/public> {
|
||||
?bestuursorgaan besluit:bestuurt <#{unit[:org]}> .
|
||||
?bestuursorgaanIntijd mandaat:isTijdspecialisatieVan ?bestuursorgaan.
|
||||
VALUES ?post { org:hasPost <http://data.lblod.info/vocabularies/leidinggevenden/heeftBestuursfunctie>}
|
||||
?bestuursorgaanIntijd ?post ?mandaat.
|
||||
?mandaat ?p ?o.
|
||||
}
|
||||
};
|
||||
DELETE {
|
||||
GRAPH <http://mu.semte.ch/graphs/public> {
|
||||
?bestuursorgaanIntijd ?p ?o
|
||||
}
|
||||
} WHERE {
|
||||
GRAPH <http://mu.semte.ch/graphs/public> {
|
||||
?bestuursorgaan besluit:bestuurt <#{unit[:org]}> .
|
||||
?bestuursorgaanIntijd mandaat:isTijdspecialisatieVan ?bestuursorgaan.
|
||||
VALUES ?post { org:hasPost <http://data.lblod.info/vocabularies/leidinggevenden/heeftBestuursfunctie>}
|
||||
?bestuursorgaanIntijd ?p ?o
|
||||
}
|
||||
};
|
||||
DELETE {
|
||||
GRAPH <http://mu.semte.ch/graphs/public> {
|
||||
?bestuursorgaan ?p ?o
|
||||
}
|
||||
} WHERE {
|
||||
GRAPH <http://mu.semte.ch/graphs/public> {
|
||||
?bestuursorgaan besluit:bestuurt <#{unit[:org]}>; ?p ?o
|
||||
}
|
||||
}
|
||||
EOF
|
||||
end
|
||||
timestamp+=1
|
||||
path = File.join(output_dir, "#{timestamp}-#{SecureRandom.uuid}-#{file_name}.ttl")
|
||||
File.open(path, 'w') do |file|
|
||||
file.write repo.dump(:ttl)
|
||||
end
|
||||
prompt.say("Exported #{repo.size} statements to #{path}")
|
||||
end
|
||||
else
|
||||
abort("No classifications found?")
|
||||
end
|
||||
|
||||
Reference in New Issue
Block a user