felix.ruizdearcaute/app-organization-portal
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge branch 'master' into chore/bump-mandatarissen-consumer

Browse Source
This commit is contained in:
Claire Lovisa
2025-02-03 16:11:05 +01:00
committed by GitHub
parent 300e088b6c 0b8530db34
commit f79724132a
80 changed files with 1540 additions and 939 deletions
Download Patch File Download Diff File Expand all files Collapse all files

99
README.md
View File

@@ -12,6 +12,7 @@ Backend for the organization portal application, based on the mu.semte.ch micros
You can shut down using `docker-compose stop` and remove everything using `docker-compose rm`.
#### Setup consumers
The application depends on external data. You'll have to perform extra steps if you want this data to be available.
These are multiple data sets, we'll try to be exhaustive on how to ingest these. Even though the procedures are similar.
@@ -52,6 +53,104 @@ drc up -d
sh scripts/reset-elastic.sh
```
#### Setup leidinggevenden sync (http://data.lblod.info/vocabularies/leidinggevenden/Functionaris)
Note: this is assuming you have never ran this before.
```
drc down;
```
Update `docker-compose.override.yml` to:
```
leidinggevenden-consumer:
environment:
DCR_SYNC_BASE_URL: "https://loket.lokaalbestuur.vlaanderen.be" # or another endpoint
DCR_LANDING_ZONE_DATABASE: "triplestore" # for the initial sync, we go directly to virtuoso
DCR_REMAPPING_DATABASE: "triplestore" # for the initial sync, we go directly to virtuoso
DCR_DISABLE_DELTA_INGEST: "false"
DCR_DISABLE_INITIAL_SYNC: "false"
```
Then:
```
drc up -d migrations
drc up -d db leidinggevenden-consumer
# Wait until success of the previous step
```
Then, update `docker-compose.override.yml` to:
```
leidinggevenden-consumer:
environment:
DCR_SYNC_BASE_URL: "https://loket.lokaalbestuur.vlaanderen.be" # choose the correct endpoint
DCR_LANDING_ZONE_DATABASE: "db"
DCR_REMAPPING_DATABASE: "db"
DCR_DISABLE_DELTA_INGEST: "false"
DCR_DISABLE_INITIAL_SYNC: "false"
```
Finally, put the stack back up and reindex elasticsearch
```
drc up -d
sh scripts/reset-elastic.sh
```
#### Setup worship mandatees sync
Note: this is assuming you have never ran this before.
```
drc down;
```
Update `docker-compose.override.yml` to:
```
worship-services-main-info-consumer:
environment:
DCR_SYNC_BASE_URL: "https://loket.lokaalbestuur.vlaanderen.be" # or another endpoint
DCR_SYNC_LOGIN_ENDPOINT: "https://loket.lokaalbestuur.vlaanderen.be/sync/worship-services-sensitive/login" # or another endpoint
DCR_SECRET_KEY: "key-of-the-producer"
DCR_LANDING_ZONE_DATABASE: "triplestore"
DCR_REMAPPING_DATABASE: "triplestore"
DCR_DISABLE_DELTA_INGEST: "false"
DCR_DISABLE_INITIAL_SYNC: "false"
worship-services-private-info-consumer:
environment:
DCR_SYNC_BASE_URL: "https://loket.lokaalbestuur.vlaanderen.be" # or another endpoint
DCR_SYNC_LOGIN_ENDPOINT: "https://loket.lokaalbestuur.vlaanderen.be/sync/worship-services-sensitive/login" # or another endpoint
DCR_SECRET_KEY: "key-of-the-producer"
DCR_LANDING_ZONE_DATABASE: "triplestore"
DCR_REMAPPING_DATABASE: "triplestore"
DCR_DISABLE_DELTA_INGEST: "false"
DCR_DISABLE_INITIAL_SYNC: "false"
```
Then:
```
drc up -d migrations
drc up -d db worship-services-main-info-consumer worship-services-private-info-consumer
# Wait until success of the previous step
```
Then, update `docker-compose.override.yml` to:
```
worship-services-main-info-consumer:
environment:
DCR_SYNC_BASE_URL: "https://loket.lokaalbestuur.vlaanderen.be" # or another endpoint
DCR_SYNC_LOGIN_ENDPOINT: "https://loket.lokaalbestuur.vlaanderen.be/sync/worship-services-sensitive/login" # or another endpoint
DCR_SECRET_KEY: "key-of-the-producer"
DCR_LANDING_ZONE_DATABASE: "db"
DCR_REMAPPING_DATABASE: "db"
DCR_DISABLE_DELTA_INGEST: "false"
DCR_DISABLE_INITIAL_SYNC: "false"
worship-services-private-info-consumer:
environment:
DCR_SYNC_BASE_URL: "https://loket.lokaalbestuur.vlaanderen.be" # or another endpoint
DCR_SYNC_LOGIN_ENDPOINT: "https://loket.lokaalbestuur.vlaanderen.be/sync/worship-services-sensitive/login" # or another endpoint
DCR_SECRET_KEY: "key-of-the-producer"
DCR_LANDING_ZONE_DATABASE: "db"
DCR_REMAPPING_DATABASE: "db"
DCR_DISABLE_DELTA_INGEST: "false"
DCR_DISABLE_INITIAL_SYNC: "false"
```
Finally, put the stack back up and reindex elasticsearch
```
drc up -d
sh scripts/reset-elastic.sh
```
### Setting up the delta-producers related services
To make sure the app can share data, producers need to be set up. There is an intial sync, that is potentially very expensive, and must be started manually