* Set up deltas for sync with sharepoint
* add service
* add dev environment variables
* Extract config to the stack
* Adjust the config and mappings
* Filter out worship services
* Add contacten sp list service
* Add configuration for contact list
* Add config files for contacten sync
* Wire sharepoint sync on main graph instead of producer
* Add env variables
* Add op sync initiator
* Minor updates
* Improve config, add job controller rules and more
* Adjust rules
* Remove unused producer changes
* Don't restart initiator on dev setup
* Sync OP status to a dedicated column
* Add ABB_ID and split bus and house numbers
* Remove config file we will not use
* Add NIS and secondary phone number, rename columns
* Remove isInitialMatchingMapping, not used anymore
* Match on URI instead of UUID
* Adapt field names to new lists
* Specify matching field in config
* Set version of op-sync-with-sharepoint
---------
Co-authored-by: Nordine Bittich <nordine@bittich.be>
* Setting up file-service
- pre-requisite to download dumps
- pre-requiiste to download the json-diff files
- security considerations: the content of the files created here, is
- assumed to be public data, hence we allow it to be in the public graph
* Bumping deltanotifier and mu-auth, which need
to be able to work with mu-scop-id's
* Delta-producer have multiple background-jobs with different tasks
Hence the introduction of the jobs-controller.
Note: in a more advanced configuration, you can easily extend the
tasks related to one job.
* A pre-requisite to serve dumps, is DCAT api support
The delta-producers make a periodic dump of what is supposed to be
published. They are published according to the DCAT data-model.
Security-considerations: the DCAT producer by the delta-producer flow,
are considered to be public data, hence added to the public graph.
* Introduction of the delta-producer-dump-file-publisher
Service responsible for creating dumps for the (potentially
many) data delta flows.
Here only for the administrative-units, but this can be extended in
the future, in the config.
* Introduction of delta-producer-background-jobs-initiator
This service is responsible for triggering the background jobs related
to the delta-productoin of the administrative-units.
It trigger currently three jobs:
- INITIAL_SYNC: this is the first job to fill the publication graph.
In theory, this is only run once per deploy
- HEALING_JOB: This is a periodic job, which checks what information
is published vs. what information should have been
published (according to a configuration). If differences are found,
because delta-messages never arrived, they are compensated here/
- DUMP_JOB: A periodic job to create a dump of the publication
graph. This is especially useful for consumers, who don't have to
replay the whole delta-diffs, but just can start from the latest sync
point.
* Introduction of delta-producer-publication-graph-maintainer
This service is the core of the logic which determines what should be
published.
The configuration is provided here, and can be elaborated refined in
the config.
It also excutes the healing logic.
* Introduction of delta-producer-json-diff-file-publisher
This simple service, listens to the updates performed on the
publication graph, and serialized these to json-files which are
compatible with current consumers
* Missing dispatcher rules for DCAT
* Various small things:
- outline, whitespaces and comments
* Update README.md
* bump delta-producer-publication-graph-maintainer
* Add organizations (KAL) - Work in progress
* added organizations graph maintainer
* removed inverses, added all bestuur types
* refactored kalliope api
Co-authored-by: Boris De Vloed <boris.de.vloed@gmail.com>
