8 Commits

Author SHA1 Message Date
felix
dac3e2854d delete 2026-09-27 17:17:34 +02:00
felix
1f80fdc815 make sure only one spec is returned 2026-09-27 16:05:25 +02:00
felix
538a2d949e added improved discrimators 2026-09-27 15:56:31 +02:00
felix
b1fa1d1cd2 update README.md 2026-09-27 15:56:19 +02:00
felix
0802b75346 remove the tools 2026-09-27 14:36:20 +02:00
felix
78a7c42a37 clean up bug in runner and disabled two tools 2026-09-27 02:05:05 +02:00
felix
80097d3130 Further cleaning 2026-09-27 00:57:17 +02:00
felix
57292fa052 Added more exploration possbilities for LLM 2026-09-27 00:44:35 +02:00
13 changed files with 846 additions and 256 deletions

375
README.md
View File

@@ -29,8 +29,8 @@ a fixed Dutch text with the CSV and the spec as bijlagen. No spec yet, a spec
that no longer checks out, or a failed run each give a fixed Dutch failure
message. Nothing is retried and no spec is rewritten.
Only refinement has tools: `list_profiles`, `describe_profile`,
`validate_spec`, `read_spec` and `lookup_values`.
Only refinement has tools: `describe_profile`, `validate_spec`, `read_spec`
and `lookup_values`.
## How it works
@@ -41,26 +41,330 @@ Two documents drive everything:
| **profile** | the app | once, committed | `config/report-profiles/*.ttl`, mounted at `/config/profiles/` |
| **spec** | the LLM | one per question | in the request, not stored |
A **profile** is a SHACL shapes graph: one `sh:NodeShape` per entity, fields
as `sh:property`, code lists via `sh:class`, links via `sh:node`. The service
ships no profile and knows no vocabulary; profiles live in the app repo.
Profiles load at boot; a broken profile stops the service.
A **spec** is `rep:ReportSpec` + `sh:NodeShape`: `sh:property` selects the
subjects, `rep:columns` says what the CSV holds. The two never interact. The
namespace is `http://mu.semte.ch/vocabularies/reporting/` (`rep:`).
A filter tests only the end of its own path. To test a node in the middle of
a path ("the mandate whose role is Burgemeester"), a filter or column carries
`rep:where`: a condition written like a filter, from the row, that shares the
nodes of every step it has in common with its host. On a column it drops
values from the cell, never rows. The app's `report-spec-readme.md` has the
details and an example.
The profile says what the data looks like. The spec says what one report
holds. Both are Turtle, and both borrow the words of SHACL, the W3C language
to describe RDF data. Neither is used the way a SHACL tool uses it: see "How
this relates to SHACL" below.
The run: check → seed (`SELECT DISTINCT ?s`, paged) → columns (per group, per
chunk of subjects) → assemble (dedup + `rep:collect`) → CSV → register file
chunk of subjects, with the nodes on each path) → assemble (pair by the
nodes columns share, on every step; `rep:collect`) → CSV → register file
+ report resource. One run per confirmed spec.
## The profile: a map of the data
A profile (using SHACL + some extensions) lists the kinds of things in the data (entities), what each one
has (fields), and which fields lead to other entities (links). The app
writes it; the service ships no profile and knows no vocabulary.
Three parts of the service read it:
- the validator checks every step of a spec's paths against it;
- the LLM gets it as a menu (`describe_profile`), its only picture of the
data;
- the seed takes the class of the rows from it.
Nothing checks the data against a profile.
```turtle
<http://data.lblod.info/id/report-profiles/mandaten> a owl:Ontology ;
dct:title "Mandaten" .
ent:Mandataris a sh:NodeShape ;
rdfs:label "mandataris" ;
sh:targetClass mandaat:Mandataris ;
sh:property [ sh:path mandaat:start ; sh:name "start" ; sh:datatype xsd:dateTime ] ;
sh:property [ sh:path org:holds ; sh:name "bekleedt mandaat" ; sh:node ent:Mandaat ] ;
sh:property [ sh:path mandaat:isBestuurlijkeAliasVan ; sh:name "persoon" ; sh:node ent:Persoon ] .
```
| You write | It means |
|---|---|
| `owl:Ontology` + `dct:title` | the profile. A spec names this URI in `rep:profile`. |
| `sh:NodeShape` | an entity. A spec names this URI in `rep:entity` when it must pick one. |
| `rdfs:label` on the entity | its name in the menu and in error messages. Keep it unique in the profile. |
| `sh:targetClass` | the `rdf:type` of the entity's subjects |
| `sh:property` + `sh:path` | a field: one step, a predicate or `[ sh:inversePath p ]` |
| `sh:name` | the field's name in the menu and in messages |
| `sh:datatype` | a value field: a column may end here |
| `sh:class` | a code-list field: a column may end here. Lists up to `INLINE_VALUES_MAX` values show in the menu. |
| `sh:node` | a link to another entity: a spec may take this step |
| `sh:or ( [ sh:node A ] [ sh:node B ] )` | a link that reaches A or B |
The service does not read `sh:description` (write it for people; the LLM
never sees it), cardinality (every link counts as "many"), or a field path
of more than one step.
At boot each profile must parse, name itself with `owl:Ontology` and hold at
least one entity; a broken profile stops the service. A `sh:node` to an
entity that does not exist only shows when a spec takes that step.
### A link to several kinds
When one step can reach entities of different kinds, say so with `sh:or`:
```turtle
sh:property [ sh:path [ sh:inversePath besluit:bestuurt ] ; sh:name "bestuursorganen" ;
sh:or ( [ sh:node ent:Bestuursorgaan ] [ sh:node ent:LeidinggevendBestuursorgaan ] ) ] .
```
The validator follows all of them, and the next steps decide. A spec that
wants one kind adds a `rep:where` on a step only that kind has. Write one
field with `sh:or`, not two fields on one path: in SHACL, two fields mean a
value must fit both. (The validator reads two fields like `sh:or`, but a
SHACL reader would not.)
### Entities that share a class
A profile may hold several entities of one class. The portal's
`mandaten.ttl` has four kinds of `besluit:Bestuursorgaan`: political and
leidinggevend, each fixed (vast) and in time. A spec picks one with
`rep:entity`; `rep:discriminator` tells the seed which subjects belong to
it.
A discriminator is written like a spec filter (see "Which rows" below): a
`sh:path` from the subject, of any length, and a test on where it ends.
Several on one entity must all hold. `sh:and ( ... )`, `sh:or ( ... )` and
`sh:not ...` combine them, and may nest.
```turtle
# fixed, and its classification is in the list
ent:LeidinggevendBestuursorgaan
rep:discriminator [ sh:path generiek:isTijdspecialisatieVan ; sh:maxCount 0 ] ;
rep:discriminator [ sh:path org:classification ; sh:in ent:LeidinggevendeOrgaanClassificaties ] .
# in time, and the classification of its fixed organ is not in the list
ent:BestuursorgaanInTijd
rep:discriminator [ sh:path generiek:isTijdspecialisatieVan ; sh:minCount 1 ] ;
rep:discriminator [ sh:path ( generiek:isTijdspecialisatieVan org:classification ) ;
sh:in ent:LeidinggevendeOrgaanClassificaties ; sh:maxCount 0 ] .
```
A rule or a list can be a named node, written once and used in several
places. `sh:not` on a named rule gives its exact opposite, so two entities
split a class with no overlap and no gap:
```turtle
ent:IsLeidinggevendOrgaan sh:path org:classification ;
sh:in ent:LeidinggevendeOrgaanClassificaties .
ent:LeidinggevendBestuursorgaan rep:discriminator ent:IsLeidinggevendOrgaan .
ent:Bestuursorgaan rep:discriminator [ sh:not ent:IsLeidinggevendOrgaan ] .
```
Turtle cannot name a `( ... )` list, so write a named list out:
`ent:X rdf:first <a> ; rdf:rest ( <b> <c> ) .` Do not give a named rule the
type `sh:NodeShape`: it would become an entity.
Test what a thing is, not what hangs off it. The classification says what
an organ is. "Has a period with a bestuursfunctie" needs no code list, but
it trusts the data: an organ that lost its functie would move to the other
entity, and no report could find it. `sh:or` of both catches the most.
The seed turns a plain discriminator into the same SPARQL as a filter.
`sh:and`, `sh:or` and `sh:not` become one `FILTER(...)` over `EXISTS` tests;
that form is not yet tested against mu-authorization or sparql-parser. A
broken discriminator stops the service at boot and names the entity.
## The spec: one report
The LLM writes one spec per question. The user reads the proposal and
confirms; the service then checks the spec again and runs it. A spec has two
halves: which rows, and what the CSV shows about each row.
```turtle
<http://data.lblod.info/id/report-specs/burgemeesters> a rep:ReportSpec , sh:NodeShape ;
dct:title "Burgemeesters sinds 2025" ;
rep:profile <http://data.lblod.info/id/report-profiles/mandaten> ;
sh:targetClass mandaat:Mandataris ;
sh:property [ sh:path ( org:holds org:role ) ;
sh:hasValue <http://data.vlaanderen.be/id/concept/BestuursfunctieCode/5ab0e9b8a3b2ca7c5e000013> ] ;
sh:property [ sh:path mandaat:start ;
sh:minInclusive "2025-01-01T00:00:00"^^xsd:dateTime ] ;
rep:columns (
[ sh:path rep:self ; rdfs:label "mandataris" ]
[ sh:path ( mandaat:isBestuurlijkeAliasVan foaf:givenName ) ; rdfs:label "voornaam" ]
[ sh:path ( mandaat:isBestuurlijkeAliasVan foaf:familyName ) ; rdfs:label "achternaam" ]
[ sh:path ( org:holds [ sh:inversePath org:hasPost ] generiek:isTijdspecialisatieVan
besluit:bestuurt skos:prefLabel ) ; rdfs:label "gemeente" ]
) .
```
One row per mandataris whose mandate has the role Burgemeester and who
started on or after 1 January 2025, with name and gemeente.
### Which rows
| You write | It means |
|---|---|
| `rep:ReportSpec` + `sh:NodeShape` | this is a spec |
| `rep:profile` | the profile the spec was written against |
| `sh:targetClass` | one row per subject of this class |
| `rep:entity` | which entity, when more than one has this class |
| `sh:property` | a filter: keep only the subjects that pass |
A filter has a `sh:path` from the row and a test. When the path reaches
several values, the test passes if one of them passes.
| Test | Keeps the row when |
|---|---|
| `sh:hasValue x` | the path reaches x |
| `sh:in ( x y )` | the path reaches x or y (all URIs, or all text) |
| `sh:minInclusive`, `sh:maxInclusive`, `sh:minExclusive`, `sh:maxExclusive` | a date or number at the end lies in range; the path must end on a value field |
| `sh:pattern` (+ `sh:flags`) | a value matches the regex |
| `rep:anyOf ( "woord" ... )` | a value holds one of these words, upper or lower case; at most 24 words, no URIs |
| `sh:minCount 1` | the path leads somewhere |
| `sh:minCount 2` | the path reaches two different values |
| `sh:maxCount 0` | the path leads nowhere |
| `sh:maxCount 0` with `sh:in` or `sh:hasValue` | the path reaches none of these values |
The service cannot count: `sh:minCount` above 2 and `sh:maxCount` above 0 are
refused.
### A test on the way: `rep:where`
A filter tests the end of its path. `rep:where` tests a node on the way.
Write it like a filter, with its path from the row. The steps it shares with
its column or filter are the same nodes; its test applies where its own path
ends. On a filter it drops rows; on a column it drops values from the cell,
never rows.
```turtle
<http://data.lblod.info/id/report-specs/eenheden-burgemeester> a rep:ReportSpec , sh:NodeShape ;
rep:profile <http://data.lblod.info/id/report-profiles/mandaten> ;
sh:targetClass besluit:Bestuurseenheid ;
rep:columns (
[ sh:path skos:prefLabel ; rdfs:label "eenheid" ]
[ sh:path ( [ sh:inversePath besluit:bestuurt ] [ sh:inversePath generiek:isTijdspecialisatieVan ]
org:hasPost [ sh:inversePath org:holds ] mandaat:isBestuurlijkeAliasVan foaf:familyName ) ;
rdfs:label "burgemeester" ;
rep:where [ sh:path ( [ sh:inversePath besluit:bestuurt ] [ sh:inversePath generiek:isTijdspecialisatieVan ]
org:hasPost org:role ) ;
sh:hasValue <http://data.vlaanderen.be/id/concept/BestuursfunctieCode/5ab0e9b8a3b2ca7c5e000013> ] ]
) .
```
Per eenheid, the family name of its mayors. The `rep:where` shares three
steps with the column, so it tests the mandate, not the organ or the
person. `validate_spec` says so: `rep:where 1 on column "burgemeester"
applies at "mandaat", after step 3`. The proposal repeats that line to the
user.
A `rep:where` must share at least one step with its column or filter, and
holds no `rep:where` of its own.
### What the CSV shows
`rep:columns` is a list; each item becomes a column, in that order.
| You write | It means |
|---|---|
| `sh:path ( ... )` | the steps from the row to the value |
| `sh:path rep:self` | the row's own URI |
| `rdfs:label` | the column header; required and unique |
| `sh:nodeKind sh:IRI` | show the URI of the node the path ends on (only then may a path end on a link) |
| no `rep:collect`, or `rep:collect rep:row` | each value gets its own row; at most one column may say `rep:row` |
| `rep:collect sh:groupConcat` (+ `sh:separator`) | all values in one cell |
| `rep:collect sh:min` / `sh:max` | the lowest / highest date or number |
A column holds no tests; those go in `sh:property`, or in the column's
`rep:where`.
### How values become rows
Columns that walk the same steps (with the same `rep:where`) share the nodes
on those steps, and their values pair by those nodes: one mayor's first and
last name stay on one row. Where paths split and both sides have several
values, every combination becomes a row. A missing value is an empty cell;
identical rows show once. `sh:groupConcat`, `sh:min` and `sh:max` fold a
column into one cell per node it shares with the other columns.
### What the validator checks
The validator (`src/runner/check.js`) runs on every proposal and again
before every run. It returns the first problem, in words the LLM can act on.
On top of the rules above, it checks that:
- the Turtle holds one `rep:ReportSpec`, no more: one answer is one report;
- every step of every path exists in the profile;
- no path is longer than `MAX_PATH_DEPTH` steps;
- a date or number test ends on a value field, not on a link;
- a class that several entities share comes with `rep:entity`.
## How this relates to SHACL
SHACL is the W3C language to check RDF data against shapes: a SHACL tool
takes data and shapes, and reports which nodes fail. We borrow its words and
its layout, so both documents are valid SHACL syntax and a SHACL tool can
read them. We use them for other jobs.
### The profile: SHACL, with one extension
Read as SHACL, a profile says what good data looks like: nodes of this
class, values of this datatype, links to nodes of that shape. We read the
same statements as a map, and never check data with them. That is a use of
SHACL, not a change to it. One thing does change the meaning:
- **`rep:discriminator` narrows `sh:targetClass`.** In SHACL, four shapes
with `sh:targetClass besluit:Bestuursorgaan` mean that every organ must
fit all four. Here each organ belongs only to the shapes whose
discriminators it meets. A SHACL tool skips `rep:` terms, so to it the
profile still says "all four". SHACL-AF has a tool for this job, custom
targets (see below); we keep `sh:targetClass` so a spec can still name its
class, and write the rule in the same terms as a spec filter.
Two smaller points:
- In SHACL, a node reached through `sh:node B` must pass B's field rules. We
go one step further and treat it as "a B" for the next steps. If the data
breaks that, the query finds nothing there; it never finds a wrong row.
- Links run in circles (bestuursorgaan to orgaan in tijd and back). SHACL
leaves such shapes to each tool. That only matters if someone checks data
with a profile.
### The spec: SHACL syntax, our own meaning
A spec is valid SHACL syntax, but it does not mean what SHACL says. A SHACL
tool reads a spec as a check and reports which subjects fail it. We read it
as a query: the rows are the subjects that pass. For some terms the two
readings agree; for others they do not.
| Term | In SHACL | In a spec | Same? |
|---|---|---|---|
| `sh:targetClass`, `sh:path` | which nodes; which values | same | yes |
| `sh:hasValue` | one value equals | same | yes |
| `sh:minCount 1` / `2` | at least 1 / 2 values | same | yes |
| `sh:maxCount 0` | no values | same | yes |
| `sh:in`, ranges, `sh:pattern` | every value passes, and a node without values passes too | one value passes | no |
| `sh:maxCount 0` with a test | no values at all | no value that passes | no |
| `rep:anyOf`, `rep:where` | skipped | tests of our own | added |
| `rep:columns`, `rep:self`, `rep:collect`, `sh:nodeKind` and `rdfs:label` on a column | skipped | the CSV | added |
So a spec means the same in both readings only if it uses the "yes" terms
alone; then its rows are exactly the subjects a SHACL tool finds good. Most
specs use `sh:in` or a date range, so most specs do not.
Why "one value passes": reports ask for "organs with classification X" or
"persons with a burgemeester mandate". With SHACL's `sh:in`, the first
would also keep organs with no classification, and the second would drop a
person who holds a second mandate too. SHACL can say "one value"
(`sh:qualifiedValueShape` with `sh:qualifiedMinCount 1`), but the LLM writes
the specs, and each extra construct is one more way to get it wrong.
The CSV half has no SHACL meaning at all: SHACL has no notion of output.
### SHACL-AF
SHACL Advanced Features is a W3C Working Group Note from 2017: a proposal,
not a standard. It adds custom targets (`sh:SPARQLTarget`: pick nodes with a
SPARQL query), values computed from the data, rules and functions. A later
draft adds `sh:count`, `sh:sum`, `sh:min`, `sh:max`, `sh:groupConcat` and
`sh:orderBy`. We borrow `sh:groupConcat`, `sh:min`, `sh:max` and
`sh:separator` as values of `rep:collect`, and keep `sh:count`, `sh:sum` and
`sh:orderBy` for when counting and sorting arrive. We do not use its custom
targets: a `sh:SPARQLTarget` is a SPARQL string that the service could not
check or combine with a spec's filters.
## One identity: mu-authorization's
All SPARQL goes through the template's own helpers (`mu`'s `query` and
@@ -89,10 +393,11 @@ node. That check stays on the profile: `validate_spec` says where each
### The service scope
The LLM's own reads during refinement (`describe_profile`'s code lists and
`lookup_values`) run under a service scope, not the caller's session. mu's
`lookup_values`) run under a service scope, not the caller's session:
`publicQuery` in `src/llm/explore.js`. mu's
`query(q, { scope })` sends `mu-auth-scope`; the sparql-parser config grants
that scope read access to `http://mu.semte.ch/graphs/public` only. So
refinement reads only public code lists, regardless of who calls. Execution
refinement reads only the public graph, regardless of who calls. Execution
still runs as the caller, so the report contains what they may see, and the
LLM sees none of it.
@@ -106,6 +411,7 @@ The scope URI is `SERVICE_SCOPE` (default
```
POST /assistant/conversations/:id/turns { content } → 202 { id }
DELETE /assistant/conversations/:id → 204
```
This is the chat. The route is in `app.js`: it records the question, answers
@@ -119,6 +425,12 @@ that way. Failure is a message, never a silent loader.
The chat needs `LLM_BASE_URL`; without it every turn answers with a failure
message. A full-org report run takes minutes.
The delete is the trash bin in the chat's sidebar. It removes the
conversation, its messages, their bijlagen (the triples and the files on
the share) and the reports of those files. It runs as the caller: the auth
layer keeps a user to their own conversations, and a conversation the caller
cannot read is a 404.
### The internal tools
The refinement loop drives one tool set (`src/llm/tools.js`). The tools are
@@ -126,6 +438,27 @@ machinery, not an external surface. All LLM-facing text (system prompts,
profile menu, tool descriptions, validator errors) is English; the answer the
user sees is Dutch.
### What the model sees
The system prompt ends with the profile list and, when there is a current
spec, the menu of its profile (`src/llm/loop.js`). A turn that stays on that
profile needs no `describe_profile` call; the model calls it only for
another profile.
Each request to the model holds one spec in full: the newest.
- The history is text only. A spec bijlage shows as its file name, and a
spec the model printed in its answer is cut out.
- The current spec, the last spec bijlage in the conversation, comes in full
under the user's new message. When the user refines, the model changes it;
when the user asks for a report that lists another kind of thing per row,
it starts a new spec.
- Within a turn, each newer spec (a `validate_spec` call, or an older spec
opened with `read_spec`) replaces the older one with "Left out: a newer
spec follows."
- The model opens an older spec with `read_spec` only when the user asks for
it.
## Environment
| Var | Default | Meaning |
@@ -144,7 +477,7 @@ user sees is Dutch.
| `SEED_PAGE_SIZE` | `5000` | seed page size |
| `SUBJECT_CHUNK_SIZE` | `100` | see above |
| `SHARE_DIR` | `/share` | where CSVs are written |
| `CHAT_ASSISTANT_URI` | — | the `prov:SoftwareAgent` that signs assistant messages; found by lookup when unset |
| `CHAT_ASSISTANT_URI` | `http://data.lblod.info/id/chat-agents/rapportassistent` | the `prov:SoftwareAgent` that signs assistant messages (the portal's chat-assistant migration seeds it) |
| `CHAT_HISTORY_LIMIT` | `20` | messages of history a turn reads |
| `SERVICE_SCOPE` | `http://services.semantic.works/natural-language-report` | the mu-auth-scope for the LLM's own reads (code lists, `lookup_values`). Must match the `with-scope` grant in the app's sparql-parser config. |

32
app.js
View File

@@ -1,12 +1,14 @@
import { app, errorHandler } from 'mu';
import bodyParser from 'body-parser';
import { loadProfiles } from './src/runner/profile.js';
import { readConversation, readHistory, findAssistant, writeMessage } from './src/chat.js';
import { readConversation, readHistory, writeMessage, deleteConversation } from './src/chat.js';
import { wantsExecution } from './src/llm/mode.js';
import { executeSpec, refineSpec } from './src/report-assistant.js';
const PROFILE_DIR = process.env.PROFILE_DIR || '/config/profiles';
const CHAT_ASSISTANT_URI = process.env.CHAT_ASSISTANT_URI;
// The agent that signs the assistant's messages, seeded by the portal's
// chat-assistant migration.
const CHAT_ASSISTANT_URI = process.env.CHAT_ASSISTANT_URI || 'http://data.lblod.info/id/chat-agents/rapportassistent';
const TITLE_LENGTH = 80;
const FAILED = 'Er ging iets mis. Probeer het opnieuw.';
@@ -22,13 +24,12 @@ app.post('/assistant/conversations/:id/turns', bodyParser.json(), async function
const content = String(req.body?.content ?? '').trim();
if (!content) return res.status(400).json({ error: 'content is required' });
let conversation, assistant, history, question;
let conversation, history, question;
try {
// the access check: the caller's session decides what is readable
conversation = await readConversation(req.params.id);
if (!conversation) return res.status(404).json({ error: 'no such conversation' });
assistant = CHAT_ASSISTANT_URI || await findAssistant();
history = await readHistory(conversation.uri, assistant);
history = await readHistory(conversation.uri, CHAT_ASSISTANT_URI);
question = await writeMessage({
conversationUri: conversation.uri,
content,
@@ -50,17 +51,34 @@ app.post('/assistant/conversations/:id/turns', bodyParser.json(), async function
await writeMessage({
conversationUri: conversation.uri,
content: answer.text,
maker: assistant,
maker: CHAT_ASSISTANT_URI,
attachments: answer.attachments,
});
} catch (e) {
console.error('[chat] turn failed:', e);
try {
await writeMessage({ conversationUri: conversation.uri, content: FAILED, maker: assistant });
await writeMessage({ conversationUri: conversation.uri, content: FAILED, maker: CHAT_ASSISTANT_URI });
} catch (e2) {
console.error('[chat] could not write the failure message:', e2);
}
}
});
// Delete a conversation with everything that hangs off it: the messages,
// their bijlagen (triples and share files) and the reports. The read is the
// access check: the caller's session decides what exists, so another user's
// conversation is a 404.
app.delete('/assistant/conversations/:id', async function(req, res) {
try {
const conversation = await readConversation(req.params.id);
if (!conversation) return res.status(404).json({ error: 'no such conversation' });
const removed = await deleteConversation(conversation.uri);
console.log(`[chat] deleted conversation ${req.params.id}, ${removed} share file(s) removed`);
res.status(204).send();
} catch (e) {
console.error('[chat] could not delete the conversation:', e);
res.status(500).json({ error: e.message });
}
});
app.use(errorHandler);

View File

@@ -6,7 +6,7 @@
// A bijlage is a real file: a logical nfo:FileDataObject the file service
// serves at /files/<uuid>/download, over a physical share:// object with
// nie:dataSource back to the logical one.
import { readFileSync, writeFileSync, statSync } from 'fs';
import { readFileSync, writeFileSync, statSync, unlinkSync } from 'fs';
import { query, update, uuid,
sparqlEscapeUri, sparqlEscapeString, sparqlEscapeInt, sparqlEscapeDateTime } from 'mu';
@@ -25,7 +25,6 @@ const T = {
created: 'http://purl.org/dc/terms/created',
maker: 'http://xmlns.com/foaf/0.1/maker',
attachment: 'https://www.w3.org/ns/activitystreams#attachment',
softwareAgent: 'http://www.w3.org/ns/prov#SoftwareAgent',
file: 'http://www.semanticdesktop.org/ontologies/2007/03/22/nfo#FileDataObject',
fileName: 'http://www.semanticdesktop.org/ontologies/2007/03/22/nfo#fileName',
fileSize: 'http://www.semanticdesktop.org/ontologies/2007/03/22/nfo#fileSize',
@@ -33,6 +32,7 @@ const T = {
format: 'http://purl.org/dc/terms/format',
fileType: 'http://purl.org/dc/terms/type',
dataSource: 'http://www.semanticdesktop.org/ontologies/2007/01/19/nie#dataSource',
generated: 'http://www.w3.org/ns/prov#generated',
};
const MESSAGE_BASE = 'http://data.lblod.info/id/chat-messages/';
const FILE_BASE = 'http://data.lblod.info/files/';
@@ -109,17 +109,6 @@ export async function readHistory(conversationUri, assistantUri) {
return messages.slice(-HISTORY_LIMIT);
}
// The one assistant the caller can read. Throws unless there is exactly one.
export async function findAssistant() {
const json = await query(`
SELECT ?agent WHERE { ?agent ${u(T.type)} ${u(T.softwareAgent)} . } LIMIT 2`);
const rows = bindings(json);
if (rows.length !== 1) {
throw new Error(`expected one prov:SoftwareAgent, found ${rows.length}; seed one or set CHAT_ASSISTANT_URI`);
}
return rows[0].agent.value;
}
// --- writes -------------------------------------------------------------------
// One message, both types, its bijlagen, and the conversation touched, in
@@ -222,3 +211,68 @@ export async function registerFile(shareName, format, type, name = shareName) {
return logical;
}
// --- deletes -------------------------------------------------------------------
// deleteConversation(uri) → the share files it removed (a number). Deletes
// the conversation, its messages, their bijlagen (the logical file and the
// physical one under it) and the reports of those files, then removes the
// share files. Everything runs as the caller, so the auth layer keeps a
// user to their own conversation.
export async function deleteConversation(conversationUri) {
// The bijlagen first: once the triples are gone their names cannot be
// found back.
const json = await query(`
SELECT DISTINCT ?doc ?name WHERE {
?message ${u(T.hasContainer)} ${u(conversationUri)} ;
${u(T.attachment)} ?doc .
OPTIONAL {
?physical ${u(T.dataSource)} ?doc ;
${u(T.fileName)} ?name .
}
}`);
const rows = bindings(json);
const docs = [...new Set(rows.map(b => b.doc.value))];
const names = [...new Set(rows.filter(b => b.name).map(b => b.name.value))];
await update(`
DELETE {
?conversation ?cp ?co .
} WHERE {
VALUES ?conversation { ${u(conversationUri)} }
?conversation ?cp ?co .
}`);
await update(`
DELETE {
?message ?mp ?mo .
} WHERE {
?message ${u(T.hasContainer)} ${u(conversationUri)} ;
?mp ?mo .
}`);
if (docs.length) {
await update(`
DELETE {
?doc ?dp ?do .
?physical ?pp ?po .
?report ?rp ?ro .
} WHERE {
VALUES ?doc { ${docs.map(u).join(' ')} }
?doc ?dp ?do .
OPTIONAL { ?physical ${u(T.dataSource)} ?doc ; ?pp ?po . }
OPTIONAL { ?report ${u(T.generated)} ?doc ; ?rp ?ro . }
}`);
}
let removed = 0;
for (const name of names) {
try {
unlinkSync(`${SHARE_DIR}/${name}`);
removed++;
} catch {
// the file was already gone
}
}
return removed;
}

View File

@@ -1,38 +1,38 @@
// A profile as a plain text menu: entities, fields, code lists. Code lists up
// to INLINE_VALUES_MAX are inlined (and cached), so the LLM never has to
// search the short ones.
import { codeList, INLINE_VALUES_MAX } from './lookup.js';
import { codeList, INLINE_VALUES_MAX } from './explore.js';
export async function describeProfile(profile) {
const lines = [];
lines.push('A spec is Turtle, in exactly this shape (copy these prefixes):');
lines.push('@prefix rep: <http://mu.semte.ch/vocabularies/reporting/> .');
lines.push('@prefix sh: <http://www.w3.org/ns/shacl#> .');
lines.push('@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .');
lines.push('@prefix dct: <http://purl.org/dc/terms/> .');
lines.push('');
lines.push('<http://data.lblod.info/id/report-specs/<your-id>> a rep:ReportSpec , sh:NodeShape ;');
lines.push(' dct:title "..." ;');
lines.push(` rep:profile <${profile.uri}> ;`);
lines.push(' sh:targetClass <the rdf:type from the list below> ;');
lines.push(' sh:property [ ... ] ; # filters, optional');
lines.push(' rep:columns ( [ sh:path ( ... ) ; rdfs:label "..." ] ... ) .');
lines.push('Every column ends on a value field. Use rep:self as sh:path for the subject URI.');
lines.push('For the URI of a linked node, end the column on that link and add sh:nodeKind sh:IRI to the column.');
lines.push('');
lines.push('sh:targetClass is the full URI in brackets after the entity below.');
lines.push('In sh:path, use the predicate URI shown after each field name, not the field name itself.');
lines.push('An inverse hop (the field shows "(inverse)") goes in the list as [ sh:inversePath <predicate> ] .');
lines.push('When two entities share one sh:targetClass, pick one with rep:entity <the URI after the entity>.');
lines.push('A date filter is sh:minInclusive or sh:maxInclusive with "yyyy-mm-dd"^^xsd:dateTime');
lines.push('(the xsd prefix above is not copied: add @prefix xsd: <http://www.w3.org/2001/XMLSchema#>).');
lines.push('');
const lines = [`A spec is Turtle, in exactly this shape (copy these prefixes):
@prefix rep: <http://mu.semte.ch/vocabularies/reporting/> .
@prefix sh: <http://www.w3.org/ns/shacl#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix dct: <http://purl.org/dc/terms/> .
<http://data.lblod.info/id/report-specs/<your-id>> a rep:ReportSpec , sh:NodeShape ;
dct:title "..." ;
rep:profile <${profile.uri}> ;
sh:targetClass <the rdf:type from the list below> ;
sh:property [ ... ] ; # filters, optional
rep:columns ( [ sh:path ( ... ) ; rdfs:label "..." ] ... ) .
Every column ends on a value field. Use rep:self as sh:path for the subject URI.
For the URI of a linked node, end the column on that link and add sh:nodeKind sh:IRI to the column.
sh:targetClass is the full URI in brackets after the entity below.
In sh:path, use the predicate URI shown after each field name, not the field name itself.
An inverse hop (the field shows "(inverse)") goes in the list as [ sh:inversePath <predicate> ] .
When two entities share one sh:targetClass, pick one with rep:entity <the URI after the entity>.
A field that links to several entities ("→ a or b") reaches all of them; the steps after it decide. To keep one kind, add a rep:where on a step only that kind has.
A date filter is sh:minInclusive or sh:maxInclusive with "yyyy-mm-dd"^^xsd:dateTime
(the xsd prefix above is not copied: add @prefix xsd: <http://www.w3.org/2001/XMLSchema#>).
`];
for (const shape of profile.shapes) {
lines.push(`${shape.label || shape.targetClass} — sh:targetClass <${shape.targetClass}> ; rep:entity <${shape.uri}>`);
for (const f of shape.fields) {
const kind = f.datatype ? datatypeName(f.datatype)
: f.class ? `→ ${conceptLabel(f.class, await codeList(f.class))}`
: f.node ? `→ ${linkedLabel(profile, f.node)}`
: f.nodes.length ? `→ ${f.nodes.map(n => linkedLabel(profile, n)).join(' or ')}`
: '?';
const dir = f.inverse ? ' (inverse)' : '';
const pathShown = f.path || '?';

View File

@@ -1,7 +1,9 @@
// The LLM's own reads: code lists. They run under the service scope, not
// the caller's session; the sparql-parser config grants that scope read
// access to the public graph only, so they read the same for every caller.
// Execution still runs as the caller.
// Everything the LLM reads from the database in refine mode: the code
// lists, inlined in describe_profile and searched by lookup_values. Every
// query goes through publicQuery: the service scope, not the caller's
// session. The sparql-parser config grants that scope read access to the
// public graph only, so these reads are the same for every caller. Nothing
// here counts, and execution still runs as the caller.
//
// lookup_values is a suggest helper, not a spec check. It reads candidate
// values the LLM can put in a filter; it never runs the spec and never sees
@@ -12,7 +14,7 @@ import { query, sparqlEscapeString } from 'mu';
const SERVICE_SCOPE = process.env.SERVICE_SCOPE || 'http://services.semantic.works/natural-language-report';
export const INLINE_VALUES_MAX = Number(process.env.INLINE_VALUES_MAX || 50);
const VALUES_TTL = Number(process.env.VALUES_TTL || 3600) * 1000;
const LIMIT = 25;
const LOOKUP_LIMIT = 25;
// class → { values, at }. Refreshed lazily when stale.
const cache = new Map();
@@ -45,7 +47,7 @@ export async function codeList(type) {
}
// lookupValues(profile, field, term) → the matches as text for the LLM: up
// to LIMIT labels and URIs, an exact label (case aside) first. field is
// to LOOKUP_LIMIT labels and URIs, an exact label (case aside) first. field is
// "shapeLabel.fieldLabel" as the LLM writes it; it resolves to the class to
// search.
export async function lookupValues(profile, field, term) {
@@ -62,12 +64,12 @@ export async function lookupValues(profile, field, term) {
// "Gent" must surface before "AGB Erfgoed Gent".
const lower = term.toLowerCase();
const exactRow = rows.find(r => r.label.toLowerCase() === lower);
let matches = (exactRow ? [exactRow, ...rows.filter(r => r !== exactRow)] : rows).slice(0, LIMIT);
let matches = (exactRow ? [exactRow, ...rows.filter(r => r !== exactRow)] : rows).slice(0, LOOKUP_LIMIT);
if (!matches.length) matches = await codeQuery(term);
if (!matches.length) return `no value matches "${term}" in ${field}.`;
const lines = matches.map(m => `${m.label} <${m.uri}>`);
if (matches.length >= LIMIT) lines.push(`(${LIMIT}+ matches, narrow the term)`);
if (matches.length >= LOOKUP_LIMIT) lines.push(`(${LOOKUP_LIMIT}+ matches, narrow the term)`);
const exact = matches.find(m => m.label.toLowerCase() === lower);
if (exact) lines.push(`exact: <${exact.uri}>`);
return lines.join('\n');
@@ -79,7 +81,7 @@ async function labelQuery(term, type) {
?uri a ${type ? `<${type}>` : '?anyType'} ;
<http://www.w3.org/2004/02/skos/core#prefLabel> ?label .
FILTER(REGEX(str(?label), ${sparqlEscapeString(escapeRegex(term))}, "i"))
} ORDER BY ?label LIMIT ${LIMIT * 8}`;
} ORDER BY ?label LIMIT ${LOOKUP_LIMIT * 8}`;
const r = await publicQuery(q);
return r.results.bindings.map(b => ({ label: b.label.value, uri: b.uri.value }));
}
@@ -92,7 +94,7 @@ async function codeQuery(term) {
?uri <http://mu.semte.ch/vocabularies/ext/supervision/nomenclatureCode> ?code ;
<http://www.w3.org/2004/02/skos/core#prefLabel> ?label .
FILTER(CONTAINS(LCASE(str(?code)), LCASE(${sparqlEscapeString(term)})))
} ORDER BY ?label LIMIT ${LIMIT}`;
} ORDER BY ?label LIMIT ${LOOKUP_LIMIT}`;
try {
const r = await publicQuery(q);
return r.results.bindings.map(b => ({ label: b.label.value, uri: b.uri.value }));
@@ -114,9 +116,10 @@ function resolveClass(profile, field) {
if (!shape) return null;
const f = shape.fields.find(x => x.name === fieldLabel);
if (f?.class) return f.class;
if (f?.node) {
const linked = profile.shapes.find(s => s.uri === f.node);
return linked?.targetClass || null;
if (f?.nodes.length) {
// sh:or of several entities: only when they share one class
const classes = new Set(f.nodes.map(n => profile.shapes.find(s => s.uri === n)?.targetClass || null));
return classes.size === 1 ? [...classes][0] : null;
}
// The model sometimes writes shape.field on a linked shape
// ("bestuurseenheid.naam"); resolve it to that shape's target class.

View File

@@ -8,10 +8,14 @@
// confirm before anything executes; execution runs the agreed spec without
// the LLM (../report-assistant.js).
import { TOOLS, runTool } from './tools.js';
import { SPEC_MEDIA_TYPE } from '../chat.js';
import { TOOLS, runTool, listProfiles } from './tools.js';
import { describeProfile } from './describe.js';
import { parseSpec } from '../runner/spec.js';
import { SPEC_MEDIA_TYPE, readSpecFile } from '../chat.js';
const MAX_ROUNDS = Number(process.env.MAX_ROUNDS || 12);
// What an older spec becomes: only the newest spec goes to the model in full.
const LEFT_OUT = 'Left out: a newer spec follows.';
const LLM_BASE_URL = process.env.LLM_BASE_URL;
const LLM_MODEL = process.env.LLM_MODEL;
const LLM_API_KEY = process.env.LLM_API_KEY;
@@ -25,7 +29,10 @@ const LOG_LLM = /^(true|1)$/i.test(process.env.LOG_LLM || '');
// Dutch and asks the user to confirm.
const REFINE_PROMPT = `You refine report specs with the user. You never write SPARQL and you never run a report.
The profile lists what can be asked. Compose paths by chaining fields.
The profile lists what can be asked. Compose paths by chaining fields. The
profiles are listed at the end of this prompt. When there is a current spec,
the menu of its profile follows them: use it, and call describe_profile only
for another profile.
Rules:
- A column must end on a value, not on a link. Chain one more hop. Only when
@@ -48,6 +55,9 @@ Rules:
applies. Say that in Dutch in your proposal, so the user can check it.
- Only offer what validate_spec accepted. Before you suggest an alternative,
write it and validate it; if it does not validate, say it cannot be done.
- One answer carries one spec, and one spec is one report on one profile. If
the question needs two reports, say so, propose the first, and offer the
second as the next step.
- You cannot count, sort, take the first N, or compare two subjects. Say so.
- The report's title is the dct:title you write in the spec. There is no other
title input.
@@ -58,9 +68,14 @@ for a filter, so your proposal names real values instead of guesses. That is
the only database read you have, and it only suggests values: it never tells
you whether a spec is good or how much it matches. The database you read may
differ from the one the report runs on, so never treat a lookup as a check.
Iterate with the user on the spec itself: paths, filters, columns. When the
user wants an earlier spec changed, open it with read_spec and change what
they asked for. Never call
Iterate with the user on the spec itself: paths, filters, columns. The
current spec, the last one in the conversation, comes with the user's
message. When the user refines (other columns, filters or title), change the
current spec. When the user asks for a different report, one that lists
another kind of thing per row, start a new spec. Older specs show only as a
bijlage name, and a spec you wrote before in this turn shows as "left out"
once a newer one follows. Only when the user asks for an older spec, open it
with read_spec and change that one. Never call
run_report; there is no such tool here. When the spec is ready, ask the user
to confirm execution. Then stop.
@@ -82,14 +97,17 @@ Reply to the user in Dutch. Earlier turns are context only. Answer the last one.
// [{ role, content, attachments }], newest last. spec is the last spec
// validate_spec accepted in this turn, or null.
export async function ask(turns, profiles) {
const messages = [{ role: 'system', content: REFINE_PROMPT }, ...turns.map(withSpecHint)];
const current = currentSpec(turns);
const messages = [{ role: 'system', content: await systemPrompt(current, profiles) },
...withCurrentSpec(turns.map(withSpecHint), current)];
let spec = null;
for (let round = 0; round < MAX_ROUNDS; round++) {
const { message } = await chat(messages, TOOLS);
const { message } = await chat(newestSpecOnly(messages), TOOLS);
messages.push(message);
const calls = message.tool_calls || [];
if (!calls.length) return { text: message.content || '', spec };
if (calls.some(c => c.function.name === 'validate_spec')) message.leftOut = { tool_calls: calls.map(leaveOutSpec) };
for (const call of calls) {
const args = parseArgs(call.function.arguments);
@@ -101,21 +119,92 @@ export async function ask(turns, profiles) {
}
// validate_spec answers "ok" (plus notes) when the spec checks out;
// that spec becomes the bijlage of the answer.
if (call.function.name === 'validate_spec' && /^ok(\n|$)/.test(result)) spec = args.spec;
messages.push({ role: 'tool', tool_call_id: call.id, content: result });
// One answer carries one spec: say so when this one replaces another.
if (call.function.name === 'validate_spec' && /^ok(\n|$)/.test(result)) {
if (spec && spec !== args.spec) {
result += '\nThis spec replaces the one you validated before. Your answer carries only this one.';
}
spec = args.spec;
}
const reply = { role: 'tool', tool_call_id: call.id, content: result };
if (call.function.name === 'read_spec' && result.includes('ReportSpec')) reply.leftOut = { content: LEFT_OUT };
messages.push(reply);
}
}
return { text: 'Sorry, dit rapport is te moeilijk. Contacteer de developers.', spec };
}
// The history carries the spec as a bijlage. Paste one hint line per spec
// under the message text, so the model knows a spec is there and opens it
// with read_spec; the chat itself still keeps the Turtle out of the text.
// The history carries the spec as a bijlage, never as text. Paste one hint
// line per spec under the message text, so the model knows a spec is there
// and can open it with read_spec when the user asks for it. A spec the model
// printed in its answer anyway is cut from the text.
function withSpecHint(turn) {
const content = turn.role === 'assistant' ? withoutSpecText(turn.content) : turn.content;
const specs = (turn.attachments || []).filter(a => a.mediaType === SPEC_MEDIA_TYPE);
if (!specs.length) return turn;
const hints = specs.map(a => `Bijlage "${a.name}", a spec file (read it with read_spec).`);
return { ...turn, content: `${turn.content}\n\n${hints.join('\n\n')}` };
if (!specs.length) return { ...turn, content };
const hints = specs.map(a => `Bijlage "${a.name}", a spec file.`);
return { ...turn, content: `${content}\n\n${hints.join('\n\n')}` };
}
function withoutSpecText(text) {
return text.replace(/```[\s\S]*?```/g, block => (block.includes('ReportSpec') ? '(spec left out)' : block));
}
// The current spec: the last spec bijlage in the conversation, or null.
function currentSpec(turns) {
const name = turns.flatMap(t => t.attachments || [])
.filter(a => a.mediaType === SPEC_MEDIA_TYPE).at(-1)?.name;
const turtle = name && readSpecFile(name);
return turtle ? { name, turtle } : null;
}
// The system prompt also carries what the model would otherwise fetch at the
// start of every turn: the profile list, and the menu of the current spec's
// profile. Without a menu the model calls describe_profile itself.
async function systemPrompt(current, profiles) {
const parts = [REFINE_PROMPT, `The profiles:\n${listProfiles(profiles)}`];
const uri = current && profileOf(current.turtle);
if (profiles.has(uri)) {
try {
parts.push(`The menu of the current spec's profile <${uri}>:\n\n${await describeProfile(profiles.get(uri))}`);
} catch (e) {
console.error('[llm] could not describe the current profile:', e.message);
}
}
return parts.join('\n\n');
}
function profileOf(turtle) {
try {
return parseSpec(turtle).profileUri;
} catch {
return null;
}
}
// The current spec goes in full under the user's new message: the one spec
// the history holds as text.
function withCurrentSpec(messages, current) {
if (!current) return messages;
const { name, turtle } = current;
const last = messages.at(-1);
return [...messages.slice(0, -1), {
...last,
content: `${last.content}\n\nThe current spec, bijlage "${name}":\n\n${turtle}`,
leftOut: { content: `${last.content}\n\nThe current spec, bijlage "${name}": ${LEFT_OUT}` },
}];
}
// Only the newest spec goes to the model in full. A message that holds an
// older one goes in its leftOut form; read_spec opens that spec again.
function newestSpecOnly(messages) {
const newest = messages.findLastIndex(m => m.leftOut);
return messages.map((m, i) => (m.leftOut && i !== newest ? { ...m, ...m.leftOut } : m));
}
function leaveOutSpec(call) {
if (call.function.name !== 'validate_spec') return call;
return { ...call, function: { ...call.function, arguments: JSON.stringify({ spec: LEFT_OUT }) } };
}
// chat(messages, tools) → { message }. The one place that talks to the

View File

@@ -1,32 +1,29 @@
// The tools the LLM calls, refinement only: list_profiles,
// describe_profile, validate_spec, read_spec and lookup_values. Each returns
// plain text for the model. No report is created, no spec is evaluated;
// the code-list reads run under the service scope (public graph).
// The tools the LLM calls, refinement only: describe_profile,
// validate_spec, read_spec and lookup_values. Each returns
// plain text for the model. No report is created, no spec is evaluated; the
// database reads run under the service scope (public graph).
import { parseSpec } from '../runner/spec.js';
import { checkSpec, whereNotes, profileError } from '../runner/check.js';
import { describeProfile } from './describe.js';
import { lookupValues } from './lookup.js';
import { lookupValues } from './explore.js';
import { readSpecFile } from '../chat.js';
// The tool definitions sent to the model. The descriptions are what the LLM
// reads to decide which tool to call.
export const TOOLS = [
tool('list_profiles',
'List the report profiles available, with their id and title.',
{}),
tool('describe_profile',
'Describe one profile: its entities, fields, relations and labels. Inline the values of short code lists.',
{ profile_id: 'The profile URI returned by list_profiles' }),
{ profile_id: 'The profile URI from the profile list' }),
tool('validate_spec',
'Validate a report spec (Turtle). Returns "ok", followed by one line per rep:where saying where it applies, or one clear sentence saying what is wrong and what to write instead. No database is touched.',
{ spec: 'The spec as Turtle text' }),
tool('read_spec',
'Return the Turtle of a spec bijlage, a ttl file attached to an earlier message in this conversation. Use it to reuse the agreed spec, or to read back an earlier proposal before you change it. Read-only.',
'Return the Turtle of a spec bijlage, a ttl file attached to an earlier message in this conversation. The current spec already comes with the user\'s message; use this only when the user asks for an older spec. Read-only.',
{ file_name: 'The file name of the spec bijlage, e.g. specificatie-<uuid>.ttl' }),
tool('lookup_values',
'Search a code list for a term so you can suggest concrete values for a filter (for sh:hasValue or sh:in). Returns up to 25 matches with their URIs and labels, a total, and an exact flag when a label equals the term. Reads candidate values only; it never runs the spec and never tells you how many subjects a spec would match. Read-only, no report is created.',
{
profile_id: 'The profile URI returned by list_profiles',
profile_id: 'The profile URI from the profile list',
field: 'The field as "entityLabel.fieldLabel", e.g. "bestuurseenheid.naam"',
term: 'The Dutch word or name to search for',
}),
@@ -49,7 +46,6 @@ function tool(name, description, params) {
export async function runTool(name, args, profiles) {
switch (name) {
case 'list_profiles': return listProfiles(profiles);
case 'describe_profile': return describe(args, profiles);
case 'validate_spec': return validateSpec(args, profiles);
case 'read_spec': return readSpec(args);
@@ -58,7 +54,8 @@ export async function runTool(name, args, profiles) {
}
}
function listProfiles(profiles) {
// The profile list, in the system prompt: one line per profile.
export function listProfiles(profiles) {
return [...profiles.values()].map(p => `${p.title} <${p.uri}>`).join('\n');
}

View File

@@ -1,13 +1,18 @@
// Values to rows. Subjects in stable order, dedup on term, collect per cell.
// rep:self columns take the subject URI; a column with no values is an
// empty cell, never a dropped row. A column without rep:collect joins its
// values only when there is one; with several, each value gets its own row
// (the other columns repeat theirs). rep:row forces that expansion; at most
// one column per spec may do that explicitly (checked in check.js).
import { sharedSteps } from './spec.js';
// Values to rows, paired by URI on every step. Two columns that walk the
// same steps (with the same rep:where on them) share the nodes on those
// steps, the way a rep:where shares nodes with its column. Per subject the
// paths of the columns make a tree of the nodes found; every row is one way
// down that tree. Columns through the same node stay on one row; branches
// under one node multiply. A column with no value is an empty cell, never a
// dropped row. sh:groupConcat, sh:min and sh:max fold a column into one cell
// per node it shares with the other columns. rep:self takes the subject
// URI. Subjects in stable order; identical rows appear once.
const SH = 'http://www.w3.org/ns/shacl#';
const REP = 'http://mu.semte.ch/vocabularies/reporting/';
const XSD = 'http://www.w3.org/2001/XMLSchema#';
const FOLDING = [SH + 'groupConcat', SH + 'min', SH + 'max'];
export const NUMERIC_DATATYPES = new Set([
'integer', 'decimal', 'double', 'float', 'long', 'int', 'short', 'byte',
@@ -15,45 +20,98 @@ export const NUMERIC_DATATYPES = new Set([
'unsignedLong', 'unsignedInt', 'unsignedShort', 'unsignedByte',
].map(d => XSD + d));
// assemble(subjects, values, spec) → rows, the header first. values:
// subject → column index → chains, one term per step (see columns.js).
export function assemble(subjects, values, spec) {
const sorted = [...subjects].sort();
const tree = columnTree(spec.columns);
const rows = [spec.columns.map(c => c.label)];
for (const subject of sorted) {
const perColumn = values.get(subject) || new Map();
const cells = spec.columns.map((col, i) => {
if (!col.path.length) return [subject]; // rep:self
const list = dedup(perColumn.get(i) || []);
return collect(list, col);
});
const lines = Math.max(1, ...cells.map(c => c.length));
for (let r = 0; r < lines; r++) {
// the expanding cell cycles its values, every other cell keeps its
// value on each line
rows.push(cells.map(c => c[Math.min(r, c.length - 1)]));
for (const subject of [...subjects].sort()) {
const top = { term: { value: subject }, below: new Map() };
for (const [index, chains] of values.get(subject) || []) {
for (const chain of chains) addChain(top, tree.paths[index], chain);
}
const seen = new Set();
for (const cells of rowsAt(tree.root, top, tree, spec.columns)) {
const row = spec.columns.map((col, i) => cells.get(i) ?? '');
const key = JSON.stringify(row);
if (!seen.has(key)) {
seen.add(key);
rows.push(row);
}
}
}
return rows;
}
function collect(terms, col) {
switch (col.collect) {
case SH + 'groupConcat':
return [terms.map(t => t.value).join(col.separator ?? ',')];
case undefined:
case null:
// no rep:collect: one value is a normal cell, several values each
// get their own row instead of a joined string
if (terms.length <= 1) return [terms.map(t => t.value).join(',')];
return terms.map(t => t.value);
case SH + 'min':
return [extreme(terms, col, -1)];
case SH + 'max':
return [extreme(terms, col, 1)];
case REP + 'row':
return terms.length ? terms.map(t => t.value) : [''];
default:
throw new Error(`column "${col.label}" has an unknown rep:collect <${col.collect}>. Use sh:groupConcat, rep:row, sh:min or sh:max.`);
// The steps of all columns as one tree of branches. Two columns share a
// branch as long as they took the same steps with the same rep:where on
// them. paths[i] lists the branches column i goes through; a branch knows
// the columns that end on it and the columns that go through it.
function columnTree(columns) {
const newBranch = () => ({ next: new Map(), ends: [], columns: [] });
const root = newBranch();
const paths = columns.map((col, index) => {
let branch = root;
const branches = col.path.map((hop, k) => {
const where = col.where.filter(w => sharedSteps(col.path, w.path) === k + 1);
const key = `${hop.inverse ? '^' : ''}${hop.predicate} ${JSON.stringify(where)}`;
if (!branch.next.has(key)) branch.next.set(key, newBranch());
branch = branch.next.get(key);
branch.columns.push(index);
return branch;
});
branch.ends.push(index);
return branches;
});
return { root, paths };
}
// One path the store returned, subject to value, into the tree of nodes
// found for that subject. Nodes are keyed by value: a node reached twice is
// one node, a value found twice under one node is one value.
function addChain(found, branches, chain) {
branches.forEach((branch, k) => {
if (!found.below.has(branch)) found.below.set(branch, new Map());
const byValue = found.below.get(branch);
if (!byValue.has(chain[k].value)) byValue.set(chain[k].value, { term: chain[k], below: new Map() });
found = byValue.get(chain[k].value);
});
}
// The rows under one found node: its own cells, times the rows of each
// branch below it. A branch where nothing was found adds empty cells.
function rowsAt(branch, found, tree, columns) {
let rows = [new Map(branch.ends.map(i => [i, found.term.value]))];
for (const next of branch.next.values()) {
const nodes = [...(found.below.get(next)?.values() || [])];
const below = next.columns.every(i => FOLDING.includes(columns[i].collect))
? [fold(next, nodes, tree, columns)]
: nodes.length ? nodes.flatMap(node => rowsAt(next, node, tree, columns)) : [new Map()];
rows = rows.flatMap(row => below.map(cells => new Map([...row, ...cells])));
}
return rows;
}
// A branch where every column folds gives one set of cells: per column, all
// its values in the branch, folded into one cell.
function fold(branch, nodes, tree, columns) {
const cells = new Map();
for (const i of branch.columns) {
const rest = tree.paths[i].slice(tree.paths[i].indexOf(branch) + 1);
const terms = nodes.flatMap(node => termsBelow(node, rest));
const unique = [...new Map(terms.map(t => [t.value, t])).values()];
const col = columns[i];
cells.set(i, col.collect === SH + 'groupConcat'
? unique.map(t => t.value).join(col.separator ?? ',')
: extreme(unique, col, col.collect === SH + 'min' ? -1 : 1));
}
return cells;
}
function termsBelow(node, rest) {
if (!rest.length) return [node.term];
const byValue = node.below.get(rest[0]);
return byValue ? [...byValue.values()].flatMap(next => termsBelow(next, rest.slice(1))) : [];
}
// min/max. Numbers compare as numbers. Dates and times compare as text:
@@ -72,17 +130,3 @@ function extreme(terms, col, direction) {
const best = terms.reduce((a, b) => (cmp(key(b), key(a)) * direction > 0 ? b : a));
return best.value;
}
// Two terms are the same when the URIs match, or lexical form, datatype
// and language tag all match.
export function dedup(terms) {
const seen = new Set();
const out = [];
for (const t of terms) {
const key = t.termType === 'NamedNode'
? `u:${t.value}`
: `l:${t.value}|${t.datatype?.value || ''}|${t.language || ''}`;
if (!seen.has(key)) { seen.add(key); out.push(t); }
}
return out;
}

View File

@@ -1,4 +1,4 @@
import { startShape, fieldsOf, field } from './profile.js';
import { startShape, fieldsOf } from './profile.js';
import { NUMERIC_DATATYPES } from './assemble.js';
import { sharedSteps } from './spec.js';
@@ -119,14 +119,10 @@ function collectMinMaxTyped(spec, profile) {
const name = col.collect === SH + 'min' ? 'sh:min' : 'sh:max';
if (!col.path.length) return `${name} on a rep:self column does not apply. A subject URI is not a number or date.`;
const last = col.path[col.path.length - 1];
const shapeUri = shapeAfter(profile, startShapeUri, col.path);
if (!shapeUri) continue; // pathsResolve already reported the real error
const f = field(profile, shapeUri, last);
if (!f) continue;
const kind = f.datatype === XSD_DATETIME || f.datatype === XSD_DATE ? 'date'
: NUMERIC_DATATYPES.has(f.datatype) ? 'number'
: null;
if (!kind) {
// no end field: pathsResolve already reported the real error
const f = endFields(profile, startShapeUri, col.path)
.find(x => x.datatype !== XSD_DATETIME && x.datatype !== XSD_DATE && !NUMERIC_DATATYPES.has(x.datatype));
if (f) {
return `"${f.name || last.predicate}" is text, so ${name} does not apply. Use rep:row (same as leaving rep:collect out: every value gets its own row).`;
}
}
@@ -291,15 +287,15 @@ function rangeOnValue(spec, profile) {
const key = ['minInclusive', 'maxInclusive', 'minExclusive', 'maxExclusive']
.find(k => cond.constraints[k]);
if (!key || !cond.path.length) continue;
const last = cond.path[cond.path.length - 1];
const f = field(profile, shapeAfter(profile, startShapeUri, cond.path), last);
if (!f || f.datatype) continue; // pathsResolve reports a missing step
const target = f.node
? `"${profile.shapes.find(s => s.uri === f.node)?.label || f.node}"`
// pathsResolve reports a missing step
const f = endFields(profile, startShapeUri, cond.path).find(x => !x.datatype);
if (!f) continue;
const target = f.nodes.length
? f.nodes.map(uri => shapeName(profile, uri)).join(' or ')
: `the code "${f.name}"`;
const steps = fieldsOf(profile, f.node)
const steps = [...new Set(f.nodes.flatMap(uri => fieldsOf(profile, uri))
.filter(x => x.datatype === XSD_DATE || x.datatype === XSD_DATETIME || NUMERIC_DATATYPES.has(x.datatype))
.map(x => `${x.name} (${x.path})`);
.map(x => `${x.name} (${x.path})`))];
const add = steps.length ? `Add a step: ${steps.join(' or ')}.` : 'End the path on a date or number field.';
return `${what} ends on ${target}, not on a value, so sh:${key} never matches. ${add}`;
}
@@ -316,15 +312,18 @@ export function whereNotes(spec, profile) {
});
}
// The entity a path lands on, or the value field when it ends on one.
// The entity a path lands on, or the value field when it ends on one. A
// step that leads to several entities names them all.
function nodeName(profile, startShapeUri, hops) {
let shapeUri = startShapeUri;
for (const [i, hop] of hops.entries()) {
const f = field(profile, shapeUri, hop);
if (!f) return `step ${hops.length}`;
if (!f.node) return i === hops.length - 1 ? `the value "${f.name}"` : `step ${hops.length}`;
shapeUri = f.node;
}
if (!hops.length) return shapeName(profile, startShapeUri);
const ends = endFields(profile, startShapeUri, hops);
if (!ends.length) return `step ${hops.length}`;
const value = ends.find(f => !f.nodes.length);
if (value) return `the value "${value.name}"`;
return [...new Set(ends.flatMap(f => f.nodes))].map(uri => shapeName(profile, uri)).join(' or ');
}
function shapeName(profile, shapeUri) {
return `"${profile.shapes.find(s => s.uri === shapeUri)?.label || shapeUri}"`;
}
@@ -367,17 +366,24 @@ export function checkSpec(spec, profiles) {
return null;
}
// The shape a path lands in: follow sh:node through every hop but the
// last. walk() has already validated the path, so a gap here means the walk
// reported the real error and this can bail.
function shapeAfter(profile, startShapeUri, hops) {
let shapeUri = startShapeUri;
for (let i = 0; i < hops.length - 1; i++) {
const f = field(profile, shapeUri, hops[i]);
if (!f || !f.node) return null;
shapeUri = f.node;
// The fields a hop takes from any of the shapes a path has reached. A field
// may link to several entities (sh:or of sh:node: an eenheid's political and
// leidinggevende organen): the walk follows them all and the next steps
// decide. The query follows predicates only, so this is what the data does.
function hopFields(profile, shapeUris, hop) {
return shapeUris.flatMap(uri => fieldsOf(profile, uri)
.filter(f => f.path === hop.predicate && !!f.inverse === !!hop.inverse));
}
// The fields a path can end on: follow sh:node through every hop but the
// last. walk() has already validated the path, so none here means the walk
// reported the real error and the caller can bail.
function endFields(profile, startShapeUri, hops) {
let shapeUris = [startShapeUri];
for (const hop of hops.slice(0, -1)) {
shapeUris = [...new Set(hopFields(profile, shapeUris, hop).flatMap(f => f.nodes))];
}
return shapeUri;
return hopFields(profile, shapeUris, hops[hops.length - 1]);
}
function walk(profile, startShapeUri, hops, what, isColumn, wantsIri = false) {
@@ -385,30 +391,34 @@ function walk(profile, startShapeUri, hops, what, isColumn, wantsIri = false) {
return `${what} walks ${hops.length} hops. The limit is ${MAX_PATH_DEPTH}.`;
}
if (!hops.length) return null;
let shapeUri = startShapeUri;
let shapeUris = [startShapeUri];
for (let i = 0; i < hops.length; i++) {
const hop = hops[i];
const fields = fieldsOf(profile, shapeUri);
const f = fields.find(x => x.path === hop.predicate && !!x.inverse === !!hop.inverse);
if (!f) {
const have = fields.map(x => (x.inverse ? `${x.name} (${x.path}, inverse)` : `${x.name} (${x.path})`)).join(', ');
const shapeLabel = profile.shapes.find(s => s.uri === shapeUri)?.label || shapeUri;
return `at "${shapeLabel}" there is no "${hop.predicate}". It has: ${have || '(nothing)'}.`;
const found = hopFields(profile, shapeUris, hop);
if (!found.length) {
const have = [...new Set(shapeUris.flatMap(uri => fieldsOf(profile, uri))
.map(x => (x.inverse ? `${x.name} (${x.path}, inverse)` : `${x.name} (${x.path})`)))].join(', ');
const shapeLabel = shapeUris.map(uri => shapeName(profile, uri)).join(' or ');
return `at ${shapeLabel} there is no "${hop.predicate}". It has: ${have || '(nothing)'}.`;
}
if (i < hops.length - 1) {
if (!f.node) {
const links = found.filter(f => f.nodes.length);
if (!links.length) {
return `${what} hop ${i + 1} <${hop.predicate}> holds a value, not a link. You cannot walk past it.`;
}
shapeUri = f.node;
shapeUris = [...new Set(links.flatMap(f => f.nodes))];
} else if (isColumn) {
if (wantsIri && f.datatype) {
return `${what} has sh:nodeKind sh:IRI but ends on the value "${f.name}", which is no URI. Drop sh:nodeKind.`;
}
if (!wantsIri && !f.datatype && !f.class) {
const valueFields = fieldsOf(profile, f.node)
.filter(x => x.datatype || x.class)
.map(x => `${x.name} (${x.path})`);
return `${what} ends on a link. Add a hop: ${valueFields.join(' or ')}. Only if the user asked for the URI of that node itself, add sh:nodeKind sh:IRI to the column instead.`;
// every field the path can end on must suit the column
for (const f of found) {
if (wantsIri && f.datatype) {
return `${what} has sh:nodeKind sh:IRI but ends on the value "${f.name}", which is no URI. Drop sh:nodeKind.`;
}
if (!wantsIri && !f.datatype && !f.class) {
const valueFields = [...new Set(f.nodes.flatMap(uri => fieldsOf(profile, uri))
.filter(x => x.datatype || x.class)
.map(x => `${x.name} (${x.path})`))];
return `${what} ends on a link. Add a hop: ${valueFields.join(' or ')}. Only if the user asked for the URI of that node itself, add sh:nodeKind sh:IRI to the column instead.`;
}
}
}
}

View File

@@ -4,11 +4,14 @@ import { hopsPattern, wherePatterns } from './seed.js';
// Fetch column values for known subjects. Columns are grouped by their
// shared non-leaf hops; two columns share a query only when every hop
// before the leaf is identical, and the leaf is read the same way.
// rep:self columns (empty path) are filled by assemble.js.
// rep:self columns (empty path) are filled by assemble.js. Every query
// returns the nodes on the path too, so assemble.js can pair columns by the
// nodes they share.
const SUBJECT_CHUNK_SIZE = Math.max(1, Number(process.env.SUBJECT_CHUNK_SIZE) || 100);
// fetchColumns(subjects, spec) → subject → column index → [term]
// fetchColumns(subjects, spec) → subject → column index → [chain], a chain
// being the terms on the path, one per step, the value last.
export async function fetchColumns(subjects, spec) {
const values = new Map();
const groups = groupColumns(spec.columns);
@@ -49,17 +52,19 @@ export function groupQuery(group, chunk) {
// the end.
const nodeVar = (k) => (k === group.hops.length ? '?v' : `?x_${k}`);
lines.push(...wherePatterns(group.where, group.hops, nodeVar, 'c'));
return `SELECT ?s ?v WHERE {\n VALUES ?s { ${subjects} }\n ${lines.join('\n ')}\n}`;
const vars = ['?s', ...group.hops.slice(1).map((hop, i) => `?x_${i + 1}`), '?v'];
return `SELECT DISTINCT ${vars.join(' ')} WHERE {\n VALUES ?s { ${subjects} }\n ${lines.join('\n ')}\n}`;
}
function collect(result, group, values) {
for (const b of result.results.bindings) {
const chain = [...group.hops.slice(1).map((hop, i) => b[`x_${i + 1}`]), b.v];
let perColumn = values.get(b.s.value);
if (!perColumn) { perColumn = new Map(); values.set(b.s.value, perColumn); }
for (const col of group.columns) {
let list = perColumn.get(col.index);
if (!list) { list = []; perColumn.set(col.index, list); }
list.push(b.v);
list.push(chain);
}
}
}

View File

@@ -1,5 +1,6 @@
import N3 from 'n3';
import { readFileSync, readdirSync } from 'fs';
import { parseFilter, readRdfList } from './spec.js';
// Parse a profile (a SHACL shapes graph) and answer questions about it.
@@ -41,8 +42,8 @@ function loadProfile(file) {
const targetClass = store.getQuads(shape, SH + 'targetClass', null)[0]?.object.value || null;
const label = store.getQuads(shape, 'http://www.w3.org/2000/01/rdf-schema#label', null)[0]?.object.value || null;
const fields = store.getQuads(shape, SH + 'property', null).map(q => parseField(store, q.object));
const discriminator = parseDiscriminator(store, shape);
return { uri: shape.value, targetClass, label, fields, discriminator };
const discriminators = parseDiscriminators(store, shape);
return { uri: shape.value, targetClass, label, fields, discriminators };
});
return { uri, title, shapes: shapeList };
@@ -64,25 +65,49 @@ function parseField(store, p) {
name: one(SH + 'name'),
datatype: one(SH + 'datatype'),
class: one(SH + 'class'),
node: one(SH + 'node'),
nodes: linkedShapes(store, p, one(SH + 'name')),
};
}
// rep:discriminator tells shapes that share a targetClass apart. sh:minCount 1
// becomes a plain triple in the seed, sh:maxCount 0 a FILTER NOT EXISTS.
function parseDiscriminator(store, shape) {
const node = store.getQuads(shape, REP + 'discriminator', null)[0]?.object;
if (!node) return null;
const { path, inverse } = readPath(store, node);
const one = (pred) => store.getQuads(node, pred, null)[0]?.object?.value || null;
const minCount = one(SH + 'minCount');
const maxCount = one(SH + 'maxCount');
if (!path || (minCount === null && maxCount === null)) return null;
return {
path, inverse,
minCount: minCount !== null ? Number(minCount) : null,
maxCount: maxCount !== null ? Number(maxCount) : null,
// The entities a field links to: its sh:node, or the sh:node of each item of
// its sh:or (SHACL for "each value is one of these").
function linkedShapes(store, p, name) {
const node = store.getQuads(p, SH + 'node', null)[0]?.object;
if (node) return [node.value];
const or = store.getQuads(p, SH + 'or', null)[0]?.object;
if (!or) return [];
return readRdfList(store, or).map(item => {
const n = store.getQuads(item, SH + 'node', null)[0]?.object;
if (!n) throw new Error(`field "${name}": every sh:or item needs a sh:node.`);
return n.value;
});
}
// rep:discriminator tells shapes that share a targetClass apart. A shape may
// carry several; a subject meets them all.
function parseDiscriminators(store, shape) {
return store.getQuads(shape, REP + 'discriminator', null)
.map(q => parseDiscriminator(store, q.object, shape.value));
}
// One discriminator: a condition written like a spec filter (sh:path plus
// sh:minCount, sh:maxCount, sh:in, sh:hasValue, ...), or sh:and / sh:or (a
// list) or sh:not (one) over discriminators, as in SHACL.
function parseDiscriminator(store, node, shapeUri) {
const one = (pred) => store.getQuads(node, SH + pred, null)[0]?.object;
const list = (pred) => {
const items = readRdfList(store, one(pred)).map(x => parseDiscriminator(store, x, shapeUri));
if (!items.length) throw new Error(`rep:discriminator on <${shapeUri}> has an empty sh:${pred}.`);
return items;
};
if (one('and')) return { and: list('and') };
if (one('or')) return { or: list('or') };
if (one('not')) return { not: parseDiscriminator(store, one('not'), shapeUri) };
const d = parseFilter(store, node);
if (!d.path.length || !Object.keys(d.constraints).length) {
throw new Error(`rep:discriminator on <${shapeUri}> needs a sh:path with a condition, or sh:and, sh:or or sh:not.`);
}
return d;
}
// The shape a spec starts from. rep:entity wins; a targetClass only resolves
@@ -101,8 +126,3 @@ export function shape(profile, shapeUri) {
export function fieldsOf(profile, shapeUri) {
return shape(profile, shapeUri)?.fields || [];
}
// The field a hop takes from a shape, or null.
export function field(profile, shapeUri, hop) {
return fieldsOf(profile, shapeUri).find(f => f.path === hop.predicate && !!f.inverse === !!hop.inverse) || null;
}

View File

@@ -25,7 +25,9 @@ export function seedPageQuery(spec, limit, offset, shape = null) {
const parts = [];
const targetClass = spec.targetClass || shape?.targetClass;
parts.push(`?s a ${sparqlEscapeUri(targetClass)} .`);
if (shape?.discriminator) parts.push(discriminatorPattern(shape.discriminator));
(shape?.discriminators || []).forEach((d, di) => {
parts.push(discriminatorPattern(d, `disc${di}`));
});
spec.filters.forEach((filter, fi) => {
parts.push(filterPattern(filter, `f${fi}`));
});
@@ -36,13 +38,20 @@ LIMIT ${limit} OFFSET ${offset}`;
}
// A shape that shares its class with another says here what sets its
// subjects apart: sh:minCount 1 is a plain triple, sh:maxCount 0 a NOT EXISTS.
function discriminatorPattern(d) {
const p = sparqlEscapeUri(d.path);
const triple = d.inverse ? `?disc ${p} ?s .` : `?s ${p} ?disc .`;
if (d.minCount !== null && d.minCount >= 1) return triple;
if (d.maxCount === 0) return `FILTER NOT EXISTS { ${triple} }`;
return null;
// subjects apart. A plain discriminator is a filter; sh:and, sh:or and sh:not
// become one FILTER over EXISTS tests, which see ?s from outside.
function discriminatorPattern(d, varPrefix) {
if (!d.and && !d.or && !d.not) return filterPattern(d, varPrefix);
return `FILTER(${discriminatorTest(d, varPrefix)})`;
}
function discriminatorTest(d, varPrefix) {
if (d.and) return `(${d.and.map((x, i) => discriminatorTest(x, `${varPrefix}_${i}`)).join(' && ')})`;
if (d.or) return `(${d.or.map((x, i) => discriminatorTest(x, `${varPrefix}_${i}`)).join(' || ')})`;
if (d.not) return `!(${discriminatorTest(d.not, `${varPrefix}_n`)})`;
const { maxCount, ...test } = d.constraints;
if (maxCount === 0) return `NOT EXISTS { ${filterPattern({ ...d, constraints: test }, varPrefix)} }`;
return `EXISTS { ${filterPattern(d, varPrefix)} }`;
}
// Writes out every hop as its own triple pattern.
@@ -66,13 +75,15 @@ export function hopsPattern(hops, varPrefix, startVar, lastVar = `?${varPrefix}_
}
// One filter: its condition, plus its rep:where conditions on the nodes it
// walks through. With sh:maxCount 0 the whole of it becomes "none such".
// walks through. With sh:maxCount 0 the whole of it becomes "none such";
// with a value test too (sh:in, sh:hasValue, ...), "none with such a value".
function filterPattern(filter, varPrefix) {
const { maxCount, ...test } = filter.constraints;
if (maxCount === 0) {
return `FILTER NOT EXISTS {\n ${filterPattern({ ...filter, constraints: test }, varPrefix)}\n }`;
}
const nodeVar = (k) => (k === filter.path.length ? `?${varPrefix}_v` : `?${varPrefix}_${k}`);
const wheres = wherePatterns(filter.where, filter.path, nodeVar, varPrefix);
if (filter.constraints.maxCount === 0) {
return `FILTER NOT EXISTS {\n ${[hopsPattern(filter.path, varPrefix, '?s'), ...wheres].join('\n ')}\n }`;
}
return [conditionPattern(filter.constraints, varPrefix, filter.path, '?s'), ...wheres].join('\n ');
}
@@ -89,10 +100,11 @@ export function wherePatterns(where, hostHops, nodeVar, varPrefix) {
// A condition: the hops from startVar and the test on where they end. A
// filter starts at ?s; a rep:where starts at a node of its host and, with no
// hops left, tests that node itself. sh:maxCount 0 means "none such", so its
// hops only live inside the NOT EXISTS.
// hops and any value test only live inside the NOT EXISTS.
export function conditionPattern(c, varPrefix, hops, startVar) {
if (c.maxCount === 0) {
return `FILTER NOT EXISTS { ${hopsPattern(hops, varPrefix + '_nex', startVar)} }`;
const { maxCount, ...test } = c;
return `FILTER NOT EXISTS { ${conditionPattern(test, varPrefix + '_nex', hops, startVar)} }`;
}
const v = hops.length ? `?${varPrefix}_v` : startVar;
const parts = hops.length ? [hopsPattern(hops, varPrefix, startVar)] : [];

View File

@@ -13,8 +13,13 @@ export function parseSpec(turtle) {
const quads = parser.parse(turtle);
const store = new N3.Store(quads);
const specNode = store.getQuads(null, RDF + 'type', REP + 'ReportSpec')[0]?.subject;
if (!specNode) throw new Error('spec has no rep:ReportSpec');
const specNodes = store.getQuads(null, RDF + 'type', REP + 'ReportSpec').map(q => q.subject);
if (!specNodes.length) throw new Error('spec has no rep:ReportSpec');
// One spec is one report. A second one would pass unchecked and never run.
if (specNodes.length > 1) {
throw new Error(`one spec per answer. This Turtle has ${specNodes.length} rep:ReportSpec. Keep one, and offer the other as the next report.`);
}
const specNode = specNodes[0];
const one = (s, pred) => store.getQuads(s, pred, null)[0]?.object || null;
@@ -31,7 +36,7 @@ export function parseSpec(turtle) {
return { uri: specNode.value, title, profileUri, targetClass, entity, filters, columns };
}
function parseFilter(store, p) {
export function parseFilter(store, p) {
let path = [];
const pathQuad = store.getQuads(p, SH + 'path', null)[0]?.object;
if (pathQuad) path = parsePath(store, pathQuad);
@@ -134,7 +139,7 @@ function readConstraints(store, p, { withLabel = false } = {}) {
return constraints;
}
function readRdfList(store, node) {
export function readRdfList(store, node) {
const items = [];
let cur = node;
while (cur && cur.value !== RDF + 'nil') {