Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SHACL valdiation not scaling #310

Open
madiganz opened this issue Jun 27, 2023 · 0 comments
Open

SHACL valdiation not scaling #310

madiganz opened this issue Jun 27, 2023 · 0 comments

Comments

@madiganz
Copy link

n10s version: 4.4.0

I have shacl validations setup through the call:
CALL apoc.trigger.add("shacl-validate", "call n10s.validation.shacl.validateTransaction($createdNodes, $createdRelationships, $assignedLabels, {}, $assignedNodeProperties, {}, $deletedRelationships, $deletedNodes)", { phase: "before" });

The shacl file is pretty basic. It validates each property usually for datatype, min/max value, and validations relationships. When trying to insert nodes, this is extremely fast when the node count isn't very large. When there are say 5 million nodes, the schema validations take an incredible long time (over a minute). If I am not mistaken, this validation is supposed to be specific for a transaction and not the entire database so I am unsure why it's not scaling with the number of nodes. Any suggestions would be greatly appreciated.

This is a subset of the shacl file as an example

with '
@prefix sh:      <http://www.w3.org/ns/shacl#> .
@prefix xsd:     <http://www.w3.org/2001/XMLSchema#> .
@prefix rdf:     <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix neovoc:  <neo4j://vocabulary#> .
@prefix n4sch:   <neo4j://graph.schema#> .

# Node Shapes

neovoc:PersonShape
    a sh:NodeShape ;
    sh:ignoredProperties ( rdf:type ) ;
    sh:targetClass n4sch:Person;
    sh:property neovoc:IdProperty ;
    sh:property neovoc:NotRequiredExternalIdProperty ;
    sh:property neovoc:CreatedAtProperty ;
    sh:property neovoc:CreatedByProperty ;
    sh:property neovoc:UpdatedAtProperty ;
    sh:property neovoc:UpdatedByProperty .

neovoc:IdProperty
    a sh:PropertyShape ;
    sh:path n4sch:Id ;
    sh:pattern "[0-9a-fA-F]{8}\\-[0-9a-fA-F]{4}\\-[0-9a-fA-F]{4}\\-[0-9a-fA-F]{4}\\-[0-9a-fA-F]{12}" ;
    sh:minCount 1 ;
    sh:maxCount 1 ;
    sh:datatype xsd:string .

neovoc:NotRequiredExternalIdProperty
    a sh:PropertyShape ;
    sh:path n4sch:ExternalId ;
    sh:datatype xsd:string .

neovoc:CreatedAtProperty
  a sh:PropertyShape ;
  sh:path n4sch:CreatedAt ;
  sh:minCount 1 ;
  sh:maxCount 1 ;
  sh:datatype xsd:dateTime .

neovoc:CreatedByProperty
  a sh:PropertyShape ;
  sh:path n4sch:CreatedBy ;
  sh:minCount 1 ;
  sh:maxCount 1 ;
  sh:datatype xsd:string .

neovoc:UpdatedAtProperty
  a sh:PropertyShape ;
  sh:path n4sch:UpdatedAt ;
  sh:datatype xsd:dateTime .

neovoc:UpdatedByProperty
  a sh:PropertyShape ;
  sh:path n4sch:UpdatedBy ;
  sh:datatype xsd:string .
' as shacl
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant